stackchain/timmy-talking-turd
1
0
Fork 0
Code Issues 41 Pull Requests 6 Actions Packages Projects Releases 15 Wiki Activity
47eec47a31
timmy-talking-turd/research/SELF-HOSTED-STOOL-VISION.md

9.6 KiB
Raw Blame History

Self-Hosted Stool Vision Decision — 2026-08-19

Executive verdict

Self-hosting removes the provider-shyness roadblock, but it does not remove the accuracy roadblock. Timmy now has a working self-hosted profile, a local llama.cpp server path—using its OpenAI-compatible server capability[6]—provider readiness reporting, a consent-traceable training-image ingester, and a real-photo runtime receipt.

The recommended bootstrap model is SmolVLM2-2.2B-Instruct Q4_K_M plus its Q8 vision projector. The base model is Apache-2.0 and explicitly accepts image and text inputs.[11] The official GGUF repository provides the quantized main model and projector needed by llama.cpp.[10]

Do not ship this general-purpose VLM as the final Bristol classifier. On a real public CC-licensed Type 4 stool photograph, it did not refuse the image and emitted valid JSON, but it estimated Type 1 with confidence 0.5. Timmy's validator correctly abstained. That proves sovereign transport and content acceptance; it disproves product-grade classification accuracy.

What was actually executed

Host

  • 4 virtual CPU cores at 2.0 GHz
  • 7.8 GiB RAM
  • no GPU
  • no swap

Real stool input

The test image was the Wikimedia Commons file Human Feces Bristol Stool Chart Type 4, labeled Type 4 by its source and licensed CC BY-SA 3.0.[7] A derived test copy was resized, re-encoded, and stripped of EXIF before inference.

SmolVLM-500M-Instruct Q8

The 500M model is Apache-2.0, is designed for image/text input, and its card says one-image inference can use 1.23 GB of GPU RAM.[1] The official GGUF files used in the spike were a 436,806,912-byte main model and a 108,783,360-byte Q8 projector.[2]

Observed on this CPU host:

  • resident memory: 908,420 KiB
  • structured request latency: 25.606 s
  • detected stool: yes
  • Bristol type: no usable value
  • schema: malformed/incomplete
  • refusal: none

Verdict: useful only as a cheap stool/not-stool or quality-gate experiment; too weak for the product label.

SmolVLM2-2.2B-Instruct Q4_K_M

Observed on this CPU host:

  • model plus projector on disk: about 1.6 GiB
  • resident memory: 3,362,048 KiB
  • cold structured request: 89.259 s
  • subsequent Timmy end-to-end request: 15.904 s
  • detected stool: yes
  • predicted color: brown
  • predicted Bristol type: 1 (source label: Type 4)
  • confidence: 0.5
  • image quality: poor
  • Timmy result: abstained and requested user input
  • refusal: none

Verdict: adopt for bootstrap labeling experiments, reject as the final classifier. The current VPS proves the path but lacks the latency and operational headroom for a smooth production service.

Candidate decision

Candidate License observed Current weight footprint Decision
SmolVLM2-2.2B-Instruct Apache-2.0[11] 1.6 GiB GGUF bundle used Bootstrap choice; no refusal, structured output, inadequate Bristol accuracy
SmolVLM-500M-Instruct Apache-2.0[1] 521 MiB GGUF bundle used Quality/admission experiments only
Moondream 2 Apache-2.0, but its own card calls it the previous generation[5] 3.85 GB BF16 model file observed Do not start new work here
Qwen2.5-VL-3B-Instruct Qwen Research License[3] 7.51 GB BF16 shards observed Reject for product default: the license defines use as non-commercial research/evaluation and requires a separate license for commercial use.[4]

Production architecture

phone camera
  -> client resize + metadata removal
  -> Timmy ingress: decode, MIME/magic-byte check, size cap, request ID
  -> self-hosted visual gate: stool / not-stool / unusable
  -> specialist classifier: Type 1..7 probabilities + color probabilities
  -> calibration + abstention policy
  -> strict Timmy JSON schema
  -> user reviews and confirms
  -> local journal save
  -> separate, explicit opt-in training contribution

Model responsibilities

  1. General VLM: bootstrap descriptions, reject non-stool images, flag blur/occlusion, and help reviewers. It never supplies the authoritative training label.
  2. Human confirmation: the user selects or corrects Bristol type and color. A clinical reviewer adjudicates the evaluation set and ambiguous examples.
  3. Specialist discriminative model: train a small image classifier for not-stool, unknown, and Types 1–7, with separate color and quality heads. This is the production inference model.
  4. Timmy policy layer: apply calibrated confidence thresholds, abstain, require confirmation, and keep symptoms entirely user-entered.

This structure matters because published stool-image work exists—including a long-term automated smart-toilet feasibility study[9]—but accessible training data is scarce. One 2024 pilot analyzed 151 images from only five hospitalized patients, and the paper says its data is not publicly available except for a possible limited request.[8] We therefore need our own consented dataset rather than pretending a generic VLM already solves the domain.

Concrete consented-data pipeline

Implemented now: scripts/ingest_training_photo.py.

For every explicitly contributed image it:

  • requires a consent version and human-review status;
  • decodes the image rather than trusting its extension;
  • applies EXIF orientation, then strips metadata;
  • resizes to a maximum 1200-pixel dimension;
  • hashes the derived bytes;
  • creates a salted pseudonymous subject key;
  • assigns train/validation/test by subject, not by image;
  • records Bristol type, broad color, quality, and review provenance in JSONL;
  • never writes the original source path into the manifest.

Keeping every image from one subject in one split prevents the model from looking better merely because near-duplicate photos from the same person leaked into training and test sets.

Dataset ladder

Phase A — 100-image reality check

  • Collect opt-in images across all seven types and non-stool/poor-quality negatives.
  • Every label is user-confirmed; ambiguous items are clinician-reviewed.
  • Compare the VLM, a frozen image encoder plus linear head, and a small fine-tuned classifier.
  • Stop if the labels are too inconsistent to support the seven-way task.

Phase B — minimum credible training corpus

Target at least:

  • 250 independently reviewed examples per Bristol type;
  • 200+ distinct contributors;
  • hard negatives: empty toilets, toilet paper, urine-only, glare, blur, occlusion, diapers, and non-toilet contexts;
  • held-out contributors and devices;
  • intentionally varied lighting, bowl colors, water tint, distance, and camera quality.

These are engineering gates, not a guarantee of medical validity.

Phase C — release gates

A build cannot prefill unless it demonstrates on held-out contributors:

  • macro-F1 of at least 0.80 across Types 1–7;
  • at least 0.90 precision among non-abstained suggestions;
  • expected calibration error at or below 0.05;
  • valid schema on 100% of responses;
  • no original-image persistence in transient inference;
  • no training use without an explicit contribution record;
  • zero automatic symptom, disease, or bleeding conclusions.

If confidence is below the calibrated threshold, Timmy asks the user. Coverage is allowed to fall; false certainty is not.

Hardware gate

  • Current VPS: accepted for a single-user engineering spike only. It ran the model, but 89-second cold latency, 3.36 GiB model RSS, four saturated CPU cores, no swap, and no GPU leave inadequate production margin.
  • Bootstrap inference / fine-tuning: use a separate NVIDIA GPU worker with 16 GB VRAM recommended. Bind inference privately; Timmy remains the public API and release authority.
  • Later production classifier: after collecting data, export the specialist model to ONNX/TensorRT. A small classifier should be much faster and cheaper than keeping a VLM in the hot path, and may ultimately run on CPU or device.

Shipped controls

  • TIMMY_VISION_PROFILE=selfhost
  • loopback OpenAI-compatible default: http://127.0.0.1:8080/v1
  • self-hosted privacy status from /api/vision-status
  • upstream /v1/models readiness probe
  • API-key redaction from public status
  • 120-second bounded timeout for self-hosted inference
  • scripts/run_selfhost_smolvlm.sh
  • scripts/ingest_training_photo.py
  • fail-closed visual schema and manual fallback

Decision

Proceed with self-hosting, but treat SmolVLM2 as the bootstrap worker—not the product brain. It solves “the provider refuses the shit picture.” The consented dataset plus specialist classifier solves “turn the shit picture into reliable concrete data.”

Sources

[1] https://huggingface.co/HuggingFaceTB/SmolVLM-500M-Instruct — SmolVLM-500M-Instruct model card [2] https://huggingface.co/ggml-org/SmolVLM-500M-Instruct-GGUF — Official SmolVLM GGUF conversion [3] https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct — Qwen2.5-VL-3B-Instruct model card [4] https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct/blob/main/LICENSE — Qwen research license [5] https://huggingface.co/vikhyatk/moondream2 — Moondream 2 model card [6] https://github.com/ggml-org/llama.cpp — llama.cpp repository [7] https://commons.wikimedia.org/wiki/File:Human_Feces_Bristol_Stool_Chart_Type_4.jpg — Wikimedia Commons: Human Feces Bristol Stool Chart Type 4 [8] https://europepmc.org/articles/PMC11350077 — AI- and physician-interpreted stool image pilot study [9] https://europepmc.org/articles/PMC11686500 — Long-term automated stool monitoring smart toilet feasibility study [10] https://huggingface.co/ggml-org/SmolVLM2-2.2B-Instruct-GGUF — Official SmolVLM2 2.2B GGUF conversion [11] https://huggingface.co/HuggingFaceTB/SmolVLM2-2.2B-Instruct — SmolVLM2-2.2B-Instruct model card