178 lines
9.6 KiB
Markdown
178 lines
9.6 KiB
Markdown
# Self-Hosted Stool Vision Decision — 2026-08-19
|
||
|
||
## Executive verdict
|
||
|
||
**Self-hosting removes the provider-shyness roadblock, but it does not remove the accuracy roadblock.** Timmy now has a working self-hosted profile, a local `llama.cpp` server path—using its OpenAI-compatible server capability[6]—provider readiness reporting, a consent-traceable training-image ingester, and a real-photo runtime receipt.
|
||
|
||
The recommended bootstrap model is **SmolVLM2-2.2B-Instruct Q4_K_M plus its Q8 vision projector**. The base model is Apache-2.0 and explicitly accepts image and text inputs.[11] The official GGUF repository provides the quantized main model and projector needed by `llama.cpp`.[10]
|
||
|
||
Do not ship this general-purpose VLM as the final Bristol classifier. On a real public CC-licensed Type 4 stool photograph, it did not refuse the image and emitted valid JSON, but it estimated Type 1 with confidence 0.5. Timmy's validator correctly abstained. That proves sovereign transport and content acceptance; it disproves product-grade classification accuracy.
|
||
|
||
## What was actually executed
|
||
|
||
### Host
|
||
|
||
- 4 virtual CPU cores at 2.0 GHz
|
||
- 7.8 GiB RAM
|
||
- no GPU
|
||
- no swap
|
||
|
||
### Real stool input
|
||
|
||
The test image was the Wikimedia Commons file **Human Feces Bristol Stool Chart Type 4**, labeled Type 4 by its source and licensed CC BY-SA 3.0.[7] A derived test copy was resized, re-encoded, and stripped of EXIF before inference.
|
||
|
||
### SmolVLM-500M-Instruct Q8
|
||
|
||
The 500M model is Apache-2.0, is designed for image/text input, and its card says one-image inference can use 1.23 GB of GPU RAM.[1] The official GGUF files used in the spike were a 436,806,912-byte main model and a 108,783,360-byte Q8 projector.[2]
|
||
|
||
Observed on this CPU host:
|
||
|
||
- resident memory: **908,420 KiB**
|
||
- structured request latency: **25.606 s**
|
||
- detected stool: **yes**
|
||
- Bristol type: **no usable value**
|
||
- schema: malformed/incomplete
|
||
- refusal: **none**
|
||
|
||
Verdict: useful only as a cheap stool/not-stool or quality-gate experiment; too weak for the product label.
|
||
|
||
### SmolVLM2-2.2B-Instruct Q4_K_M
|
||
|
||
Observed on this CPU host:
|
||
|
||
- model plus projector on disk: about **1.6 GiB**
|
||
- resident memory: **3,362,048 KiB**
|
||
- cold structured request: **89.259 s**
|
||
- subsequent Timmy end-to-end request: **15.904 s**
|
||
- detected stool: **yes**
|
||
- predicted color: **brown**
|
||
- predicted Bristol type: **1** (source label: Type 4)
|
||
- confidence: **0.5**
|
||
- image quality: **poor**
|
||
- Timmy result: **abstained and requested user input**
|
||
- refusal: **none**
|
||
|
||
Verdict: **adopt for bootstrap labeling experiments, reject as the final classifier.** The current VPS proves the path but lacks the latency and operational headroom for a smooth production service.
|
||
|
||
## Candidate decision
|
||
|
||
| Candidate | License observed | Current weight footprint | Decision |
|
||
|---|---|---:|---|
|
||
| SmolVLM2-2.2B-Instruct | Apache-2.0[11] | 1.6 GiB GGUF bundle used | **Bootstrap choice**; no refusal, structured output, inadequate Bristol accuracy |
|
||
| SmolVLM-500M-Instruct | Apache-2.0[1] | 521 MiB GGUF bundle used | Quality/admission experiments only |
|
||
| Moondream 2 | Apache-2.0, but its own card calls it the previous generation[5] | 3.85 GB BF16 model file observed | Do not start new work here |
|
||
| Qwen2.5-VL-3B-Instruct | Qwen Research License[3] | 7.51 GB BF16 shards observed | **Reject for product default**: the license defines use as non-commercial research/evaluation and requires a separate license for commercial use.[4] |
|
||
|
||
## Production architecture
|
||
|
||
```text
|
||
phone camera
|
||
-> client resize + metadata removal
|
||
-> Timmy ingress: decode, MIME/magic-byte check, size cap, request ID
|
||
-> self-hosted visual gate: stool / not-stool / unusable
|
||
-> specialist classifier: Type 1..7 probabilities + color probabilities
|
||
-> calibration + abstention policy
|
||
-> strict Timmy JSON schema
|
||
-> user reviews and confirms
|
||
-> local journal save
|
||
-> separate, explicit opt-in training contribution
|
||
```
|
||
|
||
### Model responsibilities
|
||
|
||
1. **General VLM:** bootstrap descriptions, reject non-stool images, flag blur/occlusion, and help reviewers. It never supplies the authoritative training label.
|
||
2. **Human confirmation:** the user selects or corrects Bristol type and color. A clinical reviewer adjudicates the evaluation set and ambiguous examples.
|
||
3. **Specialist discriminative model:** train a small image classifier for `not-stool`, `unknown`, and Types 1–7, with separate color and quality heads. This is the production inference model.
|
||
4. **Timmy policy layer:** apply calibrated confidence thresholds, abstain, require confirmation, and keep symptoms entirely user-entered.
|
||
|
||
This structure matters because published stool-image work exists—including a long-term automated smart-toilet feasibility study[9]—but accessible training data is scarce. One 2024 pilot analyzed 151 images from only five hospitalized patients, and the paper says its data is not publicly available except for a possible limited request.[8] We therefore need our own consented dataset rather than pretending a generic VLM already solves the domain.
|
||
|
||
## Concrete consented-data pipeline
|
||
|
||
Implemented now: `scripts/ingest_training_photo.py`.
|
||
|
||
For every explicitly contributed image it:
|
||
|
||
- requires a consent version and human-review status;
|
||
- decodes the image rather than trusting its extension;
|
||
- applies EXIF orientation, then strips metadata;
|
||
- resizes to a maximum 1200-pixel dimension;
|
||
- hashes the derived bytes;
|
||
- creates a salted pseudonymous subject key;
|
||
- assigns train/validation/test by subject, not by image;
|
||
- records Bristol type, broad color, quality, and review provenance in JSONL;
|
||
- never writes the original source path into the manifest.
|
||
|
||
Keeping every image from one subject in one split prevents the model from looking better merely because near-duplicate photos from the same person leaked into training and test sets.
|
||
|
||
## Dataset ladder
|
||
|
||
### Phase A — 100-image reality check
|
||
|
||
- Collect opt-in images across all seven types and non-stool/poor-quality negatives.
|
||
- Every label is user-confirmed; ambiguous items are clinician-reviewed.
|
||
- Compare the VLM, a frozen image encoder plus linear head, and a small fine-tuned classifier.
|
||
- Stop if the labels are too inconsistent to support the seven-way task.
|
||
|
||
### Phase B — minimum credible training corpus
|
||
|
||
Target at least:
|
||
|
||
- **250 independently reviewed examples per Bristol type**;
|
||
- **200+ distinct contributors**;
|
||
- hard negatives: empty toilets, toilet paper, urine-only, glare, blur, occlusion, diapers, and non-toilet contexts;
|
||
- held-out contributors and devices;
|
||
- intentionally varied lighting, bowl colors, water tint, distance, and camera quality.
|
||
|
||
These are engineering gates, not a guarantee of medical validity.
|
||
|
||
### Phase C — release gates
|
||
|
||
A build cannot prefill unless it demonstrates on held-out contributors:
|
||
|
||
- macro-F1 of at least **0.80** across Types 1–7;
|
||
- at least **0.90 precision** among non-abstained suggestions;
|
||
- expected calibration error at or below **0.05**;
|
||
- valid schema on **100%** of responses;
|
||
- no original-image persistence in transient inference;
|
||
- no training use without an explicit contribution record;
|
||
- zero automatic symptom, disease, or bleeding conclusions.
|
||
|
||
If confidence is below the calibrated threshold, Timmy asks the user. Coverage is allowed to fall; false certainty is not.
|
||
|
||
## Hardware gate
|
||
|
||
- **Current VPS:** accepted for a single-user engineering spike only. It ran the model, but 89-second cold latency, 3.36 GiB model RSS, four saturated CPU cores, no swap, and no GPU leave inadequate production margin.
|
||
- **Bootstrap inference / fine-tuning:** use a separate NVIDIA GPU worker with **16 GB VRAM recommended**. Bind inference privately; Timmy remains the public API and release authority.
|
||
- **Later production classifier:** after collecting data, export the specialist model to ONNX/TensorRT. A small classifier should be much faster and cheaper than keeping a VLM in the hot path, and may ultimately run on CPU or device.
|
||
|
||
## Shipped controls
|
||
|
||
- `TIMMY_VISION_PROFILE=selfhost`
|
||
- loopback OpenAI-compatible default: `http://127.0.0.1:8080/v1`
|
||
- self-hosted privacy status from `/api/vision-status`
|
||
- upstream `/v1/models` readiness probe
|
||
- API-key redaction from public status
|
||
- 120-second bounded timeout for self-hosted inference
|
||
- `scripts/run_selfhost_smolvlm.sh`
|
||
- `scripts/ingest_training_photo.py`
|
||
- fail-closed visual schema and manual fallback
|
||
|
||
## Decision
|
||
|
||
**Proceed with self-hosting, but treat SmolVLM2 as the bootstrap worker—not the product brain.** It solves “the provider refuses the shit picture.” The consented dataset plus specialist classifier solves “turn the shit picture into reliable concrete data.”
|
||
|
||
## Sources
|
||
|
||
[1] https://huggingface.co/HuggingFaceTB/SmolVLM-500M-Instruct — SmolVLM-500M-Instruct model card
|
||
[2] https://huggingface.co/ggml-org/SmolVLM-500M-Instruct-GGUF — Official SmolVLM GGUF conversion
|
||
[3] https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct — Qwen2.5-VL-3B-Instruct model card
|
||
[4] https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct/blob/main/LICENSE — Qwen research license
|
||
[5] https://huggingface.co/vikhyatk/moondream2 — Moondream 2 model card
|
||
[6] https://github.com/ggml-org/llama.cpp — llama.cpp repository
|
||
[7] https://commons.wikimedia.org/wiki/File:Human_Feces_Bristol_Stool_Chart_Type_4.jpg — Wikimedia Commons: Human Feces Bristol Stool Chart Type 4
|
||
[8] https://europepmc.org/articles/PMC11350077 — AI- and physician-interpreted stool image pilot study
|
||
[9] https://europepmc.org/articles/PMC11686500 — Long-term automated stool monitoring smart toilet feasibility study
|
||
[10] https://huggingface.co/ggml-org/SmolVLM2-2.2B-Instruct-GGUF — Official SmolVLM2 2.2B GGUF conversion
|
||
[11] https://huggingface.co/HuggingFaceTB/SmolVLM2-2.2B-Instruct — SmolVLM2-2.2B-Instruct model card
|