# Self-Hosted Stool Vision Decision — 2026-08-19

## Executive verdict

**Self-hosting removes the provider-shyness roadblock, but it does not remove the accuracy roadblock.** Timmy now has a working self-hosted profile, a local `llama.cpp` server path—using its OpenAI-compatible server capability[6]—provider readiness reporting, a consent-traceable training-image ingester, and a real-photo runtime receipt.

The recommended bootstrap model is **SmolVLM2-2.2B-Instruct Q4_K_M plus its Q8 vision projector**. The base model is Apache-2.0 and explicitly accepts image and text inputs.[11] The official GGUF repository provides the quantized main model and projector needed by `llama.cpp`.[10]

Do not ship this general-purpose VLM as the final Bristol classifier. On a real public CC-licensed Type 4 stool photograph, it did not refuse the image and emitted valid JSON, but it estimated Type 1 with confidence 0.5. Timmy's validator correctly abstained. That proves sovereign transport and content acceptance; it disproves product-grade classification accuracy.

## What was actually executed

### Host

- 4 virtual CPU cores at 2.0 GHz
- 7.8 GiB RAM
- no GPU
- no swap

### Real stool input

The test image was the Wikimedia Commons file **Human Feces Bristol Stool Chart Type 4**, labeled Type 4 by its source and licensed CC BY-SA 3.0.[7] A derived test copy was resized, re-encoded, and stripped of EXIF before inference.

### SmolVLM-500M-Instruct Q8

The 500M model is Apache-2.0, is designed for image/text input, and its card says one-image inference can use 1.23 GB of GPU RAM.[1] The official GGUF files used in the spike were a 436,806,912-byte main model and a 108,783,360-byte Q8 projector.[2]

Observed on this CPU host:

- resident memory: **908,420 KiB**
- structured request latency: **25.606 s**
- detected stool: **yes**
- Bristol type: **no usable value**
- schema: malformed/incomplete
- refusal: **none**

Verdict: useful only as a cheap stool/not-stool or quality-gate experiment; too weak for the product label.

### SmolVLM2-2.2B-Instruct Q4_K_M

Observed on this CPU host:

- model plus projector on disk: about **1.6 GiB**
- resident memory: **3,362,048 KiB**
- cold structured request: **89.259 s**
- subsequent Timmy end-to-end request: **15.904 s**
- detected stool: **yes**
- predicted color: **brown**
- predicted Bristol type: **1** (source label: Type 4)
- confidence: **0.5**
- image quality: **poor**
- Timmy result: **abstained and requested user input**
- refusal: **none**

Verdict: **adopt for bootstrap labeling experiments, reject as the final classifier.** The current VPS proves the path but lacks the latency and operational headroom for a smooth production service.

## Candidate decision

| Candidate | License observed | Current weight footprint | Decision |
|---|---|---:|---|
| SmolVLM2-2.2B-Instruct | Apache-2.0[11] | 1.6 GiB GGUF bundle used | **Bootstrap choice**; no refusal, structured output, inadequate Bristol accuracy |
| SmolVLM-500M-Instruct | Apache-2.0[1] | 521 MiB GGUF bundle used | Quality/admission experiments only |
| Moondream 2 | Apache-2.0, but its own card calls it the previous generation[5] | 3.85 GB BF16 model file observed | Do not start new work here |
| Qwen2.5-VL-3B-Instruct | Qwen Research License[3] | 7.51 GB BF16 shards observed | **Reject for product default**: the license defines use as non-commercial research/evaluation and requires a separate license for commercial use.[4] |

## Production architecture

```text
phone camera
  -> client resize + metadata removal
  -> Timmy ingress: decode, MIME/magic-byte check, size cap, request ID
  -> self-hosted visual gate: stool / not-stool / unusable
  -> specialist classifier: Type 1..7 probabilities + color probabilities
  -> calibration + abstention policy
  -> strict Timmy JSON schema
  -> user reviews and confirms
  -> local journal save
  -> separate, explicit opt-in training contribution
```

### Model responsibilities

1. **General VLM:** bootstrap descriptions, reject non-stool images, flag blur/occlusion, and help reviewers. It never supplies the authoritative training label.
2. **Human confirmation:** the user selects or corrects Bristol type and color. A clinical reviewer adjudicates the evaluation set and ambiguous examples.
3. **Specialist discriminative model:** train a small image classifier for `not-stool`, `unknown`, and Types 1–7, with separate color and quality heads. This is the production inference model.
4. **Timmy policy layer:** apply calibrated confidence thresholds, abstain, require confirmation, and keep symptoms entirely user-entered.

This structure matters because published stool-image work exists—including a long-term automated smart-toilet feasibility study[9]—but accessible training data is scarce. One 2024 pilot analyzed 151 images from only five hospitalized patients, and the paper says its data is not publicly available except for a possible limited request.[8] We therefore need our own consented dataset rather than pretending a generic VLM already solves the domain.

## Concrete consented-data pipeline

Implemented now: `scripts/ingest_training_photo.py`.

For every explicitly contributed image it:

- requires a consent version and human-review status;
- decodes the image rather than trusting its extension;
- applies EXIF orientation, then strips metadata;
- resizes to a maximum 1200-pixel dimension;
- hashes the derived bytes;
- creates a salted pseudonymous subject key;
- assigns train/validation/test by subject, not by image;
- records Bristol type, broad color, quality, and review provenance in JSONL;
- never writes the original source path into the manifest.

Keeping every image from one subject in one split prevents the model from looking better merely because near-duplicate photos from the same person leaked into training and test sets.

## Dataset ladder

### Phase A — 100-image reality check

- Collect opt-in images across all seven types and non-stool/poor-quality negatives.
- Every label is user-confirmed; ambiguous items are clinician-reviewed.
- Compare the VLM, a frozen image encoder plus linear head, and a small fine-tuned classifier.
- Stop if the labels are too inconsistent to support the seven-way task.

### Phase B — minimum credible training corpus

Target at least:

- **250 independently reviewed examples per Bristol type**;
- **200+ distinct contributors**;
- hard negatives: empty toilets, toilet paper, urine-only, glare, blur, occlusion, diapers, and non-toilet contexts;
- held-out contributors and devices;
- intentionally varied lighting, bowl colors, water tint, distance, and camera quality.

These are engineering gates, not a guarantee of medical validity.

### Phase C — release gates

A build cannot prefill unless it demonstrates on held-out contributors:

- macro-F1 of at least **0.80** across Types 1–7;
- at least **0.90 precision** among non-abstained suggestions;
- expected calibration error at or below **0.05**;
- valid schema on **100%** of responses;
- no original-image persistence in transient inference;
- no training use without an explicit contribution record;
- zero automatic symptom, disease, or bleeding conclusions.

If confidence is below the calibrated threshold, Timmy asks the user. Coverage is allowed to fall; false certainty is not.

## Hardware gate

- **Current VPS:** accepted for a single-user engineering spike only. It ran the model, but 89-second cold latency, 3.36 GiB model RSS, four saturated CPU cores, no swap, and no GPU leave inadequate production margin.
- **Bootstrap inference / fine-tuning:** use a separate NVIDIA GPU worker with **16 GB VRAM recommended**. Bind inference privately; Timmy remains the public API and release authority.
- **Later production classifier:** after collecting data, export the specialist model to ONNX/TensorRT. A small classifier should be much faster and cheaper than keeping a VLM in the hot path, and may ultimately run on CPU or device.

## Shipped controls

- `TIMMY_VISION_PROFILE=selfhost`
- loopback OpenAI-compatible default: `http://127.0.0.1:8080/v1`
- self-hosted privacy status from `/api/vision-status`
- upstream `/v1/models` readiness probe
- API-key redaction from public status
- 120-second bounded timeout for self-hosted inference
- `scripts/run_selfhost_smolvlm.sh`
- `scripts/ingest_training_photo.py`
- fail-closed visual schema and manual fallback

## Decision

**Proceed with self-hosting, but treat SmolVLM2 as the bootstrap worker—not the product brain.** It solves “the provider refuses the shit picture.” The consented dataset plus specialist classifier solves “turn the shit picture into reliable concrete data.”

## Sources

[1] https://huggingface.co/HuggingFaceTB/SmolVLM-500M-Instruct — SmolVLM-500M-Instruct model card
[2] https://huggingface.co/ggml-org/SmolVLM-500M-Instruct-GGUF — Official SmolVLM GGUF conversion
[3] https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct — Qwen2.5-VL-3B-Instruct model card
[4] https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct/blob/main/LICENSE — Qwen research license
[5] https://huggingface.co/vikhyatk/moondream2 — Moondream 2 model card
[6] https://github.com/ggml-org/llama.cpp — llama.cpp repository
[7] https://commons.wikimedia.org/wiki/File:Human_Feces_Bristol_Stool_Chart_Type_4.jpg — Wikimedia Commons: Human Feces Bristol Stool Chart Type 4
[8] https://europepmc.org/articles/PMC11350077 — AI- and physician-interpreted stool image pilot study
[9] https://europepmc.org/articles/PMC11686500 — Long-term automated stool monitoring smart toilet feasibility study
[10] https://huggingface.co/ggml-org/SmolVLM2-2.2B-Instruct-GGUF — Official SmolVLM2 2.2B GGUF conversion
[11] https://huggingface.co/HuggingFaceTB/SmolVLM2-2.2B-Instruct — SmolVLM2-2.2B-Instruct model card