timmy-talking-turd/AI-EVIDENCE.md

46 lines
2.1 KiB
Markdown

# AI Analysis Evidence — 2026-08-19
## Hosted path
- Transport: Timmy `/api/analyze` → local `hermes proxy` → Nous Portal OpenAI-compatible API.
- Model: `stepfun/step-3.7-flash:free`.
- Browser/app screenshot: `status: needs_user_input`, `isStool: false`.
- Synthetic illustration: `status: needs_user_input`, `isStool: false`.
## Self-hosted open-weight path
- Runtime: locally compiled `llama.cpp`, commit `6d05498314db1b57f81c271080018aa2d0b89be9`.
- API: loopback OpenAI-compatible `llama-server`.
- Real input: metadata-stripped derivative of Wikimedia Commons **Human Feces Bristol Stool Chart Type 4**, SHA-256 `b3df757f75b83d20050675d14db9ac59d94e5065581748bc873bc5783782a63a`.
- The source image is CC BY-SA 3.0 and explicitly labeled Type 4 by its source.
### SmolVLM-500M-Instruct Q8
- No content refusal.
- Recognized stool, but did not produce a usable Bristol type.
- Structured request: 25.606 seconds.
- Resident memory: 908,420 KiB.
- Verdict: admission/quality experiments only.
### SmolVLM2-2.2B-Instruct Q4_K_M
- No content refusal.
- Valid schema.
- `isStool: true`.
- Color: `brown`.
- Bristol prediction: Type 1; source label: Type 4.
- Confidence: 0.5; quality: poor.
- Timmy correctly returned `needs_user_input` instead of prefilling.
- Cold structured request: 89.259 seconds.
- Subsequent Timmy end-to-end request: 15.904 seconds.
- Resident memory: 3,362,048 KiB.
- Verdict: sovereign bootstrap worker; not an accurate production classifier.
## Positive UX path
The browser path—photo selection, compression, self-hosted readiness display, self-host-specific consent copy, API response, confidence display, Type/color prefill, confirmation, and preservation of zero-valued urgency/discomfort—passes with a deterministic positive fixture.
## Honest conclusion
The provider-moderation barrier is removed: a Timmy-controlled open-weight service accepts real stool imagery. The classification barrier remains: the generic VLM failed the known Type 4 label, so real prefilling must stay behind abstention while Timmy collects consented, human-confirmed labels for a specialist model. Full decision and sources: `research/SELF-HOSTED-STOOL-VISION.md`.