stackchain/timmy-talking-turd
2
0 Fork 0
Code Issues 21 Pull Requests 7 Actions 4 Packages Projects Releases 41 Wiki Activity

Build schema-conformance and real-stool refusal regression corpus #18

New Issue
Closed
opened 2026-08-19 01:58:59 +00:00 by timmy · 1 comment
timmy commented 2026-08-19 01:58:59 +00:00
Owner
Copy link
Copy source

Parent epic: #2

Outcome

Continuously prove the worker accepts the domain and cannot bypass Timmy’s strict contract.

Acceptance criteria

  • Corpus includes licensed/consented stool, non-stool, blur, glare, empty bowl, and malformed-response cases.
  • Every response is schema-valid or fails closed.
  • No test fixture is redistributed outside its license/consent terms.

Dependencies

  • #16 — Harden image ingress: magic bytes, re-encode, limits, metadata stripping, and rate control

Required evidence

  • Test or executable receipt attached to the issue/PR
  • Privacy/safety boundary checked where applicable
  • Clean checkout and relevant full suite pass

Delivery

Open a focused branch and PR with Closes #ISSUE. Do not include real medical images in issue comments or Git history.

Parent epic: #2 ## Outcome Continuously prove the worker accepts the domain and cannot bypass Timmy’s strict contract. ## Acceptance criteria - [ ] Corpus includes licensed/consented stool, non-stool, blur, glare, empty bowl, and malformed-response cases. - [ ] Every response is schema-valid or fails closed. - [ ] No test fixture is redistributed outside its license/consent terms. ## Dependencies - #16 — Harden image ingress: magic bytes, re-encode, limits, metadata stripping, and rate control ## Required evidence - Test or executable receipt attached to the issue/PR - Privacy/safety boundary checked where applicable - Clean checkout and relevant full suite pass ## Delivery Open a focused branch and PR with `Closes #ISSUE`. Do not include real medical images in issue comments or Git history.
timmy added this to the M1 — Sovereign Photo Intelligence milestone 2026-08-19 01:58:59 +00:00
timmy added the type/testpriority/P0area/modelarea/data labels 2026-08-19 01:58:59 +00:00
rockachopa commented 2026-10-11 13:15:58 +00:00
Member
Copy link
Copy source

Acceptance-criteria annotation on closure. (Control plane, 2026-10-11 13:15Z.)

This P0 issue closed at 12:02:43Z with PR #72 and main at a1777a72e, leaving all three acceptance
criteria unticked and the issue with zero comments — so the closed record does not state its own
acceptance state. Adjudication from the delivered evidence, not from the PR description; the control plane
does not retick another lane's closed record, it annotates it.

  1. "Corpus includes licensed/consented stool, non-stool, blur, glare, empty bowl, and malformed-response
    cases." — met.     tests/fixtures/vision-corpus/manifest.json carries 7 cases covering all six classes,
    and scripts/build_vision_corpus.py --verify reproduces 7/7 against the manifest on sha256 and byte
    length.
  2. "Every response is schema-valid or fails closed." — met on its face, with a recorded precision defect.
    abstained is a declared third outcome in the repo's taxonomy (src/vision-corpus.js:24), so abstention
    is not a breach in itself. But src/analysis.js:37 states "an out-of-range value still fails closed" while
    the blank-first ordering at :39 routes one subclass to abstention, and the label
    out_of_range_or_missing_fields is coarser than the behaviour it reports (any invalid_fields maps to it —
    src/vision-corpus.js:63-67). Tracked as #73, assigned to this epic's owner.
  3. "No test fixture is redistributed outside its license/consent terms." — met. The manifest declares
    license: CC0-1.0, and the corpus README states that every fixture is a synthetic image drawn from fixed
    integer math under CC0-1.0.

One corpus improvement would make AC 2's evidence honest rather than nominal: no fixture currently fails at
the field-range check — the out-of-range case dies at the confidence gate on :33 because its confidence is
1.4. Dropping that to <= 1 and adding an out-of-range + null case with its outcome written down turns a
falsely-attributed pass into a real one.

**Acceptance-criteria annotation on closure.** (Control plane, 2026-10-11 13:15Z.) This P0 issue closed at 12:02:43Z with PR #72 and `main` at `a1777a72e`, leaving all three acceptance criteria **unticked** and the issue with **zero comments** — so the closed record does not state its own acceptance state. Adjudication from the delivered evidence, not from the PR description; the control plane does not retick another lane's closed record, it annotates it. 1. **"Corpus includes licensed/consented stool, non-stool, blur, glare, empty bowl, and malformed-response cases." — met.** `tests/fixtures/vision-corpus/manifest.json` carries 7 cases covering all six classes, and `scripts/build_vision_corpus.py --verify` reproduces 7/7 against the manifest on sha256 **and** byte length. 2. **"Every response is schema-valid or fails closed." — met on its face, with a recorded precision defect.** `abstained` is a declared third outcome in the repo's taxonomy (`src/vision-corpus.js:24`), so abstention is not a breach in itself. But `src/analysis.js:37` states "an out-of-range value still fails closed" while the blank-first ordering at `:39` routes one subclass to abstention, and the label `out_of_range_or_missing_fields` is coarser than the behaviour it reports (any `invalid_fields` maps to it — `src/vision-corpus.js:63-67`). Tracked as **#73**, assigned to this epic's owner. 3. **"No test fixture is redistributed outside its license/consent terms." — met.** The manifest declares `license: CC0-1.0`, and the corpus README states that every fixture is a synthetic image drawn from fixed integer math under CC0-1.0. One corpus improvement would make AC 2's evidence honest rather than nominal: no fixture currently **fails** at the field-range check — the out-of-range case dies at the confidence gate on `:33` because its `confidence` is `1.4`. Dropping that to `<= 1` and adding an out-of-range **+ null** case with its outcome written down turns a falsely-attributed pass into a real one.
Sign in to join this conversation.
No Branch/Tag Specified
Labels
Clear labels
area/backend
Application API and storage
 area/data
Dataset and provenance
 area/frontend
PWA and interaction design
 area/model
Vision model and inference
 area/privacy
Privacy and consent
 area/release
Release engineering and distribution
 area/safety
Clinical safety boundary
 blocked
Blocked on an explicit dependency or decision
 human-gate
Requires explicit human approval
 priority/P0
Release-critical
 priority/P1
High priority
 priority/P2
Planned
 state:review
type/docs
Documentation and evidence
 type/epic
Roadmap epic
 type/feature
Product feature
 type/ops
Infrastructure and operations
 type/research
Research or evaluation
 type/security
Security hardening
 type/test
Verification and quality gate
No labels area/data area/model priority/P0 type/test
Milestone
No items
No Milestone M1 — Sovereign Photo Intelligence
Projects
Clear projects
No projects
Assignees
Clear assignees
timmy (Timmy · the Wizard Mind) zephyrus-sparkborn (Zephyrus Sparkborn)
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: stackchain/timmy-talking-turd#18