stackchain/timmy-talking-turd
2
0 Fork 0
Code Issues 21 Pull Requests 7 Actions 4 Packages Projects Releases 41 Wiki Activity

41 Releases 41 Tags

RSS Feed
  • daily-2026-10-27 b191d0427d
    Compare

    Timmy daily-2026-10-27 — blind reviewer and adjudication ledger
    Quality gates / quality (push) Canceled after 0s
    Details
    Pre-Release

    timmy released this 2026-10-27 12:39:09 +00:00 | 21 commits to main since this release

    Prerelease for review — tag daily-2026-10-27 on commit b191d0427ddbb19f982458eeaa093aaaee31a68e (America/New_York release date 2026-10-27, built 2026-10-27T12:23:57Z).

    What changed

    Blind reviewer and adjudication tool for corpus labels (issue #26, PR #88). The corpus manifest already said what was ingested and whether the person still allows it. This release adds the labelling control the photo data-flow threat model names, as a command line with no app screen:

    • scripts/review_corpus.py — a review queue that shows the proposal (Type 4 · brown · good) and withholds every model field. Blindness is structural: build_queue() takes no predictions at all, and reveal() refuses with "stays hidden until a review is committed" while appending nothing.
    • Disagreement is preserved, not smoothed — a review is one append-only line carrying the proposal it judged beside the human decision; a correction must actually change the proposal. An adjudication is a second reviewer's line that names the record it resolves, the overruled review stays in the ledger byte for byte, nobody adjudicates their own review, and a contribution adjudicates once.
    • Withdrawal outranks everything — active_records() gates the queue, reveal, adjudicate, and trainable paths, so a revoked contribution refuses with "was withdrawn and cannot be reviewed, revealed, or trained on". trainable_records() also drops unreviewed and excluded samples, so an unreviewed proposal is never training data.
    • tests/corpus-review.test.py — 21 cases written first and watched fail before the module existed, then one behaviour slice at a time. Two real defects were found by tests, not review: a withdrawn contribution reported "already reviewed" because the duplicate check ran before the lifecycle check, and load_predictions crashed reading a pretty-printed answer file line by line.
    • Release-level gate — scripts/build_release.py now runs the review suite as a release gate (like the corpus and calibration CLIs) and requires four new on-camera captions, so a future release cannot drop blindness or the withdrawal refusal silently.
    • Demo — the recording runs the real queue → reveal (refused) → review → reveal → adjudicate → trainable → revoke sequence against the committed synthetic sample and draws a REVIEW LEDGER card from that actual output, labelled synthetic answer file · no model was measured.

    Scope note, stated plainly: the review tool is a command line, not a screen in the app. This release does not imply a reviewer screen; the recording runs the real commands and shows their real output.

    Merge hygiene: PR #88's head failed the repo's own check:diff gate on a trailing blank line in scripts/review_corpus.py. Fixed with a whitespace-only commit (d54665d) before merge, then check:diff and all suites re-verified — the gate was satisfied, not skipped.

    Completed ticket / PR

    • Issue #26 — Build reviewer/adjudication tool for Bristol, color, quality, and exclusions (closed by this release).
    • PR #88 Blind reviewer and adjudication tool for corpus labels (#26) — merged as b191d04 after diff review: 10 files, +1296/−109, text only (no binaries, weights, medical imagery, or secrets), Closes #26 linked to a non-human-gate issue.

    Gates — 25/25 passed

    npm test 227/227, tests/corpus-review.test.py 21/21, corpus manifest 8/8, calibration harness 19/19, mobile green path, photo-first acceptance, sleek Hermes chat, manual fallback + draft recovery, provisional suggestion state, hard-stool follow-up, vision corpus refusal, abstention quality gate, worker container receipts, worker post-load readiness, longitudinal pattern summary, grounded chat citations, training contribution consent, corpus manifest provenance, reviewer/adjudication blindness, calibration harness gates, portable backup restore, dependency audit (0 high), syntax (node/bash/python), secret scan, forbidden artifacts, demo H.264 720×1280 probe, complete MP4 decode, release delta verified painted on camera (83/83 captions).

    Gate note: the first gated build failed npm test once at malformed and escaping base paths are rejected at startup — that test spawns 19 servers with a fixed 800 ms deadline on a busy 4-core host (load ~11). Proven host contention rather than a regression: the file passed 5/5 isolated runs, and the whole gated build was re-run end to end and passed with 227/227 and all 25 gates. The gate was re-run, not loosened or bypassed.

    Artifacts

    • source archive timmy-talking-turd-2026-10-27-b191d0427ddb.tar.gz — 311,861 bytes — sha256 d40aa477b0aa799099d28291e264594d51f1ef68efdca6338137473dcb6e6ffa
    • demo video timmy-talking-turd-2026-10-27-b191d0427ddb-demo.mp4 — 25,456,205 bytes — H.264 720×1280 yuv420p, 25 fps, 408.2 s — sha256 4c3601c830dfb81261a0ffcba75778e753777807212d9c61ed50dd5384899470

    Model / profile

    Recorded against the real app running from the built commit with TIMMY_VISION_ENABLED=false; vision replies replay the committed synthetic corpus through the app's real analysis path. The review card is a real scripts/review_corpus.py run over the committed synthetic sample plus a synthetic answer file (no model was measured on screen). No live model inference appears in the video, no real photo is used anywhere, and the fixtures are CC0 synthetic images generated by scripts/build_vision_corpus.py.

    Demonstrated on camera

    • queue: 1 pending · proposal Type 4 · brown · good, model answer withheld from the reviewer
    • reveal before any review: refused, answer stays hidden
    • correction committed (5 / green / fair) with the original proposal kept beside it
    • model answer revealed (Type 6 · brown · good · confidence 0.71) → DISAGREE bristolType · color · quality
    • second reviewer adjudicates: 2 lines · resolves record 1 · reviewer-a's review untouched
    • trainable samples: 1 → after consent withdrawal: queue 0 · trainable 0 and review/reveal/train refused

    Known accuracy limitation

    Open-weight VLM acceptance is proven; Bristol accuracy is not clinically validated. The calibration/abstention release receipt still reports BLOCKED because there is no candidate model until issue #31 — this release adds the reviewer control, not a measured classifier. User confirmation remains mandatory everywhere: photo analysis may suggest visible Bristol form, broad color, and image quality only, and never symptoms or a diagnosis.

    Downloads