stackchain/timmy-talking-turd
2
0 Fork 0
Code Issues 21 Pull Requests 7 Actions 4 Packages Projects Releases 41 Wiki Activity

41 Releases 41 Tags

RSS Feed
  • daily-2026-10-30 1bbcc10a35
    Compare

    Timmy review release 2026-10-30-1bbcc10a356f
    Quality gates / quality (push) Canceled after 0s
    Details
    Pre-Release

    timmy released this 2026-10-30 12:38:15 +00:00 | 13 commits to main since this release

    Prerelease for review — tag daily-2026-10-30 on commit 1bbcc10a356f62d3ab2118081874cbb8ab5e7ea6 (America/New_York release date 2026-10-30, built 2026-10-30T12:36Z).

    What changed

    The release candidate is now attacked before anyone is asked to trust it (issue #40, PR #91). Timmy could describe what it does; nothing drove hostile input at the real boundary and recorded what happened. This release adds an executable abuse suite and the rule that turns its observations into a release decision.

    • scripts/security_attack_suite.mjs starts the real server.mjs on a free loopback port, points it at a counting stub worker, and drives it with synthetic hostile bytes. Nine cases: oversized body, HTML polyglot dressed as image/jpeg, undecodable bytes, request without consent, worker answer that breaks the schema, byte-identical replay, concurrent burst, captured-log scan, and deletion (out of surface — counted only while tests/training-contribution.test.js exists). It writes a JSON report, prints the verdict, and exits 1 when blocked.
    • src/release-security-gate.js owns the verdict so it is testable instead of asserted: nearest-rank percentiles, per-case pass/fail derived from the recorded observation (a partial match fails and names the broken field), and — any failing P0 case blocks RC approval; a required class with no verified evidence blocks it too. A report that still holds image bytes refuses to be written.
    • docs/RC-SECURITY-EVIDENCE.md states the procedure, the rule, and the current finding plainly, rather than storing an "all clear" that goes stale on the next commit.
    • CI regenerates the same evidence on every change (npm run test:security-attack).
    • Demo — a SECURITY ATTACK SUITE card drawn from a real run of the suite against the real server, printing the verdict it derived instead of a flattering one.

    What this run actually found — the headline is a block, not a pass:

    VERDICT: BLOCKED · 2 blocking · 1 held

    • ingress-polyglot-html P0 fail — an HTML file dressed as a JPEG returns 503 and reaches the worker (worker 1); the expectation is 400 with the worker never called.
    • ingress-garbage-bytes P0 fail — undecodable bytes reach the worker the same way.
    • flood-concurrency P1 held — the concurrent burst is not refused (refused false, 12 worker calls).
    • Passing as expected: oversized body (413, worker never called), missing consent (400, worker never called), malformed worker answer (503, provider text not leaked), byte-identical replay, and the captured-log scan (image bytes false).
    • Load phase: 8 requests, p50 15 ms, p95 19 ms against the stub on loopback.

    Bytes are still trusted from the data-URL label, so a hostile file gets through and only fails afterwards on schema. That is a finding against #16 (image ingress hardening, open as PR #82 and PR #63), not a claim that the boundary is now safe.

    Scope note, stated plainly: the suite is a command line, not a screen in the app, and the card is a CLI card — this release does not imply UI. It also does not yet run as a gate inside scripts/build_release.py: the verdict is evidence the release job must read, and it currently reads BLOCKED. Wiring it into the mandatory gate list belongs with #16 landing, not with publishing this harness.

    Not closed by this release. PR #91 deliberately does not close #40: the ticket's acceptance criteria are "P0 findings block RC approval" (delivered — the gate decides) together with a boundary that survives attack (not delivered — the two ingress holds above). The suite is honest that it is blocked.

    Completed ticket / PR

    • Issue #40 — Execute security, abuse, load, and medical-image leakage tests (open; suite and blocking rule delivered, ingress holds tracked in #16).
    • PR #91 Execute security, abuse, load, and image-leakage tests against the RC (#40) — merged as 0bf8230 after diff review: 12 files, +782/−3, text and code only (no binaries, model weights, real medical imagery, or secrets; the only data-URI in the diff is the suite's own synthetic 1×1 JPEG used as hostile input), linked to a non-human-gate issue (labels area/privacy, area/release, priority/P0, type/security — no human-gate label).
    • Follow-up commit 1bbcc10 — the first gated build's card rendered http undefined · worker 12 on the flood case and repeated the bare status on the malformed case; the card now prints only the fields each case recorded (refused false, text leaked false), with a test that fails if the old formatter returns. The unpublished build of 0bf8230 was discarded and the whole gated build was re-run end to end from 1bbcc10.
    • Pre-merge verification on the PR head f0a9a73: npm test 244/244, test:security-attack 8/8 — green before the merge was issued.

    Gates — 28/28 passed

    npm test 244/244; security-attack acceptance 8/8; corpus manifest 8/8; corpus review 21/21; dataset card 14/14; calibration harness 19/19; backup-restore 11/11; production-runbook 9/9; mobile green path; photo-first acceptance; sleek Hermes chat; manual fallback + draft recovery; provisional suggestion state; hard-stool follow-up; vision corpus refusal; abstention quality gate; worker container receipts; worker post-load readiness; longitudinal pattern summary; grounded chat citations; training contribution consent; production backup restore; production runbook commands; portable backup restore; dependency audit (0 high); syntax (node/bash/python); secret scan; forbidden artifacts; demo H.264 720×1280 probe; complete MP4 decode; release delta verified painted on camera (95/95 captions, delta caption 6 lines / padBottom 15 px).

    Artifacts

    • source archive timmy-talking-turd-2026-10-30-1bbcc10a356f.tar.gz — 357,094 bytes — sha256 04552e814171e1da6fe5101369dca5189db334d2fe2ed4de72d368209c240d29
    • demo video timmy-talking-turd-2026-10-30-1bbcc10a356f-demo.mp4 — 26,654,756 bytes — H.264 720×1280 yuv420p, 25 fps, 432.16 s — sha256 aa6383c3632aa699a46eb42f5381df03aa06bcfe198a9ea4fab32597c277e1cb

    Model / profile

    Built and recorded by Hermes (profile default, model deepseek-flash). The demo runs the real app from the built commit with TIMMY_VISION_ENABLED=false; vision replies replay the committed CC0 synthetic corpus through the app's real analysis path, and the app's configured vision profile remains self-hosted SmolVLM2-2.2B-Instruct (pinned GGUF) with the hosted profile as fallback. The SECURITY ATTACK SUITE card is a real suite run against a real server.mjs — but the worker behind that server is a counting stub, so the card proves what the HTTP boundary did and whether the model boundary was reached. It measures no model quality. No live model inference appears in the video, no real photo is used anywhere, and every upload shown is synthetic hostile bytes.

    Demonstrated on camera

    • SECURITY ATTACK SUITE — subtitle command line, not app UI — real server, counted worker calls
    • COMMAND node scripts/security_attack_suite.mjs
    • TARGET real server · counting stub worker · synthetic bytes
    • OVERSIZED P0 pass · http 413 · worker 0 — refused before any provider work
    • POLYGLOT P0 fail · http 503 · worker 1 — the hostile file reached the worker
    • TRUNCATED P0 fail · http 503 · worker 1
    • CONSENT P0 pass · http 400 · worker 0
    • MALFORMED P0 pass · http 503 · text leaked false
    • REPLAY P2 pass · http 200 · worker 1 · identical true
    • FLOOD P1 fail · refused false · worker 12
    • LOG P0 pass · image bytes false
    • DELETION P0 covered-elsewhere
    • LOAD 8 req · p50 15ms · p95 19ms · abstain 8
    • VERDICT BLOCKED · 2 blocking · 1 held, with BLOCKS ingress-polyglot-html and BLOCKS ingress-garbage-bytes

    Known accuracy limitation

    The release candidate is not proven safe by this release. The suite's own verdict is BLOCKED on two P0 ingress holds, and the fix is in review at #16 (PR #82 / PR #63), not shipped here. Separately and unchanged: open-weight VLM acceptance is proven, Bristol accuracy is not clinically validated; the calibration receipt still reports BLOCKED because no candidate model clears every gate. User confirmation remains mandatory everywhere: photo analysis may suggest visible Bristol form, broad color, and image quality only, and never symptoms or a diagnosis. Visible color is not blood.

    Downloads