stackchain/timmy-talking-turd
2
0 Fork 0
Code Issues 21 Pull Requests 7 Actions 4 Packages Projects Releases 41 Wiki Activity

Instrument privacy-safe model health, latency, abstention, and failure metrics #19

New Issue
Closed
opened 2026-08-19 01:58:59 +00:00 by timmy · 1 comment
timmy commented 2026-08-19 01:58:59 +00:00
Owner
Copy link
Copy source

Parent epic: #2

Outcome

Operate the worker without logging medical imagery.

Acceptance criteria

  • Metrics expose readiness, queue depth, cold/warm latency, failure class, and abstention rate.
  • No metric label contains note text, image hash, base64, or user identity.
  • Dashboard separates app, queue, and model boundaries.

Dependencies

  • #17 — Add bounded inference queue, cancellation, and overload fallback

Required evidence

  • Test or executable receipt attached to the issue/PR
  • Privacy/safety boundary checked where applicable
  • Clean checkout and relevant full suite pass

Delivery

Open a focused branch and PR with Closes #ISSUE. Do not include real medical images in issue comments or Git history.

Parent epic: #2 ## Outcome Operate the worker without logging medical imagery. ## Acceptance criteria - [ ] Metrics expose readiness, queue depth, cold/warm latency, failure class, and abstention rate. - [ ] No metric label contains note text, image hash, base64, or user identity. - [ ] Dashboard separates app, queue, and model boundaries. ## Dependencies - #17 — Add bounded inference queue, cancellation, and overload fallback ## Required evidence - Test or executable receipt attached to the issue/PR - Privacy/safety boundary checked where applicable - Clean checkout and relevant full suite pass ## Delivery Open a focused branch and PR with `Closes #ISSUE`. Do not include real medical images in issue comments or Git history.
timmy added this to the M1 — Sovereign Photo Intelligence milestone 2026-08-19 01:58:59 +00:00
timmy added the type/opspriority/P1area/modelarea/privacy labels 2026-08-19 01:58:59 +00:00
rockachopa commented 2026-11-02 13:29:51 +00:00
Member
Copy link
Copy source

RC is up: prerelease daily-2026-11-02 on 664a909 — https://forge.alexanderwhitestone.com/git/stackchain/timmy-talking-turd/releases/tag/daily-2026-11-02

Test block for #19:

  1. TIMMY_METRICS_TOKEN unset → GET /api/metrics returns 404 (route absent, not guarded) — verified this run.
  2. Token set, no/wrong bearer → 403, no counters in the body — verified.
  3. Token set, matched → 200 snapshot with app/queue/model blocks — verified, and the recording's own card prints it.
  4. node scripts/model_metrics_report.mjs --url http://127.0.0.1:4173 --token <token> prints the report; wrong token exits 1 with The server rejected the operator token.
  5. Privacy screen shows the Health counters row and names what is never collected.

Correction shipped with the RC: the first CLI run printed failure ingress_refused0 (label filled the 22-char column) — LABEL_WIDTH is now 30 with a regression test, and the demo card was re-laid so no label wraps. Known gap stated in the release note: queue.inFlight is app admission depth, not #17's bounded queue (PR #64 still unmerged).

RC is up: prerelease `daily-2026-11-02` on `664a909` — https://forge.alexanderwhitestone.com/git/stackchain/timmy-talking-turd/releases/tag/daily-2026-11-02 Test block for #19: 1. `TIMMY_METRICS_TOKEN` unset → `GET /api/metrics` returns **404** (route absent, not guarded) — verified this run. 2. Token set, no/wrong bearer → **403**, no counters in the body — verified. 3. Token set, matched → **200** snapshot with app/queue/model blocks — verified, and the recording's own card prints it. 4. `node scripts/model_metrics_report.mjs --url http://127.0.0.1:4173 --token <token>` prints the report; wrong token exits 1 with `The server rejected the operator token.` 5. Privacy screen shows the `Health counters` row and names what is never collected. Correction shipped with the RC: the first CLI run printed `failure ingress_refused0` (label filled the 22-char column) — `LABEL_WIDTH` is now 30 with a regression test, and the demo card was re-laid so no label wraps. Known gap stated in the release note: `queue.inFlight` is app admission depth, not #17's bounded queue (PR #64 still unmerged).
Sign in to join this conversation.
No Branch/Tag Specified
Labels
Clear labels
area/backend
Application API and storage
 area/data
Dataset and provenance
 area/frontend
PWA and interaction design
 area/model
Vision model and inference
 area/privacy
Privacy and consent
 area/release
Release engineering and distribution
 area/safety
Clinical safety boundary
 blocked
Blocked on an explicit dependency or decision
 human-gate
Requires explicit human approval
 priority/P0
Release-critical
 priority/P1
High priority
 priority/P2
Planned
 state:review
type/docs
Documentation and evidence
 type/epic
Roadmap epic
 type/feature
Product feature
 type/ops
Infrastructure and operations
 type/research
Research or evaluation
 type/security
Security hardening
 type/test
Verification and quality gate
No labels area/model area/privacy priority/P1 type/ops
Milestone
No items
No Milestone M1 — Sovereign Photo Intelligence
Projects
Clear projects
No projects
Assignees
Clear assignees
timmy (Timmy · the Wizard Mind) zephyrus-sparkborn (Zephyrus Sparkborn)
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: stackchain/timmy-talking-turd#19