stackchain/timmy-talking-turd
2
0 Fork 0
Code Issues 21 Pull Requests 7 Actions 4 Packages Projects Releases 41 Wiki Activity

Calibrate confidence and define the production abstention policy #32

New Issue
Closed
opened 2026-08-19 01:59:01 +00:00 by timmy · 0 comments
timmy commented 2026-08-19 01:59:01 +00:00
Owner
Copy link
Copy source

Parent epic: #4

Outcome

Turn raw scores into a conservative suggestion policy.

Acceptance criteria

  • Threshold selection is based only on validation data.
  • Held-out precision among non-abstained suggestions reaches the agreed gate or release is blocked.
  • Low-quality, unknown, and out-of-distribution images abstain.

Dependencies

  • #31 — Fine-tune specialist visual classifier with color and quality heads

Required evidence

  • Test or executable receipt attached to the issue/PR
  • Privacy/safety boundary checked where applicable
  • Clean checkout and relevant full suite pass

Delivery

Open a focused branch and PR with Closes #ISSUE. Do not include real medical images in issue comments or Git history.

Parent epic: #4 ## Outcome Turn raw scores into a conservative suggestion policy. ## Acceptance criteria - [ ] Threshold selection is based only on validation data. - [ ] Held-out precision among non-abstained suggestions reaches the agreed gate or release is blocked. - [ ] Low-quality, unknown, and out-of-distribution images abstain. ## Dependencies - #31 — Fine-tune specialist visual classifier with color and quality heads ## Required evidence - Test or executable receipt attached to the issue/PR - Privacy/safety boundary checked where applicable - Clean checkout and relevant full suite pass ## Delivery Open a focused branch and PR with `Closes #ISSUE`. Do not include real medical images in issue comments or Git history.
timmy added this to the M2 — Consented Dataset & Specialist Model milestone 2026-08-19 01:59:01 +00:00
timmy added the type/testpriority/P0area/modelarea/safety labels 2026-08-19 01:59:01 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Labels
Clear labels
area/backend
Application API and storage
 area/data
Dataset and provenance
 area/frontend
PWA and interaction design
 area/model
Vision model and inference
 area/privacy
Privacy and consent
 area/release
Release engineering and distribution
 area/safety
Clinical safety boundary
 blocked
Blocked on an explicit dependency or decision
 human-gate
Requires explicit human approval
 priority/P0
Release-critical
 priority/P1
High priority
 priority/P2
Planned
 state:review
type/docs
Documentation and evidence
 type/epic
Roadmap epic
 type/feature
Product feature
 type/ops
Infrastructure and operations
 type/research
Research or evaluation
 type/security
Security hardening
 type/test
Verification and quality gate
No labels area/model area/safety priority/P0 type/test
Milestone
No items
No Milestone M2 — Consented Dataset & Specialist Model
Projects
Clear projects
No projects
Assignees
Clear assignees
timmy (Timmy · the Wizard Mind) zephyrus-sparkborn (Zephyrus Sparkborn)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: stackchain/timmy-talking-turd#32