timmy
9dabad495b
The corpus had an append-only manifest, a reviewer tool, and an evaluation harness, but nothing a stranger could read: what is in the corpus, what a label means, and how leakage is prevented lived in code and in reviewers' heads. scripts/dataset_card.py generates docs/DATASET_CARD.md from manifest.jsonl. Every count is read back from real records — active samples, contributors, per-split contributor and sample counts, class balance over active samples only — and every fact the pipeline does not collect (devices, contexts, licenses) is printed as pending rather than estimated. The generator refuses to render a card for a corpus where one contributor straddles two splits or where a recorded split is not the one the subject key derives, and it never publishes a subject key or a withdrawn sample's labels. --check fails when the committed page is not the generator's own output, so the card cannot drift into a hand-written document. docs/LABELING_GUIDE.md is the reviewer's rulebook: visible form, broad visible color, and photo quality; visible color is not blood; nothing is diagnosed; a model suggestion is provisional input to a human decision. Exclusions are named, and the split policy is stated where it is enforced. The demo's new DATASET CARD scene builds a throwaway corpus with the real ingest command from committed synthetic fixtures, publishes the card, and shows the refusal a planted contributor earns, then re-checks the committed page. Its captions are release gates in build_release.py, which also runs tests/dataset-card.test.py before a release exists. tests/dataset-card.test.py covers the counts, the pending facts, the withdrawal receipt, both refusals, determinism, the empty-corpus case, the committed-page identity, and both documents' prohibitions. Every fixture is synthetic. Closes #27
3.6 KiB
Dataset card — Timmy consented stool-image corpus
Generated by scripts/dataset_card.py from the append-only
corpus manifest. Do not edit this page by hand: regenerate it, or --check fails. Every count
below is read back from manifest.jsonl; a fact the pipeline does not collect is printed as
pending rather than estimated, so nothing here is invented.
| Field | Value |
|---|---|
| Corpus | /srv/timmy-corpus |
| Corpus present on this machine | no |
| Last manifest record | pending — no record yet |
| Active samples | 0 |
| Contributors | 0 |
| Withdrawals | 0 |
| Contexts | pending — not collected by ingest — no setting, time-of-day, or location fields are recorded |
| Devices | pending — not collected by ingest — no camera, model, or OS fields are recorded |
| Licenses | pending — no contributor grant is recorded yet, so no license is asserted |
| Regenerate | python3 scripts/dataset_card.py --corpus /srv/timmy-corpus |
What this corpus is for
Consented photos of stool, labelled by a human reviewer, for research on a stool-form classifier. The corpus may not be used to diagnose disease, to identify a person, or to train a model that speaks to a patient with clinical authority. A model's output stays provisional until the person logging confirms it: see PRODUCT-DECISIONS.md.
This corpus holds no consented images yet. Every count of zero below is a real count of an empty manifest, not a placeholder, and the pending facts stay pending until the pipeline collects them.
Contributor-level split policy
Splits are derived from the salted pseudonymous subject key, never assigned per photo: the
first byte of the key decides the split, < 205 train, < 230 validation, otherwise test
(the ingestion rule in scripts/ingest_training_photo.py). One contributor's photos all land
in one split, and this generator refuses a manifest where a subject key appears in two splits
or where a recorded split is not the one its subject key derives. Both edges are stated here
so the policy on this page is the policy in the code.
| Split | Contributors | Active samples |
|---|---|---|
train |
0 | 0 |
validation |
0 | 0 |
test |
0 | 0 |
Class balance (active samples only)
Type
| Type | Active samples |
|---|---|
| Type 1 | 0 |
| Type 2 | 0 |
| Type 3 | 0 |
| Type 4 | 0 |
| Type 5 | 0 |
| Type 6 | 0 |
| Type 7 | 0 |
Color
| Color | Active samples |
|---|---|
brown |
0 |
green |
0 |
yellow |
0 |
pale |
0 |
red |
0 |
black |
0 |
Quality
| Quality | Active samples |
|---|---|
good |
0 |
fair |
0 |
poor |
0 |
Review status
| Review status | Active samples |
|---|---|
user-confirmed |
0 |
clinician-reviewed |
0 |
A withdrawn sample is excluded from every table above. Labels are what a reviewer saw, not a diagnosis: LABELING_GUIDE.md says what a label may and may not assert.
Exclusions and withdrawals
A withdrawal appends a tombstone, deletes the derived image and any raw copy, and stays in the history. Contribution ids and reasons are published here because a deletion receipt is checkable; subject keys and labels are never published.
No withdrawals are recorded.
Limitations
- Composition (devices, contexts, licenses) is pending, so the corpus is not yet a representative sample of anything and must not be described as one.
- Review status is a human decision, not a clinical gold standard;
user-confirmedmeans the person who took the photo agreed with the proposal. - A held-out split measures generalization across contributors, not across a population.