Merge pull request 'Blind reviewer and adjudication tool for corpus labels (#26)' (#88) from timmy/26-review-adjudication into main

2026-10-27 12:05:58 +00:00
commit b191d0427d
@@ -34,6 +34,7 @@ jobs:
          npm test
          python3 tests/staging-deploy.test.py -v
          python3 tests/corpus-manifest.test.py -v
+          python3 tests/corpus-review.test.py -v
          python3 tests/calibration-harness.test.py -v
      - name: Mobile browser acceptance
        run: |
@@ -101,6 +101,7 @@ npm run test:vision-refusal   # corpus refusals through a real server and a real
 npm run test:grounded-chat    # chat answers cite the ledger, or refuse to answer
 npm run test:photo-lifecycle  # photo presence, explicit reveal, remove-photo-keeps-log, delete-all
 npm run test:calibration      # the release-gate harness: gates, threshold protocol, refused corpora
+npm run test:review           # blind review queue, held-back model answer, an append-only ledger, withdrawals
 npm audit --audit-level=high
 ```

@@ -135,6 +136,8 @@ Timmy does not diagnose disease, identify bleeding, infer pain/urgency, recommen

 The calibration harness ([scripts/calibration_report.py](scripts/calibration_report.py), [contract](docs/EVALUATION-HARNESS.md)) computes the ROADMAP release gates — macro-F1, precision-at-coverage, and calibration error — from prediction files, chooses a threshold on validation only, and refuses a corpus that leaks one contributor across two splits. Its committed fixtures are invented numbers that exercise the arithmetic: **no candidate model has been measured against these gates yet, and the harness itself is not evidence about accuracy.**

+The reviewer tool ([scripts/review_corpus.py](scripts/review_corpus.py), [contract](docs/REVIEWER-TOOL.md)) is the labeling control for that corpus, and it is a command-line tool with no screen in the app. The review queue shows the contributor's proposal and withholds every model field, a reveal is refused until a human has committed their own labels, a correction must actually change the proposal, adjudication is appended beside the review it resolves instead of overwriting it, and a withdrawn contribution cannot be queued, reviewed, revealed, adjudicated, or trained on. Unreviewed and excluded samples are never training data. It records decisions; it does not measure a model.
+
 ## Architecture

 ```text
@@ -59,3 +59,11 @@ python3 scripts/ingest_training_photo.py --revoke "$CONTRIBUTION_ID" \

 `npm run test:corpus` runs the contract suite. Every fixture is synthetic; no real medical image
 is read, written, or committed.
+
+## Review and adjudication
+
+A manifest says what was ingested and whether the person still allows it. It does not say whether
+the labels are right. That decision lives in a separate append-only ledger,
+`<dataset>/reviews.jsonl`, written by [scripts/review_corpus.py](review_corpus.py) — see
+[REVIEWER-TOOL.md](REVIEWER-TOOL.md). The trainer reads `trainable_records()`, which returns only
+samples that are still active, carry a human decision, and were not excluded.
@@ -0,0 +1,93 @@
+# Blind review and adjudication tool
+
+Contract for `scripts/review_corpus.py`, the labeling control named in the
+[photo data-flow threat model](PHOTO-DATAFLOW-THREAT-MODEL.md) and the gate in front of the
+[training corpus](CORPUS-MANIFEST.md). Issue [#26](https://forge.alexanderwhitestone.com/git/stackchain/timmy-talking-turd/issues/26).
+
+The tool decides which samples a trainer is allowed to read. It is a command-line tool, not a
+screen in the app: there is no reviewer UI, and nothing here implies one.
+
+## The four invariants
+
+1. **The model answer is withheld until a human commits.** `build_queue()` takes no predictions at
+   all, so the queue cannot leak an answer it never receives. `reveal()` refuses to return anything
+   for a contribution with no committed review.
+2. **History is append-only.** A review is a line in `reviews.jsonl`. An adjudication is another
+   line that names the record it resolves (`resolvesRecord`, `resolves`). The review being
+   overruled is never edited, and no record is deleted.
+3. **A withdrawal beats every other answer.** A revoked contribution cannot be queued, reviewed,
+   revealed, adjudicated, or trained on. The refusal says it was withdrawn — not that it is
+   already reviewed — because the person's withdrawal is the fact that matters.
+4. **Unreviewed and excluded samples are not training data.** `trainable_records()` returns only
+   samples that are still active, carry a human decision, and were not excluded.
+
+## Review ledger records
+
+One JSON object per line in `<dataset>/reviews.jsonl`:
+
+```json
+{"schemaVersion":1,"recordType":"review","contributionId":"...","imageId":"<sha256>","proposed":{"bristolType":4,"color":"brown","quality":"good"},"reviewer":"reviewer-a","decision":"corrected","labels":{"bristolType":5,"color":"green","quality":"fair"},"committedAt":"2026-10-27T09:00:00+00:00"}
+{"schemaVersion":1,"recordType":"adjudication","contributionId":"...","imageId":"<sha256>","resolvesRecord":1,"resolves":{"reviewer":"reviewer-a","decision":"corrected"},"reviewer":"reviewer-b","decision":"upheld","labels":{"bristolType":5,"color":"green","quality":"fair"},"adjudicatedAt":"2026-10-27T10:00:00+00:00"}
+```
+
+- `proposed` copies the labels the reviewer judged, so a later manifest change cannot rewrite what
+  was reviewed.
+- `decision` is `agreed`, `corrected`, or `excluded` for a review, and `upheld`, `corrected`, or
+  `excluded` for an adjudication. An exclusion carries `exclusionReason` and no labels.
+- The ledger never carries `subjectKey`, note text, symptom answers, or image bytes.
+
+## A model answer file
+
+`reveal` reads predictions only to show what the reviewer could not see. A file may be one JSON
+document or one object per line:
+
+```json
+{"model":"<name>","answers":{"<contributionId>":{"bristolType":6,"color":"brown","quality":"good","confidence":0.71}}}
+```
+
+Keys may be contribution ids or derived image digests. `reveal` is a read: it appends nothing.
+When no answer file is supplied, `model` is `null` — the tool does not invent an answer.
+
+## Usage
+
+```bash
+python3 scripts/review_corpus.py queue --dataset /srv/timmy-corpus
+python3 scripts/review_corpus.py review --dataset /srv/timmy-corpus \
+  --contribution "$CONTRIBUTION_ID" --reviewer reviewer-a --bristol 5 --color green --quality fair
+python3 scripts/review_corpus.py reveal --dataset /srv/timmy-corpus \
+  --contribution "$CONTRIBUTION_ID" --predictions answers.json
+python3 scripts/review_corpus.py adjudicate --dataset /srv/timmy-corpus \
+  --contribution "$CONTRIBUTION_ID" --reviewer reviewer-b --upheld
+python3 scripts/review_corpus.py trainable --dataset /srv/timmy-corpus
+```
+
+Every mutating command prints the record it appended. A refusal exits `2` and prints one line of
+plain copy on stderr.
+
+## Refusals (all covered by `tests/corpus-review.test.py`)
+
+| Attempt | Result |
+|---|---|
+| Review or reveal a withdrawn contribution | refused: `was withdrawn and cannot be reviewed, revealed, or trained on` |
+| Reveal before any review is committed | refused: `stays hidden until a review is committed` |
+| Review the same contribution twice | refused; adjudication is the only way to change a decision |
+| Correct a label to the value it already has | refused: a correction must change it |
+| Review with a blank reviewer id | refused |
+| Exclude without a reason | refused |
+| Review a contribution that has no active sample | refused |
+| Queue an active sample whose image is missing | refused: a reviewer cannot judge a file that is not there |
+| Adjudicate your own review | refused: adjudication needs a second reviewer |
+| Adjudicate twice | refused |
+| Adjudicate a contribution with no review | refused |
+
+## Known limits
+
+- One review per contribution, resolved by at most one adjudication. Re-review after an
+  adjudication would need a new record type; it is not implemented.
+- The tool does not measure or validate a model. It records what a human decided and what a model
+  answered, so the disagreement can be counted later.
+- It does not train, split, or export a dataset; `trainable_records()` is the list a trainer may
+  read, not a trainer.
+
+`npm run test:review` runs the contract suite. Every fixture is the committed synthetic image; no
+real medical image is read, written, or committed.
@@ -18,14 +18,15 @@
    "test:grounded-chat": "node tests/grounded-chat.acceptance.mjs",
    "test:photo-lifecycle": "node tests/photo-lifecycle.acceptance.mjs",
    "test:training-contribution": "node tests/training-contribution.acceptance.mjs",
-    "check:syntax": "node --check app.js && node --check server.mjs && node --check service-worker.js && node --check src/analysis.js && node --check src/domain.js && node --check src/draft.js && node --check src/suggestion.js && node --check src/followup.js && node --check src/vision-readiness.js && node --check src/hermes-agent-service.js && node --check src/vision-config.js && node --check src/vision-service.js && node --check src/vision-corpus.js && node --check src/summary.js && node --check src/photo-lifecycle.js && node --check src/grounded-chat.js && node --check src/training-contribution.js && node --check src/abstention-policy.js && node --check scripts/record_release_demo.mjs && node --check tests/staging.acceptance.mjs && node --check tests/manual-fallback.acceptance.mjs && node --check tests/provisional-suggestion.acceptance.mjs && node --check tests/followup.acceptance.mjs && node --check tests/vision-refusal.acceptance.mjs && bash -n scripts/bootstrap_selfhost_smolvlm.sh && bash -n scripts/run_selfhost_smolvlm.sh && python3 -m py_compile scripts/corpus_manifest.py scripts/ingest_training_photo.py scripts/build_release.py scripts/deploy_staging.py scripts/build_vision_corpus.py scripts/calibration_report.py && node --check scripts/record_worker_benchmark.mjs && node --check scripts/abstention_policy_receipt.mjs && node --check tests/worker-readiness.acceptance.mjs && node --check tests/pattern-summary.acceptance.mjs && node --check tests/photo-lifecycle.acceptance.mjs && node --check tests/training-contribution.acceptance.mjs && node --check tests/portability.acceptance.mjs && node --check tests/grounded-chat.acceptance.mjs && bash -n scripts/worker_ready.sh && bash -n deploy/worker/entrypoint.sh && bash -n deploy/worker/build_image.sh && bash -n deploy/worker/run_worker_container.sh",
+    "check:syntax": "node --check app.js && node --check server.mjs && node --check service-worker.js && node --check src/analysis.js && node --check src/domain.js && node --check src/draft.js && node --check src/suggestion.js && node --check src/followup.js && node --check src/vision-readiness.js && node --check src/hermes-agent-service.js && node --check src/vision-config.js && node --check src/vision-service.js && node --check src/vision-corpus.js && node --check src/summary.js && node --check src/photo-lifecycle.js && node --check src/grounded-chat.js && node --check src/training-contribution.js && node --check src/abstention-policy.js && node --check scripts/record_release_demo.mjs && node --check tests/staging.acceptance.mjs && node --check tests/manual-fallback.acceptance.mjs && node --check tests/provisional-suggestion.acceptance.mjs && node --check tests/followup.acceptance.mjs && node --check tests/vision-refusal.acceptance.mjs && bash -n scripts/bootstrap_selfhost_smolvlm.sh && bash -n scripts/run_selfhost_smolvlm.sh && python3 -m py_compile scripts/corpus_manifest.py scripts/review_corpus.py scripts/ingest_training_photo.py scripts/build_release.py scripts/deploy_staging.py scripts/build_vision_corpus.py scripts/calibration_report.py && node --check scripts/record_worker_benchmark.mjs && node --check scripts/abstention_policy_receipt.mjs && node --check tests/worker-readiness.acceptance.mjs && node --check tests/pattern-summary.acceptance.mjs && node --check tests/photo-lifecycle.acceptance.mjs && node --check tests/training-contribution.acceptance.mjs && node --check tests/portability.acceptance.mjs && node --check tests/grounded-chat.acceptance.mjs && bash -n scripts/worker_ready.sh && bash -n deploy/worker/entrypoint.sh && bash -n deploy/worker/build_image.sh && bash -n deploy/worker/run_worker_container.sh",
    "check:diff": "bash scripts/check_diff.sh",
    "start": "node server.mjs",
    "test:worker-readiness": "node tests/worker-readiness.acceptance.mjs",
    "test:portability": "node tests/portability.acceptance.mjs",
    "test:corpus": "python3 tests/corpus-manifest.test.py",
    "test:calibration": "python3 tests/calibration-harness.test.py",
-    "test:policy": "node --test tests/abstention-policy.test.js"
+    "test:policy": "node --test tests/abstention-policy.test.js",
+    "test:review": "python3 tests/corpus-review.test.py"
  },
  "devDependencies": {
    "playwright": "^1.62.1"
@@ -68,6 +68,10 @@ DEMO_REQUIRED_CAPTIONS = (
    "An older backup is upgraded on import, and the notice says which version the file came from",
    "A file from a newer Timmy is refused with an instruction, and the ledger it would have replaced is untouched",
    "A restored log still says it began as an AI suggestion you corrected — provenance is part of the record",
+    "The review tool is a command line, not a screen in the app — this runs it against the committed synthetic sample",
+    "The queue shows the proposal and hides the model answer; revealing it is refused until a reviewer commits",
+    "The model said Type 6 and the reviewer said Type 5: the disagreement is recorded, not smoothed over",
+    "Withdrawing the contribution empties the queue and the training set, and the tool refuses to review it at all",
 )


@@ -166,6 +170,9 @@ def main() -> int:
        run(["python3", "scripts/build_vision_corpus.py", "--verify"], tree)
        # Issue #25's corpus pipeline is a CLI, so its own suite is a release gate, not just CI.
        run(["python3", "tests/corpus-manifest.test.py", "-v"], tree)
+        # Issue #26's review tool decides which samples a trainer may read, so the blindness and
+        # withdrawal rules it enforces are a release gate too.
+        run(["python3", "tests/corpus-review.test.py", "-v"], tree)
        # Issue #29's evaluation harness decides GO/BLOCKED for a model, so its refusal and gate
        # arithmetic are exercised before this release is allowed to exist.
        run(["python3", "tests/calibration-harness.test.py", "-v"], tree)
@@ -261,7 +268,7 @@ def main() -> int:
        run(["bash", "-n", "deploy/worker/entrypoint.sh"], tree)
        run(["bash", "-n", "deploy/worker/build_image.sh"], tree)
        run(["bash", "-n", "deploy/worker/run_worker_container.sh"], tree)
-        run(["python3", "-m", "py_compile", "scripts/ingest_training_photo.py", "scripts/build_release.py", "scripts/build_vision_corpus.py", "scripts/corpus_manifest.py"], tree)
+        run(["python3", "-m", "py_compile", "scripts/ingest_training_photo.py", "scripts/build_release.py", "scripts/build_vision_corpus.py", "scripts/corpus_manifest.py", "scripts/review_corpus.py"], tree)
        run(["git", "diff", "--check", commit], tree)

        names = tracked_files(tree)
@@ -297,6 +304,8 @@ def main() -> int:
            "- Training contribution is a separate opt-in (issue #24): a journal save records no contribution, the consent version and the receipt are on screen before the control can be pressed, the receipt is queued on this device only (this build transmits nothing) and carries no note text, symptom answers, or photo bytes, and a withdrawal keeps a tombstone naming every derived artifact by contribution id\n"
            "- Auditable corpus pipeline (issue #25): every ingested training sample now records its consent version and timestamp, source kind and the raw-input digest (the caller's file path is never stored), both the raw and derived hashes, the derivation pipeline, the human review status, the split, and its lifecycle state in an append-only manifest; withdrawing a contribution deletes the derived image and any raw pre-derivation copy from the active dataset, keeps the original record as its own history, and appends a tombstone that names every removed artifact by contribution id and digest while dropping labels and the subject key; illegal lifecycle moves (re-ingesting a contribution, re-labelling a derived image, revoking twice, revoking an unknown contribution, reviving a revoked one, revoking without a reason) are refused before any byte is written\n"
            "- Scope note: the corpus pipeline is a command-line tool. There is no corpus screen in the app, and this release does not imply one — the recorded demo runs the real ingest and revoke commands against the committed synthetic fixture and shows their actual output\n"
+            "- Blind reviewer and adjudication tool (issue #26): the review queue shows the proposal and withholds every model field, revealing a model answer is refused until a human has committed their own labels, a correction must actually change the proposal, the ledger is append-only, and a second reader's adjudication is appended beside the review it resolves rather than replacing it; a withdrawn contribution cannot be queued, reviewed, revealed, adjudicated, or trained on, and unreviewed or excluded samples are never training data\n"
+            "- Scope note: the review tool is a command line, not a screen in the app. There is no reviewer screen, and this release does not imply one — the recorded demo runs the real queue, review, reveal, adjudicate, and trainable commands against the committed synthetic sample and shows their actual output, with a synthetic answer file that no model produced\n"
            "- Known limitation: open-weight VLM acceptance is proven, Bristol accuracy is not clinically validated\n"
            "- Package excludes model weights, medical-image corpora, raw research pages, generated screenshots, and video production assets\n"
        )
@@ -341,7 +350,7 @@ def main() -> int:
            "codec": "h264",
            "pixel_format": "yuv420p",
            "recorded_from_working_app": True,
-            "fixture_data": "synthetic-Type-4, deterministic vision suggestion reviewed, confirmed, and corrected to Type 2, user-reported hard-stool follow-up with non-diagnostic self-care and a confirmed black/tarry red flag, the CC0 synthetic vision corpus replaying a non-stool refusal, an unreadable-fields abstention, a confident reply about a poor-quality photo abstaining, an out-of-range fail-closed reply, and a blank-plus-out-of-range fail-closed reply, offline vision status, fictional multi-day sample logs driving the explainable pattern summary and its citation sheet, a sparse fictional window showing the insufficient-data copy, a grounded local chat answer citing each counted log and opening the cited record, a refusal that borrows no journal evidence, and a deterministic Hermes chat reply whose positional refs render as sources, plus the opt-in training contribution path where nothing is queued after ordinary saves, the consent version and receipt precede the confirm control, one receipt is queued on the device only, and withdrawal leaves a tombstone naming each derived artifact, plus a versioned backup that carries the logs and the withdrawn receipt, an older version 1 file upgraded with the version named on screen, a file from a newer Timmy refused without changing the ledger, and a restored log still showing it began as a corrected AI suggestion, plus the corpus audit card, where the real ingest and revoke commands run against the committed synthetic fixture and the card shows their actual output: the provenance record with consent, source digest, both hashes, derivation, review and lifecycle; the derived image deleted from disk; the original record kept as history beside a tombstone that drops labels and the subject key; zero active samples for a trainer; and a second revocation refused without appending anything",
+            "fixture_data": "synthetic-Type-4, deterministic vision suggestion reviewed, confirmed, and corrected to Type 2, user-reported hard-stool follow-up with non-diagnostic self-care and a confirmed black/tarry red flag, the CC0 synthetic vision corpus replaying a non-stool refusal, an unreadable-fields abstention, a confident reply about a poor-quality photo abstaining, an out-of-range fail-closed reply, and a blank-plus-out-of-range fail-closed reply, offline vision status, fictional multi-day sample logs driving the explainable pattern summary and its citation sheet, a sparse fictional window showing the insufficient-data copy, a grounded local chat answer citing each counted log and opening the cited record, a refusal that borrows no journal evidence, and a deterministic Hermes chat reply whose positional refs render as sources, plus the opt-in training contribution path where nothing is queued after ordinary saves, the consent version and receipt precede the confirm control, one receipt is queued on the device only, and withdrawal leaves a tombstone naming each derived artifact, plus a versioned backup that carries the logs and the withdrawn receipt, an older version 1 file upgraded with the version named on screen, a file from a newer Timmy refused without changing the ledger, and a restored log still showing it began as a corrected AI suggestion, plus the corpus audit card, where the real ingest and revoke commands run against the committed synthetic fixture and the card shows their actual output: the provenance record with consent, source digest, both hashes, derivation, review and lifecycle; the derived image deleted from disk; the original record kept as history beside a tombstone that drops labels and the subject key; zero active samples for a trainer; and a second revocation refused without appending anything, plus the review ledger card, where the real review commands run against the committed synthetic sample and the card shows their actual output: one pending proposal with the model answer withheld, a refused reveal before commitment, a committed Type 5 correction beside the Type 4 proposal, the revealed synthetic Type 6 answer recorded as a three-field disagreement, a second reader's adjudication pointing at the review it resolved, one trainable sample, and a withdrawal that empties the queue and the training set and makes review refused",
        },
        "gates": {
            "unit_security": "passed",
@@ -359,6 +368,7 @@ def main() -> int:
            "grounded_chat_citations": "passed",
            "training_contribution_consent": "passed",
            "corpus_manifest_provenance": "passed",
+            "reviewer_adjudication_blind": "passed",
            "calibration_harness_gates": "passed",
            "portable_backup_restore": "passed",
            "dependency_audit_high": "passed",
@@ -221,6 +221,67 @@ async function indicate(selector) {
  await sleep(650);
 }

+// Command-line evidence is shown on the same card chrome in three scenes (corpus audit, review
+// ledger, calibration gate). One renderer keeps the readability gate identical for all of them.
+async function evidenceCard({ id, title, subtitle, rows, font = '600 10.5px/1.5 ui-monospace, SFMono-Regular, Menlo, monospace', keyWidth = 62, gap = '1px', rowGap = '7px' }) {
+  return page.evaluate(params => {
+    document.querySelector(`#${params.id}`)?.remove();
+    const panel = document.createElement('div');
+    panel.id = params.id;
+    Object.assign(panel.style, {
+      position: 'fixed', inset: '96px 14px 40px 14px', zIndex: '15000',
+      background: '#181412', color: '#f6f1e6', borderRadius: '18px',
+      border: '1px solid rgba(245,201,91,.45)', padding: '13px 13px 11px',
+      font: params.font,
+      display: 'flex', flexDirection: 'column', gap: params.gap, overflow: 'hidden',
+      boxShadow: '0 18px 40px rgba(0,0,0,.42)',
+    });
+    const head = document.createElement('div');
+    head.textContent = params.title;
+    Object.assign(head.style, { font: '800 14px/1.2 system-ui', letterSpacing: '.14em', color: '#f5c95b', marginBottom: '2px' });
+    const sub = document.createElement('div');
+    sub.textContent = params.subtitle;
+    Object.assign(sub.style, { font: '700 9.5px/1.3 system-ui', color: '#c9bfae', marginBottom: '9px' });
+    panel.append(head, sub);
+    for (const [key, value] of params.rows) {
+      const row = document.createElement('div');
+      if (key === '—') { row.style.height = params.rowGap; panel.append(row); continue; }
+      Object.assign(row.style, { display: 'flex', gap: '7px', alignItems: 'baseline' });
+      const k = document.createElement('span');
+      k.textContent = key;
+      k.style.cssText = `flex:0 0 ${params.keyWidth}px;color:#8fd7c8;text-transform:uppercase;font-size:9px;letter-spacing:.08em`;
+      const v = document.createElement('span');
+      v.textContent = value;
+      v.style.cssText = 'flex:1 1 auto;word-break:break-word;color:#f6f1e6';
+      row.append(k, v);
+      panel.append(row);
+    }
+    document.body.append(panel);
+    panel.animate([{ opacity: 0, transform: 'translateY(12px)' }, { opacity: 1, transform: 'translateY(0)' }], { duration: 320, fill: 'forwards' });
+    return { height: panel.clientHeight, scroll: panel.scrollHeight, text: panel.innerText };
+  }, { id, title, subtitle, rows, font, keyWidth, gap, rowGap });
+}
+
+// A card is only evidence if it is actually legible: opaque, inside the frame, not covered, and not
+// clipped. A translucent or scrolled card reads as a sloppy overlay on the thinnest part of a film.
+async function evidenceCardProof(id) {
+  return page.evaluate(cardId => {
+    const panel = document.querySelector(`#${cardId}`);
+    if (!panel) return { ok: false, reason: 'panel missing' };
+    const rect = panel.getBoundingClientRect();
+    const topmost = document.elementFromPoint(Math.round(rect.left + rect.width / 2), Math.round(rect.top + rect.height / 2));
+    const background = getComputedStyle(panel).backgroundColor;
+    const opaque = !/^rgba\(.*,\s*(0?\.\d+|0)\)$/.test(background);
+    return {
+      ok: opaque && rect.top >= 60 && rect.bottom <= window.innerHeight && rect.width > 300 && rect.height > 300
+        && (topmost === panel || panel.contains(topmost)) && panel.scrollHeight <= panel.clientHeight + 1,
+      top: Math.round(rect.top), height: Math.round(rect.height), width: Math.round(rect.width),
+      scroll: panel.scrollHeight, client: panel.clientHeight, background, opaque,
+      covered: topmost === panel || panel.contains(topmost),
+    };
+  }, id);
+}
+
 await caption(`TIMMY ${version} • RELEASE DEMO`, 1200);
 await caption(`This release: ${releaseDelta}`, 2600);
 await caption('Automated checks replay this synthetic path before review', 1200);
@@ -661,60 +722,14 @@ const auditRows = [
  ['refused', audit.refusal],
  ['manifest', 'unchanged at 2 lines'],
 ];
-const panelOk = await page.evaluate(rows => {
-  document.querySelector('#corpus-audit')?.remove();
-  const panel = document.createElement('div');
-  panel.id = 'corpus-audit';
-  Object.assign(panel.style, {
-    position: 'fixed', inset: '96px 14px 40px 14px', zIndex: '15000',
-    background: '#181412', color: '#f6f1e6', borderRadius: '18px',
-    border: '1px solid rgba(245,201,91,.45)', padding: '13px 13px 11px',
-    font: '600 10.5px/1.5 ui-monospace, SFMono-Regular, Menlo, monospace',
-    display: 'flex', flexDirection: 'column', gap: '1px', overflow: 'hidden',
-    boxShadow: '0 18px 40px rgba(0,0,0,.42)',
-  });
-  const head = document.createElement('div');
-  head.textContent = 'CORPUS AUDIT';
-  Object.assign(head.style, { font: '800 14px/1.2 system-ui', letterSpacing: '.14em', color: '#f5c95b', marginBottom: '2px' });
-  const sub = document.createElement('div');
-  sub.textContent = 'command line, not app UI — real run, synthetic fixture';
-  Object.assign(sub.style, { font: '700 9.5px/1.3 system-ui', color: '#c9bfae', marginBottom: '9px' });
-  panel.append(head, sub);
-  for (const [key, value] of rows) {
-    const row = document.createElement('div');
-    if (key === '—') { row.style.height = '7px'; panel.append(row); continue; }
-    Object.assign(row.style, { display: 'flex', gap: '7px', alignItems: 'baseline' });
-    const k = document.createElement('span');
-    k.textContent = key;
-    k.style.cssText = 'flex:0 0 62px;color:#8fd7c8;text-transform:uppercase;font-size:9px;letter-spacing:.08em';
-    const v = document.createElement('span');
-    v.textContent = value;
-    v.style.cssText = 'flex:1 1 auto;word-break:break-word;color:#f6f1e6';
-    row.append(k, v);
-    panel.append(row);
-  }
-  document.body.append(panel);
-  panel.animate([{ opacity: 0, transform: 'translateY(12px)' }, { opacity: 1, transform: 'translateY(0)' }], { duration: 320, fill: 'forwards' });
-  return { height: panel.clientHeight, scroll: panel.scrollHeight, text: panel.innerText };
-}, auditRows);
-await sleep(420);
-const panelProof = await page.evaluate(() => {
-  const panel = document.querySelector('#corpus-audit');
-  if (!panel) return { ok: false, reason: 'panel missing' };
-  const rect = panel.getBoundingClientRect();
-  const topmost = document.elementFromPoint(Math.round(rect.left + rect.width / 2), Math.round(rect.top + rect.height / 2));
-  const background = getComputedStyle(panel).backgroundColor;
-  // A translucent card lets the nav bar and body copy ghost through its empty rows, which reads as a
-  // sloppy overlay on the thinnest part of the frame. The card must be fully opaque.
-  const opaque = !/^rgba\(.*,\s*(0?\.\d+|0)\)$/.test(background);
-  return {
-    ok: opaque && rect.top >= 60 && rect.bottom <= window.innerHeight && rect.width > 300 && rect.height > 300
-      && (topmost === panel || panel.contains(topmost)) && panel.scrollHeight <= panel.clientHeight + 1,
-    top: Math.round(rect.top), height: Math.round(rect.height), width: Math.round(rect.width),
-    scroll: panel.scrollHeight, client: panel.clientHeight, background, opaque,
-    covered: topmost === panel || panel.contains(topmost),
-  };
+const panelOk = await evidenceCard({
+  id: 'corpus-audit',
+  title: 'CORPUS AUDIT',
+  subtitle: 'command line, not app UI — real run, synthetic fixture',
+  rows: auditRows,
 });
+await sleep(420);
+const panelProof = await evidenceCardProof('corpus-audit');
 if (!panelProof.ok) throw new Error(`Corpus audit card is not fully readable on screen: ${JSON.stringify(panelProof)}`);
 // The card is drawn from live command output; prove the values on it are the ones the run produced.
 if (!panelOk.text.includes(audit.record.imageId.slice(0, 12))) throw new Error('the card must show the derived digest it recorded');
@@ -724,6 +739,188 @@ await caption('Every ingested sample records consent, source digest, both hashes
 await caption('Withdrawing deletes the derived image and any raw copy, keeps the original record, and appends a tombstone — the trainer sees zero active samples', 3800);
 await caption('Revoking the same contribution twice is refused and appends nothing: the manifest only ever grows', 3200);
 await page.evaluate(() => document.querySelector('#corpus-audit')?.remove());
+
+// Issue #26 — the reviewer and adjudication tool is a command line, not a screen in the app. This
+// scene runs the real commands against the committed synthetic sample in a throwaway dataset and
+// shows the order the tool enforces: the queue hides the model answer, revealing it is refused
+// until a human commits, and a withdrawal removes the sample from the queue, the review path, and
+// the training set. Every value on the card is read back from those runs. The answer file is a
+// synthetic fixture; no model measured anything here.
+async function reviewCli(args) {
+  try {
+    const result = await execFileAsync('python3', ['scripts/review_corpus.py', ...args], {
+      cwd: process.cwd(),
+      env: { ...process.env, TIMMY_DATASET_SALT: 'demo-only-salt-not-a-secret' },
+    });
+    return { code: 0, stdout: result.stdout, stderr: result.stderr };
+  } catch (error) {
+    return {
+      code: typeof error.code === 'number' ? error.code : 1,
+      stdout: error.stdout || '',
+      stderr: error.stderr || '',
+    };
+  }
+}
+function lastLine(output) {
+  return String(output).trim().split('\n').filter(Boolean).pop() || '';
+}
+async function reviewLedger() {
+  const dataset = await mkdtemp(path.join(os.tmpdir(), 'timmy-review-demo-'));
+  const contributionId = 'demo-review-0001-abcd';
+  const predictionsPath = path.join(dataset, 'synthetic-answers.json');
+  const manifestPath = path.join(dataset, 'manifest.jsonl');
+  const reviewPath = path.join(dataset, 'reviews.jsonl');
+  const lines = async file => (await readFile(file, 'utf8').catch(() => ''))
+    .split('\n').filter(line => line.trim()).map(line => JSON.parse(line));
+  try {
+    const ingest = await corpusCli([
+      'scripts/ingest_training_photo.py',
+      '--input', 'tests/fixtures/synthetic-type4.jpg',
+      '--output-dir', dataset,
+      '--subject-id', 'demo-subject',
+      '--consent-version', 'training-consent-v1',
+      '--contribution-id', contributionId,
+      '--bristol-type', '4', '--color', 'brown', '--quality', 'good',
+      '--reviewer', 'user-confirmed',
+    ]);
+    if (JSON.parse(ingest.stdout).lifecycle?.state !== 'active') {
+      throw new Error('the review scene needs one active synthetic sample');
+    }
+
+    const before = await reviewCli(['queue', '--dataset', dataset]);
+    const queue = JSON.parse(before.stdout);
+    if (queue.count !== 1) throw new Error('one unreviewed sample must queue exactly once');
+    if (queue.items[0].proposed.bristolType !== 4) throw new Error('the queue must show the proposal');
+    if ('confidence' in queue.items[0] || /"model"/.test(JSON.stringify(queue.items[0]))) {
+      throw new Error('the queue must not carry the model answer');
+    }
+
+    const early = await reviewCli(['reveal', '--dataset', dataset, '--contribution', contributionId]);
+    if (early.code === 0) throw new Error('revealing before a review must be refused');
+    const blindRefusal = lastLine(early.stderr);
+    if (!/stays hidden until a review is committed/.test(blindRefusal)) {
+      throw new Error(`the blind refusal was not legible: ${blindRefusal}`);
+    }
+
+    await writeFile(predictionsPath, JSON.stringify({
+      model: 'synthetic answer file',
+      answers: { [contributionId]: { bristolType: 6, color: 'brown', quality: 'good', confidence: 0.71 } },
+    }, null, 2));
+
+    const committed = await reviewCli([
+      'review', '--dataset', dataset, '--contribution', contributionId,
+      '--reviewer', 'reviewer-a', '--bristol', '5', '--color', 'green', '--quality', 'fair',
+    ]);
+    if (committed.code !== 0) throw new Error(`the review was refused: ${lastLine(committed.stderr)}`);
+    const review = JSON.parse(committed.stdout);
+    if (review.decision !== 'corrected' || review.labels.bristolType !== 5) {
+      throw new Error('the committed review must record the corrected label');
+    }
+    if (review.proposed.bristolType !== 4) throw new Error('the committed review must keep the proposal it judged');
+
+    const revealed = await reviewCli(['reveal', '--dataset', dataset, '--contribution', contributionId, '--predictions', predictionsPath]);
+    if (revealed.code !== 0) throw new Error(`reveal failed after a committed review: ${lastLine(revealed.stderr)}`);
+    const reveal = JSON.parse(revealed.stdout);
+    if (reveal.model?.labels.bristolType !== 6) throw new Error('the revealed answer must be the one that was withheld');
+    if (!reveal.disagreement || reveal.disagreeFields.length !== 3) {
+      throw new Error('a Type 6 answer against a Type 5 review must be recorded as a disagreement');
+    }
+
+    const ruled = await reviewCli([
+      'adjudicate', '--dataset', dataset, '--contribution', contributionId,
+      '--reviewer', 'reviewer-b', '--upheld',
+    ]);
+    if (ruled.code !== 0) throw new Error(`adjudication was refused: ${lastLine(ruled.stderr)}`);
+    const ruling = JSON.parse(ruled.stdout);
+    if (ruling.resolvesRecord !== 1) throw new Error('the ruling must point at the record it resolves');
+    const ledgerAfterRuling = await lines(reviewPath);
+    if (ledgerAfterRuling.length !== 2) throw new Error('adjudication must append, not replace');
+    if (ledgerAfterRuling[0].reviewer !== 'reviewer-a' || ledgerAfterRuling[0].decision !== 'corrected') {
+      throw new Error('the review being adjudicated must survive untouched');
+    }
+
+    const trainableBefore = JSON.parse((await reviewCli(['trainable', '--dataset', dataset])).stdout);
+    if (trainableBefore.count !== 1) throw new Error('one reviewed sample must be trainable');
+    if (trainableBefore.items[0].labels.bristolType !== 5) {
+      throw new Error('the trainer must read the reviewed label, not the proposal');
+    }
+
+    const revoke = await corpusCli([
+      'scripts/ingest_training_photo.py', '--output-dir', dataset,
+      '--revoke', contributionId, '--reason', 'participant withdrew consent',
+    ]);
+    if (JSON.parse(revoke.stdout).recordType !== 'tombstone') throw new Error('the withdrawal must be recorded');
+
+    const afterRevoke = JSON.parse((await reviewCli(['queue', '--dataset', dataset])).stdout);
+    const trainableAfter = JSON.parse((await reviewCli(['trainable', '--dataset', dataset])).stdout);
+    if (afterRevoke.count !== 0 || trainableAfter.count !== 0) {
+      throw new Error('a withdrawn sample must leave the queue and the training set');
+    }
+    const refused = await reviewCli([
+      'review', '--dataset', dataset, '--contribution', contributionId, '--reviewer', 'reviewer-c', '--agree',
+    ]);
+    if (refused.code === 0) throw new Error('reviewing a withdrawn contribution must be refused');
+    const withdrawnRefusal = lastLine(refused.stderr);
+    if (!/was withdrawn and cannot be reviewed/.test(withdrawnRefusal)) {
+      throw new Error(`the withdrawal refusal was not legible: ${withdrawnRefusal}`);
+    }
+    if ((await lines(reviewPath)).length !== 2) throw new Error('a refused review must append nothing');
+    if ((await lines(manifestPath)).length !== 2) throw new Error('the manifest must hold the sample and its tombstone');
+    return { queue, blindRefusal, review, reveal, ruling, trainableBefore, afterRevoke, trainableAfter, withdrawnRefusal };
+  } finally {
+    await rm(dataset, { recursive: true, force: true });
+  }
+}
+const reviewEvidence = await reviewLedger();
+await tap('[data-view="home"]', 450);
+const reviewRows = [
+  ['command', 'python3 scripts/review_corpus.py queue --dataset <tmp>'],
+  ['queue', `${reviewEvidence.queue.count} pending · proposal Type ${reviewEvidence.queue.items[0].proposed.bristolType} · ${reviewEvidence.queue.items[0].proposed.color} · ${reviewEvidence.queue.items[0].proposed.quality}`],
+  ['blind', 'model answer withheld from the reviewer'],
+  ['—', ''],
+  ['command', 'reveal --contribution <id> (before any review)'],
+  ['refused', reviewEvidence.blindRefusal],
+  ['—', ''],
+  ['command', 'review --reviewer reviewer-a --bristol 5 --color green --quality fair'],
+  ['committed', `${reviewEvidence.review.decision} · proposal ${reviewEvidence.review.proposed.bristolType}/${reviewEvidence.review.proposed.color}/${reviewEvidence.review.proposed.quality} kept beside it`],
+  ['command', 'reveal --predictions <synthetic answer file>'],
+  ['model', `Type ${reviewEvidence.reveal.model.labels.bristolType} · ${reviewEvidence.reveal.model.labels.color} · ${reviewEvidence.reveal.model.labels.quality} · confidence ${reviewEvidence.reveal.model.confidence}`],
+  ['disagree', reviewEvidence.reveal.disagreeFields.join(' · ')],
+  ['—', ''],
+  ['command', 'adjudicate --reviewer reviewer-b --upheld'],
+  ['ledger', `2 lines · resolves record ${reviewEvidence.ruling.resolvesRecord} · reviewer-a's review untouched`],
+  ['trainer', `trainable samples: ${reviewEvidence.trainableBefore.count}`],
+  ['—', ''],
+  ['command', 'ingest --revoke <id> --reason "participant withdrew consent"'],
+  ['after', `queue ${reviewEvidence.afterRevoke.count} · trainable ${reviewEvidence.trainableAfter.count}`],
+  ['review', reviewEvidence.withdrawnRefusal],
+  ['data', 'synthetic answer file · no model was measured'],
+];
+const reviewCard = await evidenceCard({
+  id: 'review-audit',
+  title: 'REVIEW LEDGER',
+  subtitle: 'command line, not app UI — real run, synthetic sample',
+  rows: reviewRows,
+});
+await sleep(420);
+const reviewProof = await evidenceCardProof('review-audit');
+if (!reviewProof.ok) throw new Error(`Review card is not fully readable on screen: ${JSON.stringify(reviewProof)}`);
+if (!reviewCard.text.includes(`Type ${reviewEvidence.reveal.model.labels.bristolType}`)) {
+  throw new Error('the card must show the model answer the run revealed');
+}
+if (!reviewCard.text.includes('was withdrawn and cannot be reviewed')) {
+  throw new Error('the card must show the withdrawn refusal the run earned');
+}
+if (!reviewCard.text.includes(`trainable samples: ${reviewEvidence.trainableBefore.count}`)) {
+  throw new Error('the card must show the trainer count the run produced');
+}
+await caption('The review tool is a command line, not a screen in the app — this runs it against the committed synthetic sample', 3600);
+await caption('The queue shows the proposal and hides the model answer; revealing it is refused until a reviewer commits', 3800);
+await caption('The model said Type 6 and the reviewer said Type 5: the disagreement is recorded, not smoothed over', 3600);
+await caption('A second reviewer adjudicates, and the review being overruled is still in the ledger', 3400);
+await caption('Withdrawing the contribution empties the queue and the training set, and the tool refuses to review it at all', 3800);
+await page.evaluate(() => document.querySelector('#review-audit')?.remove());
+
 // Issue #29 — the release gate is a command-line harness, not a screen in the app. This scene runs
 // the real harness over the committed synthetic prediction fixtures and shows the receipt it wrote:
 // a blocked comparison, a passing fixture, and a refused leaked corpus. Every number and every
@@ -799,58 +996,18 @@ gateRows.push(['—', '']);
 gateRows.push(['engineered', 'GO — gate-plumbing-fixture clears every gate (fixture, not a result)']);
 gateRows.push(['leaked', gate.leak.stderr.trim().replace(/^calibration harness refused:\s*/, '')]);
 gateRows.push(['data', 'synthetic predictions · no model was measured']);
-const gateCard = await page.evaluate(rows => {
-  document.querySelector('#calibration-report')?.remove();
-  const panel = document.createElement('div');
-  panel.id = 'calibration-report';
-  Object.assign(panel.style, {
-    position: 'fixed', inset: '96px 14px 40px 14px', zIndex: '15000',
-    background: '#181412', color: '#f6f1e6', borderRadius: '18px',
-    border: '1px solid rgba(245,201,91,.45)', padding: '13px 13px 11px',
-    font: '600 9.5px/1.45 ui-monospace, SFMono-Regular, Menlo, monospace',
-    display: 'flex', flexDirection: 'column', gap: '1px', overflow: 'hidden',
-    boxShadow: '0 18px 40px rgba(0,0,0,.42)',
-  });
-  const head = document.createElement('div');
-  head.textContent = 'CALIBRATION HARNESS';
-  Object.assign(head.style, { font: '800 13.5px/1.2 system-ui', letterSpacing: '.12em', color: '#f5c95b', marginBottom: '2px' });
-  const sub = document.createElement('div');
-  sub.textContent = 'command line, not app UI — real run, synthetic predictions';
-  Object.assign(sub.style, { font: '700 9px/1.3 system-ui', color: '#c9bfae', marginBottom: '8px' });
-  panel.append(head, sub);
-  for (const [key, value] of rows) {
-    const row = document.createElement('div');
-    if (key === '—') { row.style.height = '6px'; panel.append(row); continue; }
-    Object.assign(row.style, { display: 'flex', gap: '6px', alignItems: 'baseline' });
-    const k = document.createElement('span');
-    k.textContent = key;
-    k.style.cssText = 'flex:0 0 74px;color:#8fd7c8;text-transform:uppercase;font-size:8.5px;letter-spacing:.06em';
-    const v = document.createElement('span');
-    v.textContent = value;
-    v.style.cssText = 'flex:1 1 auto;word-break:break-word;color:#f6f1e6';
-    row.append(k, v);
-    panel.append(row);
-  }
-  document.body.append(panel);
-  panel.animate([{ opacity: 0, transform: 'translateY(12px)' }, { opacity: 1, transform: 'translateY(0)' }], { duration: 320, fill: 'forwards' });
-  return { height: panel.clientHeight, scroll: panel.scrollHeight, text: panel.innerText };
-}, gateRows);
-await sleep(420);
-const gateProof = await page.evaluate(() => {
-  const panel = document.querySelector('#calibration-report');
-  if (!panel) return { ok: false, reason: 'panel missing' };
-  const rect = panel.getBoundingClientRect();
-  const topmost = document.elementFromPoint(Math.round(rect.left + rect.width / 2), Math.round(rect.top + rect.height / 2));
-  // Same rule as the corpus card: a translucent card lets the screen behind it ghost through.
-  const opaque = !/^rgba\(.*,\s*(0?\.\d+|0)\)$/.test(getComputedStyle(panel).backgroundColor);
-  return {
-    ok: opaque && rect.top >= 60 && rect.bottom <= window.innerHeight && rect.width > 300 && rect.height > 300
-      && (topmost === panel || panel.contains(topmost)) && panel.scrollHeight <= panel.clientHeight + 1,
-    top: Math.round(rect.top), height: Math.round(rect.height), width: Math.round(rect.width),
-    scroll: panel.scrollHeight, client: panel.clientHeight, opaque,
-    covered: topmost === panel || panel.contains(topmost),
-  };
+const gateCard = await evidenceCard({
+  id: 'calibration-report',
+  title: 'CALIBRATION HARNESS',
+  subtitle: 'command line, not app UI — real run, synthetic predictions',
+  rows: gateRows,
+  font: '600 9.5px/1.45 ui-monospace, SFMono-Regular, Menlo, monospace',
+  keyWidth: 74,
+  gap: '1px',
+  rowGap: '6px',
 });
+await sleep(420);
+const gateProof = await evidenceCardProof('calibration-report');
 if (!gateProof.ok) throw new Error(`Calibration card is not fully readable on screen: ${JSON.stringify(gateProof)}`);
 // The card is drawn from the report; prove the values on it are the ones the harness measured.
 for (const row of gate.report.models) {
@@ -0,0 +1,461 @@
+#!/usr/bin/env python3
+"""Blind reviewer and adjudication tool for the Timmy training corpus (issue #26).
+
+A reviewer reads one active sample, commits their own label without seeing what the model
+said, and only then can the model answer be revealed. Adjudication is a second reader's
+ruling, appended after the review it resolves; nothing in the review ledger is ever rewritten.
+A withdrawn contribution cannot be queued, reviewed, revealed, adjudicated, or trained on.
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import re
+import sys
+from datetime import datetime, timezone
+from pathlib import Path
+
+# The tool sits beside corpus_manifest.py and is also imported by path from the test suite.
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+from corpus_manifest import (  # noqa: E402
+    COLORS,
+    QUALITIES,
+    CorpusError,
+    active_records,
+    load_manifest,
+)
+
+SCHEMA_VERSION = 1
+REVIEW = "review"
+ADJUDICATION = "adjudication"
+DECISIONS = ("agreed", "corrected", "excluded")
+ADJUDICATIONS = ("upheld", "corrected", "excluded")
+LABEL_FIELDS = ("bristolType", "color", "quality")
+_TIMESTAMP = re.compile(r"^\d{4}-\d{2}-\d{2}T")
+
+
+class ReviewError(Exception):
+    """A review action that must never happen is being attempted."""
+
+
+def _now() -> str:
+    return datetime.now(timezone.utc).isoformat()
+
+
+def _manifest_path(dataset_root: Path) -> Path:
+    return Path(dataset_root) / "manifest.jsonl"
+
+
+def review_ledger_path(dataset_root: Path) -> Path:
+    return Path(dataset_root) / "reviews.jsonl"
+
+
+def load_reviews(dataset_root: Path) -> list[dict]:
+    """Read the append-only review ledger in order. A missing ledger is an unreviewed corpus."""
+    path = review_ledger_path(dataset_root)
+    if not path.exists():
+        return []
+    with path.open("r", encoding="utf-8") as ledger:
+        return [json.loads(line) for line in ledger.read().splitlines() if line.strip()]
+
+
+def _history(reviews: list[dict], contribution_id: str) -> list[dict]:
+    return [record for record in reviews if record.get("contributionId") == contribution_id]
+
+
+def _append_review(dataset_root: Path, record: dict) -> dict:
+    path = review_ledger_path(dataset_root)
+    path.parent.mkdir(parents=True, exist_ok=True)
+    with path.open("a", encoding="utf-8") as ledger:
+        ledger.write(json.dumps(record, separators=(",", ":"), sort_keys=False) + "\n")
+    return record
+
+
+def _active_sample(dataset_root: Path, contribution_id: str) -> dict:
+    """The one active sample for this contribution, or a refusal that names why there is none."""
+    records = load_manifest(_manifest_path(dataset_root))
+    for sample in active_records(records):
+        if sample.get("contributionId") == contribution_id:
+            return sample
+    if any(record.get("contributionId") == contribution_id for record in records):
+        raise ReviewError(
+            f"contribution {contribution_id} was withdrawn and cannot be reviewed, revealed, or trained on"
+        )
+    raise ReviewError(f"no active sample carries contribution {contribution_id}")
+
+
+def _validate_labels(labels: object) -> dict:
+    if not isinstance(labels, dict) or set(labels) != set(LABEL_FIELDS):
+        raise ReviewError("a decided label must carry bristolType, color, and quality")
+    if labels["bristolType"] not in range(1, 8):
+        raise ReviewError("bristolType must be 1-7")
+    if labels["color"] not in COLORS or labels["quality"] not in QUALITIES:
+        raise ReviewError("color and quality must be from the documented vocabularies")
+    return {field: labels[field] for field in LABEL_FIELDS}
+
+
+def _decide(
+    *,
+    reviewer: str,
+    decision: str,
+    labels,
+    exclusion_reason,
+    proposed: dict,
+    allowed: tuple[str, ...],
+    change_subject: str = "proposed label",
+) -> dict:
+    """Validate one human decision and return the fields it contributes to the ledger."""
+    reviewer_id = reviewer.strip() if isinstance(reviewer, str) else ""
+    if not reviewer_id:
+        raise ReviewError("reviewer is required")
+    if decision not in allowed:
+        raise ReviewError(f"decision must be one of {', '.join(allowed)}")
+    if decision in ("agreed", "upheld"):
+        return {"reviewer": reviewer_id, "decision": decision, "labels": dict(proposed)}
+    if decision == "excluded":
+        if not isinstance(exclusion_reason, str) or not exclusion_reason.strip():
+            raise ReviewError("exclusion needs a recorded reason")
+        return {
+            "reviewer": reviewer_id,
+            "decision": decision,
+            "exclusionReason": exclusion_reason.strip(),
+        }
+    decided = _validate_labels(labels)
+    if decided == proposed:
+        raise ReviewError(f"a corrected decision must change the {change_subject}")
+    return {"reviewer": reviewer_id, "decision": decision, "labels": decided}
+
+
+def commit_review(
+    dataset_root: Path,
+    contribution_id: str,
+    reviewer: str,
+    decision: str,
+    labels: dict | None = None,
+    exclusion_reason: str | None = None,
+    at: str | None = None,
+) -> dict:
+    """Append one human label decision for an active sample, before any model answer is shown."""
+    dataset_root = Path(dataset_root)
+    reviews = load_reviews(dataset_root)
+    # A withdrawal outranks every other refusal: nothing may be reviewed out of a corpus that no
+    # longer holds the sample, and the refusal has to say so.
+    sample = _active_sample(dataset_root, contribution_id)
+    if _history(reviews, contribution_id):
+        raise ReviewError(
+            f"contribution {contribution_id} is already reviewed; "
+            "adjudication is the only way to change a decision"
+        )
+    proposed = dict(sample["labels"])
+    committed = _decide(
+        reviewer=reviewer,
+        decision=decision,
+        labels=labels,
+        exclusion_reason=exclusion_reason,
+        proposed=proposed,
+        allowed=DECISIONS,
+    )
+    stamped = at or _now()
+    if not _TIMESTAMP.match(stamped):
+        raise ReviewError("committed timestamp must be ISO-8601")
+    record = {
+        "schemaVersion": SCHEMA_VERSION,
+        "recordType": REVIEW,
+        "contributionId": contribution_id,
+        "imageId": sample["imageId"],
+        "proposed": proposed,
+        **committed,
+        "committedAt": stamped,
+    }
+    return _append_review(dataset_root, record)
+
+
+def load_predictions(path: Path) -> dict:
+    """Read a model answer file: one JSON document, or one JSON object per line.
+
+    A pretty-printed object is a document, not a set of lines, so it is parsed whole; only a file
+    that is not valid JSON as a document is read line by line.
+    """
+    text = Path(path).read_text(encoding="utf-8")
+    if not text.strip():
+        return {"answers": {}}
+    try:
+        document = json.loads(text)
+    except json.JSONDecodeError:
+        document = None
+    if isinstance(document, dict):
+        if "answers" in document:
+            return document
+        if document and all(isinstance(value, dict) for value in document.values()):
+            return {"answers": document}
+        raise ReviewError("every prediction needs a contributionId or imageId")
+    answers: dict[str, dict] = {}
+    for line in text.splitlines():
+        if not line.strip():
+            continue
+        entry = json.loads(line)
+        key = entry.get("contributionId") or entry.get("imageId")
+        if not key:
+            raise ReviewError("every prediction needs a contributionId or imageId")
+        answers[key] = entry
+    return {"answers": answers}
+
+
+def _model_answer(predictions: dict | None, contribution_id: str, image_id: str) -> dict | None:
+    if not isinstance(predictions, dict):
+        return None
+    answers = predictions.get("answers", predictions)
+    for key in (contribution_id, image_id):
+        entry = answers.get(key)
+        if isinstance(entry, dict):
+            labels = {field: entry.get(field) for field in LABEL_FIELDS}
+            if all(value is not None for value in labels.values()):
+                answer = {"labels": labels}
+                if "confidence" in entry:
+                    answer["confidence"] = entry["confidence"]
+                if "model" in predictions:
+                    answer["model"] = predictions["model"]
+                return answer
+    return None
+
+
+def reveal(dataset_root: Path, contribution_id: str, predictions: dict | None = None) -> dict:
+    """Show the model's answer for a sample, but only after a human has committed their own.
+
+    The order is the whole point: the reviewer's label is already fixed in the ledger when this
+    runs, so a model answer can be read for disagreement but cannot steer the human into agreeing.
+    """
+    dataset_root = Path(dataset_root)
+    sample = _active_sample(dataset_root, contribution_id)
+    history = _history(load_reviews(dataset_root), contribution_id)
+    if not history:
+        raise ReviewError(
+            f"the model answer for contribution {contribution_id} stays hidden until a review is committed"
+        )
+    reviewed = history[0]
+    human_labels = dict(reviewed["labels"]) if "labels" in reviewed else None
+    answer = _model_answer(predictions, contribution_id, sample["imageId"])
+    disagree = []
+    if answer and human_labels:
+        disagree = [field for field in LABEL_FIELDS if answer["labels"][field] != human_labels[field]]
+    return {
+        "contributionId": contribution_id,
+        "human": {"reviewer": reviewed["reviewer"], "decision": reviewed["decision"], "labels": human_labels},
+        "model": answer,
+        "disagreeFields": disagree,
+        "disagreement": bool(disagree),
+    }
+
+
+def adjudicate(
+    dataset_root: Path,
+    contribution_id: str,
+    reviewer: str,
+    decision: str,
+    labels: dict | None = None,
+    exclusion_reason: str | None = None,
+    at: str | None = None,
+) -> dict:
+    """Append a second reader's ruling on a review. The review it resolves stays in the ledger."""
+    dataset_root = Path(dataset_root)
+    reviews = load_reviews(dataset_root)
+    sample = _active_sample(dataset_root, contribution_id)
+    history = _history(reviews, contribution_id)
+    if not history:
+        raise ReviewError(f"no review to adjudicate for contribution {contribution_id}")
+    if any(record.get("recordType") == ADJUDICATION for record in history):
+        raise ReviewError(f"contribution {contribution_id} is already adjudicated")
+    reviewed = history[0]
+    if isinstance(reviewer, str) and reviewer.strip() == reviewed["reviewer"]:
+        raise ReviewError("adjudication needs a second reviewer, not the reviewer who decided")
+    reviewed_labels = dict(reviewed.get("labels") or {})
+    ruling = _decide(
+        reviewer=reviewer,
+        decision=decision,
+        labels=labels,
+        exclusion_reason=exclusion_reason,
+        proposed=reviewed_labels,
+        allowed=ADJUDICATIONS,
+        change_subject="reviewed label",
+    )
+    stamped = at or _now()
+    if not _TIMESTAMP.match(stamped):
+        raise ReviewError("committed timestamp must be ISO-8601")
+    record = {
+        "schemaVersion": SCHEMA_VERSION,
+        "recordType": ADJUDICATION,
+        "contributionId": contribution_id,
+        "imageId": sample["imageId"],
+        "resolvesRecord": reviews.index(reviewed) + 1,
+        "resolves": {"reviewer": reviewed["reviewer"], "decision": reviewed["decision"]},
+        **ruling,
+        "adjudicatedAt": stamped,
+    }
+    return _append_review(dataset_root, record)
+
+
+def trainable_records(dataset_root: Path) -> list[dict]:
+    """The training set this corpus is allowed to produce right now.
+
+    A sample is trainable only if it is still active, a human reviewed it, and the final decision
+    was not an exclusion. Unreviewed proposals are never training data, and an adjudication
+    replaces the label it resolved without deleting it.
+    """
+    dataset_root = Path(dataset_root)
+    reviews = load_reviews(dataset_root)
+    trainable = []
+    for sample in active_records(load_manifest(_manifest_path(dataset_root))):
+        history = _history(reviews, sample["contributionId"])
+        if not history:
+            continue
+        final = history[-1]
+        if final["decision"] == "excluded":
+            continue
+        entry = {
+            "contributionId": sample["contributionId"],
+            "imageId": sample["imageId"],
+            "relativePath": sample["relativePath"],
+            "split": sample["split"],
+            "labels": dict(final["labels"]),
+            "reviewedBy": history[0]["reviewer"],
+        }
+        if final.get("recordType") == ADJUDICATION:
+            entry["adjudicatedBy"] = final["reviewer"]
+        trainable.append(entry)
+    return trainable
+
+
+def _decision_from(args: argparse.Namespace, *, uphold: bool) -> tuple[str, dict | None, str | None]:
+    """Translate the flag shape into one decision, refusing a half-specified review."""
+    label_flags = [args.bristol, args.color, args.quality]
+    if args.exclude:
+        if any(flag is not None for flag in label_flags):
+            raise ReviewError("an exclusion cannot carry labels")
+        return "excluded", None, args.exclude
+    if uphold and args.upheld:
+        if any(flag is not None for flag in label_flags):
+            raise ReviewError("an upheld review keeps the reviewed label; pass no labels with it")
+        return "upheld", None, None
+    if not uphold and args.agree:
+        if any(flag is not None for flag in label_flags):
+            raise ReviewError("agreeing keeps the proposed label; pass no labels with it")
+        return "agreed", None, None
+    if all(flag is not None for flag in label_flags):
+        return "corrected", {"bristolType": args.bristol, "color": args.color, "quality": args.quality}, None
+    if any(flag is not None for flag in label_flags):
+        raise ReviewError("a corrected decision needs --bristol, --color, and --quality together")
+    hint = "--upheld, a corrected label, or --exclude" if uphold else "--agree, a corrected label, or --exclude"
+    raise ReviewError(f"a decision is required: pass {hint}")
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(
+        description="Blind review and adjudication for the Timmy training corpus.",
+    )
+    commands = parser.add_subparsers(dest="command", required=True)
+
+    queue = commands.add_parser("queue", help="list the blind review queue")
+    queue.add_argument("--dataset", required=True)
+    queue.add_argument("--limit", type=int)
+
+    review = commands.add_parser("review", help="commit a human label decision")
+    review.add_argument("--dataset", required=True)
+    review.add_argument("--contribution", required=True)
+    review.add_argument("--reviewer", required=True)
+    review.add_argument("--agree", action="store_true", help="the proposed label is correct")
+    review.add_argument("--bristol", type=int, choices=range(1, 8))
+    review.add_argument("--color", choices=COLORS)
+    review.add_argument("--quality", choices=QUALITIES)
+    review.add_argument("--exclude", metavar="REASON")
+
+    reveal = commands.add_parser("reveal", help="show the model answer after the human commits")
+    reveal.add_argument("--dataset", required=True)
+    reveal.add_argument("--contribution", required=True)
+    reveal.add_argument("--predictions", help="JSON or JSONL of model answers; omit if none exist")
+
+    adjudicate_parser = commands.add_parser("adjudicate", help="append a second reader's ruling")
+    adjudicate_parser.add_argument("--dataset", required=True)
+    adjudicate_parser.add_argument("--contribution", required=True)
+    adjudicate_parser.add_argument("--reviewer", required=True)
+    adjudicate_parser.add_argument("--bristol", type=int, choices=range(1, 8))
+    adjudicate_parser.add_argument("--color", choices=COLORS)
+    adjudicate_parser.add_argument("--quality", choices=QUALITIES)
+    adjudicate_parser.add_argument("--exclude", metavar="REASON")
+    adjudicate_parser.add_argument("--upheld", action="store_true", help="keep the reviewed label")
+
+    trainable = commands.add_parser("trainable", help="list what a trainer is allowed to read")
+    trainable.add_argument("--dataset", required=True)
+
+    return parser
+
+
+def main(argv: list[str] | None = None) -> int:
+    args = build_parser().parse_args(argv)
+    try:
+        if args.command == "queue":
+            items = build_queue(Path(args.dataset), limit=args.limit)
+            print(json.dumps({"count": len(items), "items": items}, indent=2))
+            return 0
+        if args.command == "review":
+            decision, labels, exclusion_reason = _decision_from(args, uphold=False)
+            record = commit_review(
+                Path(args.dataset), args.contribution, args.reviewer,
+                decision, labels, exclusion_reason,
+            )
+            print(json.dumps(record, indent=2))
+            return 0
+        if args.command == "reveal":
+            predictions = load_predictions(args.predictions) if args.predictions else None
+            print(json.dumps(reveal(Path(args.dataset), args.contribution, predictions), indent=2))
+            return 0
+        if args.command == "adjudicate":
+            decision, labels, exclusion_reason = _decision_from(args, uphold=True)
+            record = adjudicate(
+                Path(args.dataset), args.contribution, args.reviewer,
+                decision, labels, exclusion_reason,
+            )
+            print(json.dumps(record, indent=2))
+            return 0
+        if args.command == "trainable":
+            items = trainable_records(Path(args.dataset))
+            print(json.dumps({"count": len(items), "items": items}, indent=2))
+            return 0
+    except (ReviewError, CorpusError) as error:
+        print(str(error), file=sys.stderr)
+        return 2
+    return 0
+
+
+def build_queue(dataset_root: Path, limit: int | None = None) -> list[dict]:
+    """Blind review queue: what a reviewer must judge, and nothing about what a model thinks.
+
+    This function deliberately takes no predictions. Blindness is structural here: the queue
+    cannot leak a model answer because it never receives one.
+    """
+    dataset_root = Path(dataset_root)
+    reviews = load_reviews(dataset_root)
+    reviewed = {record.get("contributionId") for record in reviews}
+    queue: list[dict] = []
+    for sample in active_records(load_manifest(_manifest_path(dataset_root))):
+        contribution_id = sample["contributionId"]
+        if contribution_id in reviewed:
+            continue
+        image = dataset_root / sample["relativePath"]
+        if not image.is_file():
+            raise ReviewError(f"the dataset lists an active sample whose image is missing: {sample['relativePath']}")
+        queue.append({
+            "schemaVersion": SCHEMA_VERSION,
+            "contributionId": contribution_id,
+            "imageId": sample["imageId"],
+            "relativePath": sample["relativePath"],
+            "split": sample["split"],
+            "proposed": dict(sample["labels"]),
+            "blindToModel": True,
+        })
+        if limit is not None and len(queue) >= limit:
+            break
+    return queue
+
+
+if __name__ == "__main__":
+    sys.exit(main())
@@ -0,0 +1,419 @@
+#!/usr/bin/env python3
+"""Tests for the blind reviewer and adjudication tool (issue #26).
+
+The tool decides what a training sample is allowed to say, so these tests attack the two
+properties that make the labels trustworthy:
+
+* a reviewer never sees the model's answer before committing their own, and
+* a withdrawn contribution can never be shown, reviewed, or trained on.
+
+Every fixture is the committed synthetic image; no real photo bytes are read or written.
+"""
+from __future__ import annotations
+
+import importlib.util
+import json
+import os
+import subprocess
+import sys
+import tempfile
+import unittest
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parents[1]
+INGEST = ROOT / "scripts" / "ingest_training_photo.py"
+FIXTURE = ROOT / "tests" / "fixtures" / "synthetic-type4.jpg"
+FIRST = "6f1d0c9e-2c4a-4f3b-9d1e-8a7b6c5d4e3f"
+SECOND = "0b9a8c7d-6e5f-4a3b-2c1d-0e9f8a7b6c5d"
+
+
+def load_tool():
+    spec = importlib.util.spec_from_file_location("review_corpus", ROOT / "scripts" / "review_corpus.py")
+    assert spec and spec.loader
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+    return module
+
+
+tool = load_tool()
+
+
+def ingest(dataset: Path, contribution: str, *, bristol: str = "4", color: str = "brown", quality: str = "good") -> dict:
+    done = subprocess.run(
+        [
+            sys.executable, str(INGEST),
+            "--input", str(FIXTURE),
+            "--output-dir", str(dataset),
+            "--subject-id", "subject-123",
+            "--consent-version", "research-v1",
+            "--contribution-id", contribution,
+            "--bristol-type", bristol,
+            "--color", color,
+            "--quality", quality,
+            "--reviewer", "user-confirmed",
+        ],
+        cwd=ROOT,
+        capture_output=True,
+        text=True,
+        env={**os.environ, "TIMMY_DATASET_SALT": "review-test-salt"},
+    )
+    assert done.returncode == 0, done.stderr
+    return json.loads(done.stdout)
+
+
+def revoke(dataset: Path, contribution: str) -> dict:
+    done = subprocess.run(
+        [sys.executable, str(INGEST), "--output-dir", str(dataset), "--revoke", contribution, "--reason", "participant withdrew consent"],
+        cwd=ROOT,
+        capture_output=True,
+        text=True,
+        env={**os.environ, "TIMMY_DATASET_SALT": "review-test-salt"},
+    )
+    assert done.returncode == 0, done.stderr
+    return json.loads(done.stdout)
+
+
+def ledger(dataset: Path) -> list[dict]:
+    path = dataset / "reviews.jsonl"
+    if not path.exists():
+        return []
+    return [json.loads(line) for line in path.read_text(encoding="utf-8").splitlines() if line.strip()]
+
+
+PREDICTIONS = {
+    "version": "synthetic-model-placeholder",
+    "answers": {
+        FIRST: {"bristolType": 6, "color": "brown", "quality": "good", "confidence": 0.71},
+        SECOND: {"bristolType": 4, "color": "brown", "quality": "good", "confidence": 0.9},
+    },
+}
+
+
+class QueueBlindnessTests(unittest.TestCase):
+    def test_the_queue_shows_the_proposal_and_withholds_every_model_field(self):
+        with tempfile.TemporaryDirectory() as raw:
+            dataset = Path(raw)
+            sample = ingest(dataset, FIRST)
+
+            queue = tool.build_queue(dataset)
+
+            self.assertEqual(len(queue), 1)
+            packet = queue[0]
+            self.assertEqual(packet["contributionId"], FIRST)
+            self.assertEqual(packet["imageId"], sample["imageId"])
+            self.assertEqual(packet["relativePath"], sample["relativePath"])
+            self.assertEqual(packet["proposed"], {"bristolType": 4, "color": "brown", "quality": "good"})
+            self.assertTrue(packet["blindToModel"])
+
+            # Not a naming convention: nothing model-shaped and nothing identity-bearing may ride along.
+            text = json.dumps(packet)
+            for forbidden in ("confidence", "modelSuggestion", "prediction", "subjectKey", "data:image", "base64"):
+                self.assertNotIn(forbidden, text)
+
+    def test_resolved_samples_leave_the_queue(self):
+        with tempfile.TemporaryDirectory() as raw:
+            dataset = Path(raw)
+            ingest(dataset, FIRST)
+            self.assertEqual(len(tool.build_queue(dataset)), 1)
+
+            tool.commit_review(dataset, FIRST, "reviewer-a", "agreed")
+
+            self.assertEqual(tool.build_queue(dataset), [])
+
+    def test_the_queue_refuses_an_active_sample_whose_image_is_gone(self):
+        with tempfile.TemporaryDirectory() as raw:
+            dataset = Path(raw)
+            sample = ingest(dataset, FIRST)
+            (dataset / sample["relativePath"]).unlink()
+
+            with self.assertRaises(tool.ReviewError) as raised:
+                tool.build_queue(dataset)
+            self.assertRegex(str(raised.exception), "image is missing")
+
+
+class BlindRevealTests(unittest.TestCase):
+    def test_the_model_answer_stays_hidden_until_a_review_is_committed(self):
+        with tempfile.TemporaryDirectory() as raw:
+            dataset = Path(raw)
+            ingest(dataset, FIRST)
+
+            with self.assertRaises(tool.ReviewError) as raised:
+                tool.reveal(dataset, FIRST, PREDICTIONS)
+            self.assertRegex(str(raised.exception), "hidden until a review is committed")
+
+    def test_reveal_reports_the_disagreement_after_the_human_commits(self):
+        with tempfile.TemporaryDirectory() as raw:
+            dataset = Path(raw)
+            ingest(dataset, FIRST)
+            tool.commit_review(dataset, FIRST, "reviewer-a", "corrected", {"bristolType": 5, "color": "green", "quality": "fair"})
+
+            revealed = tool.reveal(dataset, FIRST, PREDICTIONS)
+
+            self.assertEqual(revealed["human"]["labels"], {"bristolType": 5, "color": "green", "quality": "fair"})
+            self.assertEqual(revealed["model"]["labels"], {"bristolType": 6, "color": "brown", "quality": "good"})
+            self.assertEqual(revealed["disagreeFields"], ["bristolType", "color", "quality"])
+            self.assertTrue(revealed["disagreement"])
+            # Revealing is a read: it must not quietly rewrite the review ledger.
+            self.assertEqual(len(ledger(dataset)), 1)
+
+
+class CommitTests(unittest.TestCase):
+    def test_a_committed_review_keeps_the_proposal_and_the_human_decision(self):
+        with tempfile.TemporaryDirectory() as raw:
+            dataset = Path(raw)
+            sample = ingest(dataset, FIRST)
+
+            record = tool.commit_review(dataset, FIRST, "reviewer-a", "corrected", {"bristolType": 2, "color": "brown", "quality": "poor"}, at="2026-10-27T09:00:00+00:00")
+
+            self.assertEqual(record["recordType"], "review")
+            self.assertEqual(record["contributionId"], FIRST)
+            self.assertEqual(record["imageId"], sample["imageId"])
+            self.assertEqual(record["reviewer"], "reviewer-a")
+            self.assertEqual(record["decision"], "corrected")
+            self.assertEqual(record["labels"], {"bristolType": 2, "color": "brown", "quality": "poor"})
+            self.assertEqual(record["proposed"], {"bristolType": 4, "color": "brown", "quality": "good"})
+            self.assertEqual(record["committedAt"], "2026-10-27T09:00:00+00:00")
+            self.assertEqual(ledger(dataset), [record])
+            self.assertNotIn("subjectKey", json.dumps(record))
+
+    def test_a_contribution_cannot_be_reviewed_twice(self):
+        with tempfile.TemporaryDirectory() as raw:
+            dataset = Path(raw)
+            ingest(dataset, FIRST)
+            tool.commit_review(dataset, FIRST, "reviewer-a", "agreed")
+            history = ledger(dataset)
+
+            with self.assertRaises(tool.ReviewError) as raised:
+                tool.commit_review(dataset, FIRST, "reviewer-b", "corrected", {"bristolType": 2, "color": "black", "quality": "poor"})
+            self.assertRegex(str(raised.exception), "already reviewed")
+            self.assertEqual(ledger(dataset), history)
+
+    def test_a_correction_that_changes_nothing_is_refused(self):
+        with tempfile.TemporaryDirectory() as raw:
+            dataset = Path(raw)
+            ingest(dataset, FIRST)
+
+            with self.assertRaises(tool.ReviewError) as raised:
+                tool.commit_review(dataset, FIRST, "reviewer-a", "corrected", {"bristolType": 4, "color": "brown", "quality": "good"})
+            self.assertRegex(str(raised.exception), "must change the proposed label")
+            self.assertEqual(ledger(dataset), [])
+
+    def test_invalid_labels_and_blank_reviewers_are_refused(self):
+        with tempfile.TemporaryDirectory() as raw:
+            dataset = Path(raw)
+            ingest(dataset, FIRST)
+
+            for decision, labels, reviewer, expected in (
+                ("corrected", {"bristolType": 9, "color": "brown", "quality": "good"}, "reviewer-a", "bristolType must be 1-7"),
+                ("corrected", {"bristolType": 4, "color": "purple", "quality": "good"}, "reviewer-a", "color and quality"),
+                ("agreed", None, "   ", "reviewer is required"),
+                ("excluded", None, "reviewer-a", "exclusion needs a recorded reason"),
+                ("approved", None, "reviewer-a", "decision must be one of"),
+            ):
+                with self.assertRaises(tool.ReviewError) as raised:
+                    tool.commit_review(dataset, FIRST, reviewer, decision, labels)
+                self.assertRegex(str(raised.exception), expected)
+            self.assertEqual(ledger(dataset), [])
+
+    def test_reviewing_an_unknown_contribution_is_refused(self):
+        with tempfile.TemporaryDirectory() as raw:
+            dataset = Path(raw)
+            ingest(dataset, FIRST)
+
+            with self.assertRaises(tool.ReviewError) as raised:
+                tool.commit_review(dataset, SECOND, "reviewer-a", "agreed")
+            self.assertRegex(str(raised.exception), "no active sample carries contribution")
+            self.assertEqual(ledger(dataset), [])
+
+
+class WithdrawnContributionTests(unittest.TestCase):
+    def test_a_withdrawn_contribution_never_reaches_the_queue_review_or_the_trainer(self):
+        with tempfile.TemporaryDirectory() as raw:
+            dataset = Path(raw)
+            ingest(dataset, FIRST)
+            tool.commit_review(dataset, FIRST, "reviewer-a", "corrected", {"bristolType": 5, "color": "brown", "quality": "good"})
+            self.assertEqual(len(tool.trainable_records(dataset)), 1)
+
+            revoke(dataset, FIRST)
+
+            self.assertEqual(tool.build_queue(dataset), [])
+            self.assertEqual(tool.trainable_records(dataset), [])
+            for call in (
+                lambda: tool.commit_review(dataset, FIRST, "reviewer-b", "agreed"),
+                lambda: tool.adjudicate(dataset, FIRST, "reviewer-b", "upheld"),
+                lambda: tool.reveal(dataset, FIRST, PREDICTIONS),
+            ):
+                with self.assertRaises(tool.ReviewError) as raised:
+                    call()
+                self.assertRegex(str(raised.exception), "withdrawn")
+
+
+class AdjudicationTests(unittest.TestCase):
+    def test_adjudication_is_appended_and_the_first_review_is_preserved(self):
+        with tempfile.TemporaryDirectory() as raw:
+            dataset = Path(raw)
+            ingest(dataset, FIRST)
+            first = tool.commit_review(dataset, FIRST, "reviewer-a", "corrected", {"bristolType": 5, "color": "brown", "quality": "good"}, at="2026-10-27T09:00:00+00:00")
+
+            ruling = tool.adjudicate(dataset, FIRST, "reviewer-b", "corrected", {"bristolType": 4, "color": "brown", "quality": "fair"}, at="2026-10-27T10:00:00+00:00")
+
+            self.assertEqual(ruling["recordType"], "adjudication")
+            self.assertEqual(ruling["decision"], "corrected")
+            self.assertEqual(ruling["labels"], {"bristolType": 4, "color": "brown", "quality": "fair"})
+            self.assertEqual(ruling["resolvesRecord"], 1)
+            self.assertEqual(ruling["resolves"], {"reviewer": "reviewer-a", "decision": "corrected"})
+
+            history = ledger(dataset)
+            self.assertEqual(len(history), 2)
+            self.assertEqual(history[0], first, "the review being overruled must survive untouched")
+            self.assertEqual(history[1], ruling)
+
+            # The trainer follows the ruling, not the review it replaced.
+            trainable = tool.trainable_records(dataset)
+            self.assertEqual(len(trainable), 1)
+            self.assertEqual(trainable[0]["labels"], {"bristolType": 4, "color": "brown", "quality": "fair"})
+            self.assertEqual(trainable[0]["adjudicatedBy"], "reviewer-b")
+
+    def test_an_upheld_review_keeps_the_label_and_records_the_second_reader(self):
+        with tempfile.TemporaryDirectory() as raw:
+            dataset = Path(raw)
+            ingest(dataset, FIRST)
+            tool.commit_review(dataset, FIRST, "reviewer-a", "agreed")
+
+            ruling = tool.adjudicate(dataset, FIRST, "reviewer-b", "upheld")
+
+            self.assertEqual(ruling["labels"], {"bristolType": 4, "color": "brown", "quality": "good"})
+            self.assertEqual(ruling["resolves"], {"reviewer": "reviewer-a", "decision": "agreed"})
+
+    def test_a_reviewer_cannot_adjudicate_their_own_review(self):
+        with tempfile.TemporaryDirectory() as raw:
+            dataset = Path(raw)
+            ingest(dataset, FIRST)
+            tool.commit_review(dataset, FIRST, "reviewer-a", "agreed")
+
+            with self.assertRaises(tool.ReviewError) as raised:
+                tool.adjudicate(dataset, FIRST, "reviewer-a", "corrected", {"bristolType": 2, "color": "brown", "quality": "good"})
+            self.assertRegex(str(raised.exception), "second reviewer")
+            self.assertEqual(len(ledger(dataset)), 1)
+
+    def test_a_second_adjudication_is_refused(self):
+        with tempfile.TemporaryDirectory() as raw:
+            dataset = Path(raw)
+            ingest(dataset, FIRST)
+            tool.commit_review(dataset, FIRST, "reviewer-a", "agreed")
+            tool.adjudicate(dataset, FIRST, "reviewer-b", "upheld")
+            history = ledger(dataset)
+
+            with self.assertRaises(tool.ReviewError) as raised:
+                tool.adjudicate(dataset, FIRST, "reviewer-c", "excluded", exclusion_reason="unclear")
+            self.assertRegex(str(raised.exception), "already adjudicated")
+            self.assertEqual(ledger(dataset), history)
+
+    def test_adjudicating_an_unreviewed_contribution_is_refused(self):
+        with tempfile.TemporaryDirectory() as raw:
+            dataset = Path(raw)
+            ingest(dataset, FIRST)
+
+            with self.assertRaises(tool.ReviewError) as raised:
+                tool.adjudicate(dataset, FIRST, "reviewer-b", "upheld")
+            self.assertRegex(str(raised.exception), "no review to adjudicate")
+            self.assertEqual(ledger(dataset), [])
+
+
+class TrainableSetTests(unittest.TestCase):
+    def test_only_human_reviewed_and_unexcluded_samples_reach_the_trainer(self):
+        with tempfile.TemporaryDirectory() as raw:
+            dataset = Path(raw)
+            ingest(dataset, FIRST)
+            ingest(dataset, SECOND)
+
+            # Unreviewed samples are not training data, however confident the proposal is.
+            self.assertEqual(tool.trainable_records(dataset), [])
+
+            tool.commit_review(dataset, FIRST, "reviewer-a", "agreed")
+            tool.commit_review(dataset, SECOND, "reviewer-a", "excluded", exclusion_reason="cannot tell stool from photo edge")
+
+            trainable = tool.trainable_records(dataset)
+            self.assertEqual([item["contributionId"] for item in trainable], [FIRST])
+            self.assertEqual(trainable[0]["labels"], {"bristolType": 4, "color": "brown", "quality": "good"})
+            self.assertEqual(trainable[0]["reviewedBy"], "reviewer-a")
+
+            # Excluded is a labeling judgement, not a withdrawal: the provenance record still exists.
+            history = [json.loads(line) for line in (dataset / "manifest.jsonl").read_text(encoding="utf-8").splitlines() if line.strip()]
+            self.assertEqual(len(history), 2)
+
+
+class PredictionFileTests(unittest.TestCase):
+    def test_a_pretty_printed_answer_file_is_read_whole_not_line_by_line(self):
+        with tempfile.TemporaryDirectory() as raw:
+            path = Path(raw) / "answers.json"
+            path.write_text(json.dumps({"model": "synthetic answer file", "answers": {FIRST: {"bristolType": 6, "color": "brown", "quality": "good"}}}, indent=2), encoding="utf-8")
+
+            loaded = tool.load_predictions(path)
+
+            self.assertEqual(loaded["answers"][FIRST]["bristolType"], 6)
+
+    def test_a_line_delimited_answer_file_is_read_by_contribution(self):
+        with tempfile.TemporaryDirectory() as raw:
+            path = Path(raw) / "answers.jsonl"
+            path.write_text(
+                json.dumps({"contributionId": FIRST, "bristolType": 6, "color": "brown", "quality": "good"}) + "\n"
+                + json.dumps({"contributionId": SECOND, "bristolType": 2, "color": "black", "quality": "poor"}) + "\n",
+                encoding="utf-8",
+            )
+
+            loaded = tool.load_predictions(path)
+
+            self.assertEqual(loaded["answers"][FIRST]["bristolType"], 6)
+            self.assertEqual(loaded["answers"][SECOND]["bristolType"], 2)
+
+    def test_an_answer_without_an_identifier_is_refused(self):
+        with tempfile.TemporaryDirectory() as raw:
+            path = Path(raw) / "answers.jsonl"
+            path.write_text(json.dumps({"bristolType": 6}) + "\n", encoding="utf-8")
+
+            with self.assertRaises(tool.ReviewError) as raised:
+                tool.load_predictions(path)
+            self.assertRegex(str(raised.exception), "contributionId or imageId")
+
+
+class CliTests(unittest.TestCase):
+    def run_cli(self, *args: str) -> subprocess.CompletedProcess:
+        return subprocess.run(
+            [sys.executable, str(ROOT / "scripts" / "review_corpus.py"), *args],
+            cwd=ROOT, capture_output=True, text=True,
+        )
+
+    def test_the_cli_keeps_the_blind_order_and_reports_refusals_non_zero(self):
+        with tempfile.TemporaryDirectory() as raw:
+            dataset = Path(raw)
+            ingest(dataset, FIRST)
+
+            queue = self.run_cli("queue", "--dataset", str(dataset))
+            self.assertEqual(queue.returncode, 0, queue.stderr)
+            packet = json.loads(queue.stdout)
+            self.assertEqual(packet["count"], 1)
+            self.assertNotIn("confidence", queue.stdout)
+
+            early = self.run_cli("reveal", "--dataset", str(dataset), "--contribution", FIRST)
+            self.assertNotEqual(early.returncode, 0)
+            self.assertRegex(early.stderr, "hidden until a review is committed")
+
+            committed = self.run_cli(
+                "review", "--dataset", str(dataset), "--contribution", FIRST,
+                "--reviewer", "reviewer-a", "--bristol", "5", "--color", "green", "--quality", "fair",
+            )
+            self.assertEqual(committed.returncode, 0, committed.stderr)
+            self.assertEqual(json.loads(committed.stdout)["decision"], "corrected")
+
+            reveal = self.run_cli("reveal", "--dataset", str(dataset), "--contribution", FIRST)
+            self.assertEqual(reveal.returncode, 0, reveal.stderr)
+            self.assertEqual(json.loads(reveal.stdout)["model"], None, "no prediction was supplied, so none is invented")
+
+            trainable = self.run_cli("trainable", "--dataset", str(dataset))
+            self.assertEqual(trainable.returncode, 0, trainable.stderr)
+            self.assertEqual(json.loads(trainable.stdout)["count"], 1)
+
+
+if __name__ == "__main__":
+    unittest.main(verbosity=2)
@@ -412,7 +412,7 @@ test('release demo replays the calibration harness against synthetic predictions
  assert.match(demo, /the engineered fixture must prove the passing branch/);
  assert.match(demo, /a leaked corpus must be refused before scoring/);
  // The card is drawn from the report the run wrote, and gated for readability like the corpus card.
-  assert.match(demo, /id = 'calibration-report'/);
+  assert.match(demo, /id: 'calibration-report'/);
  assert.match(demo, /Calibration card is not fully readable on screen/);
  assert.match(demo, /the card must show the macro-F1 the harness measured/);
  assert.match(demo, /the card must show the refusal the leaked corpus earned/);
@@ -425,3 +425,36 @@ test('release demo replays the calibration harness against synthetic predictions
  assert.match(builder, /Each candidate is scored on test at a threshold chosen on validation only/);
  assert.match(builder, /One contributor spread across two splits is refused before anything is scored/);
 });
+
+test('release demo runs the blind review tool in its enforced order instead of describing it', async () => {
+  const demo = await readFile(demoPath, 'utf8');
+  const builder = await readFile(builderPath, 'utf8');
+
+  // Issue #26: the review tool is a CLI. The scene runs the real commands and shows their output.
+  assert.match(demo, /scripts\/review_corpus\.py/);
+  assert.match(demo, /tests\/fixtures\/synthetic-type4\.jpg/);
+  // Blindness is proven by the refusal, not by a caption: reveal is attempted before any commit.
+  assert.match(demo, /stays hidden until a review is committed/);
+  // The reviewer's labels are committed before the model answer is shown, and both are recorded.
+  assert.match(demo, /--reviewer'?,? ?'reviewer-a'/);
+  assert.match(demo, /disagreeFields/);
+  // A second reader adjudicates; the review it resolves must still be in the ledger afterwards.
+  assert.match(demo, /'adjudicate'/);
+  assert.match(demo, /resolvesRecord/);
+  // A withdrawn contribution leaves the queue, the training set, and the review path entirely.
+  assert.match(demo, /was withdrawn and cannot be reviewed/);
+  assert.match(demo, /id: 'review-audit'/);
+  assert.match(demo, /Review card is not fully readable on screen/);
+  assert.match(demo, /the card must show the model answer the run revealed/);
+  assert.match(demo, /the card must show the withdrawn refusal the run earned/);
+  // The card must not claim a model measured anything: the answer file is a synthetic fixture.
+  assert.match(demo, /synthetic answer file · no model was measured/);
+
+  // The scene is a release gate, so a later release cannot quietly drop it.
+  assert.match(builder, /tests\/corpus-review\.test\.py/);
+  assert.match(builder, /"reviewer_adjudication_blind": "passed"/);
+  assert.match(builder, /The review tool is a command line, not a screen in the app/);
+  assert.match(builder, /The queue shows the proposal and hides the model answer; revealing it is refused until a reviewer commits/);
+  assert.match(builder, /The model said Type 6 and the reviewer said Type 5: the disagreement is recorded, not smoothed over/);
+  assert.match(builder, /Withdrawing the contribution empties the queue and the training set, and the tool refuses to review it at all/);
+});