feat(metrics): read the privacy-safe health snapshot without jq (#19) #92

2026-11-02T10:56:37Z

rockachopa commented

2026-11-02 10:56:37 +00:00

Closes #19

What this adds

The counters from #19 had an endpoint and no way to read them without jq. This adds the operator report:
a pure formatter (src/model-metrics-report.js) and a CLI (scripts/model_metrics_report.mjs) that prints
the app, queue, and model boundaries, the readiness phase the worker actually reported, the
abstention rate, latency, and every enumerated failure class.

The privacy property is structural: the formatter reads only the snapshot and the three frozen
vocabularies, so a failure class outside the enum is not printed rather than echoed. Nothing about a
request — note text, image bytes, hash, caller identity — has a path to the terminal.

Refusal paths are explicit instead of generic: 404 says metrics are disabled because no token is
configured, 403 says the token was rejected, 2 means the command was misused. The token is read from
the environment or --token and never appears in the report or in an error line.

RED receipts

tests/model-metrics-report-format.test.js → ERR_MODULE_NOT_FOUND: .../src/model-metrics-report.js before the module existed.
refusal-path test → failed with The server answered 404 for /api/metrics. and the message
a server with no metrics route must say so, not "unauthorized" before the 404 branch was written.
tests/model-metrics-report.acceptance.test.js was already RED on the preserved snapshot branch
(MODULE_NOT_FOUND: scripts/model_metrics_report.mjs); it is green here.

Verification (this branch, clean checkout)

npm test → 298/298 pass, 0 fail
npm run test:metrics-report → 8/8 pass (spawns the real server; a real 200, a real 404, a real 403)
npm run check:syntax → pass · npm run check:diff → pass
tests/ui.acceptance.mjs in real Chromium → PASS, with [data-health-disclosure] visible on the
privacy screen and artifacts/privacy-health-disclosure.png regenerated

Demonstration path

scripts/record_release_demo.mjs reads the recorded sentence back from the real privacy screen, fails the
recording if the disclosure copy changes, captions it ("Health counters are disclosed where you read
privacy facts — they never include notes, photos, image fingerprints, or identity"), and puts a touch
indicator on the disclosure. tests/release-demo.test.js gates that scene and the
privacy_safe_health_metrics release-gate entry, so a later release cannot drop it silently.

Known gap (unchanged, stated in docs)

queue.inFlight is the app's own admission depth for POST /api/analyze. The full bounded queue —
concurrency enforcement, cancellation, overload fallback — is #17 (PR #64, unmerged). This snapshot
reports the slot accounting that exists on main and does not claim the rest; docs/MODEL-METRICS.md
says so in the operator's own words.

Not merged: left for human RC review.

Closes #19 ## What this adds The counters from #19 had an endpoint and no way to read them without `jq`. This adds the operator report: a pure formatter (`src/model-metrics-report.js`) and a CLI (`scripts/model_metrics_report.mjs`) that prints the **app**, **queue**, and **model** boundaries, the readiness phase the worker actually reported, the abstention rate, latency, and every enumerated failure class. The privacy property is structural: the formatter reads only the snapshot and the three frozen vocabularies, so a failure class outside the enum is *not printed* rather than echoed. Nothing about a request — note text, image bytes, hash, caller identity — has a path to the terminal. Refusal paths are explicit instead of generic: `404` says metrics are disabled because no token is configured, `403` says the token was rejected, `2` means the command was misused. The token is read from the environment or `--token` and never appears in the report or in an error line. ## RED receipts - `tests/model-metrics-report-format.test.js` → `ERR_MODULE_NOT_FOUND: .../src/model-metrics-report.js` before the module existed. - refusal-path test → failed with `The server answered 404 for /api/metrics.` and the message `a server with no metrics route must say so, not "unauthorized"` before the 404 branch was written. - `tests/model-metrics-report.acceptance.test.js` was already RED on the preserved snapshot branch (`MODULE_NOT_FOUND: scripts/model_metrics_report.mjs`); it is green here. ## Verification (this branch, clean checkout) - `npm test` → **298/298 pass**, 0 fail - `npm run test:metrics-report` → **8/8 pass** (spawns the real server; a real 200, a real 404, a real 403) - `npm run check:syntax` → pass · `npm run check:diff` → pass - `tests/ui.acceptance.mjs` in real Chromium → **PASS**, with `[data-health-disclosure]` visible on the privacy screen and `artifacts/privacy-health-disclosure.png` regenerated ## Demonstration path `scripts/record_release_demo.mjs` reads the recorded sentence back from the real privacy screen, fails the recording if the disclosure copy changes, captions it (*"Health counters are disclosed where you read privacy facts — they never include notes, photos, image fingerprints, or identity"*), and puts a touch indicator on the disclosure. `tests/release-demo.test.js` gates that scene and the `privacy_safe_health_metrics` release-gate entry, so a later release cannot drop it silently. ## Known gap (unchanged, stated in docs) `queue.inFlight` is the app's own admission depth for `POST /api/analyze`. The full bounded queue — concurrency enforcement, cancellation, overload fallback — is #17 (PR #64, unmerged). This snapshot reports the slot accounting that exists on `main` and does not claim the rest; `docs/MODEL-METRICS.md` says so in the operator's own words. Not merged: left for human RC review.

rockachopa added 2 commits 2026-11-02 10:56:37 +00:00

wip(#19 ): privacy-safe model metrics in progress — preserved snapshot before nightly release 3e2253a052

Local work in progress, not reviewed and not part of any PR. Committed only so the
nightly release can build from a clean main without discarding it.

feat(metrics): render the operator health snapshot as a report (#19 )

Quality gates / quality (pull_request) Waiting to run

Details

e08b537866

The counters had an endpoint and no way to read them without jq. Adds a pure
formatter plus a CLI that prints the app, queue, and model boundaries.

The formatter is a function of the snapshot and the three frozen vocabularies,
so a caller-supplied failure label cannot reach the terminal: unknown classes
are simply not printed. Refusal paths are explicit — 404 means the route is off
because no token is configured, 403 means the token was rejected — and the token
is never echoed in the report or an error line.

RED: tests/model-metrics-report-format.test.js failed with ERR_MODULE_NOT_FOUND
before src/model-metrics-report.js existed; the refusal-path test failed with a
generic 'answered 404' before the 404 branch was written.

Tests: npm test 298/298, npm run test:metrics-report 8/8, syntax and diff gates
clean, real-Chromium ui acceptance green with the privacy disclosure visible.

rockachopa merged commit 48979bf744 into main

2026-11-02 12:07:00 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: stackchain/timmy-talking-turd#92