Add bounded inference queue, cancellation, and overload fallback #17

New Issue

timmy · 2026-08-19T01:58:59Z

timmy commented

2026-08-19 01:58:59 +00:00

Parent epic: #2

Outcome

Keep photo logging responsive under slow or saturated inference.

Acceptance criteria

Queue has concurrency, deadline, and max-depth limits.
Client receives a stable retry/manual response on timeout or overload.
Cancelled requests release buffers and worker slots.

Dependencies

#15 — Containerize the llama.cpp GPU worker with pinned model receipts

Required evidence

Test or executable receipt attached to the issue/PR
Privacy/safety boundary checked where applicable
Clean checkout and relevant full suite pass

Delivery

Open a focused branch and PR with Closes #ISSUE. Do not include real medical images in issue comments or Git history.

Parent epic: #2 ## Outcome Keep photo logging responsive under slow or saturated inference. ## Acceptance criteria - [ ] Queue has concurrency, deadline, and max-depth limits. - [ ] Client receives a stable retry/manual response on timeout or overload. - [ ] Cancelled requests release buffers and worker slots. ## Dependencies - #15 — Containerize the llama.cpp GPU worker with pinned model receipts ## Required evidence - Test or executable receipt attached to the issue/PR - Privacy/safety boundary checked where applicable - Clean checkout and relevant full suite pass ## Delivery Open a focused branch and PR with `Closes #ISSUE`. Do not include real medical images in issue comments or Git history.

timmy added this to the M1 — Sovereign Photo Intelligence milestone 2026-08-19 01:58:59 +00:00

timmy added the

labels 2026-08-19 01:58:59 +00:00

timmy referenced this issue

2026-08-19 01:58:59 +00:00

Instrument privacy-safe model health, latency, abstention, and failure metrics #19

timmy referenced this issue

2026-08-19 01:59:03 +00:00

EPIC: Sovereign vision inference #2

rockachopa referenced this issue from a commit

2026-08-22 23:05:14 +00:00

feat: bounded inference queue with deterministic cancellation

rockachopa commented

2026-08-22 23:05:52 +00:00

Implementation pushed on branch timmy/17-bounded-inference-queue (PR to follow). Evidence from a clean checkout of the branch:

Unit + security suite: npm test → 79 tests, 79 pass, 0 fail (3 consecutive full runs; includes new suites: inference-queue, inference-queue-cancellation, agent-queue, agent-turn-timeout, inference-zero-call).
Python staging-deploy suite: 20/20 OK.
Acceptance (real HTTP + real subprocesses):

npm run test:queue-acceptance — overload returns sanitized manual-fallback copy with zero provider contact, urgent text bypasses a saturated queue (safetyOverride, zero Hermes calls), abandoned client frees its queue slot, all surviving requests complete after recovery, and every spawned fixture PID is reaped (no orphans).
npm run test:shutdown-acceptance — SIGTERM during an in-flight turn kills the child process; no orphan survives.
Browser acceptance: test:ui / test:photo / test:sleek PASS against a live server.
Syntax gates: npm run check:syntax OK. Diff hygiene: check:diff OK. Dependency audit: npm audit --audit-level=high → 0 vulnerabilities.
Privacy/safety boundary: urgent-symptom interception still precedes any queue admission and any Hermes/vision call — proven under saturation (zero-call invariant); no credential/browser-controlled options accepted; error copy sanitized (no session ids, argv, paths).
No real medical images in commits or comments.

Implementation pushed on branch `timmy/17-bounded-inference-queue` (PR to follow). Evidence from a clean checkout of the branch: **Unit + security suite:** `npm test` → 79 tests, 79 pass, 0 fail (3 consecutive full runs; includes new suites: inference-queue, inference-queue-cancellation, agent-queue, agent-turn-timeout, inference-zero-call). **Python staging-deploy suite:** 20/20 OK. **Acceptance (real HTTP + real subprocesses):** - `npm run test:queue-acceptance` — overload returns sanitized manual-fallback copy with zero provider contact, urgent text bypasses a saturated queue (safetyOverride, zero Hermes calls), abandoned client frees its queue slot, all surviving requests complete after recovery, and every spawned fixture PID is reaped (no orphans). - `npm run test:shutdown-acceptance` — SIGTERM during an in-flight turn kills the child process; no orphan survives. **Browser acceptance:** test:ui / test:photo / test:sleek PASS against a live server. **Syntax gates:** npm run check:syntax OK. **Diff hygiene:** check:diff OK. **Dependency audit:** npm audit --audit-level=high → 0 vulnerabilities. **Privacy/safety boundary:** urgent-symptom interception still precedes any queue admission and any Hermes/vision call — proven under saturation (zero-call invariant); no credential/browser-controlled options accepted; error copy sanitized (no session ids, argv, paths). No real medical images in commits or comments.

rockachopa referenced a pull request that will close this issue

2026-08-22 23:06:12 +00:00

Add bounded inference queue, cancellation, and overload fallback #64

Sign in to join this conversation.