Containerize the llama.cpp GPU worker with pinned model receipts #15

New Issue

timmy · 2026-08-19T01:58:59Z

timmy commented

2026-08-19 01:58:59 +00:00

Parent epic: #2

Outcome

Run the open-weight worker on separate GPU compute with reproducible artifacts.

Acceptance criteria

Image pins llama.cpp commit and model hashes.
Health/readiness only pass after model load.
No public bind, default credentials, or bundled user images.
Cold/warm latency and VRAM are recorded on the target GPU.

Dependencies

#10 — Package a reproducible self-hosted SmolVLM2 bootstrap

Required evidence

Test or executable receipt attached to the issue/PR
Privacy/safety boundary checked where applicable
Clean checkout and relevant full suite pass

Delivery

Open a focused branch and PR with Closes #ISSUE. Do not include real medical images in issue comments or Git history.

Parent epic: #2 ## Outcome Run the open-weight worker on separate GPU compute with reproducible artifacts. ## Acceptance criteria - [ ] Image pins llama.cpp commit and model hashes. - [ ] Health/readiness only pass after model load. - [ ] No public bind, default credentials, or bundled user images. - [ ] Cold/warm latency and VRAM are recorded on the target GPU. ## Dependencies - #10 — Package a reproducible self-hosted SmolVLM2 bootstrap ## Required evidence - Test or executable receipt attached to the issue/PR - Privacy/safety boundary checked where applicable - Clean checkout and relevant full suite pass ## Delivery Open a focused branch and PR with `Closes #ISSUE`. Do not include real medical images in issue comments or Git history.

timmy added this to the M1 — Sovereign Photo Intelligence milestone 2026-08-19 01:58:59 +00:00

timmy added the

type/ops

priority/P0

area/model

labels 2026-08-19 01:58:59 +00:00

timmy referenced this issue

2026-08-19 01:58:59 +00:00

Add bounded inference queue, cancellation, and overload fallback #17

timmy referenced this issue

2026-08-19 01:59:02 +00:00

Define production topology, secrets, backup, and rollback runbook #39