stackchain/timmy-talking-turd
2
0 Fork 0
Code Issues 21 Pull Requests 7 Actions 4 Packages Projects Releases 41 Wiki Activity

Containerize the llama.cpp GPU worker with pinned model receipts #15

New Issue
Open
opened 2026-08-19 01:58:59 +00:00 by timmy · 1 comment
timmy commented 2026-08-19 01:58:59 +00:00
Owner
Copy link
Copy source

Parent epic: #2

Outcome

Run the open-weight worker on separate GPU compute with reproducible artifacts.

Acceptance criteria

  • Image pins llama.cpp commit and model hashes.
  • Health/readiness only pass after model load.
  • No public bind, default credentials, or bundled user images.
  • Cold/warm latency and VRAM are recorded on the target GPU.

Dependencies

  • #10 — Package a reproducible self-hosted SmolVLM2 bootstrap

Required evidence

  • Test or executable receipt attached to the issue/PR
  • Privacy/safety boundary checked where applicable
  • Clean checkout and relevant full suite pass

Delivery

Open a focused branch and PR with Closes #ISSUE. Do not include real medical images in issue comments or Git history.

Parent epic: #2 ## Outcome Run the open-weight worker on separate GPU compute with reproducible artifacts. ## Acceptance criteria - [ ] Image pins llama.cpp commit and model hashes. - [ ] Health/readiness only pass after model load. - [ ] No public bind, default credentials, or bundled user images. - [ ] Cold/warm latency and VRAM are recorded on the target GPU. ## Dependencies - #10 — Package a reproducible self-hosted SmolVLM2 bootstrap ## Required evidence - Test or executable receipt attached to the issue/PR - Privacy/safety boundary checked where applicable - Clean checkout and relevant full suite pass ## Delivery Open a focused branch and PR with `Closes #ISSUE`. Do not include real medical images in issue comments or Git history.
timmy added this to the M1 — Sovereign Photo Intelligence milestone 2026-08-19 01:58:59 +00:00
timmy added the type/opspriority/P0area/model labels 2026-08-19 01:58:59 +00:00
rockachopa commented 2026-10-13 08:21:13 +00:00
Member
Copy link
Copy source

[EVIDENCE] agent=timmy

PR #75 (branch timmy/15-container-worker-receipts at 46f9616) advances this issue without closing it.

Verified on the real pinned worker (llama.cpp 6d054983, SmolVLM2-2.2B-Instruct Q4_K_M + mmproj): the server answers /v1/models with HTTP 503 while the weights load, and scripts/worker_ready.sh reports that as its own state (exit 3, "not loaded yet") until the model is resident, then exit 0. The app previously called that first state ready and offered consent plus Analyze; readiness now requires the loaded model and falls back to manual Bristol entry in one tap.

Also in: digest-pinned container recipe with a pinned llama.cpp commit and hash-verified read-only weights, one shared post-load gate used by the host bootstrap and the image HEALTHCHECK, and a target-GPU benchmark harness. Unit 146 pass, nine browser acceptance suites pass, syntax/diff/staging gates pass, secret scan clean, and the release demo records the new scene on camera (caption added to the required on-camera gate).

Outstanding and blocking closure:

  • Criterion 4 (cold/warm latency and VRAM recorded on the target GPU) is instrumented but unmeasured. This run had no GPU and buying compute is out of bounds; the harness exits non-zero and writes nothing rather than inventing numbers.
  • Criterion 3 is enforced by contract tests on the recipe, not by a running container: the image targets a CUDA host and this host has none, so it was not built.

Not merging; awaiting review.

[EVIDENCE] agent=timmy PR #75 (branch `timmy/15-container-worker-receipts` at 46f9616) advances this issue without closing it. Verified on the real pinned worker (llama.cpp 6d054983, SmolVLM2-2.2B-Instruct Q4_K_M + mmproj): the server answers `/v1/models` with HTTP 503 while the weights load, and `scripts/worker_ready.sh` reports that as its own state (exit 3, "not loaded yet") until the model is resident, then exit 0. The app previously called that first state ready and offered consent plus Analyze; readiness now requires the loaded model and falls back to manual Bristol entry in one tap. Also in: digest-pinned container recipe with a pinned llama.cpp commit and hash-verified read-only weights, one shared post-load gate used by the host bootstrap and the image HEALTHCHECK, and a target-GPU benchmark harness. Unit 146 pass, nine browser acceptance suites pass, syntax/diff/staging gates pass, secret scan clean, and the release demo records the new scene on camera (caption added to the required on-camera gate). Outstanding and blocking closure: - Criterion 4 (cold/warm latency and VRAM recorded on the target GPU) is instrumented but unmeasured. This run had no GPU and buying compute is out of bounds; the harness exits non-zero and writes nothing rather than inventing numbers. - Criterion 3 is enforced by contract tests on the recipe, not by a running container: the image targets a CUDA host and this host has none, so it was not built. Not merging; awaiting review.
Sign in to join this conversation.
No Branch/Tag Specified
Labels
Clear labels
area/backend
Application API and storage
 area/data
Dataset and provenance
 area/frontend
PWA and interaction design
 area/model
Vision model and inference
 area/privacy
Privacy and consent
 area/release
Release engineering and distribution
 area/safety
Clinical safety boundary
 blocked
Blocked on an explicit dependency or decision
 human-gate
Requires explicit human approval
 priority/P0
Release-critical
 priority/P1
High priority
 priority/P2
Planned
 state:review
type/docs
Documentation and evidence
 type/epic
Roadmap epic
 type/feature
Product feature
 type/ops
Infrastructure and operations
 type/research
Research or evaluation
 type/security
Security hardening
 type/test
Verification and quality gate
No labels area/model priority/P0 type/ops
Milestone
No items
No Milestone M1 — Sovereign Photo Intelligence
Projects
Clear projects
No projects
Assignees
Clear assignees
timmy (Timmy · the Wizard Mind) zephyrus-sparkborn (Zephyrus Sparkborn)
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: stackchain/timmy-talking-turd#15