Timmy
46f961668f
Quality gates / quality (pull_request) Successful in 3m29s
A worker that answers /v1/models has not necessarily loaded its weights, so the app was offering an AI control that could only fail after the upload. Readiness now passes only once the pinned model is loaded, and the not-loaded state reaches manual Bristol entry in one tap. Verified against the real pinned llama.cpp build: it serves 503 while loading and the gate reports the distinct not-loaded state. Adds the reproducible container recipe (digest-pinned bases, pinned llama.cpp commit, hash-verified read-only weights, unprivileged, loopback-only publish), one shared post-load readiness gate used by the host bootstrap and the image healthcheck, and a target-GPU benchmark that refuses to write a receipt it did not measure. Refs #15
47 lines
1.6 KiB
Bash
Executable File
47 lines
1.6 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Start the pinned worker after verifying the mounted weights (issue #15).
|
|
#
|
|
# Fails closed before the server is allowed to answer: a missing pin, an unreadable model, or a
|
|
# hash mismatch stops the container instead of serving suggestions from an unknown model.
|
|
set -euo pipefail
|
|
|
|
MODEL="${TIMMY_MODEL_FILE:-/models/SmolVLM2-2.2B-Instruct-Q4_K_M.gguf}"
|
|
MMPROJ="${TIMMY_MMPROJ_FILE:-/models/mmproj-SmolVLM2-2.2B-Instruct-Q8_0.gguf}"
|
|
MODEL_SHA256="${TIMMY_MODEL_SHA256:-}"
|
|
MMPROJ_SHA256="${TIMMY_MMPROJ_SHA256:-}"
|
|
ALIAS="${TIMMY_MODEL_ALIAS:-SmolVLM2-2.2B-Instruct}"
|
|
HOST="${TIMMY_MODEL_HOST:-0.0.0.0}"
|
|
PORT="${TIMMY_MODEL_PORT:-8080}"
|
|
THREADS="${TIMMY_MODEL_THREADS:-4}"
|
|
|
|
for pin in "$MODEL_SHA256" "$MMPROJ_SHA256"; do
|
|
if [[ ! "$pin" =~ ^[0-9a-f]{64}$ ]]; then
|
|
printf 'Refusing to start: the pinned model hashes are missing or malformed.\n' >&2
|
|
exit 78
|
|
fi
|
|
done
|
|
|
|
for file in "$MODEL" "$MMPROJ"; do
|
|
if [[ ! -f "$file" ]]; then
|
|
printf 'Refusing to start: %s is not mounted. Mount the verified model directory read-only at /models.\n' "$file" >&2
|
|
exit 78
|
|
fi
|
|
done
|
|
|
|
printf '%s %s\n%s %s\n' "$MODEL_SHA256" "$MODEL" "$MMPROJ_SHA256" "$MMPROJ" | sha256sum --check --status || {
|
|
printf 'Refusing to start: mounted weights do not match the pinned receipts.\n' >&2
|
|
exit 78
|
|
}
|
|
|
|
printf 'Starting %s from pinned weights on %s:%s (private namespace)\n' "$ALIAS" "$HOST" "$PORT"
|
|
exec llama-server \
|
|
--model "$MODEL" \
|
|
--mmproj "$MMPROJ" \
|
|
--alias "$ALIAS" \
|
|
--host "$HOST" \
|
|
--port "$PORT" \
|
|
--ctx-size 4096 \
|
|
--parallel 1 \
|
|
--threads "$THREADS" \
|
|
--no-webui
|