Timmy
46f961668f
Quality gates / quality (pull_request) Successful in 3m29s
A worker that answers /v1/models has not necessarily loaded its weights, so the app was offering an AI control that could only fail after the upload. Readiness now passes only once the pinned model is loaded, and the not-loaded state reaches manual Bristol entry in one tap. Verified against the real pinned llama.cpp build: it serves 503 while loading and the gate reports the distinct not-loaded state. Adds the reproducible container recipe (digest-pinned bases, pinned llama.cpp commit, hash-verified read-only weights, unprivileged, loopback-only publish), one shared post-load readiness gate used by the host bootstrap and the image healthcheck, and a target-GPU benchmark that refuses to write a receipt it did not measure. Refs #15
61 lines
2.1 KiB
Bash
Executable File
61 lines
2.1 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Post-load readiness gate for Timmy's self-hosted vision worker (issue #15).
|
|
#
|
|
# A worker that answers on its port is not necessarily a worker that can answer: llama.cpp opens
|
|
# its HTTP listener before the model finishes loading and returns an error until the weights are
|
|
# resident. Readiness therefore passes only when the pinned model alias is actually served.
|
|
#
|
|
# Exit codes:
|
|
# 0 ready — the pinned model is loaded and being served
|
|
# 2 the worker is not reachable
|
|
# 3 the worker is reachable but the pinned model is not loaded yet
|
|
# 64 usage error (including a non-loopback probe host)
|
|
set -uo pipefail
|
|
|
|
HOST="${TIMMY_MODEL_HOST:-127.0.0.1}"
|
|
PORT="${TIMMY_MODEL_PORT:-8080}"
|
|
ALIAS="${TIMMY_MODEL_ALIAS:-SmolVLM2-2.2B-Instruct}"
|
|
WAIT=0
|
|
|
|
while [[ $# -gt 0 ]]; do
|
|
case "$1" in
|
|
--host) HOST="${2:-}"; shift 2 ;;
|
|
--port) PORT="${2:-}"; shift 2 ;;
|
|
--alias) ALIAS="${2:-}"; shift 2 ;;
|
|
--wait) WAIT="${2:-}"; shift 2 ;;
|
|
*)
|
|
printf 'Usage: %s [--host H] [--port P] [--alias A] [--wait SECONDS]\n' "$0" >&2
|
|
exit 64
|
|
;;
|
|
esac
|
|
done
|
|
|
|
# The worker is a private, loopback-only service. Probing anything else would be a mistake about
|
|
# exposure, so refuse rather than answer about a host this gate is not responsible for.
|
|
if [[ ! "$HOST" =~ ^(127\.0\.0\.1|localhost|::1)$ ]]; then
|
|
printf 'Readiness must be probed on loopback, got %s\n' "$HOST" >&2
|
|
exit 64
|
|
fi
|
|
|
|
deadline=$(( $(date +%s) + WAIT ))
|
|
outcome='unreachable'
|
|
while :; do
|
|
body="$(curl --silent --max-time 3 "http://$HOST:$PORT/v1/models" 2>/dev/null)"
|
|
if [[ $? -eq 0 && -n "$body" ]]; then
|
|
if printf '%s' "$body" | grep -q -- "$ALIAS"; then
|
|
printf 'Worker ready: %s is loaded on http://%s:%s/v1\n' "$ALIAS" "$HOST" "$PORT"
|
|
exit 0
|
|
fi
|
|
outcome='not-loaded'
|
|
printf 'Worker reachable on http://%s:%s/v1 but %s is not loaded yet.\n' "$HOST" "$PORT" "$ALIAS" >&2
|
|
else
|
|
outcome='unreachable'
|
|
printf 'Worker not reachable on http://%s:%s/v1.\n' "$HOST" "$PORT" >&2
|
|
fi
|
|
[[ "$(date +%s)" -ge "$deadline" ]] && break
|
|
sleep 1
|
|
done
|
|
|
|
[[ "$outcome" == 'not-loaded' ]] && exit 3
|
|
exit 2
|