Timmy
46f961668f
Quality gates / quality (pull_request) Successful in 3m29s
A worker that answers /v1/models has not necessarily loaded its weights, so the app was offering an AI control that could only fail after the upload. Readiness now passes only once the pinned model is loaded, and the not-loaded state reaches manual Bristol entry in one tap. Verified against the real pinned llama.cpp build: it serves 503 while loading and the gate reports the distinct not-loaded state. Adds the reproducible container recipe (digest-pinned bases, pinned llama.cpp commit, hash-verified read-only weights, unprivileged, loopback-only publish), one shared post-load readiness gate used by the host bootstrap and the image healthcheck, and a target-GPU benchmark that refuses to write a receipt it did not measure. Refs #15
68 lines
3.2 KiB
Docker
68 lines
3.2 KiB
Docker
# Timmy self-hosted vision worker (issue #15).
|
|
#
|
|
# Reproducible by construction: the llama.cpp revision and both model hashes are pinned here and in
|
|
# deploy/worker/receipts.json, and tests/worker-container.test.js fails if either drifts from
|
|
# scripts/bootstrap_selfhost_smolvlm.sh on the host.
|
|
#
|
|
# The weights are deliberately NOT baked into the image. They are mounted read-only at /models and
|
|
# hash-verified against the pinned receipts by the entrypoint before the server starts.
|
|
|
|
ARG LLAMA_CPP_COMMIT=6d05498314db1b57f81c271080018aa2d0b89be9
|
|
ARG MODEL_SHA256=0cf76814555b8665149075b74ab6b5c1d428ea1d3d01c1918c12012e8d7c9f58
|
|
ARG MMPROJ_SHA256=ae07ea1facd07dd3230c4483b63e8cda96c6944ad2481f33d531f79e892dd024
|
|
|
|
# Digest-pinned bases, recorded in deploy/worker/receipts.json. A tag alone can be repointed, so
|
|
# the digest is what makes this build reproducible.
|
|
FROM nvidia/cuda:12.4.1-devel-ubuntu22.04@sha256:da6791294b0b04d7e65d87b7451d6f2390b4d36225ab0701ee7dfec5769829f5 AS build
|
|
|
|
ARG LLAMA_CPP_COMMIT
|
|
RUN apt-get update \
|
|
&& apt-get install --yes --no-install-recommends git ca-certificates cmake build-essential \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
WORKDIR /src
|
|
RUN git init llama.cpp \
|
|
&& cd llama.cpp \
|
|
&& git remote add origin https://github.com/ggml-org/llama.cpp \
|
|
&& git fetch --depth 1 origin "${LLAMA_CPP_COMMIT}" \
|
|
&& git checkout --detach FETCH_HEAD
|
|
RUN cmake -S llama.cpp -B llama.cpp/build -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=OFF -DGGML_CUDA=ON \
|
|
&& cmake --build llama.cpp/build --parallel "$(nproc)" --target llama-server
|
|
|
|
FROM nvidia/cuda:12.4.1-runtime-ubuntu22.04@sha256:517da2300c184c9999ec203c2665244bdebd3578d12fcc7065e83667932643d9 AS runtime
|
|
|
|
ARG MODEL_SHA256
|
|
ARG MMPROJ_SHA256
|
|
|
|
RUN apt-get update \
|
|
&& apt-get install --yes --no-install-recommends curl ca-certificates \
|
|
&& rm -rf /var/lib/apt/lists/* \
|
|
&& mkdir -p /models
|
|
|
|
COPY --from=build /src/llama.cpp/build/bin/ /usr/local/bin/
|
|
COPY deploy/worker/entrypoint.sh /usr/local/bin/timmy-worker-entrypoint
|
|
COPY scripts/worker_ready.sh /usr/local/bin/timmy-worker-ready
|
|
RUN chmod 0755 /usr/local/bin/timmy-worker-entrypoint /usr/local/bin/timmy-worker-ready && ldconfig
|
|
|
|
# No credential is configured: the worker is a private service the app reaches over host loopback.
|
|
# 0.0.0.0 here is the address inside this container's own network namespace; the run recipe decides
|
|
# exposure and publishes on 127.0.0.1 only.
|
|
ENV TIMMY_MODEL_DIR=/models \
|
|
TIMMY_MODEL_FILE=/models/SmolVLM2-2.2B-Instruct-Q4_K_M.gguf \
|
|
TIMMY_MMPROJ_FILE=/models/mmproj-SmolVLM2-2.2B-Instruct-Q8_0.gguf \
|
|
TIMMY_MODEL_SHA256=${MODEL_SHA256} \
|
|
TIMMY_MMPROJ_SHA256=${MMPROJ_SHA256} \
|
|
TIMMY_MODEL_ALIAS=SmolVLM2-2.2B-Instruct \
|
|
TIMMY_MODEL_HOST=0.0.0.0 \
|
|
TIMMY_MODEL_PORT=8080
|
|
|
|
USER 10001:10001
|
|
EXPOSE 8080
|
|
|
|
# Health is the post-load gate: the worker is not healthy until the pinned model is being served,
|
|
# so an orchestrator cannot route to a worker that cannot answer. start-period covers a cold load.
|
|
HEALTHCHECK --interval=30s --timeout=5s --start-period=600s --retries=3 \
|
|
CMD ["/usr/local/bin/timmy-worker-ready", "--host", "127.0.0.1", "--port", "8080", "--alias", "SmolVLM2-2.2B-Instruct"]
|
|
|
|
ENTRYPOINT ["/usr/local/bin/timmy-worker-entrypoint"]
|