dmahan93 13f5459670 fix: use ManagedServer for vLLM in TBLite eval + local_vllm config ...

TBLite eval was bypassing ManagedServer and calling ServerManager
directly, which uses /v1/chat/completions — not available on the
atropos vllm_api_server (/generate only).

Now uses _use_managed_server() to detect vLLM/SGLang backends and
route through ManagedServer (Phase 2) with proper tool_parser and
/generate endpoint. Falls back to Phase 1 for OpenAI endpoints.

Also adds local_vllm.yaml config for running against a local vLLM
server with Docker sandboxes.

2026-03-11 06:52:55 -07:00

21 KiB

Raw Blame History

View Raw