Add bounded inference queue, cancellation, and overload fallback #64

Open

rockachopa wants to merge 3 commits from timmy/17-bounded-inference-queue into main

pull from: timmy/17-bounded-inference-queue

Author	SHA1	Message	Date
timmy	9118b69057	feat: graceful shutdown, disconnect-driven cancellation, and queue acceptance coverage Quality gates / quality (pull_request) Successful in 1m58s Details - server.mjs tracks in-flight chats: browser disconnects abort the queued turn (releasing its slot), and SIGTERM/SIGINT cancels every in-flight turn so no Hermes subprocess is orphaned. - vision-service forwards a per-request AbortSignal combined with the provider deadline via AbortSignal.any. - acceptance suites cover overload fallback, urgent bypass under saturation, disconnect cleanup, recovery, and shutdown orphan checks. - zero-call invariant tests prove urgent text never reaches Hermes or the vision provider under load.	2026-08-22 23:01:20 +00:00
timmy	5dcaaae4d6	feat: queue agent turns with cancellation and orphan-free timeout kills Chat turns now flow through the bounded inference queue: FIFO with configurable depth (TIMMY_AGENT_MAX_QUEUE_DEPTH) and concurrency (TIMMY_AGENT_MAX_CONCURRENT_TURNS), sanitized overload/timeout fallbacks, browser-disconnect cancellation via AbortSignal, and child process SIGTERM/SIGKILL teardown so no Hermes subprocess survives a cancelled turn.	2026-08-22 22:04:17 +00:00
timmy	38a59c8ee9	feat: bounded inference queue with deterministic cancellation Shared primitive for Timmy #17: concurrency slots, FIFO max-depth overload rejection with sanitized manual-fallback copy, queue-deadline expiry, per-request AbortSignal delivery, immediate slot release on external cancel.	2026-08-22 21:39:55 +00:00

Author

SHA1

Message

Date

timmy

9118b69057

feat: graceful shutdown, disconnect-driven cancellation, and queue acceptance coverage

Quality gates / quality (pull_request) Successful in 1m58s

Details

- server.mjs tracks in-flight chats: browser disconnects abort the
  queued turn (releasing its slot), and SIGTERM/SIGINT cancels every
  in-flight turn so no Hermes subprocess is orphaned.
- vision-service forwards a per-request AbortSignal combined with the
  provider deadline via AbortSignal.any.
- acceptance suites cover overload fallback, urgent bypass under
  saturation, disconnect cleanup, recovery, and shutdown orphan checks.
- zero-call invariant tests prove urgent text never reaches Hermes or
  the vision provider under load.

2026-08-22 23:01:20 +00:00

timmy

5dcaaae4d6

feat: queue agent turns with cancellation and orphan-free timeout kills

Chat turns now flow through the bounded inference queue: FIFO with
configurable depth (TIMMY_AGENT_MAX_QUEUE_DEPTH) and concurrency
(TIMMY_AGENT_MAX_CONCURRENT_TURNS), sanitized overload/timeout
fallbacks, browser-disconnect cancellation via AbortSignal, and child
process SIGTERM/SIGKILL teardown so no Hermes subprocess survives a
cancelled turn.

2026-08-22 22:04:17 +00:00

timmy

38a59c8ee9

feat: bounded inference queue with deterministic cancellation

Shared primitive for Timmy #17: concurrency slots, FIFO max-depth
overload rejection with sanitized manual-fallback copy, queue-deadline
expiry, per-request AbortSignal delivery, immediate slot release on
external cancel.

2026-08-22 21:39:55 +00:00