forked from Rockachopa/Timmy-time-dashboard
## What Cache the Timmy agent instance at app startup (in lifespan) instead of creating a new one per `/serve/chat` request. ## Changes - `src/timmy_serve/app.py`: Create agent in lifespan, store in `app.state.timmy` - `tests/timmy/test_timmy_serve_app.py`: Updated tests for lifespan-based caching, added `test_agent_cached_at_startup` 2085 unit tests pass. 2102 pre-push tests pass. 78.5% coverage. Closes #471 Co-authored-by: Timmy <timmy@timmytime.ai> Reviewed-on: http://localhost:3000/rockachopa/Timmy-time-dashboard/pulls/476 Co-authored-by: Timmy Time <timmy@Alexanderwhitestone.ai> Co-committed-by: Timmy Time <timmy@Alexanderwhitestone.ai>
This commit is contained in:
@@ -75,6 +75,8 @@ def create_timmy_serve_app() -> FastAPI:
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
logger.info("Timmy Serve starting")
|
||||
app.state.timmy = create_timmy()
|
||||
logger.info("Timmy agent cached in app state")
|
||||
yield
|
||||
logger.info("Timmy Serve shutting down")
|
||||
|
||||
@@ -101,7 +103,7 @@ def create_timmy_serve_app() -> FastAPI:
|
||||
async def serve_chat(request: Request, body: ChatRequest):
|
||||
"""Process a chat request."""
|
||||
try:
|
||||
timmy = create_timmy()
|
||||
timmy = request.app.state.timmy
|
||||
result = timmy.run(body.message, stream=False)
|
||||
response_text = result.content if hasattr(result, "content") else str(result)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user