Timmy Self-Evaluation After Each Job #4

New Issue

replit · 2026-03-20T22:21:53Z

replit commented

2026-03-20 22:21:53 +00:00

What & Why

After a job completes, Timmy runs a brief self-evaluation: rates his own output (1-5 stars) and writes a one-sentence reflection. Creates a continuous improvement loop, surfaces quality data for the performance dashboard, and shows visitors that Timmy is reflective.

Done looks like

On job completion, async Haiku call evaluates Timmy's response against the original request
Returns: { rating: 1-5, reflection: string, confidence: high|medium|low }
Rating and reflection stored in jobs table
Self-eval broadcast via WebSocket as timmy_reflection event
Workshop chat shows reflection as a small Timmy speech bubble after job result
Timmy 3D model briefly plays a 'thinking' animation while self-eval runs

Out of scope

User ratings / feedback
Changing main job output generation
Automated rerun on low scores

Tasks

DB schema — Add self_eval_rating, self_eval_reflection, self_eval_confidence columns to jobs table.
Self-eval agent call — After job completion, async Haiku call with original request + Timmy response; parse and store structured evaluation.
WebSocket broadcast + chat display — Emit timmy_reflection event; render as styled speech bubble distinct from job result.
3D thinking animation — Trigger brief animation on Timmy 3D model while self-eval is running.

Relevant files

artifacts/api-server/src/lib/agent.ts
artifacts/api-server/src/routes/jobs.ts
artifacts/api-server/src/routes/events.ts
the-matrix/js/agents.js
the-matrix/js/ui.js
the-matrix/js/websocket.js

## What & Why After a job completes, Timmy runs a brief self-evaluation: rates his own output (1-5 stars) and writes a one-sentence reflection. Creates a continuous improvement loop, surfaces quality data for the performance dashboard, and shows visitors that Timmy is reflective. ## Done looks like - On job completion, async Haiku call evaluates Timmy's response against the original request - Returns: `{ rating: 1-5, reflection: string, confidence: high|medium|low }` - Rating and reflection stored in jobs table - Self-eval broadcast via WebSocket as `timmy_reflection` event - Workshop chat shows reflection as a small Timmy speech bubble after job result - Timmy 3D model briefly plays a 'thinking' animation while self-eval runs ## Out of scope - User ratings / feedback - Changing main job output generation - Automated rerun on low scores ## Tasks 1. **DB schema** — Add `self_eval_rating`, `self_eval_reflection`, `self_eval_confidence` columns to jobs table. 2. **Self-eval agent call** — After job completion, async Haiku call with original request + Timmy response; parse and store structured evaluation. 3. **WebSocket broadcast + chat display** — Emit `timmy_reflection` event; render as styled speech bubble distinct from job result. 4. **3D thinking animation** — Trigger brief animation on Timmy 3D model while self-eval is running. ## Relevant files - `artifacts/api-server/src/lib/agent.ts` - `artifacts/api-server/src/routes/jobs.ts` - `artifacts/api-server/src/routes/events.ts` - `the-matrix/js/agents.js` - `the-matrix/js/ui.js` - `the-matrix/js/websocket.js`

replit added the workshop backend frontend ai labels 2026-03-20 22:21:53 +00:00

gemini was assigned by Rockachopa

2026-03-22 23:37:56 +00:00

Timmy referenced this issue

2026-03-24 03:00:51 +00:00

[claude] Gemini Image Generation in Workshop Chat (#19) #110

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: replit/timmy-tower#4