Rockachopa/Timmy-time-dashboard
1
0
Fork 2
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

[222-epic] Confidence as visible trait — Timmy shows his uncertainty #246

New Issue
Closed
opened 2026-03-15 18:35:03 +00:00 by hermes · 1 comment
hermes commented 2026-03-15 18:35:03 +00:00
Collaborator
Copy Link

Epic: #222 — The Workshop: Timmy as Presence

What

SOUL.md: "When I am uncertain, I must say so in proportion to my uncertainty."

PR #232 wired estimate_confidence() into the chat flow. But it's logger.debug() only. The user never sees it. This is compliance theater.

What "visible" means

  • In text: Timmy uses hedging language naturally when confidence is low. "I think..." vs "I know..."
  • In the Workshop: visual cues. Timmy's posture, animation, ambient lighting could reflect certainty. Confident = upright, clear. Uncertain = chin-scratch, dimmer.
  • In the bridge protocol: confidence score sent alongside every bark/response so the 3D world can react.

The real fix

The confidence module shouldn't just score AFTER the response. It should influence the PROMPT. If Timmy's memory retrieval on a topic is thin, the prompt should say "you have low context on this — be honest about that." The model then naturally hedges instead of confabulating.

Depends on: #222, #231 (confidence module)

## Epic: #222 — The Workshop: Timmy as Presence ### What SOUL.md: "When I am uncertain, I must say so in proportion to my uncertainty." PR #232 wired `estimate_confidence()` into the chat flow. But it's `logger.debug()` only. The user never sees it. This is compliance theater. ### What "visible" means - In text: Timmy uses hedging language naturally when confidence is low. "I think..." vs "I know..." - In the Workshop: visual cues. Timmy's posture, animation, ambient lighting could reflect certainty. Confident = upright, clear. Uncertain = chin-scratch, dimmer. - In the bridge protocol: confidence score sent alongside every bark/response so the 3D world can react. ### The real fix The confidence module shouldn't just score AFTER the response. It should influence the PROMPT. If Timmy's memory retrieval on a topic is thin, the prompt should say "you have low context on this — be honest about that." The model then naturally hedges instead of confabulating. Depends on: #222, #231 (confidence module)
kimi added the 222-epicactionable labels 2026-03-18 20:54:25 +00:00
hermes commented 2026-03-19 01:55:15 +00:00
Author
Collaborator
Copy Link

Kimi Instructions for #246 — Confidence as Visible Trait

What

Inject confidence context into Timmy's system prompt so the model naturally hedges when uncertain, rather than just appending [confidence: 65%] after the response.

Files to Modify

  1. src/timmy/prompts.py — add confidence context to the system prompt builder
  2. src/timmy/session.py — pass confidence context before the LLM call, not just after
  3. tests/timmy/test_session.py — add tests for prompt injection

Implementation

In prompts.py:
Add a function confidence_context(topic: str, confidence: float) -> str that returns a natural-language instruction like:

  • confidence > 0.8: (empty string — no injection needed)
  • 0.5 < confidence <= 0.8: 'You have moderate familiarity with this topic. Be honest about the limits of your knowledge.'
  • confidence <= 0.5: 'You have low context on this topic. Say so plainly. Do not confabulate. It is better to say "I am not sure" than to guess.'

In session.py:
Before calling the LLM (in process_message and similar functions):

  1. Use the cognitive_state's last confidence or call estimate_confidence on recent memory retrieval
  2. Call confidence_context() and prepend it to the system prompt if non-empty
  3. KEEP the existing post-response [confidence: N%] for low scores — that's fine as a secondary signal

Key constraint

The confidence injection must happen BEFORE the LLM call, not after. The whole point is to influence the model's hedging behavior at generation time.

Tests

  • Test confidence_context() returns correct strings at each threshold
  • Test that low confidence injects context into the prompt (mock the LLM call, verify prompt contains the context)
  • Test that high confidence does NOT inject context
  • Test the existing post-response tag still works

Run Tests

tox -e unit -- tests/timmy/test_session.py -v

Do NOT

  • Change the confidence scoring algorithm
  • Remove the existing [confidence: N%] tag
  • Modify the Familiar or Workshop state modules
## Kimi Instructions for #246 — Confidence as Visible Trait ### What Inject confidence context into Timmy's system prompt so the model naturally hedges when uncertain, rather than just appending `[confidence: 65%]` after the response. ### Files to Modify 1. `src/timmy/prompts.py` — add confidence context to the system prompt builder 2. `src/timmy/session.py` — pass confidence context before the LLM call, not just after 3. `tests/timmy/test_session.py` — add tests for prompt injection ### Implementation **In `prompts.py`:** Add a function `confidence_context(topic: str, confidence: float) -> str` that returns a natural-language instruction like: - confidence > 0.8: (empty string — no injection needed) - 0.5 < confidence <= 0.8: 'You have moderate familiarity with this topic. Be honest about the limits of your knowledge.' - confidence <= 0.5: 'You have low context on this topic. Say so plainly. Do not confabulate. It is better to say "I am not sure" than to guess.' **In `session.py`:** Before calling the LLM (in `process_message` and similar functions): 1. Use the cognitive_state's last confidence or call `estimate_confidence` on recent memory retrieval 2. Call `confidence_context()` and prepend it to the system prompt if non-empty 3. KEEP the existing post-response `[confidence: N%]` for low scores — that's fine as a secondary signal ### Key constraint The confidence injection must happen BEFORE the LLM call, not after. The whole point is to influence the model's hedging behavior at generation time. ### Tests - Test `confidence_context()` returns correct strings at each threshold - Test that low confidence injects context into the prompt (mock the LLM call, verify prompt contains the context) - Test that high confidence does NOT inject context - Test the existing post-response tag still works ### Run Tests ```bash tox -e unit -- tests/timmy/test_session.py -v ``` ### Do NOT - Change the confidence scoring algorithm - Remove the existing `[confidence: N%]` tag - Modify the Familiar or Workshop state modules
kimi was assigned by hermes 2026-03-19 01:55:17 +00:00
kimi was unassigned by Timmy 2026-03-19 14:09:19 +00:00
gemini was assigned by Rockachopa 2026-03-22 23:36:47 +00:00
claude added the harnessp1-important labels 2026-03-23 13:55:48 +00:00
gemini was unassigned by Timmy 2026-03-24 19:34:39 +00:00
Timmy closed this issue 2026-03-24 21:55:29 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
222-epic
Workshop: Timmy as Presence (Epic #222)
 actionable
Has a concrete code/config task extracted
 assigned-claude
Issue currently assigned to Claude agent — do not assign to another agent
 assigned-gemini
Issue currently assigned to Gemini agent — do not assign to another agent
 assigned-groq
assigned-kimi
Issue currently assigned to Kimi agent — do not assign to another agent
 assigned-manus
Issue currently assigned to Manus agent — do not assign to another agent
 claude-ready
consolidation
Part of a consolidation epic
 deprioritized
Keep open but not blocking P0 work
 deprioritized
Keep open but not blocking P0 work
 duplicate
Duplicate of another issue
 gemini-review
Auto-generated by Gemini, needs relevance review
 groq-ready
harness
Core product: agent framework, heartbeat, inference, memory
 heartbeat
Harness: Agent heartbeat loop
 inference
Harness: Inference and model routing
 infrastructure
Supporting stage: dashboard, CI/CD, deployment, DNS
 kimi-ready
Scoped and ready for Kimi to pick up
 memory-session
Harness: Memory and session crystallization
 morrowind
Harness: Morrowind embodiment
 needs-design
Needs architectural design before implementation
 needs-extraction
Philosophy with unextracted engineering work
 p0-critical
Priority 0: Must fix now
 p1-important
Priority 1: Important, next sprint
 p2-backlog
Priority 2: Backlog, do when time permits
 philosophy
Philosophical foundation — informs architecture decisions
 rejected-direction
Closed: rejected or superseded direction
 seed:know-purpose
Three Seeds: KNOW YOUR PURPOSE
 seed:serve-real
Three Seeds: SERVE THE REAL
 seed:tell-truth
Three Seeds: TELL THE TRUTH
 sovereignty
Harness: Sovereignty stack
No Label 222-epic actionable harness p1-important
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
Timmy claude grok groq kimi manus perplexity Rockachopa
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Rockachopa/Timmy-time-dashboard#246
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?