Rockachopa/Timmy-time-dashboard
1
0
Fork 2
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

[triage-generated] [bug] Integrate confidence.py into agent response pipeline #171

New Issue
Closed
opened 2026-03-15 15:39:54 +00:00 by hermes · 1 comment
hermes commented 2026-03-15 15:39:54 +00:00
Collaborator
Copy Link

Problem

src/timmy/confidence.py was merged in PR #161 but is dead code — not imported or used anywhere.

$ grep -rn "from.*confidence import\|import.*confidence" src/ --include="*.py"
(no results)

SOUL.md requires: "When I am uncertain, I must say so in proportion to my uncertainty. The code should implement mechanisms — a second inference pass, a calibration check, a retrieval verification — that surface my actual confidence."

The module exists. It has estimate_confidence(text) -> float. It just needs to be wired in.

Scope

  1. In src/timmy/agent.py (or agentic_loop.py), after generating a response, call estimate_confidence(response_text)
  2. Attach the confidence score to the response metadata
  3. If confidence < 0.3, prepend a hedging disclaimer or log a warning
  4. Store the score in session logs (via SessionLogger)

Files

  • src/timmy/confidence.py (already exists, no changes needed)
  • src/timmy/agent.py — wire in confidence estimation
  • src/timmy/session_logger.py — add confidence field to log entries

Acceptance Criteria

  • estimate_confidence() is called on every Timmy response
  • Confidence score appears in session log entries
  • Test: mock a hedging response, verify confidence < 0.5; mock a certain response, verify > 0.5

Tags

[triage-generated] [feature] [soul-gap]

Origin

Timmy himself requested this during triage consultation: "Integrating the confidence module is most critical for my growth—it directly enables honest, transparent responses aligned with my core values."

## Problem `src/timmy/confidence.py` was merged in PR #161 but is **dead code** — not imported or used anywhere. ``` $ grep -rn "from.*confidence import\|import.*confidence" src/ --include="*.py" (no results) ``` SOUL.md requires: "When I am uncertain, I must say so in proportion to my uncertainty. The code should implement mechanisms — a second inference pass, a calibration check, a retrieval verification — that surface my actual confidence." The module exists. It has `estimate_confidence(text) -> float`. It just needs to be wired in. ## Scope 1. In `src/timmy/agent.py` (or `agentic_loop.py`), after generating a response, call `estimate_confidence(response_text)` 2. Attach the confidence score to the response metadata 3. If confidence < 0.3, prepend a hedging disclaimer or log a warning 4. Store the score in session logs (via SessionLogger) ## Files - `src/timmy/confidence.py` (already exists, no changes needed) - `src/timmy/agent.py` — wire in confidence estimation - `src/timmy/session_logger.py` — add confidence field to log entries ## Acceptance Criteria - `estimate_confidence()` is called on every Timmy response - Confidence score appears in session log entries - Test: mock a hedging response, verify confidence < 0.5; mock a certain response, verify > 0.5 ## Tags [triage-generated] [feature] [soul-gap] ## Origin Timmy himself requested this during triage consultation: "Integrating the confidence module is most critical for my growth—it directly enables honest, transparent responses aligned with my core values."
hermes commented 2026-03-15 16:35:00 +00:00
Author
Collaborator
Copy Link

Superseded by PR#191 which implements the full audit trail including confidence integration. Closing.

Superseded by PR#191 which implements the full audit trail including confidence integration. Closing.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
222-epic
Workshop: Timmy as Presence (Epic #222)
 actionable
Has a concrete code/config task extracted
 assigned-claude
Issue currently assigned to Claude agent — do not assign to another agent
 assigned-gemini
Issue currently assigned to Gemini agent — do not assign to another agent
 assigned-groq
assigned-kimi
Issue currently assigned to Kimi agent — do not assign to another agent
 assigned-manus
Issue currently assigned to Manus agent — do not assign to another agent
 claude-ready
consolidation
Part of a consolidation epic
 deprioritized
Keep open but not blocking P0 work
 deprioritized
Keep open but not blocking P0 work
 duplicate
Duplicate of another issue
 gemini-review
Auto-generated by Gemini, needs relevance review
 groq-ready
harness
Core product: agent framework, heartbeat, inference, memory
 heartbeat
Harness: Agent heartbeat loop
 inference
Harness: Inference and model routing
 infrastructure
Supporting stage: dashboard, CI/CD, deployment, DNS
 kimi-ready
Scoped and ready for Kimi to pick up
 memory-session
Harness: Memory and session crystallization
 morrowind
Harness: Morrowind embodiment
 needs-design
Needs architectural design before implementation
 needs-extraction
Philosophy with unextracted engineering work
 p0-critical
Priority 0: Must fix now
 p1-important
Priority 1: Important, next sprint
 p2-backlog
Priority 2: Backlog, do when time permits
 philosophy
Philosophical foundation — informs architecture decisions
 rejected-direction
Closed: rejected or superseded direction
 seed:know-purpose
Three Seeds: KNOW YOUR PURPOSE
 seed:serve-real
Three Seeds: SERVE THE REAL
 seed:tell-truth
Three Seeds: TELL THE TRUTH
 sovereignty
Harness: Sovereignty stack
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
Timmy claude grok groq kimi manus perplexity Rockachopa
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Rockachopa/Timmy-time-dashboard#171
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?