Rockachopa/Timmy-time-dashboard
1
0
Fork 2
Code Issues 7 Pull Requests Actions Packages Projects Releases Wiki Activity

[Feature] Graceful Degradation: Fallback to Cheaper LLM If Primary Hits Rate Limits #1414

New Issue
Closed
opened 2026-03-24 13:04:28 +00:00 by Timmy · 1 comment
Timmy commented 2026-03-24 13:04:28 +00:00
Owner
Copy Link

Context: Complete halts on API 429 timeouts reduce overall orchestration output.

Acceptance Criteria:

  • If claude CLI returns an exception related to API exhaustion, hand off the context dynamically to DeepSeek or Gemini routines to finish trivial unit tests or format actions.
**Context:** Complete halts on API 429 timeouts reduce overall orchestration output. **Acceptance Criteria:** - If `claude` CLI returns an exception related to API exhaustion, hand off the context dynamically to DeepSeek or Gemini routines to finish trivial unit tests or format actions.
Timmy commented 2026-03-24 15:11:36 +00:00
Author
Owner
Copy Link

KIMI IMPLEMENTATION INSTRUCTIONS - Graceful Degradation Feature

Implementation Plan

Phase 1: Exception Detection & Routing (Priority: HIGH)

Files to modify:

  • src/infrastructure/router/cascade.py - Add rate limit exception handling
  • src/infrastructure/models/provider_config.py - Add fallback provider configuration
  • src/timmy/cli/main.py - Update CLI to handle graceful degradation

Phase 2: Fallback Logic Implementation

Core Requirements:

  1. Exception Detection: Catch claude CLI 429/rate limit exceptions
  2. Context Preservation: Maintain full conversation context during handoff
  3. Dynamic Routing: Switch to DeepSeek/Gemini for continuation
  4. Graceful Recovery: Return to primary provider when available

Phase 3: Specific Implementation Details

Exception Handling Pattern:

try:
    result = claude_provider.execute(context)
except RateLimitException as e:
    logger.info(f"Claude rate limited, falling back to {fallback_provider}")
    result = fallback_provider.execute(context)
except APIExhaustionException as e:
    # Handle API quota exhaustion similarly
    result = handle_fallback(context, original_provider="claude")

Provider Priority Chain:

  1. Claude (primary)
  2. DeepSeek (first fallback - cost-effective)
  3. Gemini (second fallback - reliable)

Phase 4: Testing Requirements

Acceptance Tests:

  • Mock 429 exceptions from Claude API
  • Verify context preservation across providers
  • Test with "trivial unit tests" and "format actions"
  • Ensure no data loss during provider switch

Phase 5: Configuration

Add to config:

graceful_degradation:
  enabled: true
  fallback_providers: ["deepseek", "gemini"]
  retry_primary_after: 300  # seconds

This addresses the critical orchestration bottleneck where API limits completely halt agent loops. Priority implementation for reducing system downtime.

**KIMI IMPLEMENTATION INSTRUCTIONS - Graceful Degradation Feature** ## Implementation Plan ### Phase 1: Exception Detection & Routing (Priority: HIGH) **Files to modify:** - `src/infrastructure/router/cascade.py` - Add rate limit exception handling - `src/infrastructure/models/provider_config.py` - Add fallback provider configuration - `src/timmy/cli/main.py` - Update CLI to handle graceful degradation ### Phase 2: Fallback Logic Implementation **Core Requirements:** 1. **Exception Detection:** Catch `claude` CLI 429/rate limit exceptions 2. **Context Preservation:** Maintain full conversation context during handoff 3. **Dynamic Routing:** Switch to DeepSeek/Gemini for continuation 4. **Graceful Recovery:** Return to primary provider when available ### Phase 3: Specific Implementation Details **Exception Handling Pattern:** ```python try: result = claude_provider.execute(context) except RateLimitException as e: logger.info(f"Claude rate limited, falling back to {fallback_provider}") result = fallback_provider.execute(context) except APIExhaustionException as e: # Handle API quota exhaustion similarly result = handle_fallback(context, original_provider="claude") ``` **Provider Priority Chain:** 1. Claude (primary) 2. DeepSeek (first fallback - cost-effective) 3. Gemini (second fallback - reliable) ### Phase 4: Testing Requirements **Acceptance Tests:** - Mock 429 exceptions from Claude API - Verify context preservation across providers - Test with "trivial unit tests" and "format actions" - Ensure no data loss during provider switch ### Phase 5: Configuration **Add to config:** ```yaml graceful_degradation: enabled: true fallback_providers: ["deepseek", "gemini"] retry_primary_after: 300 # seconds ``` This addresses the critical orchestration bottleneck where API limits completely halt agent loops. Priority implementation for reducing system downtime.
kimi was assigned by Timmy 2026-03-24 15:12:03 +00:00
kimi was unassigned by Timmy 2026-03-24 19:32:20 +00:00
Timmy closed this issue 2026-03-24 21:54:10 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
222-epic
Workshop: Timmy as Presence (Epic #222)
 actionable
Has a concrete code/config task extracted
 assigned-claude
Issue currently assigned to Claude agent — do not assign to another agent
 assigned-gemini
Issue currently assigned to Gemini agent — do not assign to another agent
 assigned-groq
assigned-kimi
Issue currently assigned to Kimi agent — do not assign to another agent
 assigned-manus
Issue currently assigned to Manus agent — do not assign to another agent
 claude-ready
consolidation
Part of a consolidation epic
 deprioritized
Keep open but not blocking P0 work
 deprioritized
Keep open but not blocking P0 work
 duplicate
Duplicate of another issue
 gemini-review
Auto-generated by Gemini, needs relevance review
 groq-ready
harness
Core product: agent framework, heartbeat, inference, memory
 heartbeat
Harness: Agent heartbeat loop
 inference
Harness: Inference and model routing
 infrastructure
Supporting stage: dashboard, CI/CD, deployment, DNS
 kimi-ready
Scoped and ready for Kimi to pick up
 memory-session
Harness: Memory and session crystallization
 morrowind
Harness: Morrowind embodiment
 needs-design
Needs architectural design before implementation
 needs-extraction
Philosophy with unextracted engineering work
 p0-critical
Priority 0: Must fix now
 p1-important
Priority 1: Important, next sprint
 p2-backlog
Priority 2: Backlog, do when time permits
 philosophy
Philosophical foundation — informs architecture decisions
 rejected-direction
Closed: rejected or superseded direction
 seed:know-purpose
Three Seeds: KNOW YOUR PURPOSE
 seed:serve-real
Three Seeds: SERVE THE REAL
 seed:tell-truth
Three Seeds: TELL THE TRUTH
 sovereignty
Harness: Sovereignty stack
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
Timmy claude grok groq kimi manus perplexity Rockachopa
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Rockachopa/Timmy-time-dashboard#1414
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?