docs: cycle 2 retrospective — Kimi codebase scanning bottleneck identified
Root cause: Kimi reads entire codebase before editing, consuming timeout. Fix: pre-extract context into prompt, or use Kimi for reasoning tasks only.
This commit is contained in:
54
docs/design-log/2026-03-15-cycle2-retro.md
Normal file
54
docs/design-log/2026-03-15-cycle2-retro.md
Normal file
@@ -0,0 +1,54 @@
|
||||
# Delegation Cycle 2 — Retrospective
|
||||
|
||||
## Date: 2026-03-15
|
||||
## Task: #148 split into chunks per Cycle 1 lessons
|
||||
|
||||
---
|
||||
|
||||
## What Happened
|
||||
|
||||
Split #148 into 3 chunks (A: timmy/, B: dashboard/, C: infra+spark/).
|
||||
Dispatched Chunk C (4 files, smallest) to Kimi with 180s timeout.
|
||||
|
||||
**Result: Kimi timed out again without making any changes.**
|
||||
|
||||
Kimi's --print mode spends its time budget on:
|
||||
1. Reading all files in the working directory (codebase scanning)
|
||||
2. Planning the approach (chain of thought)
|
||||
3. Searching the web for best practices
|
||||
4. Actually making edits
|
||||
|
||||
For this codebase (~8800 SLOC), steps 1-3 consume most of the timeout.
|
||||
|
||||
## Root Cause Analysis
|
||||
|
||||
**Kimi is not timing out because the task is too big. It's timing out because
|
||||
it reads the entire codebase before starting work.**
|
||||
|
||||
The 262K context is a trap — Kimi ingests everything it can find, and for a
|
||||
project with 200+ files, this planning phase alone can exceed 180s.
|
||||
|
||||
## Proposed Fix for Cycle 3
|
||||
|
||||
1. **Pre-extract context.** Instead of pointing Kimi at the full repo, give
|
||||
it ONLY the files it needs to edit, as inline code in the prompt.
|
||||
2. **Use --add-dir sparingly.** Don't let Kimi see the full src/ tree.
|
||||
3. **Consider: do mechanical refactors myself.** sqlite3 context manager
|
||||
conversion is a mechanical find-and-replace pattern. The value of Kimi
|
||||
is in tasks requiring reasoning, not search-and-replace.
|
||||
|
||||
## Deliverables
|
||||
|
||||
- No new PRs this cycle
|
||||
- Lesson: Kimi's planning overhead on large codebases is the real bottleneck
|
||||
- #148 remains open — candidate for scripted refactoring or inline-context Kimi task
|
||||
|
||||
## Cumulative Stats (Cycles 1+2)
|
||||
|
||||
| Metric | Cycle 1 | Cycle 2 | Total |
|
||||
|--------|---------|---------|-------|
|
||||
| PRs created | 1 | 0 | 1 |
|
||||
| Kimi invocations | 4 | 1 | 5 |
|
||||
| Kimi timeouts | 3 | 1 | 4 |
|
||||
| Kimi successes | 1 | 0 | 1 |
|
||||
| Success rate | 25% | 0% | 20% |
|
||||
Reference in New Issue
Block a user