diff --git a/README.md b/README.md index 4fdf5dec..c01a70a4 100644 --- a/README.md +++ b/README.md @@ -30,7 +30,8 @@ timmy-config/ │ ├── automation-inventory.md ← Live automation + stale-state inventory │ ├── ipc-hub-and-spoke-doctrine.md ← Coordinator-first, transport-agnostic fleet IPC doctrine │ ├── coordinator-first-protocol.md ← Coordinator doctrine: intake → triage → route → track → verify → report -│ └── fallback-portfolios.md ← Routing and degraded-authority doctrine +│ ├── fallback-portfolios.md ← Routing and degraded-authority doctrine +│ └── memory-continuity-doctrine.md ← File-backed continuity + pre-compaction flush rule └── training/ ← Transitional training recipes, not canonical lived data ``` @@ -53,6 +54,16 @@ current reality and stale-state risks. For fleet routing semantics over sovereign transport, read `docs/ipc-hub-and-spoke-doctrine.md`. +## Continuity + +Curated memory belongs in `memories/` inside this repo. +Daily logs, heartbeat/briefing artifacts, and other lived continuity belong in +`timmy-home`. + +Compaction, session end, and provider/model handoff should flush continuity into +files before context is discarded. See +`docs/memory-continuity-doctrine.md` for the current doctrine. + ## Orchestration: Huey All orchestration (triage, PR review, dispatch) runs via [Huey](https://github.com/coleifer/huey) with SQLite. diff --git a/docs/memory-continuity-doctrine.md b/docs/memory-continuity-doctrine.md new file mode 100644 index 00000000..3c428829 --- /dev/null +++ b/docs/memory-continuity-doctrine.md @@ -0,0 +1,221 @@ +# Memory Continuity Doctrine + +Status: doctrine for issue #158. + +## Why this exists + +Timmy should survive compaction, provider swaps, watchdog restarts, and session ends by writing continuity into durable files before context is dropped. + +A long-context provider is useful, but it is not the source of truth. +If continuity only lives inside one vendor's transcript window, we have built amnesia into the operating model. + +This doctrine defines what lives in curated memory, what lives in daily logs, what must flush before compaction, and which continuity files exist for operators versus agents. + +## Current Timmy reality + +The current split already exists: + +- `timmy-config` owns identity, curated memory, doctrine, playbooks, and harness-side orchestration glue. +- `timmy-home` owns lived artifacts: daily notes, heartbeat logs, briefings, training exports, and other workspace-native history. +- Gitea issues, PRs, and comments remain the visible execution truth for queue state and shipped work. + +Current sidecar automation already writes file-backed operational artifacts such as heartbeat logs and daily briefings. Those are useful continuity inputs, but they do not replace curated memory or operator-visible notes. + +Recommended logical roots for the first implementation pass: +- `timmy-home/daily-notes/YYYY-MM-DD.md` for the append-only daily log +- `timmy-home/continuity/active.md` for unfinished-work handoff +- existing `timmy-home/heartbeat/` and `timmy-home/briefings/` as structured automation outputs + +These are logical repo/workspace paths, not machine-specific absolute paths. + +## Core rule + +Before compaction, session end, agent handoff, or model/provider switch, the active session must flush its state to durable files. + +Compaction is not complete until the flush succeeds. + +If the flush fails, the session is in an unsafe state and should be surfaced as such instead of pretending continuity was preserved. + +## Continuity layers + +| Surface | Owner | Primary audience | Role | +|---------|-------|------------------|------| +| `memories/MEMORY.md` | `timmy-config` | agent-facing | Curated durable world-state: stable infra facts, standing rules, and long-lived truths that should survive across many sessions | +| `memories/USER.md` | `timmy-config` | agent-facing | Curated operator profile, values, and durable preferences | +| Daily notes | `timmy-home` | operator-facing first, agent-readable second | Append-only chronological log of what happened today: decisions, artifacts, blockers, links, and unresolved work | +| Heartbeat logs and daily briefings | `timmy-home` | agent-facing first, operator-inspectable | Structured operational continuity produced by automation; useful for recap and automation health | +| Session handoff note | `timmy-home` | agent-facing | Compact current-state handoff for unfinished work, especially when another agent or provider may resume it | +| Daily summary / morning report | derived from `timmy-home` and Gitea truth | operator-facing | Human-readable digest of the day or overnight state | +| Gitea issues / PRs / comments | Gitea | operator-facing and agent-facing | Execution truth: status changes, review proof, assignment changes, merge state, and externally visible decisions | + +## Daily log vs curated memory + +Daily log and curated memory serve different jobs. + +Daily log: +- append-only +- chronological +- allowed to be messy, local, and session-specific +- captures what happened, what changed, what is blocked, and what should happen next +- is the first landing zone for uncertain or fresh information + +Curated memory: +- sparse +- high-signal +- durable across days and providers +- only contains facts worth keeping available as standing context +- should be updated after a fact is validated, not every time it is mentioned + +Rule of thumb: +- if the fact answers "what happened today?", it belongs in the daily log +- if the fact answers "what should still be true next month unless explicitly changed?", it belongs in curated memory +- if unsure, log it first and promote it later + +`MEMORY.md` is not a diary. +Daily notes are not a replacement for durable memory. + +## Operator-facing vs agent-facing continuity + +Operator-facing continuity must optimize for visibility and trust. +It should answer: +- what happened +- what changed +- what is blocked +- what Timmy needs from Alexander, if anything +- where the proof lives + +Agent-facing continuity must optimize for deterministic restart and handoff. +It should answer: +- what task is active +- what facts changed +- what branch, issue, or PR is in flight +- what blockers or failing checks remain +- what exact next action should happen first + +The same event may appear in both surfaces, but in different forms. +A morning report may tell the story. +A handoff note should give the machine-readable restart point. + +Neither surface replaces the other. +Operator summaries are not the agent memory store. +Agent continuity files are not a substitute for visible operator reporting. + +## Pre-compaction flush contract + +Every compaction or session end must write the following minimum payload before context is discarded: + +1. Daily log append + - current objective + - important facts learned or changed + - decisions made + - blockers or unresolved questions + - exact next step + - pointers to artifacts, issue numbers, or PR numbers + +2. Session handoff update when work is still open + - active task or issue + - current branch or review object + - current blocker or failing check + - next action that should happen first on resume + +3. Curated memory decision + - update `MEMORY.md` and/or `USER.md` if the session produced durable facts, or + - explicitly record `curated memory changes: none` in the flush payload + +4. Operator-visible execution trail when state mutated + - if queue state, review state, or delivery state changed, that change must also exist in Gitea truth or the operator-facing daily summary + +5. Write verification + - the session must confirm the target files were written successfully + - a silent write failure is a failed flush + +## What must be flushed before compaction + +At minimum, compaction may not proceed until these categories are durable: + +- the current objective +- durable facts discovered this session +- open loops and blockers +- promised follow-ups +- artifact pointers needed to resume work +- any queue mutation or review decision not already visible in Gitea + +A WIP commit can preserve code. +It does not preserve reasoning state, decision rationale, or handoff context. +Those must still be written to continuity files. + +## Interaction with current Timmy files + +### `memories/MEMORY.md` + +Use for curated world-state: +- standing infrastructure facts +- durable operating rules +- long-lived Timmy facts that a future session should know without rereading a day's notes + +Do not use it for: +- raw session chronology +- every branch name touched that day +- speculative facts not yet validated + +### `memories/USER.md` + +Use for durable operator facts, preferences, mission context, and standing corrections. +Apply the same promotion rule as `MEMORY.md`: validated, durable, high-signal only. + +### Daily notes + +Daily notes are the chronological ledger. +They should absorb the messy middle: partial discoveries, decisions under consideration, unresolved blockers, and the exact resume point. + +If a future session needs the full story, it should be able to recover it from daily notes plus Gitea, even after provider compaction. + +### Heartbeat logs and daily briefings + +Current automation already writes heartbeat logs and a compressed daily briefing. +Treat those as structured operational continuity inputs. +They can feed summaries and operator reports, but they are not the sole memory system. + +### Daily summaries and morning reports + +Summaries are derived products. +They help Alexander understand the state of the house quickly. +They should point back to daily notes, Gitea, and structured logs when detail is needed. + +A summary is not allowed to be the only place a critical fact exists. + +## Acceptance checks for a future implementation + +A later implementation should fail closed on continuity loss. +Minimum checks: + +- compaction is blocked if the daily log append fails +- compaction is blocked if open work exists and no handoff note was updated +- compaction is blocked if the session never made an explicit curated-memory decision +- summaries are generated from file-backed continuity and Gitea truth, not only from provider transcript memory +- a new session can bootstrap from files alone without requiring one provider to remember the previous session + +## Anti-patterns + +Do not: +- rely on provider auto-summary as the only continuity mechanism +- stuff transient chronology into `MEMORY.md` +- hide queue mutations in local-only notes when Gitea is the visible execution truth +- depend on Alexander manually pasting old context as the normal recovery path +- encode local absolute paths into continuity doctrine or handoff conventions +- treat a daily summary as a replacement for raw logs and curated memory + +Human correction is valid. +Human rehydration as an invisible memory bus is not. + +## Near-term implementation path + +A practical next step is: + +1. write the flush payload into the current daily note before any compaction or explicit session end +2. maintain a small handoff file for unfinished work in `timmy-home` +3. promote durable facts into `MEMORY.md` and `USER.md` by explicit decision, not by transcript osmosis +4. keep operator-facing summaries generated from those files plus Gitea truth +5. eventually wire compaction wrappers or session-end hooks so the flush becomes enforceable instead of aspirational + +That path keeps continuity file-backed, reviewable, and independent of any single model vendor's context window.