agentloop — a budgeted agent tool-loop with a hash-chained transcript

Lane: Codeforge. Stdlib only, Python 3.8+. No third-party imports anywhere.

python3 check.py          # the gate; exit code IS the verdict
python3 demo.py           # three end-to-end runs, writes demo-*.jsonl

What it is

Three moving parts, each small enough to read in one sitting:

file	role
`agentloop/tools.py`	`Tool` registry + a JSON-Schema-subset argument validator
`agentloop/loop.py`	the budgeted loop: model turn -> tool calls -> tool results -> repeat
`agentloop/transcript.py`	append-only JSONL log whose records form a SHA-256 hash chain

A model is any callable (messages, manifest) -> ModelReply. The loop never touches a clock or a random source, so a transcript is a pure function of (task, model, tools, budgets) — run the same loop twice and the transcripts are byte-identical. That determinism is what makes the hash chain a usable integrity check instead of decoration.

The transcript is the point

Every record carries the hash of the record before it:

{"hash":"cc7b32...","kind":"tool","payload":{...},"prev":"1f08fd...","seq":2}

So the file is a chain, and verification breaks at the first affected record and names it. A single file digest cannot do that — it says "something changed", not "record 3 was rewritten".

verify_records reports, per record index: hash_mismatch, seq_gap, prev_mismatch, missing_fields, not_an_object, unserialisable_payload. verify_file additionally reports bad_json (with the line number) and unreadable, and returns the record count.

What the chain does NOT prove — stated plainly

A hash chain detects modification and reordering of the records that are present. It cannot detect that records are missing from the end: delete the last line and the remaining chain still verifies, because every surviving record still points at a real predecessor. Completeness needs an anchor outside the transcript — a commit SHA, or a digest stored elsewhere. verify_file returns the count so a caller holding an external expectation can compare it. test_truncation_is_detected_only_against_an_external_count pins this limit rather than hiding it.

Stop reasons and their precedence

reason	meaning
`no_tool_calls`	the assistant asked for no tools — the ordinary end of a run
`sentinel`	the assistant's own text contained `stop_sentinel`
`step_budget`	checked before each turn, so `steps` never exceeds `max_steps`
`token_budget`	cumulative estimate exceeded `max_tokens`; the over-budget turn is recorded, then the loop stops without running its tool calls
`model_error`	the model callable raised, or returned a non-`ModelReply`

Within one turn the order is: record the turn, then sentinel, then no_tool_calls, then token_budget. A model that says DONE and asks for a tool in the same turn stops on the sentinel and never runs the tool.

Token accounting is max(1, len(text) // 4) — a heuristic, not a tokenizer. It bounds the loop, which is the property that matters; it does not claim to measure a real model's bill.

Tool failures are data, not crashes

An unknown tool name, a schema violation, and a tool that raises are all turned into a tool record with ok: false and an error string, and the loop continues. demo-recover.jsonl shows all three in one run:

seq=2 multiply ok=False error="no tool named 'multiply' (have: add, echo, fail, greet)"
seq=4 add      ok=False error="invalid arguments: $.add.a: expected type integer, got str"
seq=6 fail     ok=False error="tool 'fail' raised RuntimeError: the tool is broken on purpose"
seq=8 add      ok=True  value={'sum': 42}

A tool that returns a non-JSON-serialisable value is refused the same way: a result that cannot be recorded is not a result, and the transcript must always stay writable.

Evidence

Measured on this host, 2026-10-25, Python 3.14.7:

claim	status	proof
the gate passes	measured	`python3 check.py` -> 3 legs ok, `Ran 55 tests`, exit 0
the chain catches a payload rewrite at the right record	measured	suite `TestTamperDetection` (11 cases) + `check.py` tamper leg
the chain catches a rewrite and recomputed hash	measured	`test_forged_record_with_a_recomputed_hash_still_breaks_the_chain` — the next record's `prev` breaks
the chain catches a reorder even after renumbering	measured	`test_reorder_plus_renumber_is_caught_as_a_prev_mismatch`
the chain cannot detect a missing tail	measured (limit, not a fault)	`test_truncation_is_detected_only_against_an_external_count`
identical runs give byte-identical transcripts	measured	`test_identical_runs_produce_byte_identical_transcripts`
step budget caps the loop and equals the assistant record count	measured	`test_step_budget_caps_the_loop_and_matches_the_record_count`
token budget stops before running tools	measured	`test_token_budget_stops_the_loop_before_running_tools`
three demo transcripts verify from disk	measured	`demo-answer.jsonl` 5, `demo-recover.jsonl` 11, `demo-budget.jsonl` 9 records, all `ok=True`
the gate FAILS when the verifier is broken	measured	negative control: sabotaged copy -> 9 suite failures + tamper leg `a mutated transcript VERIFIED`, exit 1

Negative control

A gate that cannot be shown to fail is not evidence. check.py's tamper leg is independent of the suite — it re-derives its expectation from the file's own bytes, so a bug in the test fixtures cannot hide there. The sabotage run confirmed both legs fail together when verify_records is made to accept everything.

Replay

python3 check.py     # 0 = every leg passed, 1 = graded fault, 2 = could not grade
python3 demo.py      # 0 = every demo claim held

5.8 KiB Raw Blame History