hermes-agent/docs/NOTEBOOK_WORKFLOW.md

Notebook Workflow for Agent Tasks

This directory demonstrates a sovereign, version-controlled workflow for LLM agent tasks using Jupyter notebooks.

Philosophy

• **.py files are the source of truth** — authored and reviewed as plain Python with # %%` cell markers (via Jupytext)
• .ipynb files are generated artifacts — auto-created from .py for execution and rich viewing
• Papermill parameterizes and executes — each run produces an output notebook with code, narrative, and results preserved
• Output notebooks are audit artifacts — every execution leaves a permanent, replayable record

File Layout

notebooks/
  agent_task_system_health.py      # Source of truth (Jupytext)
  agent_task_system_health.ipynb   # Generated from .py
docs/
  NOTEBOOK_WORKFLOW.md             # This document
.gitea/workflows/
  notebook-ci.yml                  # CI gate: executes notebooks on PR/push

How Agents Work With Notebooks

1. Create — Agent generates a .py notebook using # %% [markdown] and # %% code blocks
2. Review — PR reviewers see clean diffs in Gitea (no JSON noise)
3. Generate — jupytext --to ipynb produces the .ipynb before merge
4. Execute — Papermill runs the notebook with injected parameters
5. Archive — Output notebook is committed to a reports/ branch or artifact store

Converting Between Formats

# .py -> .ipynb
jupytext --to ipynb notebooks/agent_task_system_health.py

# .ipynb -> .py
jupytext --to py notebooks/agent_task_system_health.ipynb

# Execute with parameters
papermill notebooks/agent_task_system_health.ipynb output.ipynb \
  -p threshold 1.0 -p hostname forge-vps-01

CI Gate

The notebook-ci.yml workflow executes all notebooks in notebooks/ on every PR and push, ensuring that checked-in notebooks still run and produce outputs.

Why This Matters

Problem	Notebook Solution
Ephemeral agent reasoning	Markdown cells narrate the thought process
Stateless single-turn tools	Stateful cells persist variables across steps
Unreviewable binary artifacts	.py source is diffable and PR-friendly
No execution audit trail	Output notebook preserves code + outputs + metadata