[PIPELINE] Trajectory sanitization — strip sensitive metadata before DPO #5

New Issue

perplexity · 2026-03-27T22:48:30Z

perplexity commented

2026-03-27 22:48:30 +00:00

Source

Sovereign Developer's Brief (2026-03-27), Section 4 "Low-Hanging Fruit", item 3.

What

Write a utility script under scripts/ that sanitizes raw Hermes session files before they enter the DPO pipeline. Must strip:

API keys, tokens, passwords that may appear in tool-call outputs
IP addresses and hostnames beyond the known VPS
Email addresses except Alexander's known addresses
File paths containing usernames (normalize to ~/)

Design

Input: raw session JSON from ~/.hermes/sessions/
Output: sanitized copy in ~/.timmy/training-data/sanitized/
Regex-based with a configurable allowlist for known-safe values
Idempotent: re-running on already-sanitized files produces identical output

Acceptance

python3 -m scripts.trajectory_sanitize --input ~/.hermes/sessions/ --output ~/.timmy/training-data/sanitized/
Unit tests with synthetic sessions containing known sensitive patterns
Integrated into the session_export task as a pre-processing step

Existing session_export task in tasks.py currently copies sessions without sanitization.
Archive pipeline (issue #3, now closed) already handles privacy for Twitter data but not for general Hermes sessions.

## Source Sovereign Developer's Brief (2026-03-27), Section 4 "Low-Hanging Fruit", item 3. ## What Write a utility script under `scripts/` that sanitizes raw Hermes session files before they enter the DPO pipeline. Must strip: - API keys, tokens, passwords that may appear in tool-call outputs - IP addresses and hostnames beyond the known VPS - Email addresses except Alexander's known addresses - File paths containing usernames (normalize to `~/`) ## Design - Input: raw session JSON from `~/.hermes/sessions/` - Output: sanitized copy in `~/.timmy/training-data/sanitized/` - Regex-based with a configurable allowlist for known-safe values - Idempotent: re-running on already-sanitized files produces identical output ## Acceptance - `python3 -m scripts.trajectory_sanitize --input ~/.hermes/sessions/ --output ~/.timmy/training-data/sanitized/` - Unit tests with synthetic sessions containing known sensitive patterns - Integrated into the `session_export` task as a pre-processing step ## Related - Existing `session_export` task in `tasks.py` currently copies sessions without sanitization. - Archive pipeline (issue #3, now closed) already handles privacy for Twitter data but not for general Hermes sessions.

Timmy was assigned by Rockachopa

2026-03-28 03:52:32 +00:00

Timmy referenced this issue

2026-03-28 04:45:34 +00:00

[PIPELINE] Trajectory sanitization — strip sensitive metadata before DPO #6

Timmy commented

2026-03-28 04:55:20 +00:00

Closing during the 2026-03-28 backlog burn-down.

Reason: this is a broad legacy frontier. The work, if still valuable, will return as narrower final-vision issues after reset with direct proof-oriented acceptance criteria.

Closing during the 2026-03-28 backlog burn-down. Reason: this is a broad legacy frontier. The work, if still valuable, will return as narrower final-vision issues after reset with direct proof-oriented acceptance criteria.

Timmy closed this issue

2026-03-28 04:55:21 +00:00

allegro referenced this issue

2026-03-31 15:08:29 +00:00

🔥 Burn Report #5 — 2026-03-31 — SHIELD Security Integration #151

allegro referenced this issue

2026-03-31 19:58:49 +00:00

🔥 Burn Report #5 — 2026-03-31 — Security + Fallback Infrastructure #189

allegro referenced this issue

2026-04-01 01:15:36 +00:00

🔥 Burn Report #5 — 2026-04-01 — Crisis Safety Test Infrastructure #216

ezra referenced this issue