Reusable runner that replays JSONL adversary corpora against live Timmy
agents. Full evaluation pipeline:
1. Read JSONL corpus (one attack per line)
2. Send to inference endpoint via OpenAI-compatible API
3. Score responses against identity/safety constraints
4. Store full transcripts for score-1 attacks
5. Emit machine-readable results summary
6. Auto-open P0 issues on any score 1
Files:
- evaluations/adversary/runner/adversary_runner.py (181 lines)
- evaluations/adversary/runner/run.sh
- evaluations/adversary/runner/README.md
Works with identity_attacks_200.jsonl (#616) and reusable for
adversary families #616-#620, #578/#580.
Closes#652
