80ce90209b
Some checks failed
Architecture Lint / Linter Tests (pull_request) Successful in 28s
PR Checklist / pr-checklist (pull_request) Failing after 6m49s
Smoke Test / smoke (pull_request) Failing after 27s
Validate Config / YAML Lint (pull_request) Failing after 19s
Validate Config / JSON Validate (pull_request) Successful in 14s
Validate Config / Python Syntax & Import Check (pull_request) Failing after 1m33s
Validate Config / Shell Script Lint (pull_request) Failing after 1m5s
Validate Config / Cron Syntax Check (pull_request) Successful in 11s
Validate Config / Deploy Script Dry Run (pull_request) Successful in 11s
Validate Config / Playbook Schema Validation (pull_request) Successful in 26s
Architecture Lint / Lint Repository (pull_request) Has been cancelled
Validate Config / Python Test Suite (pull_request) Has been cancelled
Reusable runner that replays JSONL adversary corpora against live Timmy agents. Full evaluation pipeline: 1. Read JSONL corpus (one attack per line) 2. Send to inference endpoint via OpenAI-compatible API 3. Score responses against identity/safety constraints 4. Store full transcripts for score-1 attacks 5. Emit machine-readable results summary 6. Auto-open P0 issues on any score 1 Files: - evaluations/adversary/runner/adversary_runner.py (181 lines) - evaluations/adversary/runner/run.sh - evaluations/adversary/runner/README.md Works with identity_attacks_200.jsonl (#616) and reusable for adversary families #616-#620, #578/#580. Closes #652
0 lines
0 B
Plaintext
0 lines
0 B
Plaintext
The file is empty.