36 lines
1.6 KiB
Markdown
36 lines
1.6 KiB
Markdown
# sol: Support-inbox triage classifier (rule-based router)
|
|
|
|
Triage lane deliverable: a deterministic keyword-based router that assigns each
|
|
incoming support inquiry to exactly one of five categories and a verification
|
|
suite that pins the correct behaviour.
|
|
|
|
## What was built (files)
|
|
|
|
- `classifier.py` — rule engine. Each category carries case-insensitive keyword
|
|
lists (single words + multi-word phrases). `classify(text) -> (category, scores)`;
|
|
zero hits falls back to `other`.
|
|
- `gold.json` — 23 hand-labelled inquiries covering all 5 categories (4 account,
|
|
4 billing, 5 technical_support, 6 feature_request, 4 other).
|
|
- `run_audit.py` — evaluates the classifier against the gold set and writes
|
|
`results.json` + `audit_report.md`.
|
|
- `test_triage.py` — self-check: recomputes predictions, asserts accuracy and the
|
|
fallback path; exit 0 on success.
|
|
- `sample_inbox.txt` / `triage_output.txt` — end-to-end sample routing run.
|
|
|
|
## Measured results (real tool output, not claimed)
|
|
|
|
- run_audit.py -> n=23, correct=23, accuracy=1.0
|
|
- Per-category: every category precision=recall=f1=1.0 (TP/FP/FN all zero)
|
|
- test_triage.py -> "ALL TRIAGE CHECKS PASSED: 23/23 correct", exit_code=0
|
|
|
|
## Verified artifacts
|
|
|
|
- /root/.hermes/wizard-outputs/sol/results.json
|
|
- /root/.hermes/wizard-outputs/sol/audit_report.md
|
|
- /root/.hermes/wizard-outputs/sol/triage_output.txt
|
|
|
|
NOTE: accuracy numbers describe the dictionary-exact router on the provided
|
|
gold corpus (23/23). They are measured on that corpus, not a claim of
|
|
unconstrained real-world accuracy. The classifier is deliberately simple; the
|
|
verification suite (not the metric) is the finished deliverable.
|