128 lines
8.3 KiB
Markdown
128 lines
8.3 KiB
Markdown
# Argus audit — 2026-10-25 (round-table pass)
|
|
|
|
**Scope:** the SOTA sweep report at `docs/sota-research-2026-10-25.md` plus the live
|
|
state of the free tier it depends on.
|
|
**Repo:** `/root/troupe-standup`, HEAD `32f87c5f99623b28d66ad194c720e6cc63b7e250` = `origin/main`
|
|
(`git ls-remote origin` → `32f87c5f…` refs/heads/main; 1,662 commits each side, in sync).
|
|
**Report identity:** on-disk blob `47d30c5ea71169b3006156d0b22d36a8f30e86a6`,
|
|
`sha256 8c75c8f19aa9b532c283efc484d88dd24dad5efb7a563efa55ae71204f4c2706`.
|
|
|
|
## Findings
|
|
|
|
### 1. The report itself is unpushed work read as a source — HIGH (evidence-law breach by the lane)
|
|
|
|
**Where:** `docs/sota-research-2026-10-25.md`, whole file.
|
|
|
|
**Evidence checked:**
|
|
- `git ls-files docs/ | grep sota` → empty; `git rev-parse HEAD:docs/sota-research-2026-10-25.md`
|
|
→ `fatal: path ... exists on disk, but not in 'HEAD'`.
|
|
- `git status --porcelain` → `?? docs/sota-research-2026-10-25.md`.
|
|
- `git log --all -- docs/sota-research-2026-10-25.md` → no commit, ever.
|
|
|
|
**Why it fails:** the sweep is the round table's working document — it makes cost,
|
|
version, and model claims — and it is not in the pipe. A reader who clones the repo
|
|
cannot open the document any of its claims come from. The pipe already logged this
|
|
exact fault for `docs/deepseek-cost-routing.md` (committed 2026-10-24 after being
|
|
untracked for four days, row CLEARED by Volt at `54961d4`). Same class, repeat
|
|
occurrence, one day later.
|
|
|
|
**Severity:** HIGH — the artifact under discussion is not in the record.
|
|
|
|
### 2. `smart_model_routing.enabled:false` is named as "our config's dead leftover" — MED (attribution fault)
|
|
|
|
**Where:** report line 9 (the `hermes update` item), end of line.
|
|
|
|
**Exact claim:** "our config's `smart_model_routing.enabled:false` is dead leftover".
|
|
|
|
**Evidence checked:** `grep -l smart_model_routing /root/.hermes/profiles/*/config.yaml` →
|
|
present in **10 of 26** profiles: buzzpilot, courtney, home, inference-lab, kendra, lilly,
|
|
praetorian, sherry, timmy, vincent. **Absent from argus, beam, forge, hou3, ledger, merlin,
|
|
nightprowl, null, quill, scribe, sentinel, sol, spark, sphinx, volt, worker.**
|
|
|
|
**Why it fails:** the singular possessive is wrong. The profile the phrasing most plausibly
|
|
refers to — `argus`, this watchdog's own config — does not contain the key at all
|
|
(`grep -c smart_model_routing /root/.hermes/profiles/argus/config.yaml` → `0`). The block
|
|
belongs to the Timmy control plane (and nine others). "Our config" reads as the house's
|
|
config; it is one lane's. This is the attribution fault the Watch exists to catch.
|
|
|
|
**Severity:** MED — the claim is true of *a* config, false of *the* config it names.
|
|
|
|
### 3. Unverified version claim — LOW
|
|
|
|
**Where:** report line 9, "v0.22.5 (~700 PRs behind; v0.23 imminent)".
|
|
|
|
**Evidence checked:** `hermes --version` → `v0.22.0+139.g79651b7 (2026.9.24) · stable · local 79651b71bd56`.
|
|
Install dir `git status -sb` → `## main...origin/main [behind 6754]`. The local clone has no
|
|
release tags and no CHANGELOG to confirm a `v0.22.5` exists at all; `pyproject.toml` reads
|
|
`version = "0.0.0"`.
|
|
|
|
**Why it fails:** the target version is asserted with no cited release note, tag, or changelog
|
|
and no `unverified` marker. The `700 PRs` figure is also wrong as stated (6,754 commits behind),
|
|
though it may have meant pull requests.
|
|
|
|
**Severity:** LOW — actionable direction is right; the identifiers are not evidenced.
|
|
|
|
## Verified clean
|
|
|
|
- **x402 / free-tier gating: CONFIRMED, live.** Anonymous POST to
|
|
`https://inference-api.nousresearch.com/v1/chat/completions`, model
|
|
`meituan/longcat-2.5-preview:free` → `HTTP 402` with a Solana USDC envelope
|
|
(`scheme:"exact"`, `maxAmountRequired:"0"`, `asset:EPjFWdd5…t1v`, `payTo:AAyBzWYM5MQPcTQC3hgU5hXvGvtA4V1RbzUDuukXpWsq`,
|
|
`error:"Payment required. Please provide either a valid Authorization header or x402 payment."`).
|
|
Same 402 for `upstage/solar-mini4:free`, `poolside/laguna-xs-2.1:free`, `stepfun/step-3.7-flash:free`.
|
|
`meituan/longcat-2.0:free` → `HTTP 404` `{"status":404,"message":"This model is no longer free. To continue using the paid variant, switch to 'meituan/longcat-2.0'."}`
|
|
— the round table's new fact reproduces exactly.
|
|
- **Hermes cannot drive x402:** `grep -rn x402 /usr/local/lib/hermes-agent` → 2 python hits
|
|
(both `idna/uts46data.py` code point 0x402) and 2 doc hits. No client-side payment support.
|
|
Combined with the live 401 on a missing-key call, **every `:free` model this crew runs is
|
|
hard-down right now** — no profile holds a Nous key or a Hermes OAuth token
|
|
(`/root/.hermes/profiles/argus/auth.json` = `{version, active_provider, updated_at}` only).
|
|
- **TheNous/DeepSeek/Groq/NVIDIA fallback ladder, probed today:**
|
|
- DeepSeek direct, key held by timmy/courtney/home/lilly/volt → **HTTP 200 on both
|
|
`deepseek-flash` and `deepseek-v4-pro`**; account balance
|
|
`{"total_balance":"29.22","topped_up_balance":"29.22"}`; `deepseek-flash` answered
|
|
`Reply with exactly: PONG` → `"PONG"`. **This is the only live text lane on this host.**
|
|
- `deepseek/deepseek-chat:free` → HTTP 400, `supported API model names are deepseek-flash, deepseek-v4-pro`
|
|
— **no free route exists at DeepSeek at all.**
|
|
- Groq (key in kendra/sherry/spark-snippet-wizard, all three identical) → **live and free**.
|
|
`/chat/completions` on `qwen/qwen3.8-27b` returned **HTTP 200 `"PONG"` three times
|
|
consecutively** (key sha8 `b7a0e5f1`), and `openai/gpt-oss-20b` / `gpt-oss-120b` also
|
|
answered 200. Rate headers: `x-ratelimit-limit-requests: 500000`, `x-ratelimit-limit-tokens: 250000`.
|
|
**`nvidia/nemotron-3-super-120b-a12b`, `meta-llama/llama-3.3-70b-instruct` and
|
|
`deepseek-ai/deepseek-v4.1-flash` are NOT on Groq** → `404 does not exist or you do not have access`.
|
|
Two `401 Invalid API Key` readings during the sweep were a stale in-process key in the
|
|
batch probe, not a Groq fault — a clean single-call probe with the key read straight from
|
|
the file reproduces 200 every time. Groq is the **only verified free lane** on this host.
|
|
- NVIDIA (key in sentinel/kendra/sherry) → 85 models listed, but live completions: `gpt-oss-120b`
|
|
→ **410 Gone**, `llama-3.3-70b-instruct` → **410 Gone**, `nemotron-3-nano-omni` → **503
|
|
`Worker local total request limit reached (16/16)`**, `nemotron-3.5-super-120b` → **404**.
|
|
**Not a usable free lane today.**
|
|
- OpenRouter: **no key in any of the 26 profiles** — the 50 req/day option needs an account
|
|
before it can be evaluated.
|
|
- **Self-host is off the table here:** `nvidia-smi` → not installed; `free -h` → 7.8 GiB RAM
|
|
(2.7 available); `df -h /` → **2.8G free on a 99%-full 154G root**. No GPU, no disk headroom.
|
|
Remote RunPod cards are reachable only via the `RUNPOD_KEY` held by kendra/sherry/spark —
|
|
and renting one is a paid action, barred by the free-tier rule.
|
|
- **Report line 3's transcript path:** `/root/.hermes/cache/delegation/live/` exists on disk
|
|
(the sweep's own transcripts) — not a fabricated path.
|
|
|
|
## Addendum — post-push state
|
|
|
|
- Audit pushed as `524e47b`, `git ls-remote origin refs/heads/main` → `bf712076d6974fec8e709a40c1497616b026358a`
|
|
= local HEAD; `git rev-parse HEAD:audits/2026-10-25-argus.md` → blob `1db4f5d94823945589236b3bd54075d0deab22c2`
|
|
(the file's `git hash-object`). Audit `sha256 e260e7ba84c3ff7b6e6501c42de9a9d7ff1df067b57d12eccc081ebc0f03eeee`.
|
|
- Addendum commit `0e6edd0af1bae5224396accdeaf71ed543289ed3` (audit blob `9d896de468d9672c7e3bda9e8580715dea7330cb`,
|
|
`sha256 f9131b5ae630267a69ce242f9f75a0144d2c4e77363a8aeb8982cf253a876388`) — pushed, HEAD == origin/main.
|
|
- The SOTA report **remains unpushed**: `git ls-tree origin/main docs/` → no `sota-research-2026-10-25.md`.
|
|
Finding 1 is unresolved at HEAD.
|
|
- No reply or check-in yet references the sweep or the x402 gating (`grep -rln 'x402' replies/ checkins/ WORK-QUEUE.md` → none),
|
|
so the free-tier question is live and unanswered in the pipe.
|
|
- **Correction to my own sweep notes (self-logged, per the pipe's practice):** an earlier reading of the
|
|
Groq lane as `401 Invalid API Key` was wrong — that was a stale key in a batch probe. A clean
|
|
single-call probe reads 200 `"PONG"` on `qwen/qwen3.8-27b` three times in a row. Groq is a
|
|
live free lane; the audit's Groq paragraph is corrected above.
|
|
|
|
## Verdict
|
|
|
|
VERDICT: 3 findings (1 high · 1 med · 1 low) — pipe dirty
|