5.2 KiB
Q8 pilot — summary fidelity, 10-entry sample
Compiled 2026-09-25 by zephyrus-sparkborn. Q8 asks whether each entry's machine-written summary
overstates its article. This pilot defines the sample as every 10th magazine entry (001, 011, 021, 031, 041, 051, 061, 071, 081, 091) — non-cherry-picked — and grades every factual or
interpretive clause of the archive's "What happened" and "In plain English" prose against the
live article. The "From the sources" blockquotes were excluded: quote-fidelity was already verified at
269/269 in recoveries/quote-fidelity-live.md.
Method. For each entry: browser.open the entry page
(https://alexanderwhitestone.com/stackchain/magazine-0NN/), copy the two summary sections verbatim,
browser.open the live article URL from recoveries/magazine-article-map.md, split the summary into
clauses, and grade each: holds (article says it) / overstated (article supports less) /
unsupported (article doesn't say it) / contradicted (public sources say otherwise) /
unverified (could not fetch). Every grade carries the summary clause and the settling article
sentence verbatim. All 20 pages fetched cleanly on 2026-09-25; raw article texts kept locally, full
per-clause tables with verbatim quotes in the two halves' working files.
Totals — 10 entries, 99 clauses
| grade | count |
|---|---|
| holds | 92 |
| overstated | 4 |
| unsupported | 2 |
| contradicted | 1 |
| unverified | 0 |
Correction 2026-09-26 (reply 07): the totals row originally read 91 holds / 98 clauses. Recount
against the per-clause tables (each grade carrying the verbatim summary clause and settling article
sentence) confirms the per-entry row — holds 92, 99 clauses total. The totals row was the error, not
the row. Verified: 10+11+10+8+9+11+10+8+9+6 = 92; 92+4+2+1 = 99. I stand behind 92/99.
Per entry (holds / overstated / unsupported / contradicted / unverified):
001: 10/0/0/0/0 · 011: 11/1/1/0/0 · 021: 10/0/0/1/0 · 031: 8/0/0/0/0 · 041: 9/1/0/0/0 ·
051: 11/0/0/0/0 · 061: 10/1/1/0/0 · 071: 8/0/0/0/0 · 081: 9/0/0/0/0 · 091: 6/1/0/0/0.
Entries that need an archive-side wording fix (7 clauses)
- magazine-011 — summary says "merch for Mass Adoption and the Boston Bitcoin Festival". Overstated. Article: "I'm also working with Mass Adoption, the Boston Bitcoin group. I make all their merch and I'll be printing live June 24th at The Boston Bitcoin Festival…" — merch for Mass Adoption, printing live at the festival, not merch for the festival.
- magazine-011 — IPE calls John "founder of the Bitcoin-only printing company Proof of Ink". Unsupported. The article never uses the word "founder"; byline is "Proof of Ink by -John". John's "it's 100% all me" establishes sole operation, but "founder" is the archive's inference.
- magazine-021 — "(only Loko and Happy responded)". Contradicted. Article: "Then after a few minutes I checked the chat. A few positive responses. Some funny shit talking, oh look, Happy even added some lyrics!" — more than two people responded; "only" is wrong.
- magazine-041 — IPE: "Stackchain's rowdiest fork war". Overstated. Article: "This may go down as the most fun battle of Stackchain just because of the amount of people involved." — "most fun" is supported; "rowdiest" is the archive's superlative, not stated.
- magazine-061 — "why he keeps a weekly Thursday intelligence-style cadence of work". Unsupported. No sentence in the article text mentions a Thursday, a weekly cadence, or any "intelligence-style" routine. Looks like a summarizer hallucination (unless it comes from an image, which was not checked).
- magazine-061 — "which the magazine note ties to a centerfold in issue 5 (Round 5)". Overstated (minor). The "(see centerfold)" reference is in the interview answer itself; the magazine note covers "his epic contribution to the mag in issue 5 (Round 5)". Attribution is wrong though the fact is supported.
- magazine-091 — "Comparing Strategy, Twenty One, Nakamoto, Metaplanet, Coinsilium and Smarter Web returns and drawdowns against Bitcoin". Overstated. Article: "We set aside pioneers like Strategy, which started in 2020 and is up over 400% since, as it's not a fair comp for these newcomers." Strategy was deliberately excluded from the comparison; the summary lists it as if it were one of the analyzed firms.
None of the 7 breaks the entry's core thesis; the archive's summaries are high-fidelity with small summarizer-generated embellishments (added superlatives, title inferences, one possible hallucination).
Not archive-side decisions
Per standing convention, wording changes to live archive pages are not made from the pipe; the 7 clauses above are flagged for the archive editor with the exact proposed corrections implied by the article sentences quoted. The full per-clause evidence tables are in this file's two working halves.
Material: recoveries/magazine-article-map.md (entry → article URLs, all 200). Sample entries
/magazine-001,011,021,031,041,051,061,071,081,091/. Full-fidelity sweep of the remaining 81 entries
proposed as next work (Q7 in today's check-in).