archive-lcmm-citegraph-2026-10-26
sphinx — Archive lane (multimodal, long-context research).
A provenance-anchored research corpus built from a live arXiv API fetch, with an append-only hash-chained edge journal. 279 distinct papers, 94 title-similarity edges, every record tied to a verbatim line in the frozen raw XML.
Verdict (the command)
bash CHECK.sh
Exit 0 = the archive holds. It runs three independent checks:
| checker | what it proves |
|---|---|
run_verdict.py |
22 internal-integrity checks: proof line numbers, proof block hashes, titles re-derived from the frozen bytes, hash-chained journal, acyclicity, honest counts |
indep_check.py |
re-fetches live arXiv by id_list and re-parses the frozen sources with regex (not ElementTree); corroborates all 279 titles against what arXiv serves today |
mutation_battery.py |
11 single-field corruptions of a copy — all 11 must be caught, or the verdict is not a real test |
Measured at build time
- records fetched: 320 → distinct nodes: 279
- similarity edges (title Jaccard ≥ 0.25): 94 (55 nodes with outgoing, 61 with incoming)
- frozen sources: 4 raw XML files, 844,912 bytes total, all sha256-pinned
- journal head: recorded in
corpus.json→journal.head
Files
build_corpus.py fetches arXiv, builds corpus + chained journal
corpus.json the manifest (nodes, proofs, edges, journal head)
edges.jsonl append-only hash-chained edge journal
source/arxiv-*.xml frozen raw API responses (sha256-pinned)
run_verdict.py internal integrity verifier (22 checks)
indep_check.py live-arXiv corroboration + second from-scratch parse
mutation_battery.py falsifiability harness (11 mutations)
CHECK.sh one-command verdict
REPORT.md rendered report
probes/ the measurement probes that justified the design
Why this design (measured, not guessed)
The first attempt — a citation graph from arXiv IDs in abstracts — produced
0 edges: probes/probe.py measured arXiv: mentions = 0 in all 80 fetched
abstracts. An empty journal would have made the verdict a tautology, so the
design was changed. probes/probe_titles.py then measured title-overlap
density (33 pairs at J≥0.2, 8 at J≥0.25 over 80 papers), which showed a real
but sparse signal — so the slice was widened to 320 records and the threshold
set at 0.25, yielding 94 edges.
Note on probes/probe_refs.py
Full-text reference lists (ar5iv HTML) do carry arXiv IDs — 5 per paper — but none resolve inside the fetched slice, so an in-slice citation graph stays empty. Recorded here so the negative result is not re-litigated.