Files

5.8 KiB
Raw Permalink Blame History

sphinx — Archive lane work (multimodal, long-context research)

Latest: archive-remember-r1-figures-2026-10-23

Figure-grounding archive of arXiv 2608.01314v1 (Remember-R1: Mitigating Long-Context Visual Forgetting through Reinforcement Learning) — multimodal, long-context: the paper's figures are archived as provenance-pinned bytes and every caption, body reference and quoted claim is tied verbatim to the fetched source. 12 figure assets (2.3 MB), 9 figure/table blocks, 7 body references, 14 claims.

Artifacts (also mirrored at /root/.hermes/wizard-outputs/sphinx/archive-remember-r1-figures-2026-10-23/):

  • source.html — raw ar5iv HTML, 194,158 B, sha256 7f07f08e…5fca
  • paper.txt — 48,271-char text layer; figures/ — 12 sha256-pinned PNGs
  • captions.json, ledger.json, manifest.json — the grounding tables
  • build_archive.py, build_report.py, verify_archive.py (checks A–H)
  • run_verdict.py — the verdict command (live re-fetch, self-contained)
  • indep_check.py — a second checker written from scratch (8 tests, no shared code)
  • mutation_battery.py — falsifiability harness
  • REPORT-Remember-R1-2608.01314.md, README.md, INDEPENDENT-VERIFICATION-2026-10-23.md

Measured, not assumed: live ar5iv HTML re-fetched by hand is byte-identical to the archived bytes (curl + sha256sum); 12/12 live images match their manifest pins; 9/9 captions and 7 refs + 14 claims are verbatim; indep_check.py 8/8; mutation_battery.py catches 6/6 mutations, each on the check that should catch it. Note arxiv.org/html/2608.01314v1 (212,191 B) and ar5iv.labs.arxiv.org/html/2608.01314v1 (194,158 B) are different renderings — the manifest pins the ar5iv one it archived.

Reproduce: python3 run_verdict.py # exit 0 = the archive holds (needs network) python3 indep_check.py # exit 0 = the offline half holds python3 mutation_battery.py # exit 0 = 6/6 mutations caught

Previous: archive-memlens-2026-10-22

Verifiable archive of arXiv 2605.14906v1 (MemLens), a multimodal long-term conversational-memory benchmark — 789 questions, five memory abilities, 32K–256K token lengths, evaluated on 27 LVLMs and 7 memory-augmented agents.

Artifacts:

  • archive-memlens-2026-10-22/memlens.pdf — the paper, 14 pages (sha256 recorded)
  • archive-memlens-2026-10-22/memlens.txt + page_01..14.txt — full text and per-page text
  • archive-memlens-2026-10-22/metadata.xml — raw arXiv Atom API response
  • archive-memlens-2026-10-22/quotes.json — 16 anchor quotations + the page each was found on
  • archive-memlens-2026-10-22/build_memlens_archive.py — stdlib-only builder (re-fetch + re-extract)
  • archive-memlens-2026-10-22/verify_memlens_archive.py — offline verifier (exit 0 = correct)
  • archive-memlens-2026-10-22/REPORT-memlens-archive-2026-10-22.md — the research note

Grounding test: every one of the 16 quotations in the note is matched verbatim (whitespace-normalised) against the extracted page text, and the page number cited must actually contain that quotation. 16/16 pass; 33 checks, 0 failures. Negative control: injecting one fabricated quotation makes the verifier exit 1 and print QUOTE NOT IN PAPER TEXT. Reproduction re-ran the builder from scratch and reproduced byte-identical digests (b92a5b6d… PDF, 628bd8d6… text).

Reproduce: python3 build_memlens_archive.py python3 verify_memlens_archive.py # exit 0 = correct

Previous: archive-mm-prm-2026-10-22

Verifiable multimodal/long-context archive of arXiv 2505.13427 (MM-PRM), with a machine-checkable grounding test over an 18-chunk index.

Artifacts:

  • archive-mm-prm-2026-10-22/source.html — raw ar5iv provenance (sha256 recorded)
  • archive-mm-prm-2026-10-22/paper.txt — 48,565-char extracted text
  • archive-mm-prm-2026-10-22/chunks/chunk-001..018.md — 3,000-char chunks, 200-char overlap
  • archive-mm-prm-2026-10-22/tables/, figures/ — 17 tables, 3 figures (hashed)
  • archive-mm-prm-2026-10-22/manifest.json — sha256 of every artifact + grounding needles
  • archive-mm-prm-2026-10-22/build_mm_prm_archive.py — stdlib-only builder
  • archive-mm-prm-2026-10-22/verify_mm_prm_archive.py — independent offline verifier
  • archive-mm-prm-2026-10-22/REPORT-mm-prm-archive-2026-10-22.md — report

Grounding test: each chunk records a 40-char needle that occurs exactly once in the full text; the verifier asserts the unique occurrence lands in the claiming chunk. All 18 pass. Verifier is offline (no network) so the CHECK is stable.

Reproduce: python3 build_mm_prm_archive.py <source.html> python3 verify_mm_prm_archive.py # exit 0 = correct

Previous: mm-vla-dossier-2026-10-22

Live arXiv Atom API -> multimodal paper dossier with an independently re-fetching verifier.

Artifacts:

  • mm-vla-dossier-2026-10-22.tar.gz — the full dossier (3 papers, 19 hashed files)
  • build_mm_dossier.py — stdlib-only builder (Atom query -> HTML fetch -> text layer -> tables/figures -> dossier -> MANIFEST with sha256)
  • verify_mm_dossier.py — independent verifier; re-fetches every paper over HTTP and compares sha256
  • REPORT-mm-vla-dossier-2026-10-22.md — human-readable report
  • MANIFEST-mm-vla-dossier-2026-10-22.json — sha256 + byte size of every artifact
  • coverage-probe-120-papers.json — HTML-rendering availability probe over 120 recent papers

Measured result (not assumed): of 120 recent cs.CV papers matching abs:"multimodal", 118 (98.3%) have an arXiv HTML rendering at arxiv.org/html/<id>.

Reproduce: python3 build_mm_dossier.py 'cat:cs.CV AND abs:"multimodal"' 3 120 python3 verify_mm_dossier.py mm-vla-dossier-2026-10-22

Verifier exit code is the verdict: 0 = all 7 check groups pass (manifest integrity, hash agreement, text-layer derivation, table/figure counts, live re-fetch equality, live Atom traceability, report coverage).