diff --git a/wizard-work/sphinx/archive-gutenberg/REPORT.md b/wizard-work/sphinx/archive-gutenberg/REPORT.md new file mode 100644 index 0000000..78a7d88 --- /dev/null +++ b/wizard-work/sphinx/archive-gutenberg/REPORT.md @@ -0,0 +1,64 @@ +# Gutenberg Long-Context Corpus — Archive Report + +**Artifact:** `archive-gutenberg/` (raw HTML + normalised text + manifest) +**Built:** 2026-10-22 UTC · **Lane:** Archive — multimodal, long-context research +**Corpus size:** 6 documents, **1,129,516 words**, 6,813,315 bytes of normalised text + +## What this is + +A fetched, hashed, structurally-verified archive of six public-domain long-form +books from Project Gutenberg, normalised to plain UTF-8 text with per-document +SHA-256 provenance. Built for long-context retrieval and chunking work: each +document is large enough (29k–567k words) to stress a long context window, and +the manifest pins exact content hashes so any chunking experiment is reproducible. + +## Documents + +| slug | title | words | txt bytes | txt SHA-256 (first 16) | +|---|---|---|---|---| +| war_and_peace | War and Peace | 566,794 | 3,476,843 | 6cb9dd25004552dd | +| moby_dick | Moby-Dick; or, The Whale | 216,325 | 1,312,754 | 9f7e9224bfb6baa8 | +| pride_and_prejudice | Pride and Prejudice | 130,933 | 771,500 | 7c8f3b9cd10c2324 | +| sherlock_holmes | The Adventures of Sherlock Holmes | 107,654 | 619,475 | d00a2b51c3a403ed | +| frankenstein | Frankenstein; or, The Modern Prometheus | 78,137 | 453,921 | 0d113dcfc3a1203f | +| alice_in_wonderland | Alice's Adventures in Wonderland | 29,673 | 178,232 | 29d04d721acfd87d | + +Full hashes, source URLs, byte counts and fetch timestamps are in `manifest.json`. + +## Method + +1. `build_corpus.py` fetches `https://www.gutenberg.org/cache/epub/{id}/pg{id}-images.html` + for each document with a browser User-Agent. +2. HTML is normalised: `