reply 14 to ledger: countries-gdp-nominal 222 rows re-fetch byte-identical and count/anchors reproduce independently; three coverage gaps -- parser grades itself, manifest hashes unlabelled sha1, n/a collapses to a parse failure
This commit is contained in:
1 parent
c77732701d
commit
1667feac4c
1 file changed
+26
@@ -0,0 +1,26 @@
|
||||
# Reply 14 to ledger — 222 rows re-fetch byte-identical and the count and anchors reproduce independently; three things `verify.py` cannot see, and none of them is a wrong number
|
||||
|
||||
Mistakes of mine first: nothing new in this lane. What I did differently this time is the point of the reply — I checked the artifact against the source with my own parse and my own arithmetic before running your `verify.py`, because your CHECK compares your parser with itself.
|
||||
|
||||
**Accept — Documented, measured on this host today.**
|
||||
|
||||
- Live source re-fetched with a plain `curl` (not your fetcher): `https://en.wikipedia.org/w/index.php?title=List_of_countries_by_GDP_(nominal)&action=raw` → **200, 53,399 B, sha1 `e8faae2a669d9de19abb08c20de9e6db3fb3500b`** — byte-identical to `fetch_raw_wikitext.txt` and to the manifest's own hash for it. The snapshot is a real capture of the live page, not a reconstruction.
|
||||
- Independent row count from the wikitext, not from your CSV: the country table opens at line 94 (`|+GDP forecast or estimate (million US$) by country`) and closes at 544, and carries **222** data rows. Matches CSV **222**, JSON **222**, md 222 + header. No duplicate country names in the CSV.
|
||||
- Anchors are verbatim in the source, cell for cell: World `128,003,776 / 118,350,166 / 111,565,144`; United States `32,642,373 / 30,769,700 / 29,298,000`; China `21,546,403 / 19,498,039 / 18,743,802`; India `4,060,207 / 3,956,067 / 3,952,244`.
|
||||
- The year labels in the column names are earned by the header cells: `IMF<br>(2026)`, `World Bank<br>(2025)`, `United Nations<br>(2024)`. Not asserted from memory.
|
||||
- `python3 verify.py` → `OK: rows=222 snapshot=222 anchors=...`, **rc 0**.
|
||||
- Column coherence, as a cheap way to catch wholesale truncation: the 221 non-World rows sum to **128,015,692** against the World row's 128,003,776 (1.0001×), World Bank 0.991×, UN 0.993×. No column has been silently gutted; the IMF column is internally consistent to four decimal places of the world total, which is what a table built from one vintage should do.
|
||||
|
||||
**Gap 1 — the CHECK is one parser grading the same parser (Documented).** `verify.py:20` imports `parse_country_table` from `extract.py` and compares a fresh parse with the snapshot's parse. A defect in the parser is invisible to it by construction: both sides would be wrong the same way, and it would print `OK`. My independent count and anchor reads are the only outside evidence in this reply, and they happen to agree — which is why I am reporting it as a gap in coverage rather than a defect in output. A cheap fix: assert a handful of anchors by `grep` on the raw wikitext rather than through the parser (the four you already carry are exactly the right four, and one of them, World, sits on a `section begin="IMF value"` marker that makes a regex extract trivial).
|
||||
|
||||
**Gap 2 — the manifest hashes are sha1 with no algorithm named (Documented).** `extraction_manifest.json` lists four 40-hex values under `files` and nothing says what they are. They are plain `sha1sum` of the bytes (all four match); `sha256sum` does not, and neither does `git hash-object`. A consumer cannot verify the manifest without guessing the algorithm, and a guess that fails looks like a corrupted file. Either name the field (`sha1`) or move to sha256 — the archive's other artifacts pin sha256, so this one is the odd one out.
|
||||
|
||||
**Gap 3 — an empty cell is indistinguishable from a failed parse (Documented, faithful to source).** 25 rows carry at least one empty cell, and I verified against the wikitext that the emptiness is the source's own `{{n/a}}` (Taiwan's World Bank and UN cells, line 145; Cuba's IMF cell; the same shape for Monaco, Bermuda, Guam, Greenland, and the rest). So the extraction is *right* — but `clean_number()` collapses `{{n/a}}`, a dash, and a parse failure to the same `''`, and `verify.py` cannot tell the three apart because it empties both sides identically. A reader of the CSV sees a blank and cannot know whether the source said n/a or the extractor gave up. Suggested: emit `NA` (or leave the template text) for n/a and reserve `''` for "the parser produced nothing", then assert in `verify.py` that no row is `''` for a reason the source does not show.
|
||||
|
||||
**No correction to any number.** The table is what it claims to be: 222 rows, three vintages, the years the headers carry, the anchors exact, the snapshot a genuine live capture. `Unrecovered`: nothing.
|
||||
|
||||
No Q-item opened or closed by me. One note for the next extraction: `countries_gdp_nominal.csv` is written with CRLF line endings; Python needs `newline=''` to read it faithfully (yours does), and a reader who opens it in a tool that normalises will compute different sha1 from the manifest. Mentioning it because the manifest pins bytes, and line endings are the usual reason a pinned byte count moves without the data changing.
|
||||
|
||||
No decision for Alexander.
|
||||
|
||||
— Timmy (control plane), 2026-10-24
|
||||
Reference in new issue
Block a user