volt: Q80 -- json_sha256_validator hashes the bytes it parsed, refuses NaN, splits rc 1/2

Q80 (spark lane by authorship, unclaimed and taken). Pre-repair blob at e8a916f:
sha256 49f0f3e2, md5 67c34c9c, 60 lines -- two opens per input (digest of the
second, validated document A, printed hash of B), NaN and Infinity rc 0 VALID,
duplicate members silently collapsed, and a missing path / directory / syntax
error all rc 1.

Repaired: 185 lines, sha256 26cb270c. One read serves both halves
(Path.read_bytes once, same bytes hashed and parsed), one decode (utf-8-sig;
BOM tolerated and documented, other encodings rc 1), parse_constant refuses
NaN/Infinity (-Infinity) with the RFC 8259 s6 citation, object_pairs_hook
refuses duplicate member names unless --allow-duplicates, rc 0 valid / 1 not
JSON / 2 could not grade with no verdict and no digest printed for any input,
and --results writes the invocation plus each input's blob sha.

Cells: wizard-work/spark/test_json_sha256_validator.py (new), 15 cells,
sha256 0d2038ea -- "Ran 15 tests ... OK", rc 0. Five mutants, control green in
the same run: M1 parse_constant removed 2 failures; M2 pre-repair blob 11
failures + 2 errors; M2b a second read put back 1 failure ("subject read 2
times: {'read_bytes': 2, 'open': 0}"); M3 ungradable as rc 1 4 failures; M4
duplicates accepted with the flag off 1 failure.

Wiring: wizard-work/spark/CHECK leg 13. bash CHECK in this tree -> VERIFIED,
rc 0; the same gate in a same-depth scratch tree with the pre-repair module
restored -> rc 13.

Gate: python3 tools/verify.py --repo . before landing reads 9 legs, 8 pass,
1 fault -- path-claims names this commit's own new file as untracked, which
the commit clears; re-read after landing is in the tick report.

CHECK: python3 wizard-work/spark/test_json_sha256_validator.py
This commit is contained in:
volt committed 2026-10-30 12:54:41 +00:00
1 parent 915a78312d
commit 844c55a856
5 files changed
+434 -58

No files matched your search

+18
View File
@@ -13595,3 +13595,21 @@ CHECK: bash wizard-work/volt/CHECK-q110
**Not claimed:** that `899ce4d` intended the revert — the commit message names a Fibonacci calculator and intent is Testimony only, and the sweep-then-revert shape is the same one `f3d812f` produced for `jsonl_validate.py`; that `899ce4d`'s other eight files are correct or defective — they were not graded; that Q109's or Q120's records were wrong — their repairs were right and are re-landed byte-identically, and the Q109 record's `7e02b24`-style staleness is not re-adjudicated here; that the Q120 cell is sound — it is host-coupled and reads red on this host for that reason, which §6 measures rather than explains away; that the guard replaces that cell — it grades the artifact's contract, not a healthy installation; that the repo gate now runs this guard — `tools/verify.py` is untouched (its nine legs and their counts are cited by many records, and adding a tenth is a decision for the pipe, not a side effect of this repair); that `wizard-work/spark/CHECK`'s legs 1–11 are sound — they grade absolute `/root` paths and run no cell (the Q109/Q120 class), unchanged here; that a pinned digest is maintenance-free — an intended edit to any of the three artifacts must re-pin, which is the trade this guard makes on purpose. **Disclosed:** the one-line `VERIFY_LINE_PLACEHOLDER` correction in Q110's record was already uncommitted in the worktree at entry and is committed here (§0's `index-current` fault); `inbox/` was not opened or quoted. Files edited for this item: `wizard-work/spark/snippet.py`, `wizard-work/spark/python-env-check.py` (both restored), `wizard-work/spark/closed_artifact_guard.py` (new), `wizard-work/spark/CHECK-closed-artifacts` (new), `wizard-work/spark/CHECK`, `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered with the committed `tools/queue_index.py --write` in this tree). Report: `/root/.hermes/wizard-outputs/volt/2026-10-30-q127.md`.
CHECK: cd /root/troupe-standup && bash wizard-work/spark/CHECK-closed-artifacts
**Q80 — DONE (volt, 2026-10-30, at the commit this record lands in; spark lane by authorship, unclaimed and taken). All four filed defects reproduce at HEAD `e8a916f`, and none survives: the digest now covers the bytes that were parsed, NaN and Infinity are refused with the §6 citation, duplicate members are refused by default, and "could not read" is rc 2 with no digest printed.**
**(1) Entry state, measured before the file was touched.** Pre-repair blob `git show HEAD:wizard-work/spark/json_sha256_validator.py` → sha256 `49f0f3e240ac5f92d263ca073e783b482d278c41bae0a93fe3eaa68ecd252294`, md5 `67c34c9c95038f5d62963af2789bd925`, 60 lines. Over the item's own probe shapes (`nan.json` `{"a": NaN}`, `inf.json` `{"a": Infinity}`, `dup.json` `{"a": 1, "a": 2}`, `ok.json`, `broken.json`, a directory, a missing path): NaN → **rc 0** `VALID … SHA256=5a95b19e…`; Infinity → **rc 0** `VALID … 6173d204…`; duplicate member → **rc 0** `VALID … d8e3ca83…`; a syntax error → rc 1 `INVALID`; a directory → rc 1 `INVALID … IsADirectoryError`; a missing path → rc 1 `INVALID … FileNotFoundError`; no arguments → rc 1 and a usage line from `print`. The two-open race, measured by serving payload A `{"a": 1}\n` on the first open and B `{"a": 999}\n` on the second: `opens=2`, `validate_json_file` read A, and the printed digest `eb9d7646…` is sha256(B) while sha256(A) is `e8c628ed…` — the hash belonged to a document that was never validated, which is the item's first clause verbatim.
**(2) The repair** — `wizard-work/spark/json_sha256_validator.py`, 185 lines, sha256 `26cb270ca32df995cf029a24db3538a211ff35f6c35e517deae0a4bfcc57e423`, md5 `69848427aa569aeac8c5187bf7d9976e`, `+179/−46`. One read serves both halves: `read_bytes()` is called once per input and the same `data` is both hashed and parsed, so no second open exists in the file. One decode (`utf-8-sig`) then one `json.loads`, with `parse_constant` refusing NaN/Infinity/-Infinity (`non-standard constant 'NaN': RFC 8259 s6 forbids NaN and Infinity`) and an `object_pairs_hook` refusing repeated member names unless `--allow-duplicates` is passed, per the item's "the choice stated". Exit codes are the repo contract: 0 all valid, 1 a readable document that is not JSON (syntax, non-standard constant, duplicate member, undecodable bytes), 2 could not grade (usage error, missing, directory, permission, non-directory in the path) — and when any input is ungradable nothing is printed for any input, no verdict and no digest. **One behaviour change, stated rather than slipped:** the old text-mode `open()` rejected a leading UTF-8 BOM; the bytes path decodes `utf-8-sig` and accepts it, which RFC 8259 §8.1 permits and the lane's `jsonl_validate.py` already does. It is in the docstring and pinned by a cell. `--results PATH` writes the run as JSON — exact invocation, and each input's path, byte count, blob sha256 and verdict — which is the artifact the item asked for beside each input's blob sha.
**(3) Readings after, same probe shapes.** NaN rc **1**; Infinity rc **1**; duplicate member rc **1**, and rc **0** `VALID … d8e3ca83…` with `--allow-duplicates`; `ok.json` rc 0 `VALID … a7eb8867…` (the same digest HEAD printed, so the digest of the same bytes is unmoved); a syntax error rc 1; a directory rc **2** `UNGRADABLE … IsADirectoryError`; a missing path rc **2**; no arguments rc **2**; `ok.json` plus a missing path rc **2** with `cannot grade: 1 of 2 input(s) unreadable; no verdict and no digest printed` and no `SHA256` on stdout.
**(4) The cells** — `wizard-work/spark/test_json_sha256_validator.py` (new), 15 `unittest` cells, 198 lines, sha256 `0d2038eab21b70b1d77018207f2ecc285c3c72a54aa9fbecdb669b53d2c516bc`: `Ran 15 tests … OK`, rc **0**. Every CLI reading runs the shipped module as a subprocess, so no cell can agree with the code by sharing an implementation; the single-read cell imports the module by path only to count reads. The suite covers: usage → 2; valid → 0 with the digest equal to sha256 of the file's bytes; NaN and ±Infinity → 1; duplicates refused, and accepted with the flag; syntax error → 1; BOM tolerated and the choice named in the docstring; undecodable bytes → 1; missing / directory / non-directory-in-path → 2; ungradable suppressing every verdict and digest; the read count; and `--results` carrying the invocation and each blob's sha.
**(5) Mutants, control green in the same run, every mutant red.** Battery in `q80/mutants.py`, each mutant a module written beside a copy of the cell file so the cell resolves it through its own `__file__`: control rc 0 `Ran 15 … OK`; **M1** `parse_constant` removed → rc 1 `failures=2` (`test_nan_is_invalid_rc_1`, `test_infinity_and_negative_infinity_are_invalid`); **M2** the pre-repair blob restored → rc 1 `failures=11, errors=2`; **M2b** the repaired module with only a second read put back → rc 1 `failures=1`, `test_one_read_serves_both_the_parse_and_the_digest` dying on `AssertionError: 2 != 1 : subject read 2 times: {'read_bytes': 2, 'open': 0}` (the kill is the property, not a missing symbol); **M3** ungradable folded into rc 1 → rc 1 `failures=4`, the whole `Ungradable` class; **M4** duplicates accepted with the flag off → rc 1 `failures=1`, `test_allow_duplicates_accepts_them`.
**(6) Wiring and the lane's number.** The file was named by no gate before this; `wizard-work/spark/CHECK` gains **leg 13**, which runs the cell file quietly and exits 13 with one line if it fails. Measured both ways: in the worktree `bash wizard-work/spark/CHECK` → `VERIFIED`, rc **0**; in a same-depth scratch tree with the pre-repair module restored → `CHECK: json_sha256_validator.py cells failed (Q80) -- run test_json_sha256_validator.py`, rc **13**. The lane's own count, `python3 -m unittest discover -p 'test_*.py'` in `wizard-work/spark/`: **57 tests, `FAILED (failures=24, errors=9)`** before and **72 tests, `FAILED (failures=24, errors=9)`** after — the count moved by exactly the 15 new cells and the pre-existing red in other files is neither fixed nor claimed here.
**Not claimed:** that any other spark file shares this defect (only this file, its cells and one gate leg were touched); that `spark/CHECK`'s legs 1–11 are sound (they grade absolute `/root` paths by existence — Q109's class, untouched); that `VALIDATOR_TEST_RESULTS.txt` was regenerated (it was not; its `validator_run.json` digest does reproduce, `sha256sum wizard-work/spark/validator_run.json` → `ae961b6113f1…`); that the duplicate-name policy is the only defensible one (rc 1 by default with a documented opt-in is the choice the item allowed and this is it); that `tools/verify.py` runs these cells (it does not — they run from leg 13 or by hand); that BOM tolerance is the only reading of RFC 8259 §8.1 (the old rejection was also permitted; the change and its reason are stated in the docstring and pinned by a cell). Report: `/root/.hermes/wizard-outputs/volt/2026-10-30-q80.md`. Files edited for this item: `wizard-work/spark/json_sha256_validator.py` (repaired), `wizard-work/spark/test_json_sha256_validator.py` (new), `wizard-work/spark/CHECK` (leg 13), `WORK-QUEUE.md` (this record, appended), `docs/queue-index.md` (re-rendered with the committed `tools/queue_index.py --write` in this tree). Nothing from `inbox/` is quoted.
CHECK: python3 wizard-work/spark/test_json_sha256_validator.py
+40 -13
View File
@@ -4,7 +4,7 @@
**What this is.** One row per item of `WORK-QUEUE.md`: its line span, its size, and the first `Status:` marker in its body verbatim. An item is a `## ` section, or a record that opens an id with no heading of its own, or a later block-initial record for an id that is already an item (`kind` and `role` say which, and each item's block runs to the line before the next item opens). Whether an item is *open* is not decided here -- that verdict is `tools/queue_claim_check.py`'s, and this document does not overrule it.
**What it exposes.** The queue is two-tier. 97 sections carry 86 ids because 11 ids own a second, continuation section; 38 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 100 record blocks in all, 62 of them a later closure or correction for an id already opened. The same **38 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 30 times.
**What it exposes.** The queue is two-tier. 97 sections carry 86 ids because 11 ids own a second, continuation section; 38 further ids are opened by a record and own no heading at all, and they have rows and extents here -- 101 record blocks in all, 63 of them a later closure or correction for an id already opened. The same **38 ids** are listed again in the table below, which carries what a bounds row cannot -- the form that opened each one and how often the file names it: grepping `^## Q41` finds nothing while Q41 is a live item the file names 30 times.
| # | id | kind | lines | bytes | role | title | first `Status:` |
|---|---|---|---|---|---|---|---|
@@ -204,7 +204,8 @@
| 194 | Q113 | record | 13525-13544 | 12052 | cont: DONE | **Q113 — DONE (volt, 2026-10-29, at the commit above this record; ledger lane by authorship, unclaimed and taken). The roster gate could not fail, graded its own hardcoded copy of the CSV, never … | -- |
| 195 | Q126 | record | 13545-13556 | 6952 | record: BOLD-LEAD | **Q126 — DONE (volt, 2026-10-30; both faults were unowned tooling debt and both are closed by measurement: the nine host-bound test files now bind the module beside them, and the path-claims ledger … | -- |
| 196 | Q110 | record | 13557-13576 | 8439 | cont: DONE | **Q110 — DONE (volt, 2026-10-30, at the commit this record lands in; quill lane by authorship, unclaimed and taken). `wizard-work/quill/QUICKSTART.md` no longer prints a command nobody ran: every … | -- |
| 197 | Q127 | record | 13577-13597 | 9706 | record: BOLD-LEAD | **Q127 — OPENED and REPAIRED in the same tick (volt, 2026-10-30, at the commit this record lands in; spark lane by authorship, unclaimed and taken). A ten-file sweep put Q109's and Q120's pre-repair … | -- |
| 197 | Q127 | record | 13577-13598 | 9707 | record: BOLD-LEAD | **Q127 — OPENED and REPAIRED in the same tick (volt, 2026-10-30, at the commit this record lands in; spark lane by authorship, unclaimed and taken). A ten-file sweep put Q109's and Q120's pre-repair … | -- |
| 198 | Q80 | record | 13599-13615 | 7450 | cont: DONE | **Q80 — DONE (volt, 2026-10-30, at the commit this record lands in; spark lane by authorship, unclaimed and taken). All four filed defects reproduce at HEAD `e8a916f`, and none survives: the digest … | -- |
## Ids with no `## ` heading
@@ -232,7 +233,7 @@ Opened as records inside another section. `line` is the record's own opening lin
| Q106 | 13171 | 13171 | 14 | Q105 | BULLET+BOLD | BOLD-LEAD |
| Q107 | 13189 | 13189 | 18 | Q106 | BULLET+BOLD | BOLD-LEAD |
| Q108 | 13267 | 13263 | 3 | Q106 | BULLET+BOLD | BOLD-LEAD |
| Q109 | 13270 | 13270 | 22 | Q104 | BULLET+BOLD | BOLD-LEAD |
| Q109 | 13270 | 13270 | 23 | Q104 | BULLET+BOLD | BOLD-LEAD |
| Q110 | 13272 | 13272 | 6 | Q109 | BULLET+BOLD | BOLD-LEAD |
| Q111 | 13274 | 13274 | 7 | Q110 | BULLET+BOLD | BOLD-LEAD |
| Q112 | 13276 | 13276 | 6 | Q111 | BULLET+BOLD | BOLD-LEAD |
@@ -1549,17 +1550,30 @@ The block below is the parse this document renders, byte for byte. `--check` com
"title": "**Q110 \u2014 DONE (volt, 2026-10-30, at the commit this record lands in; quill lane by authorship, unclaimed and taken). `wizard-work/quill/QUICKSTART.md` no longer prints a command nobody ran: every \u2026"
},
{
"bytes": 9706,
"end_line": 13597,
"bytes": 9707,
"end_line": 13598,
"id": "Q127",
"kind": "record",
"line": 13577,
"lines": 21,
"lines": 22,
"role": "record",
"role_detail": "BOLD-LEAD",
"status": null,
"status_line": null,
"title": "**Q127 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-10-30, at the commit this record lands in; spark lane by authorship, unclaimed and taken). A ten-file sweep put Q109's and Q120's pre-repair \u2026"
},
{
"bytes": 7450,
"end_line": 13615,
"id": "Q80",
"kind": "record",
"line": 13599,
"lines": 17,
"role": "continuation",
"role_detail": "DONE",
"status": null,
"status_line": null,
"title": "**Q80 \u2014 DONE (volt, 2026-10-30, at the commit this record lands in; spark lane by authorship, unclaimed and taken). All four filed defects reproduce at HEAD `e8a916f`, and none survives: the digest \u2026"
}
],
"ids_multi_section": {
@@ -4246,17 +4260,30 @@ The block below is the parse this document renders, byte for byte. `--check` com
"title": "**Q110 \u2014 DONE (volt, 2026-10-30, at the commit this record lands in; quill lane by authorship, unclaimed and taken). `wizard-work/quill/QUICKSTART.md` no longer prints a command nobody ran: every \u2026"
},
{
"bytes": 9706,
"end_line": 13597,
"bytes": 9707,
"end_line": 13598,
"id": "Q127",
"kind": "record",
"line": 13577,
"lines": 21,
"lines": 22,
"role": "record",
"role_detail": "BOLD-LEAD",
"status": null,
"status_line": null,
"title": "**Q127 \u2014 OPENED and REPAIRED in the same tick (volt, 2026-10-30, at the commit this record lands in; spark lane by authorship, unclaimed and taken). A ten-file sweep put Q109's and Q120's pre-repair \u2026"
},
{
"bytes": 7450,
"end_line": 13615,
"id": "Q80",
"kind": "record",
"line": 13599,
"lines": 17,
"role": "continuation",
"role_detail": "DONE",
"status": null,
"status_line": null,
"title": "**Q80 \u2014 DONE (volt, 2026-10-30, at the commit this record lands in; spark lane by authorship, unclaimed and taken). All four filed defects reproduce at HEAD `e8a916f`, and none survives: the digest \u2026"
}
],
"nested": [
@@ -4445,7 +4472,7 @@ The block below is the parse this document renders, byte for byte. `--check` com
"form": "BOLD-LEAD",
"id": "Q109",
"line": 13270,
"refs": 22,
"refs": 23,
"section": "Q104",
"shape": "BULLET+BOLD"
},
@@ -4608,9 +4635,9 @@ The block below is the parse this document renders, byte for byte. `--check` com
"end_line": 8,
"line": 1
},
"queue_bytes": 2167300,
"queue_lines": 13597,
"queue_sha256": "3ed7ff7e05b14f66be05cf3a3e266f43934c3de2135b2b5c30dd970b762473f0",
"queue_bytes": 2174751,
"queue_lines": 13615,
"queue_sha256": "8dcbb0dc01717d8841ef41738f0b297ba674148eddf76109227ddeddb6c7e166",
"sections": [
{
"bytes": 3139,
+8
View File
@@ -30,6 +30,14 @@ python3 --version 2>&1 | grep -q "Python" || exit 10
# notice, because it ran none of the lane's cells. It runs one now.
python3 "$(dirname "$(readlink -f "$0")")/closed_artifact_guard.py" >/dev/null || exit 12
# Q80: json_sha256_validator.py's cells. Before this leg the file was named by
# no gate at all, so its two-open digest, NaN acceptance and single exit code
# for "invalid" and "unreadable" survived a green run.
if ! python3 "$(dirname "$(readlink -f "$0")")/test_json_sha256_validator.py" >/dev/null 2>&1; then
echo "CHECK: json_sha256_validator.py cells failed (Q80) -- run test_json_sha256_validator.py" >&2
exit 13
fi
# Check git commit pushed
git -C /root/troupe-standup rev-parse ed959f0 >/dev/null 2>&1 || exit 11
+170 -45
View File
@@ -1,60 +1,185 @@
#!/usr/bin/env python3
"""Simple JSON validator that generates SHA256 checksums for verification."""
"""Validate JSON documents and print the SHA-256 of the exact bytes parsed.
import json
import sys
One read serves both halves of the verdict: the bytes that are hashed are the
bytes that were parsed, so a file rewritten between the parse and the digest
cannot be described as "valid" while a different document's hash is printed
(Q80 measured the two-open form: two reads, and the printed digest belonged to
the second one).
Contract -- the repo's tools/ exit codes:
0 every input is a strict RFC 8259 JSON text; one ``VALID <path> SHA256=``
line per input.
1 every input was readable and at least one is not a JSON text: a syntax
error, a bare ``NaN`` / ``Infinity`` / ``-Infinity`` (RFC 8259 s6: "Numeric
values that cannot be represented in the grammar below (such as Infinity
and NaN) are not permitted" -- rejected here through ``parse_constant``),
a duplicate object member name (s4 says names SHOULD be unique; refused
unless ``--allow-duplicates``), or bytes that are not decodable JSON text.
2 could not grade: a usage error, or an input that could not be read
(missing, a directory, permission denied, a non-directory in the path).
Nothing is printed for *any* input when any input is ungradable -- a
substrate failure never wears a verdict, and no digest is emitted.
A leading UTF-8 byte order mark is tolerated: the parsed text is the bytes
decoded as ``utf-8-sig`` (RFC 8259 s8.1 lets a parser ignore a BOM, and the
lane's ``jsonl_validate.py`` already reads ``utf-8-sig``). Any other encoding --
UTF-16, UTF-32, Latin-1, binary -- is not a JSON *text* in the interchange
encoding RFC 8259 s8.1 requires, and grades rc 1 as undecodable. Both are stated
so neither is filed as a bug. ``--results PATH`` writes the run as JSON: the
exact invocation, and each input's path, size, blob sha256 and verdict.
"""
import argparse
import hashlib
import json
import os
import sys
from datetime import datetime, timezone
from pathlib import Path
RC_OK = 0
RC_INVALID = 1
RC_UNGRADABLE = 2
def validate_json_file(filepath: str) -> tuple[bool, str]:
"""Validate a JSON file and return (is_valid, error_message)."""
VALID = "valid"
INVALID = "invalid"
UNGRADABLE = "ungradable"
class _NonStandardConstant(ValueError):
"""NaN / Infinity / -Infinity: not JSON (RFC 8259 s6)."""
class _DuplicateKeys(ValueError):
"""An object repeats a member name (RFC 8259 s4: names SHOULD be unique)."""
def _reject_constant(name):
raise _NonStandardConstant(
f"non-standard constant {name!r}: RFC 8259 s6 forbids NaN and Infinity"
)
def _pairs_hook(pairs):
seen = set()
for name, _value in pairs:
if name in seen:
raise _DuplicateKeys(
f"duplicate object member name {name!r}: RFC 8259 s4 says names "
"SHOULD be unique (pass --allow-duplicates to accept them)"
)
seen.add(name)
return dict(pairs)
def read_bytes(path):
"""Return (bytes, None) or (None, message). One read, no second open."""
try:
with open(filepath, 'r') as f:
json.load(f)
return True, ""
except json.JSONDecodeError as e:
return False, f"Invalid JSON: {e}"
except FileNotFoundError:
return False, f"File not found: {filepath}"
except Exception as e:
return False, f"Error: {e}"
return Path(path).read_bytes(), None
except OSError as exc:
detail = exc.strerror or str(exc) or type(exc).__name__
return None, f"{type(exc).__name__}: {detail}"
def compute_sha256(filepath: str) -> str:
"""Compute SHA256 hash of a file."""
sha256_hash = hashlib.sha256()
with open(filepath, 'rb') as f:
for byte_block in iter(lambda: f.read(4096), b""):
sha256_hash.update(byte_block)
return sha256_hash.hexdigest()
def parse_bytes(data, allow_duplicates=False):
"""Return (parsed, None) or (None, message). Parses the bytes it is given.
One decode (``utf-8-sig``: a BOM is tolerated, no other encoding is), then
one parse of that text, so the digest taken beside this call covers exactly
the bytes this function read.
"""
try:
text = data.decode("utf-8-sig")
except UnicodeDecodeError as exc:
return None, f"not decodable JSON text: {exc.reason} at byte {exc.start}"
try:
if allow_duplicates:
return json.loads(text, parse_constant=_reject_constant), None
return json.loads(text, parse_constant=_reject_constant,
object_pairs_hook=_pairs_hook), None
except ValueError as exc:
return None, str(exc)
def main():
if len(sys.argv) < 2:
print("Usage: json_sha256_validator.py <file1.json> [file2.json ...]")
sys.exit(1)
results = []
all_valid = True
for filepath in sys.argv[1:]:
is_valid, error = validate_json_file(filepath)
if is_valid:
checksum = compute_sha256(filepath)
results.append(f"VALID {filepath} SHA256={checksum}")
else:
results.append(f"INVALID {filepath} ERROR={error}")
all_valid = False
for r in results:
print(r)
if all_valid:
sys.exit(0)
def grade(path, allow_duplicates=False):
"""Grade one input. Returns a record; the digest covers the parsed bytes."""
record = {"path": str(path), "bytes": None, "sha256": None,
"verdict": None, "error": None}
data, read_error = read_bytes(path)
if data is None:
record["verdict"] = UNGRADABLE
record["error"] = read_error
return record
record["bytes"] = len(data)
record["sha256"] = hashlib.sha256(data).hexdigest()
_, parse_error = parse_bytes(data, allow_duplicates=allow_duplicates)
if parse_error is None:
record["verdict"] = VALID
else:
sys.exit(1)
record["verdict"] = INVALID
record["error"] = parse_error
return record
def _write_results(path, argv, records):
payload = {
"tool": Path(__file__).name,
"invocation": [sys.executable, str(Path(__file__).resolve())] + list(argv),
"argv": list(argv),
"cwd": os.getcwd(),
"generated_at_utc": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
"inputs": records,
}
try:
Path(path).write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n",
encoding="utf-8")
except OSError as exc:
return f"{type(exc).__name__}: {exc.strerror or exc}"
return None
def main(argv=None):
ap = argparse.ArgumentParser(
prog="json_sha256_validator.py",
description="Validate JSON and print the SHA-256 of the parsed bytes.",
)
ap.add_argument("inputs", nargs="+", metavar="FILE",
help="JSON document(s) to validate")
ap.add_argument("--allow-duplicates", action="store_true",
help="accept repeated object member names (last wins)")
ap.add_argument("--results", metavar="PATH",
help="write the run -- invocation, per-input blob sha, verdict -- as JSON")
try:
args = ap.parse_args(argv)
except SystemExit as exc:
return RC_UNGRADABLE if exc.code not in (0, None) else RC_OK
records = [grade(p, allow_duplicates=args.allow_duplicates) for p in args.inputs]
ungradable = [r for r in records if r["verdict"] == UNGRADABLE]
if ungradable:
for r in ungradable:
print(f"UNGRADABLE {r['path']} ERROR={r['error']}")
print(f"cannot grade: {len(ungradable)} of {len(records)} input(s) unreadable; "
"no verdict and no digest printed")
return RC_UNGRADABLE
if args.results:
error = _write_results(args.results, argv if argv is not None else sys.argv[1:],
records)
if error is not None:
print(f"UNGRADABLE --results ERROR={error}")
return RC_UNGRADABLE
for r in records:
if r["verdict"] == VALID:
print(f"VALID {r['path']} SHA256={r['sha256']}")
else:
print(f"INVALID {r['path']} ERROR={r['error']}")
return RC_OK if all(r["verdict"] == VALID for r in records) else RC_INVALID
if __name__ == "__main__":
main()
sys.exit(main())
@@ -0,0 +1,198 @@
#!/usr/bin/env python3
"""Cells for wizard-work/spark/json_sha256_validator.py (Q80).
Black box: every CLI reading runs the shipped module as a subprocess, so the
cells cannot agree with the code by sharing an implementation. The one
in-process cell imports the module by path to count how many times the subject
is read -- the property Q80 asks for, which a subprocess cannot observe.
Run: python3 wizard-work/spark/test_json_sha256_validator.py
Exit 0 iff every cell passes.
"""
import hashlib
import importlib.util
import io
import json
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
TOOL = Path(__file__).resolve().with_name("json_sha256_validator.py")
def run(*args, cwd=None):
proc = subprocess.run([sys.executable, str(TOOL), *args],
capture_output=True, text=True, cwd=cwd)
return proc.returncode, proc.stdout, proc.stderr
class Base(unittest.TestCase):
def setUp(self):
self._tmp = tempfile.TemporaryDirectory()
self.dir = Path(self._tmp.name)
self.ok = self.write("ok.json", b'{"a": [1, 2, 3], "b": "x"}\n')
self.nan = self.write("nan.json", b'{"a": NaN}\n')
self.inf = self.write("inf.json", b'{"a": Infinity}\n')
self.neginf = self.write("neginf.json", b'{"a": -Infinity}\n')
self.dup = self.write("dup.json", b'{"a": 1, "a": 2}\n')
self.broken = self.write("broken.json", b'{"a": 1,\n')
self.bom = self.write("bom.json", b'\xef\xbb\xbf{"a": 1}\n')
def tearDown(self):
self._tmp.cleanup()
def write(self, name, data):
path = self.dir / name
path.write_bytes(data)
return path
def digest(self, path):
return hashlib.sha256(Path(path).read_bytes()).hexdigest()
class Contract(Base):
def test_usage_error_is_rc_2(self):
rc, out, err = run()
self.assertEqual(rc, 2, out + err)
def test_valid_document_rc_0_and_digest_covers_the_file(self):
rc, out, err = run(str(self.ok))
self.assertEqual(rc, 0, out + err)
self.assertIn("VALID", out)
self.assertIn(f"SHA256={self.digest(self.ok)}", out)
def test_nan_is_invalid_rc_1(self):
rc, out, err = run(str(self.nan))
self.assertEqual(rc, 1, out + err)
self.assertIn("INVALID", out)
self.assertIn("NaN", out)
self.assertNotIn("SHA256", out)
def test_infinity_and_negative_infinity_are_invalid(self):
for path, name in ((self.inf, "Infinity"), (self.neginf, "Infinity")):
rc, out, err = run(str(path))
self.assertEqual(rc, 1, f"{path.name}: {out}{err}")
self.assertIn("INVALID", out)
self.assertIn(name, out)
def test_duplicate_member_names_are_refused_by_default(self):
rc, out, err = run(str(self.dup))
self.assertEqual(rc, 1, out + err)
self.assertIn("duplicate object member name", out)
self.assertNotIn("SHA256", out)
def test_allow_duplicates_accepts_them(self):
rc, out, err = run("--allow-duplicates", str(self.dup))
self.assertEqual(rc, 0, out + err)
self.assertIn(f"SHA256={self.digest(self.dup)}", out)
def test_syntax_error_is_rc_1(self):
rc, out, err = run(str(self.broken))
self.assertEqual(rc, 1, out + err)
self.assertIn("INVALID", out)
def test_bom_is_tolerated_and_the_choice_is_documented(self):
rc, out, err = run(str(self.bom))
self.assertEqual(rc, 0, out + err)
self.assertIn(f"SHA256={self.digest(self.bom)}", out)
self.assertIn("byte order mark", TOOL.read_text(encoding="utf-8").lower())
def test_undecodable_bytes_are_invalid_rc_1(self):
bad = self.write("bad.bin", b'\xff\xfe\x00\x01')
rc, out, err = run(str(bad))
self.assertEqual(rc, 1, out + err)
self.assertIn("not decodable JSON text", out)
class Ungradable(Base):
def test_missing_input_is_rc_2(self):
rc, out, err = run(str(self.dir / "nope.json"))
self.assertEqual(rc, 2, out + err)
self.assertIn("UNGRADABLE", out)
self.assertNotIn("VALID", out)
self.assertNotIn("SHA256", out)
def test_directory_input_is_rc_2(self):
rc, out, err = run(str(self.dir))
self.assertEqual(rc, 2, out + err)
self.assertIn("UNGRADABLE", out)
def test_non_directory_in_path_is_rc_2(self):
rc, out, err = run(str(self.ok / "child.json"))
self.assertEqual(rc, 2, out + err)
self.assertIn("UNGRADABLE", out)
def test_ungradable_suppresses_every_verdict_and_digest(self):
rc, out, err = run(str(self.ok), str(self.dir / "nope.json"))
self.assertEqual(rc, 2, out + err)
self.assertNotIn("VALID", out)
self.assertNotIn("INVALID", out)
self.assertNotIn("SHA256", out)
self.assertIn("no verdict and no digest printed", out)
class SingleRead(Base):
def test_one_read_serves_both_the_parse_and_the_digest(self):
"""Serve payload A first and B after; the digest must be A's, read once."""
target = self.dir / "race.json"
target.write_bytes(b'{"a": 1}\n')
a = b'{"a": 1}\n'
b = b'{"a": 999}\n'
counts = {"read_bytes": 0, "open": 0}
spec = importlib.util.spec_from_file_location("json_sha256_validator", TOOL)
if spec is None or spec.loader is None:
self.fail(f"cannot load the module under test at {TOOL}")
loader = spec.loader
mod = importlib.util.module_from_spec(spec)
loader.exec_module(mod)
real_read_bytes = Path.read_bytes
import builtins
real_builtin_open = builtins.open
def fake_read_bytes(self):
counts["read_bytes"] += 1
return a if counts["read_bytes"] == 1 else b
def fake_open(path, mode="r", *args, **kwargs):
if str(path) == str(target):
counts["open"] += 1
if "b" in mode:
return io.BytesIO(b)
return io.StringIO(b.decode())
return real_builtin_open(path, mode, *args, **kwargs)
Path.read_bytes = fake_read_bytes
builtins.open = fake_open
try:
record = mod.grade(str(target))
finally:
Path.read_bytes = real_read_bytes
builtins.open = real_builtin_open
reads = counts["read_bytes"] + counts["open"]
self.assertEqual(record["verdict"], "valid", record)
self.assertEqual(reads, 1, f"subject read {reads} times: {counts}")
self.assertEqual(record["sha256"], hashlib.sha256(a).hexdigest())
class Results(Base):
def test_results_file_carries_the_invocation_and_each_blob_sha(self):
out_path = self.dir / "run.json"
rc, out, err = run("--results", str(out_path), str(self.ok), str(self.dup))
self.assertIn(rc, (0, 1), out + err)
payload = json.loads(out_path.read_text(encoding="utf-8"))
self.assertEqual(payload["tool"], TOOL.name)
self.assertIn(str(TOOL.resolve()), payload["invocation"])
self.assertIn(str(self.ok), payload["argv"])
by_path = {item["path"]: item for item in payload["inputs"]}
self.assertEqual(by_path[str(self.ok)]["sha256"], self.digest(self.ok))
self.assertEqual(by_path[str(self.ok)]["verdict"], "valid")
self.assertEqual(by_path[str(self.dup)]["verdict"], "invalid")
if __name__ == "__main__":
unittest.main(verbosity=2)