76 lines
3.9 KiB
Plaintext
76 lines
3.9 KiB
Plaintext
EVIDENCE — jsonpatch (RFC 6901 pointer + RFC 6902 patch engine)
|
|
Date: 2026-10-21. Every number below is tool output, not an estimate.
|
|
|
|
ORACLE
|
|
Vendored upstream suite, fetched 2026-10-21 from
|
|
https://raw.githubusercontent.com/json-patch/json-patch-tests/master/tests.json
|
|
https://raw.githubusercontent.com/json-patch/json-patch-tests/master/spec_tests.json
|
|
sha256 pins (checked by stage 1 of the battery on every run):
|
|
tests.json 18707 bytes de3dce3d0d5029fed83007e50b54607750dd3d1478d3c59ca35fdc18fb1a04ae
|
|
spec_tests.json 4031 bytes a26b050292207033e5cccc5d6102b7bd6f8add7db0d0680e5d46a7ecf40a8c7b
|
|
95 + 17 = 112 cases; 4 are marked "disabled" upstream and are skipped.
|
|
|
|
COMMANDS AND RESULTS (run from /root/troupe-standup/wizard-work/null/jsonpatch)
|
|
|
|
1. Full battery, teeth included:
|
|
$ python3 test_jsonptr_patch.py
|
|
official suite: 108 passed, 0 failed, 4 skipped (disabled)
|
|
== 54 passed, 0 failed == (engine selftest: RFC 6902 App. A + RFC 6901 + equality)
|
|
[atomicity] 118 checks, 0 failed (doc untouched after a failed patch; input never mutated on success)
|
|
fuzz: 2000 patches, 823 ok, 1177 rejected
|
|
fuzz: identity 402 ok / 0 failed / 98 skipped
|
|
fuzz: move out and back is identity -> pass
|
|
fuzz: 3000 bad patches rejected as expected
|
|
fuzz: test op 439 accepted / 393 rejected / 0 wrong
|
|
teeth: 8 killed, 0 survived
|
|
TOTAL: 126 checks, 0 failed
|
|
VERDICT: PASS
|
|
exit code 0.
|
|
|
|
2. Same battery from the wizard-outputs copy (the deliverable path):
|
|
$ python3 /root/.hermes/wizard-outputs/null/test_jsonptr_patch.py
|
|
exit code 0. (This is the CHECK line.)
|
|
|
|
3. Repo copy, teeth skipped:
|
|
$ python3 test_jsonptr_patch.py --no-teeth
|
|
TOTAL: 125 checks, 0 failed / VERDICT: PASS
|
|
|
|
4. CLI smoke test (measured):
|
|
$ python3 jsonptr_patch.py --doc cli_doc.json --patch cli_patch.json
|
|
{"foo": ["bar", "qux", "baz"], "obj": {"k": "~x/y", "n": 1}, "tail": "qux"} exit 0
|
|
$ python3 jsonptr_patch.py --doc cli_doc.json --patch <failing test op>
|
|
error: patch[0] ('test'): test failed at '/n': expected '999', got '1' exit 1
|
|
$ python3 jsonptr_patch.py --nope
|
|
usage: ... exit 2
|
|
$ python3 jsonptr_patch.py --selftest | tail -1 -> "== 54 passed, 0 failed ==" exit 0
|
|
Also measured: `copy` with `from: "/foo/-"` is rejected with exit 1 — "-" is an
|
|
add-only token, so the engine refuses it (correct per RFC 6901 §4).
|
|
|
|
5. Mutation teeth: 8 source mutants were generated by single-string edits of
|
|
jsonptr_patch.py and the whole battery was re-run against each. All 8 were killed
|
|
(battery exited non-zero), so the suite has teeth:
|
|
add index==len allowed ............ killed
|
|
missing parent tolerated .......... killed
|
|
bool is a number .................. killed
|
|
unescape ~0 wrong ................. killed
|
|
move into descendant allowed ...... killed
|
|
pointer slash not enforced ........ killed
|
|
leading zeros allowed ............. killed
|
|
remove root tolerated ............. killed
|
|
|
|
WHAT IS MEASURED vs CLAIMED
|
|
Measured: everything above (exit codes, counts, sha256s).
|
|
Claimed (spec reading, not machine-checked): that RFC 6902 §5's "the entire patch
|
|
document MUST NOT be applied" reading matches this implementation's atomic
|
|
behaviour. The vendored suite has no multi-op error case that pins this down, so
|
|
the atomicity stage asserts it directly instead of relying on the suite.
|
|
|
|
KNOWN LIMITATIONS
|
|
* `test` equality is implemented structurally (1 == 1.0); the official suite
|
|
contains no 1-vs-1.0 case, so that behaviour is asserted only by the engine's
|
|
own selftest.
|
|
* Duplicate keys in a patch object (upstream case A.13) are marked "disabled"
|
|
upstream and are skipped; Python's json.loads keeps the last one, so a duplicate
|
|
`op` is not detectable here.
|
|
* `disabled` cases: 4 skipped, by upstream's own marking.
|