57 lines
7.3 KiB
Plaintext
57 lines
7.3 KiB
Plaintext
PASS c1 parse plain cells line
|
|
PASS c2 parse fixtures with parenthetical
|
|
PASS c3 parse hyphenated self-test
|
|
PASS c4 parse thousands separator
|
|
PASS c5 no count line -> None
|
|
PASS c6 claim on the --selftest usage line
|
|
PASS c7 prose beside the word selftest is not a claim
|
|
PASS c8 claim with a unit the tool never prints is still a claim
|
|
PASS c9 advertised 18 vs printed 13 -> COUNT_MISMATCH
|
|
PASS c10 advertised 18 vs printed 18 -> OK
|
|
PASS c11 unit mismatch is a mismatch, not a pass
|
|
PASS c12 two advertised values -> SPLIT
|
|
PASS c13 failed>0 is fatal even at rc=0
|
|
PASS c14 rc!=0 with no count line -> RUN_FAILED
|
|
PASS c15 rc=0 with no count line -> NO_COUNT_LINE
|
|
sweep: 3 tools, 1 OK, 2 fatal (COUNT_MISMATCH, NO_SELFTEST_FLAG)
|
|
read-refusal: 30 probes, 0 ok, 30 fault (NO_TOOL append_only_audit.py/missing, NO_TOOL append_only_owners.py/missing, NO_TOOL census_row_delta.py/missing, NO_TOOL check_lint.py/missing, NO_TOOL checkin_check.py/missing, NO_TOOL citation_hash_class.py/missing, NO_TOOL citation_resolve_check.py/missing, NO_TOOL doc_command_check.py/missing, NO_TOOL doc_census_check.py/missing, NO_TOOL gate_blame.py/missing, NO_TOOL landing_check.py/missing, NO_TOOL manifest_audit.py/missing, NO_TOOL path_claim_check.py/missing, NO_TOOL queue_claim_check.py/missing, NO_TOOL queue_index.py/missing, NO_TOOL queue_append_audit.py/missing, NO_TOOL reply_check.py/missing, NO_TOOL selftest_sweep.py/missing, NO_TOOL served_drift_check.py/missing, NO_TOOL tool_map.py/missing, NO_TOOL verify.py/missing, NO_TOOL write_probe.py/missing, NO_TOOL append_only_owners.py/not-utf8, NO_TOOL census_row_delta.py/not-utf8, NO_TOOL checkin_check.py/not-utf8, NO_TOOL queue_claim_check.py/not-utf8, NO_TOOL queue_index.py/not-utf8, NO_PROBE fake_bad.py/missing, NO_PROBE fake_flagless.py/missing, NO_PROBE fake_good.py/missing)
|
|
malformed-input: 29 probes, 0 ok, 29 fault (NO_TOOL append_only_audit.py/not-a-repo, NO_TOOL append_only_owners.py/malformed-json, NO_TOOL append_only_owners.py/wrong-shape, NO_TOOL census_row_delta.py/malformed-json, NO_TOOL census_row_delta.py/wrong-shape, NO_TOOL check_lint.py/junk-log, NO_TOOL check_lint.py/junk-log-gated, NO_TOOL checkin_check.py/no-headings, NO_TOOL citation_hash_class.py/not-a-repo, NO_TOOL citation_resolve_check.py/not-a-repo, NO_TOOL doc_census_check.py/not-a-repo, NO_TOOL doc_command_check.py/not-a-repo, NO_TOOL gate_blame.py/not-a-repo, NO_TOOL landing_check.py/not-a-repo, NO_TOOL manifest_audit.py/not-a-dir, NO_TOOL manifest_audit.py/junk-manifest, NO_TOOL path_claim_check.py/not-a-repo, NO_TOOL queue_claim_check.py/junk-queue, NO_TOOL queue_index.py/junk-index, NO_TOOL queue_append_audit.py/not-a-repo, NO_TOOL reply_check.py/not-a-dir, NO_TOOL served_drift_check.py/not-a-repo, NO_TOOL selftest_sweep.py/not-a-repo, NO_TOOL tool_map.py/not-a-repo, NO_TOOL verify.py/not-a-repo, NO_TOOL write_probe.py/not-a-repo, NO_PROBE fake_bad.py/malformed, NO_PROBE fake_flagless.py/malformed, NO_PROBE fake_good.py/malformed)
|
|
|
|
verdict: rc 1 -- red: tools 2 fatal, read-refusal 30 fault, malformed-input 29 fault (tools 2 fatal, read-refusal 30 fault, malformed-input 29 fault, pin 0 drift)
|
|
PASS c16 end to end: a wrong claim is fatal
|
|
PASS c17 end to end: correct claim OK, flagless caught, exit 1
|
|
sweep: 3 tools, 1 OK, 2 fatal (COUNT_MISMATCH, NO_SELFTEST_FLAG)
|
|
read-refusal: 30 probes, 0 ok, 30 fault (NO_TOOL append_only_audit.py/missing, NO_TOOL append_only_owners.py/missing, NO_TOOL census_row_delta.py/missing, NO_TOOL check_lint.py/missing, NO_TOOL checkin_check.py/missing, NO_TOOL citation_hash_class.py/missing, NO_TOOL citation_resolve_check.py/missing, NO_TOOL doc_command_check.py/missing, NO_TOOL doc_census_check.py/missing, NO_TOOL gate_blame.py/missing, NO_TOOL landing_check.py/missing, NO_TOOL manifest_audit.py/missing, NO_TOOL path_claim_check.py/missing, NO_TOOL queue_claim_check.py/missing, NO_TOOL queue_index.py/missing, NO_TOOL queue_append_audit.py/missing, NO_TOOL reply_check.py/missing, NO_TOOL selftest_sweep.py/missing, NO_TOOL served_drift_check.py/missing, NO_TOOL tool_map.py/missing, NO_TOOL verify.py/missing, NO_TOOL write_probe.py/missing, NO_TOOL append_only_owners.py/not-utf8, NO_TOOL census_row_delta.py/not-utf8, NO_TOOL checkin_check.py/not-utf8, NO_TOOL queue_claim_check.py/not-utf8, NO_TOOL queue_index.py/not-utf8, NO_PROBE fake_bad.py/missing, NO_PROBE fake_flagless.py/missing, NO_PROBE fake_good.py/missing)
|
|
malformed-input: 29 probes, 0 ok, 29 fault (NO_TOOL append_only_audit.py/not-a-repo, NO_TOOL append_only_owners.py/malformed-json, NO_TOOL append_only_owners.py/wrong-shape, NO_TOOL census_row_delta.py/malformed-json, NO_TOOL census_row_delta.py/wrong-shape, NO_TOOL check_lint.py/junk-log, NO_TOOL check_lint.py/junk-log-gated, NO_TOOL checkin_check.py/no-headings, NO_TOOL citation_hash_class.py/not-a-repo, NO_TOOL citation_resolve_check.py/not-a-repo, NO_TOOL doc_census_check.py/not-a-repo, NO_TOOL doc_command_check.py/not-a-repo, NO_TOOL gate_blame.py/not-a-repo, NO_TOOL landing_check.py/not-a-repo, NO_TOOL manifest_audit.py/not-a-dir, NO_TOOL manifest_audit.py/junk-manifest, NO_TOOL path_claim_check.py/not-a-repo, NO_TOOL queue_claim_check.py/junk-queue, NO_TOOL queue_index.py/junk-index, NO_TOOL queue_append_audit.py/not-a-repo, NO_TOOL reply_check.py/not-a-dir, NO_TOOL served_drift_check.py/not-a-repo, NO_TOOL selftest_sweep.py/not-a-repo, NO_TOOL tool_map.py/not-a-repo, NO_TOOL verify.py/not-a-repo, NO_TOOL write_probe.py/not-a-repo, NO_PROBE fake_bad.py/malformed, NO_PROBE fake_flagless.py/malformed, NO_PROBE fake_good.py/malformed)
|
|
|
|
verdict: rc 1 -- red: tools 2 fatal, read-refusal 30 fault, malformed-input 29 fault (tools 2 fatal, read-refusal 30 fault, malformed-input 29 fault, pin 0 drift)
|
|
PASS c18 --json carries every row and the fatal count
|
|
PASS c19 a refusal at rc 2 with a message -> OK
|
|
PASS c20 a graded fault's rc 1 on an unreadable input -> RC_NOT_2
|
|
PASS c21 a traceback is fatal even when it exits 2
|
|
PASS c22 rc 2 with no message -> SILENT
|
|
PASS c23 a documented rc 0 grade on malformed input -> OK
|
|
PASS c24 the silent-zero: rc 0 where the contract says 2 -> RC_WRONG
|
|
PASS c25 a refusal with no message -> SILENT
|
|
PASS c26 a refusal at rc 2 whose prose is not the pinned phrase -> OK (contract)
|
|
PASS c27 a traceback is fatal even at the documented code
|
|
PASS c28 every tool with a missing-probe also has a malformed probe
|
|
PASS c29 an unprobed tool is NO_PROBE on both axes and is a fault
|
|
PASS c30 every FATAL_READ verdict is red and OK is not
|
|
PASS c31 a printed count that matches its pinned row -> PIN_OK
|
|
PASS c32 four cells against a pin of seven -> PIN_DRIFT, fatal
|
|
PASS c33 a tool with no pinned row -> PIN_UNPINNED, fatal
|
|
PASS c34 a pinned row whose tool is gone -> PIN_ORPHAN, fatal
|
|
PASS c35 a tool with no --selftest is PIN_NA, not a fault
|
|
PASS c36 render/load round-trip keeps both rows and their units
|
|
PASS c37 a non-integer count is refused, naming the line
|
|
PASS c38 --write-baseline pins exactly what the run printed
|
|
PASS c39 end to end: a drifted pin names PIN_DRIFT and turns the run red
|
|
PASS c40 a GRADED row still needs its own verdict word -> UNEXPLAINED
|
|
PASS c41 end to end: a refusal at rc 2 with unpinned prose -> OK (contract)
|
|
PASS c42 end to end: a tool that exits 0 on a malformed input -> RC_WRONG, fatal
|
|
PASS c43 a faulting axis names its probe and tool on the summary line
|
|
PASS c44 the verdict line states the rc the same call returns
|
|
selftest: 44 cells, 44 passed, 0 failed
|
|
SELFTEST_EXIT=0
|