Files
troupe-standup/wizard-work/zeph/q60_enforce.py
T

473 lines
24 KiB
Python

#!/usr/bin/env python3
"""Q60 enforcement check (Zeph lane, 2026-10-26).
Implements the narrowed Q60 residue from reply 100 section 6 and the
WORK-QUEUE.md Q60 addendum: the census half is landed; this is the check half.
What it does, on a checkout at a NAMED head:
1. Population: count truncated-identifier occurrences / distinct tokens /
files under the pinned pattern ``[0-9a-f]{8,}(-[0-9a-f]{4})*-…``.
The head and the exclusion list are printed beside every number, because
a recount at the head that carries the report re-counts the report itself
(reply 100 section 2: 77/19 at 0d294b8 collapses to 49/17 once the census
file and the check-in that quotes it are excluded). Occurrence and
distinct counts are printed side by side, because the 31 -> 35 -> 16
drift was an occurrence/distinct confusion and the distinct count was
constant at 16 (reply 100 section 3).
2. Recovery: for each distinct truncated stem, look for a longer string in
the tree (same exclusions) beginning with that stem. Stems with no
in-tree longer string fall back to the KNOWN tables below, which are
census rows (in-tree recoveries) and reply-100 section 4 (external
recoveries). A stem in neither place FAILS the gate: it is a new
unrecovered identifier-subject token.
3. Complete-but-wrong: any full UUID-shaped value in the tree whose first
8 hex chars match a KNOWN full value's prefix but whose value differs
FAILS the gate. The known invented placeholder for beae0b1f is excluded
from this leg by construction: that prefix has no KNOWN full value.
Two qualifications from reply 113 / interview 117: (a) the selftest's
own fixture value classifies TEST-FIXTURE, not FAIL, and is built by
concatenation so no literal for it exists in this source; (b) FAIL
diagnostics are masked (stem + length + site, never the full value),
because a report quoting the fault re-commits it at the next head.
The beae0b1f stem rule keys on the 8-char stem, narrowed per reply
130 section 2 / interview 117 section 1.1 (Q96): a truncated token
sharing that stem skips the longer-string look and stays
UNRECOVERED-JUSTIFIED, but a full UUID-shaped value arriving under
that stem is reported RECOVERED with its site. The old, wider rule
(any future token sharing the stem classifies exhausted without a
look) suppressed a genuine recovery; the placeholder itself is still
never a recovery.
Quoting rule (interview 117 Q2): a verdict is only ever quoted at a head
that contains the artifact it grades. Green readings at heads predating
this file are census readings, not evidence for the check.
Exit status: 0 when no FAIL line is printed, 1 otherwise.
``--selftest`` builds fixture trees and asserts the negative cells: an
unknown truncated token fails, a real complete-but-wrong value fails, the
fixture value classifies TEST-FIXTURE, a self-quoting report file excluded
from the population collapses the count, the check over the checker's
own directory passes (cell 7, the self-scan leg), the checker's own
source bytes carry no fixture value -- each table entry tested on its own,
never as a joined string (cell 8, the absence cell, Q96; per-entry form
per Q121) -- a full value under the exhausted beae0b1f stem reports
RECOVERED (cell 9, Q96), and a two-entry table with one entry's literal
inserted names exactly that entry (cell 10, the Q121 cell: the joined form
named nothing once the table held two values) -- and a record-quoted
probe value under that stem does NOT report RECOVERED (cell 11, Q117).
Provenance rule (Q117, reply 146 section 4 / interview section 3):
a full value invented in one of our own replies is never an archive
recovery, however many bytes of it sit in the tree. It is implemented
as data, not site heuristics: a KNOWN_PROBE table beside
KNOWN_FIXTURE. Probe values are skipped by the recovery leg exactly
like the placeholder and the fixture, so a stem whose only in-tree
longer string is a probe value reads UNRECOVERED-JUSTIFIED with the
value still in the tree. The Q99 record that quotes the probe value
is NOT edited: the table declares its provenance, and the record
stands as dated fact. Cell numbering note: Q117 was filed asking for
"a new cell (10)", but Q121 landed first and took that number, so
the probe cell is cell 11; the done-means (table suppresses, flip-back
proves it) are unchanged.
Correction, reply 130 probe D / interview 117 section 1.4: the
2026-10-27 check-in credited cell 7 with proving that no UUID-shaped
literal for the fixture exists in this source. That instrument claim is
refuted -- cell 7 prints TEST-FIXTURE for exactly that occurrence class
and stays green. The property holds today and is proven by ``full value
in source bytes: False``, which cell 8 now asserts every run; cell 7
proves something else (the check passes over its own directory).
Usage:
python3 q60_enforce.py --tree <checkout-path> [--exclude <relpath> ...]
python3 q60_enforce.py --selftest
"""
import argparse
import os
import re
import subprocess
import sys
import tempfile
TRUNC_RE = re.compile("([0-9a-f]{8,}(?:-[0-9a-f]{4})*)-\u2026")
# Mutant note (reply 146 section 1): an injected token must be
# token-shaped -- stem + hyphen + U+2026 ellipsis. A bare stem, or a stem
# with three ASCII dots, never matches TRUNC_RE and leaves --selftest
# green, which reads exactly like a mutant that failed to bite.
FULL_RE = re.compile(
r"[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}"
)
# Longer-string search: a UUID-shaped continuation of the stem (one or more
# hyphen groups of 4-12 hex, so the 8-4-4-4-12 shape matches in full).
# The truncated form itself (stem + hyphen, before the ellipsis) must NOT
# count: a bare "stem-" is not a longer string.
LONG_RE_TEMPLATE = r"{stem}(-[0-9a-f]{{4,12}})+"
# Census in-tree recoveries (recoveries/truncated-identifier-census.md rows
# 2, 3, 4, 6, 7, 8, 10, 13, 14, 15), keyed by first 8 hex chars.
KNOWN_IN_TREE = {
"3d79ba82": "3d79ba82-926b-48c0-9d15-8791a364ea81",
"34413f8d": "34413f8d-26bf-4bbc-8ade-eb309a0e2245",
"a92b55d0": "a92b55d0-1d36-4884-8e65-f456450b3a74",
"3c67f712": "3c67f712-697d-48d8-b05c-01be896e61da",
"29ba0d51": "29ba0d51-2d9f-412e-ad19-5f28a76ca96e",
"e33f87f8": "e33f87f8-88ec-4e1a-9a13-df9bbb1d8120",
"6ec57334": "6ec57334-604d-42b5-a2e7-54662739c504",
"51a65c2c": "51a65c2c-07dd-4307-aba6-f7c4af17311c",
"20c08939": "20c08939-56c2-4351-a47a-efcdf022eca5",
"18a809aa": "18a809aa-0b7f-4523-ac0e-508c9f0f6f66",
}
# External recoveries (reply 100 section 4, Timmy's fetches, 2026-10-25).
KNOWN_EXTERNAL = {
"6b5089a8": "6b5089a8-2daf-4b4e-a739-bcf762cfe29a",
"10dfc8bc": "10dfc8bc-99ad-4ca2-ab76-853cb90a92c2",
"a90cdb7c": "a90cdb7c-1c33-4fca-9a96-26c236006f14",
"6c573a9c": "6c573a9c-1d6f-4ae6-a93e-ff79868456f4",
"67cbefc2": "67cbefc2-d94f-4f31-9e6e-b6210a60ad5e",
}
KNOWN_FULL = dict(KNOWN_IN_TREE)
KNOWN_FULL.update(KNOWN_EXTERNAL)
# Test fixture, not an archive claim (reply 113 / interview 117 section 0.7):
# the complete-but-wrong value used by selftest cell 4's predecessor. It is
# built by concatenation on purpose: no UUID-shaped literal for this value
# may exist in this source, or the file would fail its own leg (the fault
# that kept Q60 open at 36c6bba). Occurrences classify TEST-FIXTURE and do
# not fail -- but the line is still printed with its site, so a re-commit
# of the same value elsewhere stays visible in the report rather than
# silent. Value alone cannot distinguish fixture from re-commit; site can.
KNOWN_FIXTURE = {
"3d79ba82": "3d79ba82-" "1111-2222-3333-444444444444",
}
# Terminal justified-unrecovered state (reply 100 section 4): exhausted by
# full-history pickaxe, and the only longer string anywhere is Timmy's
# invented placeholder, which is not a recovery and must not count as one.
KNOWN_UNRECOVERED = {"beae0b1f"}
PLACEHOLDER = "beae0b1f-0000-0000-0000-000000000000"
# Probe values invented in our own investigation (Q117): Timmy's probe-C
# value, quoted in full by his own Q99 record (WORK-QUEUE.md) to
# demonstrate the over-wide stem rule. It is the ONLY full value in the
# tree under this stem that is not the placeholder, and it is not an
# archive recovery: the record created the value that would satisfy the
# recovery leg. Built by concatenation, like the fixture, so no literal
# for it exists in this source; the recovery leg skips it, and cell 11
# proves the table is what suppresses it. The Q99 record is not edited.
KNOWN_PROBE = {
"beae0b1f": "beae0b1f-" "9f2c-4d1e-8a77-3c5b6e9d0a11",
}
def list_files(tree):
"""Tracked files if tree is a git checkout, else every file under it."""
git_dir = os.path.join(tree, ".git")
if os.path.isdir(git_dir) or os.path.isfile(git_dir):
out = subprocess.run(
["git", "-C", tree, "ls-files"],
capture_output=True, text=True, check=True,
)
return [p for p in out.stdout.splitlines() if p]
found = []
for root, _dirs, names in os.walk(tree):
for name in names:
found.append(os.path.relpath(os.path.join(root, name), tree))
return sorted(found)
def read_text(path):
with open(path, "r", encoding="utf-8", errors="replace") as fh:
return fh.read()
def head_of(tree):
try:
out = subprocess.run(
["git", "-C", tree, "rev-parse", "HEAD"],
capture_output=True, text=True, check=True,
)
return out.stdout.strip()
except Exception:
return "(not a git checkout)"
def run_check(tree, excludes):
"""Return (report_lines, failed_bool). Pure function over the tree."""
lines = []
failed = False
head = head_of(tree)
lines.append("HEAD=%s" % head)
lines.append("EXCLUDE=%s" % ",".join(excludes) if excludes else "EXCLUDE=(none)")
files = [f for f in list_files(tree) if f not in set(excludes)]
contents = {}
for rel in files:
try:
contents[rel] = read_text(os.path.join(tree, rel))
except OSError:
continue
occ = 0
per_token = {}
files_with = set()
for rel, text in contents.items():
for m in TRUNC_RE.finditer(text):
occ += 1
per_token.setdefault(m.group(1), []).append(rel)
files_with.add(rel)
lines.append("OCC=%d DISTINCT=%d FILES=%d" % (occ, len(per_token), len(files_with)))
# Recovery + complete-but-wrong, per distinct stem.
for stem in sorted(per_token):
stem8 = stem[:8]
longer = None
if stem8 not in KNOWN_UNRECOVERED:
pat = re.compile(LONG_RE_TEMPLATE.format(stem=re.escape(stem)))
for rel, text in contents.items():
for m in pat.finditer(text):
cand = m.group(0)
# The placeholder, the test fixture and any probe
# value are not recoveries: skip all three, or the
# recovery line would name one of them as a recovery.
if (cand != PLACEHOLDER
and cand not in KNOWN_FIXTURE.values()
and cand not in KNOWN_PROBE.values()):
longer = (cand, rel)
break
if longer:
break
else:
# Q96 narrowing: a truncated token under an exhausted stem
# skips the longer-string look, but a full value that arrives
# under that stem is a recovery, not an exhaustion. The
# placeholder (and the fixture, by symmetry) never counts;
# since Q117 neither does a record-quoted probe value.
for rel, text in contents.items():
for val in FULL_RE.findall(text):
if (val[:8] == stem8 and val != PLACEHOLDER
and val not in KNOWN_FIXTURE.values()
and val not in KNOWN_PROBE.values()):
longer = (val, rel)
break
if longer:
break
if longer:
lines.append("TOKEN %s: RECOVERED-IN-TREE %s at %s" % (stem, longer[0], longer[1]))
elif stem8 in KNOWN_EXTERNAL:
lines.append("TOKEN %s: RECOVERED-EXTERNAL %s (reply 100 s4)" % (stem, KNOWN_EXTERNAL[stem8]))
elif stem8 in KNOWN_IN_TREE:
lines.append("TOKEN %s: RECOVERED-IN-TREE-KNOWN %s" % (stem, KNOWN_IN_TREE[stem8]))
elif stem8 in KNOWN_UNRECOVERED:
lines.append("TOKEN %s: UNRECOVERED-JUSTIFIED (exhausted; placeholder is not a recovery)" % stem)
else:
lines.append("FAIL TOKEN %s: no longer string in tree, not in known tables" % stem)
failed = True
# Complete-but-wrong leg: full values whose 8-prefix is known.
# Diagnostics are MASKED (interview 117 section 0.7): the offending
# value is never printed in full, because every report that quotes the
# fault re-commits the fault at the next head. Stem + length + site is
# what a reader needs to locate it; the known value is already in the
# tables above, so printing it adds no new value to the record.
# Masking note (reply 146 section 3): set() dedupes by VALUE, not by
# (prefix, length), so two distinct values sharing prefix and length
# print two identical masked lines -- multiplicity stays visible, and
# a change that deduped by (prefix, len) "to cut noise" would delete
# the last evidence of such a pair. Do not make that change.
for rel, text in contents.items():
for val in set(FULL_RE.findall(text)):
prefix = val[:8]
if prefix in KNOWN_FULL and val != KNOWN_FULL[prefix]:
if val == KNOWN_FIXTURE.get(prefix):
lines.append("TEST-FIXTURE %s-\u2026 (len %d) at %s" % (prefix, len(val), rel))
else:
lines.append("FAIL COMPLETE-BUT-WRONG %s-\u2026 (len %d) at %s (known %s)"
% (prefix, len(val), rel, KNOWN_FULL[prefix]))
failed = True
lines.append("VERDICT=%s" % ("FAIL" if failed else "PASS"))
return lines, failed
def fixture_literals_in_source(blob, table=None):
"""Return the stems whose full fixture value occurs in ``blob``, one per entry.
Q121: the value must be tested per entry. Joining the table's values into
one string and testing that string works only while the table has exactly
one entry -- with two it tests ``value1value2``, which occurs nowhere, so
both values pass unchecked and the cell stops working by passing.
Diagnostics name the stem and the length, never the value (the file's
masking rule, so a red report does not re-commit the fault it reports).
"""
table = KNOWN_FIXTURE if table is None else table
faults = []
for stem in sorted(table):
if table[stem].encode("utf-8") in blob:
faults.append("%s (len %d)" % (stem, len(table[stem])))
return faults
def selftest():
"""Fixture trees; each cell asserts a behaviour, not a tautology."""
passed = []
# Cell 8 (Q96 absence cell, repaired for Q121): every value in the
# fixture table must be absent from this file's own bytes, tested one
# entry at a time. The values are assembled in the table from fragments,
# never written as literals, so a literal re-commit of any one of them
# in this source fails this cell while cell 7 stays green (TEST-FIXTURE)
# -- the two cells red on distinguishable faults. Per-entry testing is
# what Q121 filed: the joined-string form was vacuous from the second
# table entry onward, and cell 10 now measures that case directly.
with open(os.path.abspath(__file__), "rb") as fh:
source_bytes = fh.read()
faults = fixture_literals_in_source(source_bytes)
assert not faults, "fixture value literal(s) present in checker source: %s" % ", ".join(faults)
passed.append("source-absence-passes")
def check(tree, excludes, want_fail, label):
_lines, failed = run_check(tree, excludes)
assert failed == want_fail, "%s: want fail=%s" % (label, want_fail)
passed.append(label)
with tempfile.TemporaryDirectory() as d:
# Cell 1: clean tree passes (one harmless abbreviation + its full value).
# Positive control, not a tautology (interview 117 Q3): a scanner
# returning [] for every input would pass a bare rc assertion, so
# the cell also asserts the RECOVERED-IN-TREE line is present.
with open(os.path.join(d, "a.md"), "w") as fh:
fh.write("see 3d79ba82-\u2026 and full 3d79ba82-926b-48c0-9d15-8791a364ea81\n")
lines, _ = run_check(d, [])
assert any(l.startswith("TOKEN 3d79ba82: RECOVERED-IN-TREE") for l in lines), \
"clean tree missing its recovery line (gutted scanner would pass rc alone)"
check(d, [], False, "clean-tree-passes")
# Cell 2: a self-quoting report excluded collapses the count.
with open(os.path.join(d, "report.md"), "w") as fh:
fh.write("3d79ba82-\u2026 3d79ba82-\u2026 3d79ba82-\u2026\n")
lines, _ = run_check(d, [])
assert any(l.startswith("OCC=4 ") for l in lines), "self-quote counted without exclusion"
lines, _ = run_check(d, ["report.md"])
assert any(l.startswith("OCC=1 ") for l in lines), "exclusion did not collapse count"
passed.append("self-quotation-excluded")
# Cell 3: unknown truncated token fails.
with open(os.path.join(d, "b.md"), "w") as fh:
fh.write("dead link deadbeef-\u2026 with no longer string\n")
check(d, ["report.md"], True, "unknown-token-fails")
os.remove(os.path.join(d, "b.md"))
# Cell 4: a REAL complete-but-wrong value fails. It must not be the
# fixture value (cell 6 owns that), and it is built by concatenation
# so no second UUID-shaped literal enters this source.
with open(os.path.join(d, "c.md"), "w") as fh:
fh.write("34413f8d-" "0000-0000-0000-000000000000\n")
check(d, ["report.md"], True, "complete-but-wrong-fails")
os.remove(os.path.join(d, "c.md"))
# Cell 5: justified-unrecovered placeholder does not count as recovery
# and does not fail.
with open(os.path.join(d, "e.md"), "w") as fh:
fh.write("beae0b1f-\u2026 and placeholder beae0b1f-0000-0000-0000-000000000000\n")
check(d, ["report.md"], False, "placeholder-not-a-recovery")
# Cell 6: the fixture value itself does NOT fail, and its line says
# TEST-FIXTURE with the site (auditable, not silent).
with open(os.path.join(d, "f.md"), "w") as fh:
fh.write("3d79ba82-" "1111-2222-3333-444444444444\n")
lines, _ = run_check(d, ["report.md"])
assert any(l.startswith("TEST-FIXTURE 3d79ba82-") and l.endswith("at f.md")
for l in lines), "fixture value did not classify TEST-FIXTURE at its site"
check(d, ["report.md"], False, "fixture-value-passes")
os.remove(os.path.join(d, "f.md"))
# Cell 9 (Q96 exhausted-stem flip): a full value arriving under
# the exhausted beae0b1f stem reports RECOVERED with its site.
# Built by concatenation, as in cell 4, so no second UUID-shaped
# literal enters this source. Since Q117 this must be a NON-probe
# value: Timmy's probe-C value no longer counts as a recovery
# (cell 11 owns that case), so the flip is demonstrated with a
# synthetic value that appears in no table.
with open(os.path.join(d, "g.md"), "w") as fh:
fh.write("beae0b1f-" "1111-2222-3333-444444444444\n")
lines, _ = run_check(d, ["report.md"])
assert any(l.startswith("TOKEN beae0b1f: RECOVERED-IN-TREE")
and l.endswith("at g.md") for l in lines), \
"full value under exhausted stem did not report RECOVERED"
check(d, ["report.md"], False, "exhausted-stem-recovers")
os.remove(os.path.join(d, "g.md"))
# Cell 10 (Q121): the absence leg must see each table entry on its
# own. This builds the case the joined-string form could not express:
# a two-entry table (the real fixture plus one synthetic stem) with
# an entry's literal present in the source. The per-entry scan must
# name exactly the inserted stem; the joined form named nothing,
# because value1+value2 occurs nowhere. Both insertions are measured
# separately, so the cell cannot pass by checking one table position.
two_entry = dict(KNOWN_FIXTURE)
two_entry["5e0c7a91"] = "5e0c7a91-" "abcd-1234-5678-90abcdef1234"
assert len(two_entry) == 2, "cell 10 needs a two-entry table"
assert fixture_literals_in_source(source_bytes, two_entry) == [], \
"clean source already reports a fixture literal under a two-entry table"
for stem in sorted(two_entry):
want = "%s (len %d)" % (stem, len(two_entry[stem]))
mutant = source_bytes + ("\n# stray re-commit: %s\n" % two_entry[stem]).encode("utf-8")
got = fixture_literals_in_source(mutant, two_entry)
assert got == [want], "insertion of %s reported %s, want [%s]" % (stem, got, want)
both = source_bytes + ("\n".join("# %s" % v for v in two_entry.values())).encode("utf-8")
want_both = sorted("%s (len %d)" % (k, len(two_entry[k])) for k in two_entry)
assert fixture_literals_in_source(both, two_entry) == want_both, \
"both-inserted scan did not name both stems"
passed.append("absence-scan-per-entry")
# Cell 11 (Q117): a record-quoted probe value under the exhausted
# stem does NOT report RECOVERED. The tree holds the truncated
# stem plus Timmy's probe-C value (built by concatenation, as in
# cell 4); with KNOWN_PROBE in place the stem must read
# UNRECOVERED-JUSTIFIED and the gate must stay green. Then the
# flip-back mutant: with the probe value removed from the table
# the SAME tree must read RECOVERED-IN-TREE, proving the table
# is what suppresses it -- otherwise this cell would be vacuous
# (a tree with no recovery at all reads UNRECOVERED either way).
probe_val = KNOWN_PROBE["beae0b1f"]
with open(os.path.join(d, "h.md"), "w") as fh:
fh.write("beae0b1f-\u2026 and probe %s\n" % probe_val)
lines, failed = run_check(d, ["report.md"])
assert any(l.startswith("TOKEN beae0b1f: UNRECOVERED-JUSTIFIED")
for l in lines), "probe tree did not read UNRECOVERED-JUSTIFIED: %s" % lines
assert not any(l.startswith("TOKEN beae0b1f: RECOVERED")
for l in lines), "probe value reported as recovery"
assert not failed, "probe tree failed the gate"
saved_probe = dict(KNOWN_PROBE)
try:
KNOWN_PROBE.clear()
lines, _ = run_check(d, ["report.md"])
finally:
KNOWN_PROBE.update(saved_probe)
assert any(l.startswith("TOKEN beae0b1f: RECOVERED-IN-TREE")
and l.endswith("at h.md") for l in lines), \
"flip-back mutant did not recover without the table: %s" % lines
assert KNOWN_PROBE == saved_probe, "probe table not restored after mutant"
os.remove(os.path.join(d, "h.md"))
passed.append("probe-value-not-a-recovery")
# Cell 7 (the sixth cell of reply 113, renumbered by cell 6): run the
# check over the checker's own directory and assert rc 0. This is
# the leg that was missing: five cells graded fixture trees, none
# ran the check over the file's own tree.
own_dir = os.path.dirname(os.path.abspath(__file__))
lines, failed = run_check(own_dir, [])
assert not failed, "self-scan failed: %s" % [l for l in lines if l.startswith("FAIL")]
passed.append("self-scan-passes")
for label in passed:
print("SELFTEST PASS %s" % label)
print("SELFTEST: %d cells passed, 0 failed" % len(passed))
return 0
def main(argv=None):
ap = argparse.ArgumentParser()
ap.add_argument("--tree", default=".")
ap.add_argument("--exclude", action="append", default=[])
ap.add_argument("--selftest", action="store_true")
args = ap.parse_args(argv)
if args.selftest:
return selftest()
lines, failed = run_check(args.tree, args.exclude)
for line in lines:
print(line)
return 1 if failed else 0
if __name__ == "__main__":
sys.exit(main())