volt
ed12a9b9d9
The ledger attach loop already visits every MISSING row, and the old return statement then re-walked missing.values() three more times to recover matched, new and tolerated. Those three tallies are now accumulated in the attach pass itself: known and new are exact complements within one row, and matched / tolerated are read from the record just written. Four traversals of the row set become one. Output is unchanged, and that is measured rather than asserted. Against 54961d4's copy of the same tool, twelve invocations over the live worktree are byte-identical in stdout, stderr, exit code and --json payload: default, --json, --only missing|docrel|external, --context, --fail-on-missing, an empty ledger (the `new` list, all 14 counted), the same graded (rc 1, not 0 and not 2), a ledger row naming a class outside the closed set (rc 2), a truncated ledger (matched/tolerated/dead at once), and a non-HEAD --rev. Selftest is 31/31 on both copies. No speed claim: five timed runs put HEAD at 6.94/6.92s median and this at 6.70/6.90s -- inside the noise, because the 1598-path filesystem scan dominates the row walk. The change buys passes, not milliseconds.
893 lines
45 KiB
Python
893 lines
45 KiB
Python
#!/usr/bin/env python3
|
|
"""path_claim_check.py -- does every in-namespace path this archive names exist
|
|
at this revision?
|
|
|
|
WHY THIS EXISTS
|
|
`citation_resolve_check.py` grades `<path>:<line>` citations: it can resolve
|
|
them because a line number makes the claim decidable. A path named WITHOUT a
|
|
line -- "`recoveries/foo.md` (10,232 B)", "the record `docs/bar.md`" -- is the
|
|
same kind of claim with one fewer field, and nothing here graded it. The
|
|
defect is on the record: checkins/2026-10-19-volt.md records a report that
|
|
"named `check.sh` and `files.sha256`" and never wrote them, and the ledger's
|
|
rule is that a reader must be able to go and look. A path a reader cannot
|
|
open is a citation that fails the only test a stranger can run.
|
|
|
|
The measurement that made this tool possible also bounded it. Over the whole
|
|
corpus, most distinct path tokens are not candidates for "this archive holds
|
|
that file" at all: they begin with a component this repo does not track --
|
|
scratch trees (`after/verify.txt`), host artifacts (`/root/crucible/...`),
|
|
other repositories, served routes and URLs. Grading those as missing would be
|
|
hundreds of false defects, so they are COUNTED and never graded. What is left
|
|
is small enough for a reader: every MISSING path at the revision the ledger
|
|
was last reviewed against is in `tools/path_claim_baseline.tsv` with the class
|
|
that explains it, and `--fail-on-missing` fails on any the ledger does not
|
|
name -- so a revision that moves the corpus reds the gate until a reader has
|
|
looked. The census print is what says which revision that was.
|
|
|
|
This docstring deliberately carries no census figures. The first draft did,
|
|
and the leading-slash fix in the matcher moved every one of them (1,024
|
|
distinct tokens became 1,036) -- the defect class this archive grades as a
|
|
stale claim. The tool prints its own census on every run; that print is the
|
|
only place those numbers live.
|
|
|
|
WHAT IT MEASURES -- one verdict per DISTINCT path (unit: distinct paths)
|
|
EXISTS the path is in the tree at --rev.
|
|
DOCREL the path is NOT in the tree at the repo root, but it resolves
|
|
relative to the document that cites it -- an artifact's own
|
|
`tools/fetch.py` named beside its `REPORT.md`. Printed with the
|
|
path it resolves to and the cite that resolved it, so a reader can
|
|
go and look at the base the tool used. Counted as resolved, not as
|
|
a defect: the file it names exists.
|
|
MISSING the path's first component IS a tracked top-level entry, so the
|
|
prose is naming something inside this repository, and neither base
|
|
holds it -- not the tree at --rev, and not the citing document's
|
|
own directory. This is the graded class.
|
|
EXTERNAL the first component is not tracked here (scratch tree, host path,
|
|
URL, other repo). OUT OF SCOPE: the path may be perfectly true and
|
|
this tool cannot see it. Counted, never graded -- the same
|
|
vocabulary `citation_resolve_check.py` uses for EXTERNAL_PATH.
|
|
TEMPLATE the path carries a placeholder (`YYYY`, `<name>`, `{n}`), so it is
|
|
a pattern rather than a claim about one file.
|
|
|
|
Occurrences are counted beside distinct paths, because one wrong path cited
|
|
16 times is one defect and 16 chances for a reader to trip on it.
|
|
|
|
WHY THERE ARE TWO BASES, AND WHY THE ROOT ONE WINS
|
|
The corpus writes most paths relative to the repository root, so the root base
|
|
is the contract and it is tried first, always. But an artifact inside
|
|
`wizard-work/` writes its own inventory the way a reader sitting in that
|
|
directory would: `tools/fetch.py` beside `REPORT.md` is the artifact's fetch
|
|
script. Graded at the root alone, exactly those citations read MISSING while
|
|
the files sat in that artifact's own `tools/` directory (the live case:
|
|
`wizard-work/sphinx/longctx-needle-grounding-2026-10-22/REPORT.md:49-52`) --
|
|
a tool reporting a base mismatch as a defect teaches writers to stop naming
|
|
files, which is the worse failure. So the second base exists, it is used ONLY
|
|
when the root base missed, and the verdict it produces is a different word
|
|
(DOCREL) that carries the resolved path and the resolving cite. A root-
|
|
relative citation is never rewritten by it, and a token that resolves under
|
|
neither base is still MISSING.
|
|
|
|
A FENCED BLOCK IS QUOTED MATERIAL, NOT THIS DOCUMENT'S PROSE -- the rule
|
|
`queue_index.py`, `checkin_check.py` and `citation_resolve_check.py` already
|
|
apply to headings and citations. A fenced listing that quotes a path is not
|
|
the containing document asserting that path exists. Skipped lines are printed
|
|
on every run, never silently dropped.
|
|
|
|
THE BASELINE, AND WHY THE FIRST CENSUS IS ITS LEDGER
|
|
`<path>\\t<CLASS>\\t<reason>`, one row per reviewed missing path, class from a
|
|
closed set (PLANTED, NOT_ADDED, RENAMED, SHORT_NAME, FOREIGN_TREE, EXAMPLE,
|
|
TARGET).
|
|
A row is a stated reading, not an exemption: `--fail-on-missing` fails only on
|
|
a MISSING path the ledger does not name, and every tolerated row is still
|
|
PRINTED with its class on every run, so a reader sees the tolerated rows
|
|
rather than trusting that there are none. A ledger row whose path now exists
|
|
is reported CLEARED, and a row naming a path the census no longer produces is
|
|
reported as a dead row -- the ledger cannot rot invisibly.
|
|
|
|
TARGET is the seventh class, added 2026-10-21 for the case the first six did
|
|
not cover: a path named in an IMPERATIVE sentence -- "Post
|
|
`checkins/2026-10-21-spark.md` (Did / Blocked / Next / Ask)" -- which asks its
|
|
owner to create the file rather than asserting that one exists. `git log --all
|
|
--diff-filter=A -- checkins/2026-10-21-spark.md` returns 0 commits, so no ref
|
|
ever held it, and the artifact is genuinely owed; but a repository whose
|
|
deliverables cannot be named without reddening the census teaches writers to
|
|
stop naming them, which is the worse failure. The row must name the asking
|
|
file and line and the zero-commit reading, so it is a citation a reader can
|
|
check rather than a word that closes a red. A claim that something EXISTS is
|
|
still MISSING until the file does.
|
|
|
|
REVISIONS -- AND WHY `WORKTREE` IS NOT ONE OF THEM
|
|
`--rev` selects the tree the corpus is read from AND the tree existence is
|
|
judged against, so the claim and its oracle come from the same revision. The
|
|
default is `HEAD`.
|
|
|
|
`--rev WORKTREE` reads the files on disk and lists them from the INDEX, so it
|
|
grades the tree you are about to commit. It is not a revision and has no tree
|
|
object: `git rev-parse --verify WORKTREE^{commit}` fails, which is why this
|
|
flag could not be spelled until now. The two modes were measured apart in the
|
|
selftest before this paragraph existed, on one fixture document: a path claim
|
|
that lives only in the staged tree is invisible at `HEAD` (rc 0, no MISSING
|
|
row) and is a graded MISSING row under `WORKTREE` (rc 1 under
|
|
--fail-on-missing). That window -- the tick between writing prose and
|
|
committing it -- is exactly where the archive writes every claim it makes, so
|
|
a grader that could only read HEAD could not grade a lane's own tick. The
|
|
convention is `doc_command_check.py`'s and `tool_map.py`'s, and the reason is
|
|
the same in all three: grade the tree you are about to commit, not the last
|
|
one.
|
|
|
|
A path that exists on disk but is in neither the index nor HEAD is still
|
|
MISSING under `WORKTREE`: the inventory is the index, so `git add` before the
|
|
gate is the discipline, not a workaround. Untracked debris under a scratch
|
|
tree therefore cannot make a claim read EXISTS.
|
|
|
|
EXIT CODES
|
|
0 the census ran. Without --fail-on-missing a MISSING path is not a fault.
|
|
1 --fail-on-missing and at least one MISSING path is not in the baseline, or
|
|
names a class the ledger does not tolerate.
|
|
2 could not grade: not a git work tree, a revision that does not resolve, a
|
|
corpus with no markdown, a NAMED baseline that cannot be read or names an
|
|
unknown class. Never 0, and never a silent empty census. A repo that
|
|
carries no ledger of its own is not this: `--baseline` defaults to
|
|
tools/path_claim_baseline.tsv resolved against --repo, and a tree without
|
|
that file is graded with no ledger after one note on stderr.
|
|
|
|
USAGE
|
|
python3 path_claim_check.py [--repo DIR] [--rev HEAD|WORKTREE] [--corpus PATH...]
|
|
[--json OUT] [--only missing|exists|docrel|external|template|all]
|
|
[--context] [--fail-on-missing] [--baseline PATH]
|
|
--baseline defaults to tools/path_claim_baseline.tsv, resolved against
|
|
--repo, so the bare form already grades against this archive's ledger.
|
|
python3 path_claim_check.py --selftest # 31 cells
|
|
python3 path_claim_check.py --repo . --fail-on-missing
|
|
python3 path_claim_check.py --repo . --rev WORKTREE --fail-on-missing
|
|
python3 path_claim_check.py --repo . --only docrel --context
|
|
"""
|
|
import argparse
|
|
import collections
|
|
import json
|
|
import os
|
|
import posixpath
|
|
import re
|
|
import subprocess
|
|
import sys
|
|
|
|
DEFAULT_REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
|
|
|
# The ledger this archive carries, REPO-relative: resolved against --repo, not
|
|
# against the caller's cwd, so a bare run against a clone reads that clone's
|
|
# ledger. Until 2026-10-21 this was advertised in USAGE and never applied --
|
|
# `--baseline` had no default, so a bare invocation graded against an empty
|
|
# ledger and, under --fail-on-missing, returned rc 1 for an archive whose own
|
|
# rows excuse seven of the ten MISSING paths. A default a reader can cite and
|
|
# the code does not apply is the worst kind: it makes the tool lie in the
|
|
# direction of a defect.
|
|
DEFAULT_BASELINE = "tools/path_claim_baseline.tsv"
|
|
|
|
# The pseudo-revision `doc_command_check.py` and `tool_map.py` already carry.
|
|
# Spelled once, here, so a reader greps one name and finds every branch that
|
|
# treats it as a state rather than as a commit.
|
|
WORKTREE = "WORKTREE"
|
|
|
|
# The extensions this archive actually names. A closed list, for the reason
|
|
# citation_resolve_check.py gives: an open `\\.[a-z]+` swallows clock times,
|
|
# decimals and domain names.
|
|
EXT = (r"md|markdown|py|json|txt|js|mjs|cjs|sh|bash|yml|yaml|html|css|csv|tsv|"
|
|
r"jsonl|sql|db|sqlite|age|wav|mp3|mp4|gz|tgz|toml|cfg|ini|log|out|bak")
|
|
|
|
# At least one '/' is REQUIRED: a bare filename ("index.html") is not a claim
|
|
# about a location, and the archive names dozens of those for pages it does not
|
|
# hold. The trailing guard stops `x.md` from matching the front of `x.md5`.
|
|
PATH = re.compile(r"(?P<path>/?(?:[A-Za-z0-9_.@+-]+/)+[A-Za-z0-9_.@+-]+\.(?:%s))"
|
|
r"(?![A-Za-z0-9])" % EXT)
|
|
FENCE = re.compile(r"^\s*(```|~~~)")
|
|
HOST = re.compile(r"^(?:/|~/|\.\./)")
|
|
TEMPLATE = re.compile(r"YYYY|<[^>]*>|\{[^}]*\}|NNN")
|
|
STRIP = "`*'\"()[]<>,;:."
|
|
|
|
# The closed class set a baseline row may name. Each is a reason a MISSING path
|
|
# is not a live defect; adding one is a deliberate edit to this tuple.
|
|
# EXAMPLE was added after the tool's own construction record landed and the next
|
|
# census found 3 new MISSING paths -- two quoted metasyntactic illustrations
|
|
# ("recoveries/foo.md", "the record docs/bar.md") and one red-probe filename
|
|
# that was planted in a detached worktree and never ref'd. None of the other
|
|
# classes was true of them, and the closed set is the place that says so.
|
|
# TARGET was added 2026-10-21 by the same rule: the census's next tick found
|
|
# `checkins/2026-10-21-spark.md`, cited by replies/2026-10-21-to-spark-06.md:26
|
|
# in an imperative ("Post ... (Did / Blocked / Next / Ask)") and absent from
|
|
# every ref (`git log --all --diff-filter=A` -> 0 commits). It is not a broken
|
|
# citation; it is a deliverable somebody owes, which is a different thing.
|
|
BASELINE_CLASSES = ("PLANTED", "NOT_ADDED", "RENAMED", "SHORT_NAME",
|
|
"FOREIGN_TREE", "EXAMPLE", "TARGET")
|
|
# DOCREL sits between the two graded outcomes on purpose: it is a resolution,
|
|
# reported beside EXISTS rather than folded into it, because the base it used is
|
|
# part of the claim and a reader has to be able to see that.
|
|
VERDICTS = ("EXISTS", "DOCREL", "MISSING", "EXTERNAL", "TEMPLATE")
|
|
# Which resolution wins when one path is cited from more than one document: a
|
|
# root hit beats a document-relative hit beats no hit. EXTERNAL/TEMPLATE never
|
|
# meet the others (they are a property of the token, not of the tree).
|
|
RESOLUTION_RANK = {"MISSING": 0, "DOCREL": 1, "EXISTS": 2}
|
|
|
|
|
|
class CannotGrade(Exception):
|
|
"""An input this tool refuses to grade: the caller turns it into rc 2."""
|
|
|
|
|
|
def git(repo, *args):
|
|
return subprocess.run(["git", "-C", repo, *args], capture_output=True)
|
|
|
|
|
|
def precondition(repo, rev):
|
|
"""None when `repo` is a git work tree carrying `rev`, else why not.
|
|
|
|
Every reader below swallows git's status, so an unreadable repo or a
|
|
mistyped revision would come back as an empty tree and print a clean census
|
|
of nothing -- the silent-pass class selftest_sweep.py's malformed axis
|
|
exists to catch.
|
|
"""
|
|
if not os.path.isdir(repo):
|
|
return "cannot read repo %s: no such directory" % repo
|
|
p = git(repo, "rev-parse", "--is-inside-work-tree")
|
|
if p.returncode != 0 or p.stdout.strip() != b"true":
|
|
return "cannot read repo %s: not a git work tree" % repo
|
|
if rev == WORKTREE:
|
|
# WORKTREE is a state, not a revision: it has no commit to resolve to.
|
|
# Everything below reads the index (inventory) and the disk (bytes), so
|
|
# the work-tree test just made is the whole precondition.
|
|
return None
|
|
q = git(repo, "rev-parse", "--verify", rev + "^{commit}")
|
|
if q.returncode != 0:
|
|
why = q.stderr.decode("utf-8", "replace").strip().splitlines()
|
|
return "cannot read %s at %s: %s" % (repo, rev, why[-1] if why else "git rev-parse failed")
|
|
return None
|
|
|
|
|
|
def clean(token):
|
|
"""Strip the punctuation a path wears when it sits in prose or in a span."""
|
|
p = token.strip(STRIP)
|
|
while p.startswith("./"):
|
|
p = p[2:].strip(STRIP)
|
|
return p
|
|
|
|
|
|
def classify(path, tracked, tops):
|
|
"""The verdict at the repo-root base alone. DOCREL needs a second base."""
|
|
if TEMPLATE.search(path):
|
|
return "TEMPLATE"
|
|
if HOST.match(path) or path.split("/")[0] not in tops:
|
|
return "EXTERNAL"
|
|
return "EXISTS" if path in tracked else "MISSING"
|
|
|
|
|
|
def doc_relative(path, where, tracked):
|
|
"""The tracked path this document's own directory resolves `path` to, or None.
|
|
|
|
`where` is the citing document's repo-relative name; a document at the root
|
|
has no second base distinct from the first, so it returns None and root
|
|
documents keep the single-base contract. A candidate that climbs out of the
|
|
tree is not a base this tool will stand on.
|
|
"""
|
|
if not where or "/" not in where:
|
|
return None
|
|
cand = posixpath.normpath(posixpath.join(posixpath.dirname(where), path))
|
|
if cand == ".." or cand.startswith("../"):
|
|
return None
|
|
return cand if cand in tracked else None
|
|
|
|
|
|
def fence_mask(text):
|
|
"""1-based line numbers that are a fence marker or sit inside a fence."""
|
|
out, in_fence = set(), False
|
|
for i, raw in enumerate(text.split("\n"), 1):
|
|
if FENCE.match(raw):
|
|
out.add(i)
|
|
in_fence = not in_fence
|
|
continue
|
|
if in_fence:
|
|
out.add(i)
|
|
return out
|
|
|
|
|
|
def scan_text(text, where, tracked, tops):
|
|
"""Rows for one document -> ({path: row}, counter).
|
|
|
|
`where` is the document's name, used for the first-cite handle and as the
|
|
second base: a token the root base misses is tried beside the document
|
|
(`doc_relative`) before it is called MISSING. Line numbers are the
|
|
document's own, so a row can be read back at --rev.
|
|
"""
|
|
rows, stats = {}, collections.Counter()
|
|
mask = fence_mask(text)
|
|
stats["fenced_lines"] = len(mask)
|
|
for i, line in enumerate(text.split("\n"), 1):
|
|
if i in mask:
|
|
continue
|
|
for m in PATH.finditer(line):
|
|
p = clean(m.group("path"))
|
|
if not p:
|
|
continue
|
|
stats["occurrences"] += 1
|
|
verdict = classify(p, tracked, tops)
|
|
resolved = None
|
|
if verdict == "MISSING":
|
|
resolved = doc_relative(p, where, tracked)
|
|
if resolved:
|
|
verdict = "DOCREL"
|
|
row = rows.get(p)
|
|
if row is None:
|
|
row = rows[p] = {"path": p, "mentions": 0,
|
|
"first": "%s:%d" % (where, i),
|
|
"context": line.strip()[:180],
|
|
"verdict": verdict,
|
|
"resolved": resolved,
|
|
"resolved_from": ("%s:%d" % (where, i)
|
|
if resolved else None),
|
|
"unresolved": 0}
|
|
row["mentions"] += 1
|
|
if verdict == "MISSING":
|
|
# One resolving cite does not erase the cites that did not
|
|
# resolve at their own base; the count travels with the row.
|
|
row["unresolved"] += 1
|
|
if (RESOLUTION_RANK.get(verdict, -1) >
|
|
RESOLUTION_RANK.get(row["verdict"], -1)):
|
|
row["verdict"] = verdict
|
|
if resolved:
|
|
row["resolved"] = resolved
|
|
row["resolved_from"] = "%s:%d" % (where, i)
|
|
return rows, stats
|
|
|
|
|
|
def scan(repo, rev, corpus, store):
|
|
"""Every document's rows, merged by path -- the same scan_text() the fixtures
|
|
are graded by, so no second matcher exists to drift from this one."""
|
|
all_rows, stats = {}, collections.Counter()
|
|
for f in corpus:
|
|
text = store.read(f).decode("utf-8", "replace")
|
|
rows, st = scan_text(text, f, store.tree, store.tops)
|
|
stats.update(st)
|
|
for p, row in rows.items():
|
|
if p in all_rows:
|
|
cur = all_rows[p]
|
|
cur["mentions"] += row["mentions"]
|
|
cur["unresolved"] += row["unresolved"]
|
|
if (RESOLUTION_RANK.get(row["verdict"], -1) >
|
|
RESOLUTION_RANK.get(cur["verdict"], -1)):
|
|
cur["verdict"] = row["verdict"]
|
|
cur["resolved"] = row.get("resolved")
|
|
cur["resolved_from"] = row.get("resolved_from")
|
|
else:
|
|
all_rows[p] = row
|
|
return all_rows, stats
|
|
|
|
|
|
class Store:
|
|
"""The inventory and the bytes, at one revision or at the staged work tree.
|
|
|
|
WORKTREE is two reads, not one: the INVENTORY is the index (`ls-files`), so
|
|
only what will actually be committed can make a claim read EXISTS, and the
|
|
BYTES are the disk, so an edit that is not staged yet is still graded as the
|
|
text a reader will see. A path the index lists but the disk has lost falls
|
|
back to the index blob -- that is byte-for-byte what the commit would carry,
|
|
and it is the only honest answer to "does this exist in the tree I am about
|
|
to commit".
|
|
"""
|
|
|
|
def __init__(self, repo, rev):
|
|
self.repo, self.rev = repo, rev
|
|
if rev == WORKTREE:
|
|
raw = git(repo, "ls-files", "-z").stdout.decode("utf-8", "replace")
|
|
names = [p for p in raw.split("\0") if p]
|
|
else:
|
|
names = git(repo, "ls-tree", "-r", "--name-only", rev) \
|
|
.stdout.decode("utf-8", "replace").splitlines()
|
|
self.tree = set(names)
|
|
self.tops = {p.split("/")[0] for p in self.tree}
|
|
|
|
def read(self, path):
|
|
if self.rev == WORKTREE:
|
|
try:
|
|
with open(os.path.join(self.repo, path), "rb") as fh:
|
|
return fh.read()
|
|
except OSError:
|
|
p = git(self.repo, "show", ":" + path)
|
|
if p.returncode != 0:
|
|
raise CannotGrade("cannot read %s at WORKTREE" % path)
|
|
return p.stdout
|
|
p = git(self.repo, "show", "%s:%s" % (self.rev, path))
|
|
if p.returncode != 0:
|
|
raise CannotGrade("cannot read %s at %s" % (path, self.rev))
|
|
return p.stdout
|
|
|
|
|
|
def load_baseline(path):
|
|
"""`<path>\\t<CLASS>\\t<reason>` -> {path: (class, reason)}. Unreadable is rc 2."""
|
|
try:
|
|
with open(path, "r", encoding="utf-8") as fh:
|
|
text = fh.read()
|
|
except OSError as exc:
|
|
raise CannotGrade("cannot read baseline %s: %s" % (path, exc))
|
|
except UnicodeDecodeError as exc:
|
|
raise CannotGrade("cannot read baseline %s: not UTF-8: %s" % (path, exc))
|
|
out = {}
|
|
for n, line in enumerate(text.splitlines(), 1):
|
|
s = line.rstrip("\r")
|
|
if not s.strip() or s.lstrip().startswith("#"):
|
|
continue
|
|
parts = s.split("\t")
|
|
if len(parts) != 3:
|
|
raise CannotGrade("cannot read baseline %s:%d: not "
|
|
"<path>\\t<CLASS>\\t<reason>: %r" % (path, n, s))
|
|
name, cls, reason = (x.strip() for x in parts)
|
|
if not name or not reason:
|
|
raise CannotGrade("cannot read baseline %s:%d: empty path or reason: %r"
|
|
% (path, n, s))
|
|
if cls not in BASELINE_CLASSES:
|
|
raise CannotGrade("cannot read baseline %s:%d: unknown class %r "
|
|
"(known: %s)" % (path, n, cls, ", ".join(BASELINE_CLASSES)))
|
|
out[name] = (cls, reason)
|
|
return out
|
|
|
|
|
|
def apply_baseline(rows, ledger):
|
|
"""Attach the ledger to each MISSING row; report what moved on both sides."""
|
|
missing = {p: r for p, r in rows.items() if r["verdict"] == "MISSING"}
|
|
# A single FUSED census pass does both jobs the original split across four
|
|
# traversals: it attaches each row's baseline record AND accumulates the
|
|
# matched count, the new-path list, and the tolerated class Counter in the
|
|
# same iteration. `known` and `new` are exact complements within one row, so
|
|
# the tally is read from the record just written -- no second pass is needed.
|
|
# The report renders `new` sorted and `tolerated` via sorted(.items()), so
|
|
# neither traversal order nor the merge can move a byte of output.
|
|
matched = 0
|
|
new = []
|
|
tolerated = collections.Counter()
|
|
for p, row in missing.items():
|
|
if p in ledger:
|
|
cls, reason = ledger[p]
|
|
row["baseline"] = {"known": True, "class": cls,
|
|
"reason": reason, "new": False}
|
|
matched += 1
|
|
tolerated[cls] += 1
|
|
else:
|
|
row["baseline"] = {"known": False, "class": None,
|
|
"reason": None, "new": True}
|
|
new.append(p)
|
|
cleared = sorted(p for p in ledger if p in rows and rows[p]["verdict"] == "EXISTS")
|
|
# A row the second base now resolves is cleared too, but it says so: the
|
|
# path is true at the citing document's directory, not at the repo root, and
|
|
# a reader who deletes the row should know which base made it go.
|
|
cleared_docrel = sorted(p for p in ledger
|
|
if p in rows and rows[p]["verdict"] == "DOCREL")
|
|
dead = sorted(p for p in ledger if p not in rows)
|
|
return {"entries": len(ledger), "matched": matched,
|
|
"new": sorted(new),
|
|
"cleared": cleared, "cleared_docrel": cleared_docrel, "dead": dead,
|
|
"tolerated": tolerated}
|
|
|
|
|
|
def report(rows, stats, rev, sha, corpus, ledger, path, only, context):
|
|
counts = collections.Counter(r["verdict"] for r in rows.values())
|
|
# WORKTREE has no commit of its own, so the sha printed is the HEAD it sits
|
|
# on, labelled as such: a reader must not mistake it for the graded tree.
|
|
where = ("HEAD " + sha[:7] + ", worktree graded") if rev == WORKTREE else sha[:7]
|
|
print("path_claim_check: %s at %s (%s)" % ("repo", rev, where))
|
|
print("matcher: /?(?:[A-Za-z0-9_.@+-]+/)+[A-Za-z0-9_.@+-]+\\.(closed ext list), "
|
|
"at least one '/' required")
|
|
print(" graded only when the first component is a tracked top-level entry; "
|
|
"fenced blocks skipped")
|
|
print("corpus: %d markdown file(s)" % len(corpus))
|
|
print("paths: %d distinct / %d occurrence(s) (unit: distinct paths; "
|
|
"occurrences beside)" % (len(rows), stats["occurrences"]))
|
|
for v in VERDICTS:
|
|
print(" %-9s %4d distinct" % (v, counts[v]))
|
|
print("fenced skipped: %d line(s)" % stats["fenced_lines"])
|
|
if ledger is not None:
|
|
print("baseline %s: %d entries, %d matched, %d new, %d cleared, %d dead row(s)"
|
|
% (path, ledger["entries"], ledger["matched"], len(ledger["new"]),
|
|
len(ledger["cleared"]), len(ledger["dead"])))
|
|
for cls, n in sorted(ledger["tolerated"].items()):
|
|
print(" tolerated %-13s %d" % (cls, n))
|
|
for p in ledger["cleared"]:
|
|
print(" CLEARED %s -- ledger row, now in the tree; the row can go" % p)
|
|
for p in ledger["cleared_docrel"]:
|
|
row = rows[p]
|
|
print(" CLEARED_DOCREL %s -- ledger row, now resolves at %s from %s; "
|
|
"the row can go" % (p, row.get("resolved"), row.get("resolved_from")))
|
|
for p in ledger["dead"]:
|
|
print(" DEAD_ROW %s -- ledger row, no longer in the census; the row can go" % p)
|
|
|
|
sel = {"missing": lambda r: r["verdict"] == "MISSING",
|
|
"exists": lambda r: r["verdict"] == "EXISTS",
|
|
"docrel": lambda r: r["verdict"] == "DOCREL",
|
|
"external": lambda r: r["verdict"] == "EXTERNAL",
|
|
"template": lambda r: r["verdict"] == "TEMPLATE",
|
|
"all": lambda r: True}[only]
|
|
show = sorted((r for r in rows.values() if sel(r)),
|
|
key=lambda r: (-r["mentions"], r["path"]))
|
|
print("--- %s: %d distinct path(s) (showing first 300) ---" % (only, len(show)))
|
|
for r in show[:300]:
|
|
tag = ""
|
|
if r["verdict"] == "MISSING" and r.get("baseline"):
|
|
b = r["baseline"]
|
|
tag = (" [%s: %s]" % (b["class"], b["reason"])) if b["known"] \
|
|
else " [NEW -- not in the baseline]"
|
|
elif r["verdict"] == "DOCREL":
|
|
tag = (" [DOCREL: resolves at %s, from %s; %d cite(s) did not resolve "
|
|
"at their own base]"
|
|
% (r.get("resolved"), r.get("resolved_from"), r.get("unresolved", 0)))
|
|
print(" %3dx %-58s first %s%s" % (r["mentions"], r["path"], r["first"], tag))
|
|
if context:
|
|
print(" ...%s..." % r["context"])
|
|
|
|
|
|
def resolve_baseline(repo, path):
|
|
"""The ledger to grade against, or None when this repo carries none.
|
|
|
|
`path` is repo-relative by contract (DEFAULT_BASELINE), so it is joined onto
|
|
`repo` before the existence test: a bare run from another cwd must read the
|
|
tree it is grading, never a file beside the caller. An absolute path is
|
|
taken as given. A caller who NAMED a ledger that is not there gets the path
|
|
back unchanged, so load_baseline refuses it with rc 2 -- a mistyped ledger
|
|
must never read as "no ledger". A repo that simply carries none (a fixture
|
|
tree) is a stated reading: one note on stderr, no ledger, and the census
|
|
still prints, because the count is the finding.
|
|
"""
|
|
if not path:
|
|
return None
|
|
cand = path if os.path.isabs(path) else os.path.join(repo, path)
|
|
if os.path.exists(cand):
|
|
return cand
|
|
if path == DEFAULT_BASELINE:
|
|
print("note: no ledger at %s -- every MISSING path counts as new" % cand,
|
|
file=sys.stderr)
|
|
return None
|
|
return cand
|
|
|
|
|
|
def main():
|
|
ap = argparse.ArgumentParser(description=__doc__,
|
|
formatter_class=argparse.RawDescriptionHelpFormatter)
|
|
ap.add_argument("--repo", default=DEFAULT_REPO)
|
|
ap.add_argument("--rev", default="HEAD")
|
|
ap.add_argument("--corpus", nargs="*")
|
|
ap.add_argument("--json")
|
|
ap.add_argument("--only", choices=["missing", "exists", "docrel", "external",
|
|
"template", "all"], default="missing")
|
|
ap.add_argument("--context", action="store_true")
|
|
ap.add_argument("--fail-on-missing", action="store_true")
|
|
ap.add_argument("--baseline", default=DEFAULT_BASELINE,
|
|
help="ledger to grade MISSING rows against (default: "
|
|
"%s, resolved against --repo)" % DEFAULT_BASELINE)
|
|
ap.add_argument("--selftest", action="store_true")
|
|
a = ap.parse_args()
|
|
|
|
if a.selftest:
|
|
return selftest()
|
|
|
|
problem = precondition(a.repo, a.rev)
|
|
if problem:
|
|
print(problem, file=sys.stderr)
|
|
return 2
|
|
try:
|
|
store = Store(a.repo, a.rev)
|
|
sha = git(a.repo, "rev-parse",
|
|
"HEAD" if a.rev == WORKTREE else a.rev).stdout.decode().strip()
|
|
corpus = a.corpus if a.corpus is not None else \
|
|
sorted(p for p in store.tree if p.endswith((".md", ".markdown")))
|
|
if not corpus:
|
|
raise CannotGrade("cannot grade: no markdown corpus at %s in %s"
|
|
% (a.rev, a.repo))
|
|
ledger = resolve_baseline(a.repo, a.baseline)
|
|
ledger = load_baseline(ledger) if ledger else None
|
|
rows, stats = scan(a.repo, a.rev, corpus, store)
|
|
except CannotGrade as exc:
|
|
print(str(exc), file=sys.stderr)
|
|
return 2
|
|
led = apply_baseline(rows, ledger) if ledger is not None else None
|
|
if led is None:
|
|
# No ledger: every MISSING path is new, and the count is the finding.
|
|
led_view = None
|
|
else:
|
|
led_view = led
|
|
report(rows, stats, a.rev, sha, corpus, led_view, a.baseline, a.only, a.context)
|
|
|
|
missing = [r for r in rows.values() if r["verdict"] == "MISSING"]
|
|
new = len(missing) if led is None else len(led["new"])
|
|
print("\npath_claim_check: %d distinct paths, %d missing, %d new against the baseline"
|
|
% (len(rows), len(missing), new))
|
|
if a.json:
|
|
try:
|
|
with open(a.json, "w") as fh:
|
|
json.dump({"repo": a.repo, "rev": a.rev, "sha": sha, "corpus": corpus,
|
|
"stats": dict(stats), "baseline": led,
|
|
"paths": sorted(rows.values(), key=lambda r: r["path"])},
|
|
fh, indent=1)
|
|
except OSError as exc:
|
|
print("cannot write --json " + a.json + ": " + str(exc), file=sys.stderr)
|
|
return 2
|
|
print("wrote %s" % a.json)
|
|
if a.fail_on_missing and new:
|
|
print("FAIL: %d missing path(s) the baseline does not name" % new)
|
|
return 1
|
|
return 0
|
|
|
|
|
|
# --------------------------------------------------------------- self-test
|
|
# The fixtures are graded by the same scan_text() the corpus is, so a cell that
|
|
# passes here is a statement about the shipped matcher, not about a copy of it.
|
|
TRACKED = {"docs/real.md", "tools/t.py", "notes/n.md", "WORK-QUEUE.md"}
|
|
TOPS = {"docs", "tools", "notes"}
|
|
|
|
|
|
def _verdicts(text):
|
|
rows, _ = scan_text(text, "fixture.md", TRACKED, TOPS)
|
|
return {p: r["verdict"] for p, r in rows.items()}
|
|
|
|
|
|
def _rows(text):
|
|
return scan_text(text, "fixture.md", TRACKED, TOPS)[0]
|
|
|
|
|
|
def selftest():
|
|
n = bad = 0
|
|
|
|
def cell(name, ok, detail=""):
|
|
nonlocal n, bad
|
|
n += 1
|
|
bad += 0 if ok else 1
|
|
print("%s %s%s" % ("PASS" if ok else "FAIL", name,
|
|
"" if ok else " <- " + detail))
|
|
|
|
cell("c1 a tracked path is EXISTS",
|
|
_verdicts("see docs/real.md for the table") == {"docs/real.md": "EXISTS"},
|
|
str(_verdicts("see docs/real.md for the table")))
|
|
cell("c2 an in-namespace path not in the tree is MISSING",
|
|
_verdicts("the record `docs/absent.md` was written") == {"docs/absent.md": "MISSING"},
|
|
str(_verdicts("the record `docs/absent.md` was written")))
|
|
cell("c3 a component this repo does not track is EXTERNAL, not MISSING",
|
|
_verdicts("kept at crucible/q51/helm-selftest-q51.py") ==
|
|
{"crucible/q51/helm-selftest-q51.py": "EXTERNAL"},
|
|
str(_verdicts("kept at crucible/q51/helm-selftest-q51.py")))
|
|
cell("c4 an absolute host path is EXTERNAL",
|
|
_verdicts("under /root/crucible/q39/gate.py today") ==
|
|
{"/root/crucible/q39/gate.py": "EXTERNAL"},
|
|
str(_verdicts("under /root/crucible/q39/gate.py today")))
|
|
cell("c5 a placeholder path is TEMPLATE, not MISSING",
|
|
_verdicts("one file per day: checkins/YYYY-MM-DD-volt.md") ==
|
|
{"checkins/YYYY-MM-DD-volt.md": "TEMPLATE"},
|
|
str(_verdicts("one file per day: checkins/YYYY-MM-DD-volt.md")))
|
|
cell("c6 a fenced path is quoted material, not a claim",
|
|
_verdicts("```\ndocs/absent.md\n```") == {},
|
|
str(_verdicts("```\ndocs/absent.md\n```")))
|
|
cell("c7 the fence closes: the same path in prose is graded again",
|
|
_verdicts("```\ndocs/absent.md\n```\nsee docs/absent.md") ==
|
|
{"docs/absent.md": "MISSING"},
|
|
str(_verdicts("```\ndocs/absent.md\n```\nsee docs/absent.md")))
|
|
cell("c8 a bare filename with no directory is not a path claim",
|
|
_verdicts("the page index.html and blocks/index.html") ==
|
|
{"blocks/index.html": "EXISTS" if "blocks/index.html" in TRACKED else "EXTERNAL"},
|
|
str(_verdicts("the page index.html and blocks/index.html")))
|
|
cell("c9 trailing punctuation and backticks are stripped",
|
|
list(_verdicts("`docs/absent.md`.").keys()) == ["docs/absent.md"],
|
|
str(_verdicts("`docs/absent.md`.")))
|
|
r = _rows("docs/absent.md once, then docs/absent.md again")
|
|
cell("c10 occurrences aggregate onto one row with its first cite",
|
|
len(r) == 1 and r["docs/absent.md"]["mentions"] == 2
|
|
and r["docs/absent.md"]["first"] == "fixture.md:1", str(r))
|
|
|
|
# --- the CLI contract, on a real fixture repo -------------------------
|
|
import tempfile
|
|
td = tempfile.mkdtemp(prefix="pathclaim-")
|
|
repo = os.path.join(td, "repo")
|
|
os.makedirs(os.path.join(repo, "docs"))
|
|
os.makedirs(os.path.join(repo, "tools"))
|
|
with open(os.path.join(repo, "docs", "real.md"), "w") as fh:
|
|
fh.write("real\n")
|
|
with open(os.path.join(repo, "tools", "t.py"), "w") as fh:
|
|
fh.write("# t\n")
|
|
with open(os.path.join(repo, "WORK-QUEUE.md"), "w") as fh:
|
|
fh.write("the record `docs/absent.md` and the real docs/real.md\n")
|
|
subprocess.run(["git", "init", "-q", repo], capture_output=True)
|
|
subprocess.run(["git", "-C", repo, "add", "-A"], capture_output=True)
|
|
subprocess.run(["git", "-C", repo, "-c", "user.email=t@t", "-c", "user.name=t",
|
|
"commit", "-qm", "fixture"], capture_output=True)
|
|
|
|
def run(*args):
|
|
p = subprocess.run([sys.executable, os.path.abspath(__file__), *args],
|
|
capture_output=True, text=True)
|
|
return p.returncode, p.stdout + p.stderr
|
|
|
|
rc, out = run("--repo", repo)
|
|
cell("c11 the census alone is rc 0 even with a missing path",
|
|
rc == 0 and "1 missing" in out, "rc=%d %s" % (rc, out[-200:]))
|
|
rc, out = run("--repo", repo, "--fail-on-missing")
|
|
cell("c12 --fail-on-missing is rc 1 on an unbaselined missing path",
|
|
rc == 1 and "FAIL: 1 missing path" in out, "rc=%d %s" % (rc, out[-200:]))
|
|
bl = os.path.join(td, "baseline.tsv")
|
|
with open(bl, "w") as fh:
|
|
fh.write("# fixture\n" + "docs/absent.md\tPLANTED\tfixture row\n")
|
|
rc, out = run("--repo", repo, "--fail-on-missing", "--baseline", bl)
|
|
cell("c13 a baselined missing path is tolerated, and still printed",
|
|
rc == 0 and "PLANTED" in out and "1 matched" in out,
|
|
"rc=%d %s" % (rc, out[-300:]))
|
|
with open(os.path.join(td, "cleared.tsv"), "w") as fh:
|
|
fh.write("docs/real.md\tRENAMED\tfixture: the row is now in the tree\n")
|
|
rc, out = run("--repo", repo, "--baseline", os.path.join(td, "cleared.tsv"))
|
|
cell("c14 a ledger row whose path now exists is CLEARED and named",
|
|
rc == 0 and "CLEARED" in out and "docs/real.md" in out,
|
|
"rc=%d %s" % (rc, out[-300:]))
|
|
rc, out = run("--repo", td)
|
|
cell("c15 a directory that is not a git work tree is rc 2",
|
|
rc == 2 and "not a git work tree" in out, "rc=%d %s" % (rc, out[-200:]))
|
|
rc, out = run("--repo", repo, "--rev", "nosuchrev")
|
|
cell("c16 a revision that does not resolve is rc 2",
|
|
rc == 2 and "cannot read" in out, "rc=%d %s" % (rc, out[-200:]))
|
|
empty = os.path.join(td, "empty")
|
|
os.makedirs(empty)
|
|
with open(os.path.join(empty, "a.txt"), "w") as fh:
|
|
fh.write("no markdown here\n")
|
|
subprocess.run(["git", "init", "-q", empty], capture_output=True)
|
|
subprocess.run(["git", "-C", empty, "add", "-A"], capture_output=True)
|
|
subprocess.run(["git", "-C", empty, "-c", "user.email=t@t", "-c", "user.name=t",
|
|
"commit", "-qm", "fixture"], capture_output=True)
|
|
rc, out = run("--repo", empty)
|
|
cell("c17 a corpus with no markdown is rc 2, never an empty clean census",
|
|
rc == 2 and "no markdown corpus" in out, "rc=%d %s" % (rc, out[-200:]))
|
|
|
|
# --- WORKTREE: the tree you are about to commit, not the last commit ----
|
|
# One fixture document in three states: the committed claim (HEAD), an
|
|
# unstaged edit on disk, and a claim that lives only in the index. Each cell
|
|
# names which of the two reads it measures -- the inventory or the bytes.
|
|
q = os.path.join(repo, "WORK-QUEUE.md")
|
|
with open(q) as fh:
|
|
committed_txt = fh.read()
|
|
with open(q, "w") as fh:
|
|
fh.write("the real docs/real.md, with the committed claim gone\n")
|
|
_, out_h = run("--repo", repo)
|
|
_, out_w = run("--repo", repo, "--rev", "WORKTREE")
|
|
cell("c17a WORKTREE reads the bytes on disk; HEAD reads the blob",
|
|
"1 missing" in out_h and "0 missing" in out_w,
|
|
"HEAD %r | WORKTREE %r" % (out_h[-140:], out_w[-140:]))
|
|
with open(os.path.join(repo, "docs", "new.md"), "w") as fh:
|
|
fh.write("new\n")
|
|
with open(q, "w") as fh:
|
|
fh.write("the record now cites docs/new.md\n")
|
|
subprocess.run(["git", "-C", repo, "add", "-A"], capture_output=True)
|
|
_, out_h = run("--repo", repo, "--only", "exists")
|
|
_, out_w = run("--repo", repo, "--rev", "WORKTREE", "--only", "exists")
|
|
cell("c17b a path staged but never committed is invisible at HEAD, EXISTS under WORKTREE",
|
|
"docs/new.md" not in out_h and "docs/new.md" in out_w,
|
|
"HEAD %r | WORKTREE %r" % (out_h[-200:], out_w[-200:]))
|
|
with open(q, "w") as fh:
|
|
fh.write("the record cites docs/ghost.md\n")
|
|
subprocess.run(["git", "-C", repo, "add", "-A"], capture_output=True)
|
|
rc_h, out_h = run("--repo", repo, "--fail-on-missing", "--baseline", bl)
|
|
rc_w, out_w = run("--repo", repo, "--rev", "WORKTREE",
|
|
"--fail-on-missing", "--baseline", bl)
|
|
cell("c17c a claim only the staged tree makes: rc 0 at HEAD, rc 1 under WORKTREE",
|
|
rc_h == 0 and rc_w == 1 and "FAIL: 1 missing path" in out_w,
|
|
"HEAD rc=%d | WORKTREE rc=%d %s" % (rc_h, rc_w, out_w[-200:]))
|
|
rc, out = run("--repo", td, "--rev", "WORKTREE")
|
|
cell("c17d WORKTREE outside a git work tree is rc 2, not a crash",
|
|
rc == 2 and "not a git work tree" in out, "rc=%d %s" % (rc, out[-200:]))
|
|
# Put the fixture back the way the committed cells below expect it.
|
|
with open(q, "w") as fh:
|
|
fh.write(committed_txt)
|
|
os.remove(os.path.join(repo, "docs", "new.md"))
|
|
subprocess.run(["git", "-C", repo, "add", "-A"], capture_output=True)
|
|
|
|
# --- the ledger's closed set is enforced, not decorative ----------------
|
|
badcls = os.path.join(td, "badclass.tsv")
|
|
with open(badcls, "w") as fh:
|
|
fh.write("docs/absent.md\tMADE_UP\tfixture: a class the tool does not know\n")
|
|
rc, out = run("--repo", repo, "--fail-on-missing", "--baseline", badcls)
|
|
cell("c18 a ledger row naming a class outside the closed set is rc 2",
|
|
rc == 2 and "unknown class" in out and "MADE_UP" in out,
|
|
"rc=%d %s" % (rc, out[-300:]))
|
|
excls = os.path.join(td, "example.tsv")
|
|
with open(excls, "w") as fh:
|
|
fh.write("docs/absent.md\tEXAMPLE\tfixture: a metasyntactic illustration\n")
|
|
rc, out = run("--repo", repo, "--fail-on-missing", "--baseline", excls)
|
|
cell("c19 the EXAMPLE class is accepted and its row still printed",
|
|
rc == 0 and "tolerated EXAMPLE" in out and "1 matched" in out,
|
|
"rc=%d %s" % (rc, out[-300:]))
|
|
tgtcls = os.path.join(td, "target.tsv")
|
|
with open(tgtcls, "w") as fh:
|
|
fh.write("docs/absent.md\tTARGET\tfixture: an owed deliverable\n")
|
|
rc, out = run("--repo", repo, "--fail-on-missing", "--baseline", tgtcls)
|
|
cell("c21 the TARGET class is accepted and its row still printed",
|
|
rc == 0 and "tolerated TARGET" in out and "1 matched" in out,
|
|
"rc=%d %s" % (rc, out[-300:]))
|
|
|
|
# --- the default ledger is the one the tree carries ---------------------
|
|
# Until 2026-10-21 `--baseline` had NO default while USAGE advertised one:
|
|
# a bare run graded against an empty ledger, so `--fail-on-missing` returned
|
|
# rc 1 for a tree whose own rows excused the path. This pins bare == named.
|
|
with open(os.path.join(repo, "tools", "path_claim_baseline.tsv"), "w") as fh:
|
|
fh.write("# fixture: the ledger this tree carries\n"
|
|
"docs/absent.md\tPLANTED\tfixture row, carried by the tree\n")
|
|
subprocess.run(["git", "-C", repo, "add", "-A"], capture_output=True)
|
|
subprocess.run(["git", "-C", repo, "-c", "user.email=t@t", "-c", "user.name=t",
|
|
"commit", "-qm", "ledger"], capture_output=True)
|
|
|
|
def summary(text):
|
|
return [ln for ln in text.splitlines() if ln.startswith("path_claim_check:")]
|
|
|
|
bare_rc, bare_out = run("--repo", repo, "--fail-on-missing")
|
|
named_rc, named_out = run("--repo", repo, "--fail-on-missing", "--baseline",
|
|
os.path.join(repo, "tools", "path_claim_baseline.tsv"))
|
|
cell("c20 the repo's own ledger is the default: bare run == named run",
|
|
bare_rc == named_rc == 0 and summary(bare_out) == summary(named_out)
|
|
and "1 matched" in bare_out,
|
|
"bare rc=%d %r | named rc=%d %r"
|
|
% (bare_rc, summary(bare_out), named_rc, summary(named_out)))
|
|
|
|
# --- the second base: DOCREL, and the root base still winning -------------
|
|
# These are statements about the shipped scan_text()/scan(), not about a
|
|
# copy: the merge is exercised through the same function the corpus uses.
|
|
ART = "wizard-work/art/REPORT.md"
|
|
art_tracked = {"docs/real.md", "tools/t.py", "wizard-work/art/tools/fetch.py"}
|
|
|
|
def _doc(text, where, tracked):
|
|
rows, _ = scan_text(text, where, tracked,
|
|
{"docs", "tools", "notes", "wizard-work"})
|
|
return rows
|
|
|
|
r = _doc("inventory: `tools/fetch.py` beside this report", ART, art_tracked)
|
|
cell("c22 a path the root base misses but the document's own directory holds is DOCREL",
|
|
r.get("tools/fetch.py", {}).get("verdict") == "DOCREL"
|
|
and r["tools/fetch.py"].get("resolved") == "wizard-work/art/tools/fetch.py"
|
|
and r["tools/fetch.py"].get("resolved_from") == ART + ":1", str(r))
|
|
r = _doc("inventory: `tools/fetch.py` at the record", "WORK-QUEUE.md", art_tracked)
|
|
cell("c23 the second base is not available to a document at the repo root",
|
|
r.get("tools/fetch.py", {}).get("verdict") == "MISSING", str(r))
|
|
both = art_tracked | {"tools/fetch.py"}
|
|
r = _doc("inventory: `tools/fetch.py`", ART, both)
|
|
cell("c24 a root-relative hit is never rewritten by the second base",
|
|
r.get("tools/fetch.py", {}).get("verdict") == "EXISTS"
|
|
and r["tools/fetch.py"].get("resolved") is None, str(r))
|
|
r = _doc("inventory: `tools/nope.py`", ART, art_tracked)
|
|
cell("c25 a candidate that resolves under neither base is still MISSING",
|
|
r.get("tools/nope.py", {}).get("verdict") == "MISSING", str(r))
|
|
|
|
class _FakeStore:
|
|
def __init__(self, tree, docs):
|
|
self.tree, self.docs = set(tree), docs
|
|
self.tops = {p.split("/")[0] for p in self.tree}
|
|
|
|
def read(self, path):
|
|
return self.docs[path].encode()
|
|
|
|
merged, _ = scan(".", "HEAD", ["WORK-QUEUE.md", ART],
|
|
_FakeStore(art_tracked,
|
|
{"WORK-QUEUE.md": "quotes `tools/fetch.py`\n",
|
|
ART: "inventory: `tools/fetch.py`\n"}))
|
|
row = merged.get("tools/fetch.py", {})
|
|
cell("c26 one path cited from two documents keeps the resolving base and counts both",
|
|
row.get("verdict") == "DOCREL" and row.get("mentions") == 2
|
|
and row.get("unresolved") == 1 and row.get("resolved_from") == ART + ":1",
|
|
str(row))
|
|
|
|
# --- DOCREL end to end, through the CLI ----------------------------------
|
|
os.makedirs(os.path.join(repo, "wizard-work", "art", "tools"))
|
|
with open(os.path.join(repo, "wizard-work", "art", "tools", "fetch.py"), "w") as fh:
|
|
fh.write("# fetch\n")
|
|
with open(os.path.join(repo, "wizard-work", "art", "REPORT.md"), "w") as fh:
|
|
fh.write("## Evidence chain\n\n- `tools/fetch.py` -- fetch + normalise\n")
|
|
subprocess.run(["git", "-C", repo, "add", "-A"], capture_output=True)
|
|
subprocess.run(["git", "-C", repo, "-c", "user.email=t@t", "-c", "user.name=t",
|
|
"commit", "-qm", "artifact"], capture_output=True)
|
|
rc, out = run("--repo", repo, "--fail-on-missing")
|
|
rc2, out2 = run("--repo", repo, "--only", "docrel")
|
|
cell("c27 the CLI reports a document-relative citation as DOCREL and does not fail on it",
|
|
rc == 0 and "DOCREL 1 distinct" in out
|
|
and "0 new against the baseline" in out
|
|
and rc2 == 0 and "1 distinct path(s)" in out2 and "tools/fetch.py" in out2,
|
|
"rc=%d %r | only rc=%d %r" % (rc, out[-300:], rc2, out2[-300:]))
|
|
|
|
print("selftest: %d cells, %d passed, %d failed" % (n, n - bad, bad))
|
|
return 0 if bad == 0 else 1
|
|
|
|
|
|
if __name__ == "__main__":
|
|
sys.exit(main())
|