stackchain/troupe-standup
2
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

tools (volt): citation_resolve_check.py -- a fenced block is quoted material, not a citation

Browse Source 
The scanner had no fence state, so a fenced code listing carrying `:line` rows
was graded as the containing document's own citation. checkin_check.py and
queue_index.py already refuse to read STRUCTURE out of a fence (a quoted
heading is not a heading); this applies the same rule to a quoted citation.

Measured before the fix, over the whole corpus at a2d9bc0 (345 .md), with an
outside probe that imports nothing from the tool: 2,273 citation rows, 6 of
them on fenced lines (0.26%) -- UNRESOLVED 5, AS_SELF 1, every other class 0 --
and 398 of 7,890 file mentions. The class is real and bounded, which is what
licenses a six-row change.

The mask is per FILE, not per paragraph: a fence can contain blank lines and so
straddle two of this scanner's blank-line blocks, and absolute line numbers make
the join sound. A matched citation or mention on a masked line is counted and
skipped, never silently dropped -- the run prints
  `fenced skipped: N citations, M mentions (unit: rows)`.

Same corpus, same revision, pre-fix build vs post-fix build (witness:
/root/.hermes/wizard-outputs/volt/2026-10-19-fence-cites/verify_fix.py, rc 0):
  citations 2273 -> 2267 ; mentions 7890 -> 7492
  UNRESOLVED 570 -> 565 ; AS_SELF 776 -> 775
  RESOLVES 151->151  STALE_LINE 0->0  EXTERNAL_PATH 318->318
  AS_NAMED 143->143  AMBIGUOUS 57->57  EXTERNAL_NAMED 258->258
  0 rows exist only after the fix, 0 surviving rows changed verdict or target,
  and the 6 removed rows are exactly the 6 fenced rows -- not one prose row.

Selftest 11 -> 14 fixtures: a fenced citation is not graded; the fence CLOSES
(the same citation in prose is graded again, the positive control); an
unterminated fence quotes to end of file. doc_census on this tool: 22 claims,
0 INTERNAL SPLIT, 0 STALE, rc 0. verify.py 3/3 legs rc 0.
This commit is contained in:
timmy committed 2026-10-19 05:07:44 +00:00
1 parent a2d9bc0654
commit efe7ec60d7
1 file changed
+61 -5
Download Patch File Download Diff File Expand all files Collapse all files
tools/citation_resolve_check.py
+61 -5
View File
@@ -101,17 +101,30 @@ WORKED CATCH -- this tool's first finding is one of MY OWN artifacts
python3 citation_resolve_check.py --only unresolved --context | grep volt.md
NOT in scope, stated rather than implied: whether the cited LINE contains the
evidence the sentence claims. That needs the claim, not just the pointer. And a
citation REPRODUCED INSIDE QUOTES is still a citation to this scanner, which
does not parse quotation marks -- so a document that reports a defective
evidence the sentence claims. That needs the claim, not just the pointer. A
citation reproduced inside INLINE quotes is still a citation to this scanner,
which does not parse quotation marks -- so a document that reports a defective
citation verbatim appears to contain it. Measured instance: the landing
check-in for this tool reproduces the defective citations it is reporting,
verbatim, and the scanner counts every one of them.
A FENCED BLOCK IS QUOTED MATERIAL, NOT THIS DOCUMENT'S PROSE -- made 2026-10-19
`tools/checkin_check.py` and `tools/queue_index.py` both refuse to read
structure out of a ``` fence, on the rule that a quoted heading is not a
heading. This scanner had no fence state, so a fenced listing that carries
`path:line` rows was graded as the containing document's own citation. The
question was left open on 2026-10-19 ("a quoted tool output is not a claim")
and is closed here by measurement rather than taste: over the whole corpus at
HEAD the fenced rows are 0.26% of the census -- six out of 2,273. Every one
is a bare `:line` inside a code listing -- a number the listing quotes, not
one the document asserts. The exclusion removes no prose verdict (pre/post row
diff: exactly those six rows, no verdict changed) and the count is PRINTED
rather than silently dropped, so a reader sees what was set aside.
USAGE
python3 citation_resolve_check.py [--repo DIR] [--rev HEAD] [--corpus PATH...]
[--json OUT] [--only stale|asself|unresolved|ambiguous|external|externalnamed|named|all]
[--context] [--selftest] # 11 fixtures
[--context] [--selftest] # 14 fixtures
"""
import argparse
import collections
@@ -139,6 +152,28 @@ PAT = re.compile(
r"(?:`?:(?P<line>[0-9]{1,7})(?:\s*[-\u2013,]\s*(?P<end>[0-9]{1,7}))?)?"
r"|(?<![\dA-Za-z]):(?P<bline>[0-9]{3,7})(?![0-9])" % EXT)
# A FENCED BLOCK IS QUOTED MATERIAL, NOT THIS DOCUMENT'S PROSE (2026-10-19).
# The same rule `queue_index.py` and `checkin_check.py` already apply to headings:
# a quoted structure is not structure. Here the quoted thing is a citation. The
# mask is computed per FILE (not per block) because a fence can contain blank
# lines and so straddle two of `blocks()`' paragraphs; line numbers stay absolute,
# which is what makes the join sound. The count is reported, never silently
# dropped: `fenced skipped` is printed on every run.
FENCE = re.compile(r"^\s*(```|~~~)")
def fence_mask(text):
"""1-based line numbers that are a fence marker or sit inside a fence."""
out, in_fence = set(), False
for i, raw in enumerate(text.split("\n"), 1):
if FENCE.match(raw):
out.add(i)
in_fence = not in_fence
continue
if in_fence:
out.add(i)
return out
def git(repo, *a):
return subprocess.run(["git", "-C", repo, *a], capture_output=True).stdout
@@ -233,10 +268,15 @@ def scan(repo, rev, corpus, store):
stats["corpus_missing"] += 1
continue
text = store.text(p)
mask = fence_mask(text)
for blk_first, blk in blocks(text):
last = None
for m in PAT.finditer(blk):
abs_line = blk_first + blk.count("\n", 0, m.start())
if abs_line in mask: # quoted, not asserted
stats["fenced_mentions" if m.group("path") and m.group("line") is None
else "fenced_citations"] += 1
continue
ctx = blk[max(0, m.start() - 70):m.end() + 70].replace("\n", " ")
if m.group("path"):
last = m.group("path")
@@ -310,14 +350,27 @@ FIXTURES = [
# a range is reported with its end line
(QUEUE, "recoveries/helm-pin-hash-convention.md:85-90",
[("explicit", "RESOLVES", "recoveries/helm-pin-hash-convention.md", 85)]),
# a fenced block is quoted material, not the containing document's prose --
# the same rule queue_index.py and checkin_check.py apply to headings
(QUEUE, "```\ncheckins/2026-10-15-volt.md:12\n```", []),
# ... and the fence CLOSES: the same citation in prose is graded again, the
# positive control that shows the mask is a state and not a blanket refusal
(QUEUE, "```\ncheckins/2026-10-15-volt.md:12\n```\n"
"see checkins/2026-10-15-volt.md:12",
[("explicit", "RESOLVES", "checkins/2026-10-15-volt.md", 12)]),
# an unterminated fence quotes to end of file, the fence's documented cost
(QUEUE, "```\nWORK-QUEUE.md:999999", []),
]
def _expect(here, text, store):
got = []
for _first, blk in blocks(text):
mask = fence_mask(text)
for first, blk in blocks(text):
last = None
for m in PAT.finditer(blk):
if first + blk.count("\n", 0, m.start()) in mask:
continue # quoted, not asserted
if m.group("path"):
last = m.group("path")
if m.group("line") is None:
@@ -388,6 +441,9 @@ def main():
f"check them without inferring the referent)")
print(f"file mentions with no line: {stats['mentions']} (unit: occurrences) "
f"-- these seed the inheritance scope")
print(f"fenced skipped: {stats['fenced_citations']} citations, "
f"{stats['fenced_mentions']} mentions (unit: rows) -- a fenced block is "
f"quoted material, not this document's prose")
by_target = collections.Counter(r["target"] for r in rows
if r["kind"] == "explicit" and r["target"])
print(f"distinct explicitly cited targets: {len(by_target)} (unit: distinct paths)")