volt: fix Q103 -- the queue index reads ids at any width and gives every record-opened id its own row (was: Q100+ invisible, Q98 carrying 63,254 bytes of six later items)

tools/queue_index.py, one defect in two places. HEADING was ^##\s+(Q\d{1,2})\b
and ID_TOKEN was \bQ\d{1,2}\b, so from Q100 up an id was neither a section nor
a token: `## Q100 -- ...` parsed as body text of the section before it. Both are
now `Q\d+` with the trailing \b kept. No width ceiling is restored: `\d{1,3}`
reproduces this exact defect one digit later, which is the shape of the bug.

Second half, same item: an id opened by a `**Qnn -- ...**` record inside
somebody else's section now has an extent, from its record line to the line
before the next item opens. Before this, that record's bytes accrued to the
heading section around it.

Measured at c703338 (the commit this lands on) against the fixed tool:

  --index at c703338   96 sections, 85 ids, 11 nested ids; Q98 = 12616-13117,
                       63,254 B (Q99-Q104 inside it); no row for any id >= Q100
  --index, fixed       112 items (96 sections, 16 record blocks), 85 ids,
                       16 nested ids; Q98 = 12616-12733, 11,601 B; Q99 12,385,
                       Q100 4,375, Q101 8,386, Q102 4,216, Q103 16,301,
                       Q104 5,990 -- items tile end+1 == next line, byte sum
                       equals queue_bytes exactly
  --check at c703338   rc 1 (the Q104 append in c703338 was never rendered)
  --check, this tree   rc 0, "index current: 112 items (96 sections, 16 record
                       blocks), 85 ids, 11 continuation sections, 16 nested ids"
  --selftest           58 cells -> 73 cells, 73 passed, 0 failed, rc 0
                       (tools/selftest_baseline.tsv follows: queue_index.py
                       58 -> 73)

The 15 new cells are falsified, not asserted: the same selftest run with the two
patterns mutated back to `\d{1,2}` in memory reads 62 passed, 11 failed, rc 1 --
three-digit heading is a section / is an id, three-digit bounds do not overlap,
a three-digit bold record stays a nested id with its form, four-digit heading is
its own id, a longer id is one token not a prefix match, and the four
block-bound cells (their fixture opens Q102 on purpose). Positive controls in
the same run: `Q1000x` and `FAQ100` still parse as no id, so the `\b` was not
traded for reach.

Two documentation defects fixed in the same file, both measured, neither
cosmetic-only:

1. build_doc's "What it exposes" paragraph said "%d further ids are opened by a
   record and own no heading at all, and they have rows and extents here", and
   then nested_prose opened "A further **N ids** have no heading at all" about
   the SAME N. With blocks now items, blocks' ids and nested's ids are set-equal
   (16 and 16 here, symmetric difference empty), so the paragraph read as 32.
   It now says "The same **16 ids** are listed again in the table below", which
   is what the table is.
2. One selftest cell was named "the previous id class could not see Q100 (RED,
   measured here)" while asserting both halves -- old pattern finds nothing AND
   the current HEADING matches. Renamed to say both.

Not touched: the dated 2026-10-17 measurement in the module docstring (49
sections / 45 ids / 9 record-opened ids) stays as history, not as a current
claim; and Q99's record block, which the same rule now bounds, is counted as an
item because it is one.

docs/queue-index.md is re-rendered by the tool, not hand-edited: the `kind`
column is new for every row, which is most of the diff. verify.py's
index-current leg reads it.

Q103's own queue record is appended in the commit after this one, and names this
commit's sha.
This commit is contained in:
volt committed 2026-10-28 00:36:16 +00:00
1 parent c703338267
commit e3bbe875a5
3 files changed
+2204 -236

No files matched your search

+1963 -133
View File
File diff suppressed because it is too large. Load diff
+240 -102
View File
@@ -28,7 +28,22 @@ WHY THIS EXISTS
line and the form that admitted the id, so the difference is visible instead of line and the form that admitted the id, so the difference is visible instead of
implied. implied.
WHAT IT MEASURES -- structural facts only, one row per `## ` section An id is `Q` and one or more digits, no width (Q103, 2026-10-27). The pattern
read `Q` plus one or two digits, so from Q100 up an id was not a section and
not even a token: a
`## Q100 -- ...` heading parsed as body text of the section before it and that
section's byte count absorbed the whole tail behind it. Three-digit ids are now
sections, tokens and record ids alike, with no ceiling restored at three digits
-- a fixed width is the defect, so a three-digit ceiling would only postpone it.
The trailing word boundary is what stops `Q100` matching inside `Q1000`.
TIER TWO HAS BOUNDS NOW (Q103). A `**Qnn -- ...**` record that opens an id with
no heading of its own is an item too: its block runs from the record line to
the line before the next item opens, so `Q98`'s row stopped carrying the 40,957
bytes of `Q100`-`Q103` and each of those ids has a row and an extent.
WHAT IT MEASURES -- structural facts only, one row per item -- a `## ` section,
or (Q103) a record block that opens an id with no heading of its own
id, line, end_line, role, title, lines, bytes, first `Status:` marker. id, line, end_line, role, title, lines, bytes, first `Status:` marker.
role=opening the first section carrying that id role=opening the first section carrying that id
role=continuation a later section for an id already seen (a closure or role=continuation a later section for an id already seen (a closure or
@@ -94,7 +109,7 @@ USAGE
python3 tools/queue_index.py --json out.json python3 tools/queue_index.py --json out.json
python3 tools/queue_index.py --write docs/queue-index.md python3 tools/queue_index.py --write docs/queue-index.md
python3 tools/queue_index.py --check docs/queue-index.md python3 tools/queue_index.py --check docs/queue-index.md
python3 tools/queue_index.py --selftest # fixture corpus, 58 cells python3 tools/queue_index.py --selftest # fixture corpus, 73 cells
Re-run `--write` after any append to the queue. The document carries the Re-run `--write` after any append to the queue. The document carries the
queue's sha256, so a stale copy is a RED check, never a silent lie. queue's sha256, so a stale copy is a RED check, never a silent lie.
@@ -114,8 +129,16 @@ DOC_PATH = "docs/queue-index.md"
BEGIN = "<!-- queue-index:begin -->" BEGIN = "<!-- queue-index:begin -->"
END = "<!-- queue-index:end -->" END = "<!-- queue-index:end -->"
HEADING = re.compile(r"^##\s+(Q\d{1,2})\b(.*)$") # THE ID CLASS (Q103, 2026-10-27). An id is `Q` and one or more digits -- no
ID_TOKEN = re.compile(r"\bQ\d{1,2}\b") # width. It was `\d{1,2}`, so from Q100 up an item was not a section and not even
# a token: `## Q100 -- ...` parsed as body text of whatever section preceded it
# and the eight-figure byte count of a whole tail accrued to its neighbour. The
# trailing `\b` is load-bearing in the other direction: without it `Q100` is a
# prefix-match inside `Q1000` and `Q1000` would be indexed as the item Q100.
# A width ceiling is deliberately NOT restored: `\d{1,3}` reproduces this exact
# defect one digit later, which is the shape of the bug, not a fix for it.
HEADING = re.compile(r"^##\s+(Q\d+)\b(.*)$")
ID_TOKEN = re.compile(r"\bQ\d+\b")
STATUS = re.compile(r"^\s*\*{0,2}Status:\s*(.*)$") STATUS = re.compile(r"^\s*\*{0,2}Status:\s*(.*)$")
FENCE = re.compile(r"^(```|~~~)") FENCE = re.compile(r"^(```|~~~)")
@@ -214,7 +237,17 @@ def fence_mask(lines):
def parse(text): def parse(text):
"""Return the structural parse: sections, nested ids, id map, byte accounting.""" """Return the structural parse: items, nested ids, id map, byte accounting.
An *item* is a heading section OR a record block (Q103, 2026-10-27). An id
opened by a record -- `**Q100 -- OPEN ...**` inside somebody else's section --
is an item with its own extent, from its record line to the line before the
next item opens, instead of being counted as prose inside its neighbour. The
two tiers are one table here and `kind` says which: before this, a record
opening an id with no heading handed its bytes to the section around it (Q98's
row carried 40,957 bytes of four later items) and an id from Q100 up was not
even a token.
"""
lines, trailing_newline = split_lines(text) lines, trailing_newline = split_lines(text)
n = len(lines) n = len(lines)
sizes = [line_bytes(ln, i == n - 1, trailing_newline) for i, ln in enumerate(lines)] sizes = [line_bytes(ln, i == n - 1, trailing_newline) for i, ln in enumerate(lines)]
@@ -227,43 +260,7 @@ def parse(text):
m = HEADING.match(ln) m = HEADING.match(ln)
if m: if m:
headings.append((i, m.group(1), m.group(2).strip())) headings.append((i, m.group(1), m.group(2).strip()))
heading_ids = {h[1] for h in headings}
sections = []
seen = {}
for idx, (line_no, qid, rest) in enumerate(headings):
end = headings[idx + 1][0] - 1 if idx + 1 < len(headings) else n
role = "opening" if qid not in seen else "continuation"
detail = ""
if role == "continuation":
words = rest.lstrip("—- ").split()
detail = words[0].strip("*(") if words else ""
seen.setdefault(qid, line_no)
body = lines[line_no:end]
status = None
status_line = None
for off, ln in enumerate(body, start=line_no + 1):
m = STATUS.match(ln)
if m:
status = status_value(m.group(1)) or None
status_line = off
break
title = rest.lstrip("—- ").strip()
sections.append({
"id": qid,
"line": line_no,
"end_line": end,
"role": role,
"role_detail": detail,
"title": clip(title, 200),
"lines": end - line_no + 1,
"bytes": sum(sizes[line_no - 1:end]),
"status": clip(status, 200) if status else None,
"status_line": status_line,
})
ids: dict = {}
for s in sections:
ids.setdefault(s["id"], []).append(s["line"])
# every id token in the queue, counted as a mention, plus the first line that # every id token in the queue, counted as a mention, plus the first line that
# opens it as a record. Fenced lines are skipped: an id in a code block is # opens it as a record. Fenced lines are skipped: an id in a code block is
@@ -290,20 +287,73 @@ def parse(text):
if form: if form:
hit["record"] = i hit["record"] = i
hit["form"] = form hit["form"] = form
nested = []
for qid in sorted(mentioned, key=lambda s: int(s[1:])): # tier two: ids opened by a record and owning no heading of their own
if qid in ids: records = [(mentioned[qid]["record"], qid, mentioned[qid]["form"])
continue for qid in sorted(mentioned, key=lambda s: int(s[1:]))
rec = mentioned[qid] if qid not in heading_ids and mentioned[qid]["record"] is not None]
if rec["record"] is None:
continue # a mention, not a record -- not an item # Every item opens somewhere and owns the lines up to the next opening: a
line_no = rec["record"] # heading, or a record that opens an id with no heading. Nothing else cuts a
# section, so a bold *mention* inside a section does not split it.
boundaries = sorted({h[0] for h in headings} | {r[0] for r in records})
def end_of(line_no):
nxt = [b for b in boundaries if b > line_no]
return (nxt[0] - 1) if nxt else n
items = []
seen = {}
for line_no, qid, rest in headings:
role = "opening" if qid not in seen else "continuation"
detail = ""
if role == "continuation":
words = rest.lstrip("—- ").split()
detail = words[0].strip("*(") if words else ""
seen.setdefault(qid, line_no)
items.append({"id": qid, "line": line_no, "kind": "heading", "role": role,
"role_detail": detail, "title": clip(rest.lstrip("—- ").strip(), 200)})
for line_no, qid, form in records:
raw = lines[line_no - 1] raw = lines[line_no - 1]
owner = None items.append({"id": qid, "line": line_no, "kind": "record", "role": "record",
for s in sections: "role_detail": form,
if s["line"] <= line_no <= s["end_line"]: "title": clip(BULLET.sub("", raw).strip(), 200)})
owner = s["id"] items.sort(key=lambda it: it["line"])
for it in items:
end = end_of(it["line"])
status = None
status_line = None
for off, ln in enumerate(lines[it["line"]:end], start=it["line"] + 1):
m = STATUS.match(ln)
if m:
status = status_value(m.group(1)) or None
status_line = off
break break
it["end_line"] = end
it["lines"] = end - it["line"] + 1
it["bytes"] = sum(sizes[it["line"] - 1:end])
it["status"] = clip(status, 200) if status else None
it["status_line"] = status_line
sections = [it for it in items if it["kind"] == "heading"]
blocks = [it for it in items if it["kind"] == "record"]
ids: dict = {}
for s in sections:
ids.setdefault(s["id"], []).append(s["line"])
# `nested` carries what a bounds row cannot: the record's form, how often the
# id is named, the first line that names it at all, and the item that opens
# before it (its parent). A block is the same id with bounds; both are
# reported because a reader needs the line and the shape.
nested = []
for it in blocks:
qid = it["id"]
rec = mentioned[qid]
idx = [k for k, cand in enumerate(items) if cand["line"] == it["line"]][0]
owner = items[idx - 1]["id"] if idx else None
raw = lines[it["line"] - 1]
stripped = raw.lstrip() stripped = raw.lstrip()
if stripped.startswith("#"): if stripped.startswith("#"):
shape = "HEADING" shape = "HEADING"
@@ -315,7 +365,7 @@ def parse(text):
shape = "PLAIN" shape = "PLAIN"
if shape not in ("HEADING", "BOLD") and "**" in raw: if shape not in ("HEADING", "BOLD") and "**" in raw:
shape += "+BOLD" shape += "+BOLD"
nested.append({"id": qid, "line": line_no, "section": owner, "shape": shape, nested.append({"id": qid, "line": it["line"], "section": owner, "shape": shape,
"refs": rec["refs"], "form": rec["form"], "refs": rec["refs"], "form": rec["form"],
"first_mention": rec["line"]}) "first_mention": rec["line"]})
@@ -327,7 +377,9 @@ def parse(text):
"queue_sha256": hashlib.sha256(text.encode("utf-8")).hexdigest(), "queue_sha256": hashlib.sha256(text.encode("utf-8")).hexdigest(),
"preamble": {"line": 1, "end_line": preamble_end, "preamble": {"line": 1, "end_line": preamble_end,
"bytes": sum(sizes[:preamble_end])}, "bytes": sum(sizes[:preamble_end])},
"items": items,
"sections": sections, "sections": sections,
"blocks": blocks,
"ids_with_section": sorted(ids, key=lambda s: int(s[1:])), "ids_with_section": sorted(ids, key=lambda s: int(s[1:])),
"ids_multi_section": {k: v for k, v in sorted( "ids_multi_section": {k: v for k, v in sorted(
ids.items(), key=lambda kv: int(kv[0][1:])) if len(v) > 1}, ids.items(), key=lambda kv: int(kv[0][1:])) if len(v) > 1},
@@ -379,10 +431,12 @@ def outside_ids(repo, queue_path, exclude=(), exclude_tools=True):
def summary(payload): def summary(payload):
secs = payload["sections"] secs = payload["sections"]
blocks = payload.get("blocks", [])
cont = [s for s in secs if s["role"] == "continuation"] cont = [s for s in secs if s["role"] == "continuation"]
return ("index: %d sections, %d ids, %d continuation sections, %d nested ids, " return ("index: %d items (%d sections, %d record blocks), %d ids, "
"%d lines / %d bytes" % (len(secs), len(payload["ids_with_section"]), "%d continuation sections, %d nested ids, %d lines / %d bytes"
len(cont), len(payload["nested"]), % (len(payload["items"]), len(secs), len(blocks),
len(payload["ids_with_section"]), len(cont), len(payload["nested"]),
payload["queue_lines"], payload["queue_bytes"])) payload["queue_lines"], payload["queue_bytes"]))
@@ -403,17 +457,24 @@ def nested_prose(nested):
with IndexError -- leaving a zero-byte document and rc 1, the code that with IndexError -- leaving a zero-byte document and rc 1, the code that
means "graded a fault" -- on any queue whose ids all carry headings (a fresh means "graded a fault" -- on any queue whose ids all carry headings (a fresh
queue, a fixture, a partial extract). Found 2026-10-18 by the fixture in queue, a fixture, a partial extract). Found 2026-10-18 by the fixture in
`tools/queue_append_audit.py --selftest`. The non-empty branch reproduces `tools/queue_append_audit.py --selftest`. The non-empty branch reproduced
the previous sentence byte for byte, so the real document's render did not that fix's sentence byte for byte so the render did not move.
move and `--check` stayed green across the fix.
Q103 (2026-10-27): the sentence used to open "A further **N ids**", which
double-counted -- the clause before it already said those N ids are items
with rows and extents, so the paragraph read as 2N. The ids in `blocks` and
the ids in `nested` are the same set now that a record block is an item (16
and 16 in this tree, set-equal); the sentence says so instead of implying a
second population, and the table below is described as the second view of
the same ids it is.
""" """
if not nested: if not nested:
return (" No id is opened as a record without a heading of its own, so the " return (" No id is opened as a record without a heading of its own, so the "
"table below is empty.") "table below is empty.")
return (" A further **%d ids have no heading " return (" The same **%d ids** are listed again in the table below, which carries "
"at all** and are opened as records nested inside another item's section -- so " "what a bounds row cannot -- the form that opened each one and how often the "
"grepping `^## %s` finds nothing while %s is a live item the file names %d times. " "file names it: grepping `^## %s` finds nothing while %s is a live item the "
"Those are listed in their own table below." "file names %d times."
% (len(nested), nested[0]["id"], nested[0]["id"], nested[0]["refs"])) % (len(nested), nested[0]["id"], nested[0]["id"], nested[0]["refs"]))
@@ -426,24 +487,35 @@ def build_doc(payload):
"goes RED when this document and `WORK-QUEUE.md` disagree, which is what an " "goes RED when this document and `WORK-QUEUE.md` disagree, which is what an "
"append to the queue produces. Regenerate, commit, move on.*") "append to the queue produces. Regenerate, commit, move on.*")
lines.append("") lines.append("")
lines.append("**What this is.** One row per `## ` section of `WORK-QUEUE.md`: its line " lines.append("**What this is.** One row per item of `WORK-QUEUE.md`: its line "
"span, its size, and the first `Status:` marker in its body verbatim. " "span, its size, and the first `Status:` marker in its body verbatim. "
"Whether an item is *open* is not decided here -- that verdict is " "An item is a `## ` section, or a record that opens an id with no heading "
"`tools/queue_claim_check.py`'s, and this document does not overrule it.") "of its own (`kind` says which, and such an id's block runs to the line "
"before the next item opens). Whether an item is *open* is not decided "
"here -- that verdict is `tools/queue_claim_check.py`'s, and this document "
"does not overrule it.")
lines.append("") lines.append("")
lines.append("**What it exposes.** The queue is two-tier. %d sections carry %d ids because " lines.append("**What it exposes.** The queue is two-tier. %d sections carry %d ids because "
"%d ids own a second, continuation section.%s" "%d ids own a second, continuation section; %d further ids are opened by a "
"record and own no heading at all, and they have rows and extents here.%s"
% (len(payload["sections"]), len(payload["ids_with_section"]), % (len(payload["sections"]), len(payload["ids_with_section"]),
len(payload["ids_multi_section"]), nested_prose(payload["nested"]))) len(payload["ids_multi_section"]), len(payload.get("blocks", [])),
nested_prose(payload["nested"])))
lines.append("") lines.append("")
lines.append("| # | id | lines | bytes | role | title | first `Status:` |") lines.append("| # | id | kind | lines | bytes | role | title | first `Status:` |")
lines.append("|---|---|---|---|---|---|---|") lines.append("|---|---|---|---|---|---|---|---|")
for i, s in enumerate(payload["sections"], 1): for i, s in enumerate(payload["items"], 1):
status = (s["status"] or "--").replace("|", "\\|") status = (s["status"] or "--").replace("|", "\\|")
title = s["title"].replace("|", "\\|") title = s["title"].replace("|", "\\|")
role = s["role"] if s["role"] == "opening" else "cont: %s" % (s["role_detail"] or "?") if s["role"] == "opening":
lines.append("| %d | %s | %d-%d | %d | %s | %s | %s |" role = "opening"
% (i, s["id"], s["line"], s["end_line"], s["bytes"], role, title, status)) elif s["role"] == "record":
role = "record: %s" % (s["role_detail"] or "?")
else:
role = "cont: %s" % (s["role_detail"] or "?")
lines.append("| %d | %s | %s | %d-%d | %d | %s | %s | %s |"
% (i, s["id"], s["kind"], s["line"], s["end_line"], s["bytes"],
role, title, status))
lines.append("") lines.append("")
lines.append("## Ids with no `## ` heading") lines.append("## Ids with no `## ` heading")
lines.append("") lines.append("")
@@ -496,22 +568,36 @@ def extract_pinned(text):
def delta(pinned, fresh): def delta(pinned, fresh):
"""A short, specific account of how a pinned parse and a fresh one differ.""" """A short, specific account of how a pinned parse and a fresh one differ."""
out = [] out = []
p_sec = {(s["id"], s["line"]): s for s in pinned.get("sections", [])} p_it = {(s["id"], s["line"]): s for s in pinned.get("items", pinned.get("sections", []))}
f_sec = {(s["id"], s["line"]): s for s in fresh.get("sections", [])} f_it = {(s["id"], s["line"]): s for s in fresh.get("items", [])}
added = [k for k in f_sec if k not in p_sec] added = [k for k in f_it if k not in p_it]
removed = [k for k in p_sec if k not in f_sec] removed = [k for k in p_it if k not in f_it]
moved = [k for k in f_sec if k in p_sec and moved = [k for k in f_it if k in p_it and
(f_sec[k]["end_line"], f_sec[k]["bytes"], f_sec[k]["status"]) (f_it[k]["end_line"], f_it[k]["bytes"], f_it[k]["status"])
!= (p_sec[k]["end_line"], p_sec[k]["bytes"], p_sec[k]["status"])] != (p_it[k]["end_line"], p_it[k]["bytes"], p_it[k]["status"])]
if added: # Q103: an item is a heading section or a record block, and the account names
out.append("%d new section(s): %s" % (len(added), ", ".join("%s@%d" % k for k in added))) # which -- a block that moved is not reported as a section.
if removed: def kind_of(table, key):
out.append("%d gone section(s): %s" % (len(removed), ", ".join("%s@%d" % k for k in removed))) return table[key].get("kind", "heading")
add_h = [k for k in added if kind_of(f_it, k) == "heading"]
add_b = [k for k in added if kind_of(f_it, k) != "heading"]
if add_h:
out.append("%d new section(s): %s" % (len(add_h), ", ".join("%s@%d" % k for k in add_h)))
if add_b:
out.append("%d new record block(s): %s"
% (len(add_b), ", ".join("%s@%d" % k for k in add_b)))
gone_h = [k for k in removed if kind_of(p_it, k) == "heading"]
gone_b = [k for k in removed if kind_of(p_it, k) != "heading"]
if gone_h:
out.append("%d gone section(s): %s" % (len(gone_h), ", ".join("%s@%d" % k for k in gone_h)))
if gone_b:
out.append("%d gone record block(s): %s"
% (len(gone_b), ", ".join("%s@%d" % k for k in gone_b)))
if moved: if moved:
out.append("%d section(s) changed extent or status, e.g. %s@%d %d->%d lines" out.append("%d item(s) changed extent or status, e.g. %s@%d %d->%d lines"
% (len(moved), moved[0][0], moved[0][1], % (len(moved), moved[0][0], moved[0][1],
p_sec[moved[0]]["end_line"] - p_sec[moved[0]]["line"] + 1, p_it[moved[0]]["end_line"] - p_it[moved[0]]["line"] + 1,
f_sec[moved[0]]["end_line"] - f_sec[moved[0]]["line"] + 1)) f_it[moved[0]]["end_line"] - f_it[moved[0]]["line"] + 1))
p_nest = [r["id"] for r in pinned.get("nested", [])] p_nest = [r["id"] for r in pinned.get("nested", [])]
f_nest = [r["id"] for r in fresh.get("nested", [])] f_nest = [r["id"] for r in fresh.get("nested", [])]
if p_nest != f_nest: if p_nest != f_nest:
@@ -608,12 +694,12 @@ def run(args):
return 2 return 2
if args.index: if args.index:
print("%-4s %-5s %-11s %-7s %-13s %-22s %s" print("%-4s %-5s %-7s %-11s %-7s %-10s %s"
% ("#", "id", "lines", "bytes", "role", "status", "title")) % ("#", "id", "kind", "lines", "bytes", "role", "title"))
for i, s in enumerate(payload["sections"], 1): for i, s in enumerate(payload["items"], 1):
print("%-4d %-5s %-11s %-7d %-13s %-22s %s" print("%-4d %-5s %-7s %-11s %-7d %-10s %s"
% (i, s["id"], "%d-%d" % (s["line"], s["end_line"]), s["bytes"], % (i, s["id"], s["kind"], "%d-%d" % (s["line"], s["end_line"]), s["bytes"],
s["role"][:13], clip((s["status"] or "--"), 22), clip(s["title"], 72))) s["role"][:10], clip(s["title"], 72)))
print() print()
for r in payload["nested"]: for r in payload["nested"]:
print("nested %-4s line %-5d refs %-3d inside %-4s %s" print("nested %-4s line %-5d refs %-3d inside %-4s %s"
@@ -712,16 +798,68 @@ def selftest():
check("quoted tool output opens nothing (Q77/Q99)", check("quoted tool output opens nothing (Q77/Q99)",
{"Q77", "Q99"} & {r["id"] for r in p["nested"]}, set()) {"Q77", "Q99"} & {r["id"] for r in p["nested"]}, set())
check("mentions are still counted as refs", p["nested"][0]["refs"], 2) check("mentions are still counted as refs", p["nested"][0]["refs"], 2)
check("sections tile the file with no gap", check("items tile the file with no gap",
all(secs[i]["end_line"] + 1 == secs[i + 1]["line"] for i in range(len(secs) - 1)), True) all(p["items"][i]["end_line"] + 1 == p["items"][i + 1]["line"]
check("last section ends at the last line", secs[-1]["end_line"], p["queue_lines"]) for i in range(len(p["items"]) - 1)), True)
check("a record block carves its own bounds out of its neighbour",
[(b["id"], b["line"], b["end_line"]) for b in p["blocks"]],
[("Q41", 24, 25), ("Q43", 26, 28), ("Q45", 29, 29)])
check("the section that used to swallow them now stops before the block",
(secs[3]["id"], secs[3]["line"], secs[3]["end_line"]), ("Q3", 21, 23))
check("last item ends at the last line", p["items"][-1]["end_line"], p["queue_lines"])
check("preamble ends before the first heading", p["preamble"]["end_line"], 4) check("preamble ends before the first heading", p["preamble"]["end_line"], 4)
check("byte accounting is exact", check("byte accounting is exact",
p["preamble"]["bytes"] + sum(s["bytes"] for s in secs), p["queue_bytes"]) p["preamble"]["bytes"] + sum(s["bytes"] for s in p["items"]), p["queue_bytes"])
# a block between two headings keeps both headings as sections and takes the
# lines between them: the case that made `sections tile` a false statement
gap = parse("## Q99 — a\n\nbody\n\n- **Q102 — OPEN (pipe).**\n\nprose\n\n## Q100 — b\n")
check("a block between two headings does not dissolve them",
[s["id"] for s in gap["sections"]], ["Q99", "Q100"])
check("the section before the block stops at it",
[(s["id"], s["line"], s["end_line"]) for s in gap["sections"][:1]],
[("Q99", 1, 4)])
check("the block owns the lines up to the next heading",
[(b["id"], b["line"], b["end_line"]) for b in gap["blocks"]],
[("Q102", 5, 8)])
check("items still tile across a block boundary",
[(i["id"], i["line"], i["end_line"]) for i in gap["items"]],
[("Q99", 1, 4), ("Q102", 5, 8), ("Q100", 9, 9)])
check("bytes are the file's own length", p["queue_bytes"], len(FIXTURE.encode())) check("bytes are the file's own length", p["queue_bytes"], len(FIXTURE.encode()))
check("sha256 is the file's own", p["queue_sha256"], check("sha256 is the file's own", p["queue_sha256"],
hashlib.sha256(FIXTURE.encode()).hexdigest()) hashlib.sha256(FIXTURE.encode()).hexdigest())
# --- the id class: three digits are an id (Q103, 2026-10-27) -------------
# The fixture above stops at Q99, exactly like the real queue did: `\d{1,2}`
# made every item from Q100 up body text of the section before it. These
# cells fail on the pre-fix pattern, which is why they are here.
three = parse("## Q99 — a\n\nbody\n\n## Q100 — b\n\nQ101 is named here.\n\n"
"## Q101 — c\n\n- **Q102 — OPEN (pipe).**\n")
check("three-digit heading is a section",
[s["id"] for s in three["sections"]], ["Q99", "Q100", "Q101"])
check("three-digit heading is an id",
three["ids_with_section"], ["Q99", "Q100", "Q101"])
pairs = [(a["end_line"], b["line"]) for a, b in
zip(three["sections"], three["sections"][1:])]
check("three-digit section bounds do not overlap the one before",
pairs[:1], [(4, 5)])
check("sections still tile across the id boundary",
all(three["sections"][i]["end_line"] + 1 == three["sections"][i + 1]["line"]
for i in range(len(three["sections"]) - 1)), True)
check("a three-digit bold record is a nested id (form kept)",
[(r["id"], r["form"]) for r in three["nested"]], [("Q102", "BOLD-LEAD")])
# No width ceiling: a fourth digit is a new id, never a truncation of a
# three-digit one. `\d{1,3}` would fail both cells below -- the ceiling is
# the bug one digit later, so it is not restored.
four = parse("## Q1000 — d\n")
check("four-digit heading is its own id", four["ids_with_section"], ["Q1000"])
check("a longer id is one token, not a prefix match",
ID_TOKEN.findall("see Q1000 and Q100, plus Q1000x"), ["Q1000", "Q100"])
check("an id glued to a word is not an id",
ID_TOKEN.findall("FAQ100 / xQ100 / Q100y / Q"), [])
check("the old id class could not see Q100; this one does (RED -> GREEN)",
(re.compile(r"\bQ\d{1,2}\b").findall("## Q100 — b"),
HEADING.match("## Q100 — b") is not None), ([], True))
# --- the gate, round trip and both directions of falsification ---------- # --- the gate, round trip and both directions of falsification ----------
root = tempfile.mkdtemp(prefix="queue-index-selftest-") root = tempfile.mkdtemp(prefix="queue-index-selftest-")
repo = os.path.join(root, "repo") repo = os.path.join(root, "repo")
@@ -802,7 +940,7 @@ def selftest():
json=None, write=None, check=None, outside=False)) json=None, write=None, check=None, outside=False))
check("--index outside a git work tree exits 0", rc, 0) check("--index outside a git work tree exits 0", rc, 0)
check("--index outside a git work tree still summarizes", check("--index outside a git work tree still summarizes",
"index: 4 sections" in buf.getvalue(), True) "index: 7 items (4 sections, 3 record blocks)" in buf.getvalue(), True)
# falsify 1: an append moves the last section's extent # falsify 1: an append moves the last section's extent
with open(queue, "a", encoding="utf-8") as fh: with open(queue, "a", encoding="utf-8") as fh:
@@ -889,7 +1027,7 @@ def main(argv=None):
ap.add_argument("--check", metavar="PATH", help="re-derive and compare a rendered index") ap.add_argument("--check", metavar="PATH", help="re-derive and compare a rendered index")
ap.add_argument("--outside", action="store_true", ap.add_argument("--outside", action="store_true",
help="report ids the prose corpus names that the queue never mentions") help="report ids the prose corpus names that the queue never mentions")
ap.add_argument("--selftest", action="store_true", help="fixture corpus, 58 cells") ap.add_argument("--selftest", action="store_true", help="fixture corpus, 73 cells")
args = ap.parse_args(argv) args = ap.parse_args(argv)
if args.selftest: if args.selftest:
return selftest() return selftest()
+1 -1
View File
@@ -25,7 +25,7 @@ manifest_audit.py 17 cells
path_claim_check.py 31 cells path_claim_check.py 31 cells
queue_append_audit.py 25 cells queue_append_audit.py 25 cells
queue_claim_check.py 24 fixtures queue_claim_check.py 24 fixtures
queue_index.py 58 cells queue_index.py 73 cells
reply_check.py 49 cells reply_check.py 49 cells
selftest_sweep.py 44 cells selftest_sweep.py 44 cells
served_drift_check.py 25 cells served_drift_check.py 25 cells
1 # selftest_baseline.tsv -- the cell count each tool's selftest prints, pinned.
25 path_claim_check.py
26 queue_append_audit.py
27 queue_claim_check.py
28 queue_index.py
29 reply_check.py
30 selftest_sweep.py
31 served_drift_check.py