volt: fix Q103 -- the queue index reads ids at any width and gives every record-opened id its own row (was: Q100+ invisible, Q98 carrying 63,254 bytes of six later items)
tools/queue_index.py, one defect in two places. HEADING was ^##\s+(Q\d{1,2})\b
and ID_TOKEN was \bQ\d{1,2}\b, so from Q100 up an id was neither a section nor
a token: `## Q100 -- ...` parsed as body text of the section before it. Both are
now `Q\d+` with the trailing \b kept. No width ceiling is restored: `\d{1,3}`
reproduces this exact defect one digit later, which is the shape of the bug.
Second half, same item: an id opened by a `**Qnn -- ...**` record inside
somebody else's section now has an extent, from its record line to the line
before the next item opens. Before this, that record's bytes accrued to the
heading section around it.
Measured at c703338 (the commit this lands on) against the fixed tool:
--index at c703338 96 sections, 85 ids, 11 nested ids; Q98 = 12616-13117,
63,254 B (Q99-Q104 inside it); no row for any id >= Q100
--index, fixed 112 items (96 sections, 16 record blocks), 85 ids,
16 nested ids; Q98 = 12616-12733, 11,601 B; Q99 12,385,
Q100 4,375, Q101 8,386, Q102 4,216, Q103 16,301,
Q104 5,990 -- items tile end+1 == next line, byte sum
equals queue_bytes exactly
--check at c703338 rc 1 (the Q104 append in c703338 was never rendered)
--check, this tree rc 0, "index current: 112 items (96 sections, 16 record
blocks), 85 ids, 11 continuation sections, 16 nested ids"
--selftest 58 cells -> 73 cells, 73 passed, 0 failed, rc 0
(tools/selftest_baseline.tsv follows: queue_index.py
58 -> 73)
The 15 new cells are falsified, not asserted: the same selftest run with the two
patterns mutated back to `\d{1,2}` in memory reads 62 passed, 11 failed, rc 1 --
three-digit heading is a section / is an id, three-digit bounds do not overlap,
a three-digit bold record stays a nested id with its form, four-digit heading is
its own id, a longer id is one token not a prefix match, and the four
block-bound cells (their fixture opens Q102 on purpose). Positive controls in
the same run: `Q1000x` and `FAQ100` still parse as no id, so the `\b` was not
traded for reach.
Two documentation defects fixed in the same file, both measured, neither
cosmetic-only:
1. build_doc's "What it exposes" paragraph said "%d further ids are opened by a
record and own no heading at all, and they have rows and extents here", and
then nested_prose opened "A further **N ids** have no heading at all" about
the SAME N. With blocks now items, blocks' ids and nested's ids are set-equal
(16 and 16 here, symmetric difference empty), so the paragraph read as 32.
It now says "The same **16 ids** are listed again in the table below", which
is what the table is.
2. One selftest cell was named "the previous id class could not see Q100 (RED,
measured here)" while asserting both halves -- old pattern finds nothing AND
the current HEADING matches. Renamed to say both.
Not touched: the dated 2026-10-17 measurement in the module docstring (49
sections / 45 ids / 9 record-opened ids) stays as history, not as a current
claim; and Q99's record block, which the same rule now bounds, is counted as an
item because it is one.
docs/queue-index.md is re-rendered by the tool, not hand-edited: the `kind`
column is new for every row, which is most of the diff. verify.py's
index-current leg reads it.
Q103's own queue record is appended in the commit after this one, and names this
commit's sha.
This commit is contained in:
volt committed
1 parent
c703338267
commit
e3bbe875a5
3 files changed
+2205
-237
No files matched your search
+1963
-133
File diff suppressed because it is too large.
Load diff
+241
-103
@@ -28,7 +28,22 @@ WHY THIS EXISTS
|
||||
line and the form that admitted the id, so the difference is visible instead of
|
||||
implied.
|
||||
|
||||
WHAT IT MEASURES -- structural facts only, one row per `## ` section
|
||||
An id is `Q` and one or more digits, no width (Q103, 2026-10-27). The pattern
|
||||
read `Q` plus one or two digits, so from Q100 up an id was not a section and
|
||||
not even a token: a
|
||||
`## Q100 -- ...` heading parsed as body text of the section before it and that
|
||||
section's byte count absorbed the whole tail behind it. Three-digit ids are now
|
||||
sections, tokens and record ids alike, with no ceiling restored at three digits
|
||||
-- a fixed width is the defect, so a three-digit ceiling would only postpone it.
|
||||
The trailing word boundary is what stops `Q100` matching inside `Q1000`.
|
||||
|
||||
TIER TWO HAS BOUNDS NOW (Q103). A `**Qnn -- ...**` record that opens an id with
|
||||
no heading of its own is an item too: its block runs from the record line to
|
||||
the line before the next item opens, so `Q98`'s row stopped carrying the 40,957
|
||||
bytes of `Q100`-`Q103` and each of those ids has a row and an extent.
|
||||
|
||||
WHAT IT MEASURES -- structural facts only, one row per item -- a `## ` section,
|
||||
or (Q103) a record block that opens an id with no heading of its own
|
||||
id, line, end_line, role, title, lines, bytes, first `Status:` marker.
|
||||
role=opening the first section carrying that id
|
||||
role=continuation a later section for an id already seen (a closure or
|
||||
@@ -94,7 +109,7 @@ USAGE
|
||||
python3 tools/queue_index.py --json out.json
|
||||
python3 tools/queue_index.py --write docs/queue-index.md
|
||||
python3 tools/queue_index.py --check docs/queue-index.md
|
||||
python3 tools/queue_index.py --selftest # fixture corpus, 58 cells
|
||||
python3 tools/queue_index.py --selftest # fixture corpus, 73 cells
|
||||
|
||||
Re-run `--write` after any append to the queue. The document carries the
|
||||
queue's sha256, so a stale copy is a RED check, never a silent lie.
|
||||
@@ -114,8 +129,16 @@ DOC_PATH = "docs/queue-index.md"
|
||||
BEGIN = "<!-- queue-index:begin -->"
|
||||
END = "<!-- queue-index:end -->"
|
||||
|
||||
HEADING = re.compile(r"^##\s+(Q\d{1,2})\b(.*)$")
|
||||
ID_TOKEN = re.compile(r"\bQ\d{1,2}\b")
|
||||
# THE ID CLASS (Q103, 2026-10-27). An id is `Q` and one or more digits -- no
|
||||
# width. It was `\d{1,2}`, so from Q100 up an item was not a section and not even
|
||||
# a token: `## Q100 -- ...` parsed as body text of whatever section preceded it
|
||||
# and the eight-figure byte count of a whole tail accrued to its neighbour. The
|
||||
# trailing `\b` is load-bearing in the other direction: without it `Q100` is a
|
||||
# prefix-match inside `Q1000` and `Q1000` would be indexed as the item Q100.
|
||||
# A width ceiling is deliberately NOT restored: `\d{1,3}` reproduces this exact
|
||||
# defect one digit later, which is the shape of the bug, not a fix for it.
|
||||
HEADING = re.compile(r"^##\s+(Q\d+)\b(.*)$")
|
||||
ID_TOKEN = re.compile(r"\bQ\d+\b")
|
||||
STATUS = re.compile(r"^\s*\*{0,2}Status:\s*(.*)$")
|
||||
FENCE = re.compile(r"^(```|~~~)")
|
||||
|
||||
@@ -214,7 +237,17 @@ def fence_mask(lines):
|
||||
|
||||
|
||||
def parse(text):
|
||||
"""Return the structural parse: sections, nested ids, id map, byte accounting."""
|
||||
"""Return the structural parse: items, nested ids, id map, byte accounting.
|
||||
|
||||
An *item* is a heading section OR a record block (Q103, 2026-10-27). An id
|
||||
opened by a record -- `**Q100 -- OPEN ...**` inside somebody else's section --
|
||||
is an item with its own extent, from its record line to the line before the
|
||||
next item opens, instead of being counted as prose inside its neighbour. The
|
||||
two tiers are one table here and `kind` says which: before this, a record
|
||||
opening an id with no heading handed its bytes to the section around it (Q98's
|
||||
row carried 40,957 bytes of four later items) and an id from Q100 up was not
|
||||
even a token.
|
||||
"""
|
||||
lines, trailing_newline = split_lines(text)
|
||||
n = len(lines)
|
||||
sizes = [line_bytes(ln, i == n - 1, trailing_newline) for i, ln in enumerate(lines)]
|
||||
@@ -227,43 +260,7 @@ def parse(text):
|
||||
m = HEADING.match(ln)
|
||||
if m:
|
||||
headings.append((i, m.group(1), m.group(2).strip()))
|
||||
|
||||
sections = []
|
||||
seen = {}
|
||||
for idx, (line_no, qid, rest) in enumerate(headings):
|
||||
end = headings[idx + 1][0] - 1 if idx + 1 < len(headings) else n
|
||||
role = "opening" if qid not in seen else "continuation"
|
||||
detail = ""
|
||||
if role == "continuation":
|
||||
words = rest.lstrip("—- ").split()
|
||||
detail = words[0].strip("*(") if words else ""
|
||||
seen.setdefault(qid, line_no)
|
||||
body = lines[line_no:end]
|
||||
status = None
|
||||
status_line = None
|
||||
for off, ln in enumerate(body, start=line_no + 1):
|
||||
m = STATUS.match(ln)
|
||||
if m:
|
||||
status = status_value(m.group(1)) or None
|
||||
status_line = off
|
||||
break
|
||||
title = rest.lstrip("—- ").strip()
|
||||
sections.append({
|
||||
"id": qid,
|
||||
"line": line_no,
|
||||
"end_line": end,
|
||||
"role": role,
|
||||
"role_detail": detail,
|
||||
"title": clip(title, 200),
|
||||
"lines": end - line_no + 1,
|
||||
"bytes": sum(sizes[line_no - 1:end]),
|
||||
"status": clip(status, 200) if status else None,
|
||||
"status_line": status_line,
|
||||
})
|
||||
|
||||
ids: dict = {}
|
||||
for s in sections:
|
||||
ids.setdefault(s["id"], []).append(s["line"])
|
||||
heading_ids = {h[1] for h in headings}
|
||||
|
||||
# every id token in the queue, counted as a mention, plus the first line that
|
||||
# opens it as a record. Fenced lines are skipped: an id in a code block is
|
||||
@@ -290,20 +287,73 @@ def parse(text):
|
||||
if form:
|
||||
hit["record"] = i
|
||||
hit["form"] = form
|
||||
nested = []
|
||||
for qid in sorted(mentioned, key=lambda s: int(s[1:])):
|
||||
if qid in ids:
|
||||
continue
|
||||
rec = mentioned[qid]
|
||||
if rec["record"] is None:
|
||||
continue # a mention, not a record -- not an item
|
||||
line_no = rec["record"]
|
||||
|
||||
# tier two: ids opened by a record and owning no heading of their own
|
||||
records = [(mentioned[qid]["record"], qid, mentioned[qid]["form"])
|
||||
for qid in sorted(mentioned, key=lambda s: int(s[1:]))
|
||||
if qid not in heading_ids and mentioned[qid]["record"] is not None]
|
||||
|
||||
# Every item opens somewhere and owns the lines up to the next opening: a
|
||||
# heading, or a record that opens an id with no heading. Nothing else cuts a
|
||||
# section, so a bold *mention* inside a section does not split it.
|
||||
boundaries = sorted({h[0] for h in headings} | {r[0] for r in records})
|
||||
|
||||
def end_of(line_no):
|
||||
nxt = [b for b in boundaries if b > line_no]
|
||||
return (nxt[0] - 1) if nxt else n
|
||||
|
||||
items = []
|
||||
seen = {}
|
||||
for line_no, qid, rest in headings:
|
||||
role = "opening" if qid not in seen else "continuation"
|
||||
detail = ""
|
||||
if role == "continuation":
|
||||
words = rest.lstrip("—- ").split()
|
||||
detail = words[0].strip("*(") if words else ""
|
||||
seen.setdefault(qid, line_no)
|
||||
items.append({"id": qid, "line": line_no, "kind": "heading", "role": role,
|
||||
"role_detail": detail, "title": clip(rest.lstrip("—- ").strip(), 200)})
|
||||
for line_no, qid, form in records:
|
||||
raw = lines[line_no - 1]
|
||||
owner = None
|
||||
for s in sections:
|
||||
if s["line"] <= line_no <= s["end_line"]:
|
||||
owner = s["id"]
|
||||
items.append({"id": qid, "line": line_no, "kind": "record", "role": "record",
|
||||
"role_detail": form,
|
||||
"title": clip(BULLET.sub("", raw).strip(), 200)})
|
||||
items.sort(key=lambda it: it["line"])
|
||||
|
||||
for it in items:
|
||||
end = end_of(it["line"])
|
||||
status = None
|
||||
status_line = None
|
||||
for off, ln in enumerate(lines[it["line"]:end], start=it["line"] + 1):
|
||||
m = STATUS.match(ln)
|
||||
if m:
|
||||
status = status_value(m.group(1)) or None
|
||||
status_line = off
|
||||
break
|
||||
it["end_line"] = end
|
||||
it["lines"] = end - it["line"] + 1
|
||||
it["bytes"] = sum(sizes[it["line"] - 1:end])
|
||||
it["status"] = clip(status, 200) if status else None
|
||||
it["status_line"] = status_line
|
||||
|
||||
sections = [it for it in items if it["kind"] == "heading"]
|
||||
blocks = [it for it in items if it["kind"] == "record"]
|
||||
|
||||
ids: dict = {}
|
||||
for s in sections:
|
||||
ids.setdefault(s["id"], []).append(s["line"])
|
||||
|
||||
# `nested` carries what a bounds row cannot: the record's form, how often the
|
||||
# id is named, the first line that names it at all, and the item that opens
|
||||
# before it (its parent). A block is the same id with bounds; both are
|
||||
# reported because a reader needs the line and the shape.
|
||||
nested = []
|
||||
for it in blocks:
|
||||
qid = it["id"]
|
||||
rec = mentioned[qid]
|
||||
idx = [k for k, cand in enumerate(items) if cand["line"] == it["line"]][0]
|
||||
owner = items[idx - 1]["id"] if idx else None
|
||||
raw = lines[it["line"] - 1]
|
||||
stripped = raw.lstrip()
|
||||
if stripped.startswith("#"):
|
||||
shape = "HEADING"
|
||||
@@ -315,7 +365,7 @@ def parse(text):
|
||||
shape = "PLAIN"
|
||||
if shape not in ("HEADING", "BOLD") and "**" in raw:
|
||||
shape += "+BOLD"
|
||||
nested.append({"id": qid, "line": line_no, "section": owner, "shape": shape,
|
||||
nested.append({"id": qid, "line": it["line"], "section": owner, "shape": shape,
|
||||
"refs": rec["refs"], "form": rec["form"],
|
||||
"first_mention": rec["line"]})
|
||||
|
||||
@@ -327,7 +377,9 @@ def parse(text):
|
||||
"queue_sha256": hashlib.sha256(text.encode("utf-8")).hexdigest(),
|
||||
"preamble": {"line": 1, "end_line": preamble_end,
|
||||
"bytes": sum(sizes[:preamble_end])},
|
||||
"items": items,
|
||||
"sections": sections,
|
||||
"blocks": blocks,
|
||||
"ids_with_section": sorted(ids, key=lambda s: int(s[1:])),
|
||||
"ids_multi_section": {k: v for k, v in sorted(
|
||||
ids.items(), key=lambda kv: int(kv[0][1:])) if len(v) > 1},
|
||||
@@ -379,11 +431,13 @@ def outside_ids(repo, queue_path, exclude=(), exclude_tools=True):
|
||||
|
||||
def summary(payload):
|
||||
secs = payload["sections"]
|
||||
blocks = payload.get("blocks", [])
|
||||
cont = [s for s in secs if s["role"] == "continuation"]
|
||||
return ("index: %d sections, %d ids, %d continuation sections, %d nested ids, "
|
||||
"%d lines / %d bytes" % (len(secs), len(payload["ids_with_section"]),
|
||||
len(cont), len(payload["nested"]),
|
||||
payload["queue_lines"], payload["queue_bytes"]))
|
||||
return ("index: %d items (%d sections, %d record blocks), %d ids, "
|
||||
"%d continuation sections, %d nested ids, %d lines / %d bytes"
|
||||
% (len(payload["items"]), len(secs), len(blocks),
|
||||
len(payload["ids_with_section"]), len(cont), len(payload["nested"]),
|
||||
payload["queue_lines"], payload["queue_bytes"]))
|
||||
|
||||
|
||||
def clip(text, limit):
|
||||
@@ -403,17 +457,24 @@ def nested_prose(nested):
|
||||
with IndexError -- leaving a zero-byte document and rc 1, the code that
|
||||
means "graded a fault" -- on any queue whose ids all carry headings (a fresh
|
||||
queue, a fixture, a partial extract). Found 2026-10-18 by the fixture in
|
||||
`tools/queue_append_audit.py --selftest`. The non-empty branch reproduces
|
||||
the previous sentence byte for byte, so the real document's render did not
|
||||
move and `--check` stayed green across the fix.
|
||||
`tools/queue_append_audit.py --selftest`. The non-empty branch reproduced
|
||||
that fix's sentence byte for byte so the render did not move.
|
||||
|
||||
Q103 (2026-10-27): the sentence used to open "A further **N ids**", which
|
||||
double-counted -- the clause before it already said those N ids are items
|
||||
with rows and extents, so the paragraph read as 2N. The ids in `blocks` and
|
||||
the ids in `nested` are the same set now that a record block is an item (16
|
||||
and 16 in this tree, set-equal); the sentence says so instead of implying a
|
||||
second population, and the table below is described as the second view of
|
||||
the same ids it is.
|
||||
"""
|
||||
if not nested:
|
||||
return (" No id is opened as a record without a heading of its own, so the "
|
||||
"table below is empty.")
|
||||
return (" A further **%d ids have no heading "
|
||||
"at all** and are opened as records nested inside another item's section -- so "
|
||||
"grepping `^## %s` finds nothing while %s is a live item the file names %d times. "
|
||||
"Those are listed in their own table below."
|
||||
return (" The same **%d ids** are listed again in the table below, which carries "
|
||||
"what a bounds row cannot -- the form that opened each one and how often the "
|
||||
"file names it: grepping `^## %s` finds nothing while %s is a live item the "
|
||||
"file names %d times."
|
||||
% (len(nested), nested[0]["id"], nested[0]["id"], nested[0]["refs"]))
|
||||
|
||||
|
||||
@@ -426,24 +487,35 @@ def build_doc(payload):
|
||||
"goes RED when this document and `WORK-QUEUE.md` disagree, which is what an "
|
||||
"append to the queue produces. Regenerate, commit, move on.*")
|
||||
lines.append("")
|
||||
lines.append("**What this is.** One row per `## ` section of `WORK-QUEUE.md`: its line "
|
||||
lines.append("**What this is.** One row per item of `WORK-QUEUE.md`: its line "
|
||||
"span, its size, and the first `Status:` marker in its body verbatim. "
|
||||
"Whether an item is *open* is not decided here -- that verdict is "
|
||||
"`tools/queue_claim_check.py`'s, and this document does not overrule it.")
|
||||
"An item is a `## ` section, or a record that opens an id with no heading "
|
||||
"of its own (`kind` says which, and such an id's block runs to the line "
|
||||
"before the next item opens). Whether an item is *open* is not decided "
|
||||
"here -- that verdict is `tools/queue_claim_check.py`'s, and this document "
|
||||
"does not overrule it.")
|
||||
lines.append("")
|
||||
lines.append("**What it exposes.** The queue is two-tier. %d sections carry %d ids because "
|
||||
"%d ids own a second, continuation section.%s"
|
||||
"%d ids own a second, continuation section; %d further ids are opened by a "
|
||||
"record and own no heading at all, and they have rows and extents here.%s"
|
||||
% (len(payload["sections"]), len(payload["ids_with_section"]),
|
||||
len(payload["ids_multi_section"]), nested_prose(payload["nested"])))
|
||||
len(payload["ids_multi_section"]), len(payload.get("blocks", [])),
|
||||
nested_prose(payload["nested"])))
|
||||
lines.append("")
|
||||
lines.append("| # | id | lines | bytes | role | title | first `Status:` |")
|
||||
lines.append("|---|---|---|---|---|---|---|")
|
||||
for i, s in enumerate(payload["sections"], 1):
|
||||
lines.append("| # | id | kind | lines | bytes | role | title | first `Status:` |")
|
||||
lines.append("|---|---|---|---|---|---|---|---|")
|
||||
for i, s in enumerate(payload["items"], 1):
|
||||
status = (s["status"] or "--").replace("|", "\\|")
|
||||
title = s["title"].replace("|", "\\|")
|
||||
role = s["role"] if s["role"] == "opening" else "cont: %s" % (s["role_detail"] or "?")
|
||||
lines.append("| %d | %s | %d-%d | %d | %s | %s | %s |"
|
||||
% (i, s["id"], s["line"], s["end_line"], s["bytes"], role, title, status))
|
||||
if s["role"] == "opening":
|
||||
role = "opening"
|
||||
elif s["role"] == "record":
|
||||
role = "record: %s" % (s["role_detail"] or "?")
|
||||
else:
|
||||
role = "cont: %s" % (s["role_detail"] or "?")
|
||||
lines.append("| %d | %s | %s | %d-%d | %d | %s | %s | %s |"
|
||||
% (i, s["id"], s["kind"], s["line"], s["end_line"], s["bytes"],
|
||||
role, title, status))
|
||||
lines.append("")
|
||||
lines.append("## Ids with no `## ` heading")
|
||||
lines.append("")
|
||||
@@ -496,22 +568,36 @@ def extract_pinned(text):
|
||||
def delta(pinned, fresh):
|
||||
"""A short, specific account of how a pinned parse and a fresh one differ."""
|
||||
out = []
|
||||
p_sec = {(s["id"], s["line"]): s for s in pinned.get("sections", [])}
|
||||
f_sec = {(s["id"], s["line"]): s for s in fresh.get("sections", [])}
|
||||
added = [k for k in f_sec if k not in p_sec]
|
||||
removed = [k for k in p_sec if k not in f_sec]
|
||||
moved = [k for k in f_sec if k in p_sec and
|
||||
(f_sec[k]["end_line"], f_sec[k]["bytes"], f_sec[k]["status"])
|
||||
!= (p_sec[k]["end_line"], p_sec[k]["bytes"], p_sec[k]["status"])]
|
||||
if added:
|
||||
out.append("%d new section(s): %s" % (len(added), ", ".join("%s@%d" % k for k in added)))
|
||||
if removed:
|
||||
out.append("%d gone section(s): %s" % (len(removed), ", ".join("%s@%d" % k for k in removed)))
|
||||
p_it = {(s["id"], s["line"]): s for s in pinned.get("items", pinned.get("sections", []))}
|
||||
f_it = {(s["id"], s["line"]): s for s in fresh.get("items", [])}
|
||||
added = [k for k in f_it if k not in p_it]
|
||||
removed = [k for k in p_it if k not in f_it]
|
||||
moved = [k for k in f_it if k in p_it and
|
||||
(f_it[k]["end_line"], f_it[k]["bytes"], f_it[k]["status"])
|
||||
!= (p_it[k]["end_line"], p_it[k]["bytes"], p_it[k]["status"])]
|
||||
# Q103: an item is a heading section or a record block, and the account names
|
||||
# which -- a block that moved is not reported as a section.
|
||||
def kind_of(table, key):
|
||||
return table[key].get("kind", "heading")
|
||||
add_h = [k for k in added if kind_of(f_it, k) == "heading"]
|
||||
add_b = [k for k in added if kind_of(f_it, k) != "heading"]
|
||||
if add_h:
|
||||
out.append("%d new section(s): %s" % (len(add_h), ", ".join("%s@%d" % k for k in add_h)))
|
||||
if add_b:
|
||||
out.append("%d new record block(s): %s"
|
||||
% (len(add_b), ", ".join("%s@%d" % k for k in add_b)))
|
||||
gone_h = [k for k in removed if kind_of(p_it, k) == "heading"]
|
||||
gone_b = [k for k in removed if kind_of(p_it, k) != "heading"]
|
||||
if gone_h:
|
||||
out.append("%d gone section(s): %s" % (len(gone_h), ", ".join("%s@%d" % k for k in gone_h)))
|
||||
if gone_b:
|
||||
out.append("%d gone record block(s): %s"
|
||||
% (len(gone_b), ", ".join("%s@%d" % k for k in gone_b)))
|
||||
if moved:
|
||||
out.append("%d section(s) changed extent or status, e.g. %s@%d %d->%d lines"
|
||||
out.append("%d item(s) changed extent or status, e.g. %s@%d %d->%d lines"
|
||||
% (len(moved), moved[0][0], moved[0][1],
|
||||
p_sec[moved[0]]["end_line"] - p_sec[moved[0]]["line"] + 1,
|
||||
f_sec[moved[0]]["end_line"] - f_sec[moved[0]]["line"] + 1))
|
||||
p_it[moved[0]]["end_line"] - p_it[moved[0]]["line"] + 1,
|
||||
f_it[moved[0]]["end_line"] - f_it[moved[0]]["line"] + 1))
|
||||
p_nest = [r["id"] for r in pinned.get("nested", [])]
|
||||
f_nest = [r["id"] for r in fresh.get("nested", [])]
|
||||
if p_nest != f_nest:
|
||||
@@ -608,12 +694,12 @@ def run(args):
|
||||
return 2
|
||||
|
||||
if args.index:
|
||||
print("%-4s %-5s %-11s %-7s %-13s %-22s %s"
|
||||
% ("#", "id", "lines", "bytes", "role", "status", "title"))
|
||||
for i, s in enumerate(payload["sections"], 1):
|
||||
print("%-4d %-5s %-11s %-7d %-13s %-22s %s"
|
||||
% (i, s["id"], "%d-%d" % (s["line"], s["end_line"]), s["bytes"],
|
||||
s["role"][:13], clip((s["status"] or "--"), 22), clip(s["title"], 72)))
|
||||
print("%-4s %-5s %-7s %-11s %-7s %-10s %s"
|
||||
% ("#", "id", "kind", "lines", "bytes", "role", "title"))
|
||||
for i, s in enumerate(payload["items"], 1):
|
||||
print("%-4d %-5s %-7s %-11s %-7d %-10s %s"
|
||||
% (i, s["id"], s["kind"], "%d-%d" % (s["line"], s["end_line"]), s["bytes"],
|
||||
s["role"][:10], clip(s["title"], 72)))
|
||||
print()
|
||||
for r in payload["nested"]:
|
||||
print("nested %-4s line %-5d refs %-3d inside %-4s %s"
|
||||
@@ -712,16 +798,68 @@ def selftest():
|
||||
check("quoted tool output opens nothing (Q77/Q99)",
|
||||
{"Q77", "Q99"} & {r["id"] for r in p["nested"]}, set())
|
||||
check("mentions are still counted as refs", p["nested"][0]["refs"], 2)
|
||||
check("sections tile the file with no gap",
|
||||
all(secs[i]["end_line"] + 1 == secs[i + 1]["line"] for i in range(len(secs) - 1)), True)
|
||||
check("last section ends at the last line", secs[-1]["end_line"], p["queue_lines"])
|
||||
check("items tile the file with no gap",
|
||||
all(p["items"][i]["end_line"] + 1 == p["items"][i + 1]["line"]
|
||||
for i in range(len(p["items"]) - 1)), True)
|
||||
check("a record block carves its own bounds out of its neighbour",
|
||||
[(b["id"], b["line"], b["end_line"]) for b in p["blocks"]],
|
||||
[("Q41", 24, 25), ("Q43", 26, 28), ("Q45", 29, 29)])
|
||||
check("the section that used to swallow them now stops before the block",
|
||||
(secs[3]["id"], secs[3]["line"], secs[3]["end_line"]), ("Q3", 21, 23))
|
||||
check("last item ends at the last line", p["items"][-1]["end_line"], p["queue_lines"])
|
||||
check("preamble ends before the first heading", p["preamble"]["end_line"], 4)
|
||||
check("byte accounting is exact",
|
||||
p["preamble"]["bytes"] + sum(s["bytes"] for s in secs), p["queue_bytes"])
|
||||
p["preamble"]["bytes"] + sum(s["bytes"] for s in p["items"]), p["queue_bytes"])
|
||||
# a block between two headings keeps both headings as sections and takes the
|
||||
# lines between them: the case that made `sections tile` a false statement
|
||||
gap = parse("## Q99 — a\n\nbody\n\n- **Q102 — OPEN (pipe).**\n\nprose\n\n## Q100 — b\n")
|
||||
check("a block between two headings does not dissolve them",
|
||||
[s["id"] for s in gap["sections"]], ["Q99", "Q100"])
|
||||
check("the section before the block stops at it",
|
||||
[(s["id"], s["line"], s["end_line"]) for s in gap["sections"][:1]],
|
||||
[("Q99", 1, 4)])
|
||||
check("the block owns the lines up to the next heading",
|
||||
[(b["id"], b["line"], b["end_line"]) for b in gap["blocks"]],
|
||||
[("Q102", 5, 8)])
|
||||
check("items still tile across a block boundary",
|
||||
[(i["id"], i["line"], i["end_line"]) for i in gap["items"]],
|
||||
[("Q99", 1, 4), ("Q102", 5, 8), ("Q100", 9, 9)])
|
||||
check("bytes are the file's own length", p["queue_bytes"], len(FIXTURE.encode()))
|
||||
check("sha256 is the file's own", p["queue_sha256"],
|
||||
hashlib.sha256(FIXTURE.encode()).hexdigest())
|
||||
|
||||
# --- the id class: three digits are an id (Q103, 2026-10-27) -------------
|
||||
# The fixture above stops at Q99, exactly like the real queue did: `\d{1,2}`
|
||||
# made every item from Q100 up body text of the section before it. These
|
||||
# cells fail on the pre-fix pattern, which is why they are here.
|
||||
three = parse("## Q99 — a\n\nbody\n\n## Q100 — b\n\nQ101 is named here.\n\n"
|
||||
"## Q101 — c\n\n- **Q102 — OPEN (pipe).**\n")
|
||||
check("three-digit heading is a section",
|
||||
[s["id"] for s in three["sections"]], ["Q99", "Q100", "Q101"])
|
||||
check("three-digit heading is an id",
|
||||
three["ids_with_section"], ["Q99", "Q100", "Q101"])
|
||||
pairs = [(a["end_line"], b["line"]) for a, b in
|
||||
zip(three["sections"], three["sections"][1:])]
|
||||
check("three-digit section bounds do not overlap the one before",
|
||||
pairs[:1], [(4, 5)])
|
||||
check("sections still tile across the id boundary",
|
||||
all(three["sections"][i]["end_line"] + 1 == three["sections"][i + 1]["line"]
|
||||
for i in range(len(three["sections"]) - 1)), True)
|
||||
check("a three-digit bold record is a nested id (form kept)",
|
||||
[(r["id"], r["form"]) for r in three["nested"]], [("Q102", "BOLD-LEAD")])
|
||||
# No width ceiling: a fourth digit is a new id, never a truncation of a
|
||||
# three-digit one. `\d{1,3}` would fail both cells below -- the ceiling is
|
||||
# the bug one digit later, so it is not restored.
|
||||
four = parse("## Q1000 — d\n")
|
||||
check("four-digit heading is its own id", four["ids_with_section"], ["Q1000"])
|
||||
check("a longer id is one token, not a prefix match",
|
||||
ID_TOKEN.findall("see Q1000 and Q100, plus Q1000x"), ["Q1000", "Q100"])
|
||||
check("an id glued to a word is not an id",
|
||||
ID_TOKEN.findall("FAQ100 / xQ100 / Q100y / Q"), [])
|
||||
check("the old id class could not see Q100; this one does (RED -> GREEN)",
|
||||
(re.compile(r"\bQ\d{1,2}\b").findall("## Q100 — b"),
|
||||
HEADING.match("## Q100 — b") is not None), ([], True))
|
||||
|
||||
# --- the gate, round trip and both directions of falsification ----------
|
||||
root = tempfile.mkdtemp(prefix="queue-index-selftest-")
|
||||
repo = os.path.join(root, "repo")
|
||||
@@ -802,7 +940,7 @@ def selftest():
|
||||
json=None, write=None, check=None, outside=False))
|
||||
check("--index outside a git work tree exits 0", rc, 0)
|
||||
check("--index outside a git work tree still summarizes",
|
||||
"index: 4 sections" in buf.getvalue(), True)
|
||||
"index: 7 items (4 sections, 3 record blocks)" in buf.getvalue(), True)
|
||||
|
||||
# falsify 1: an append moves the last section's extent
|
||||
with open(queue, "a", encoding="utf-8") as fh:
|
||||
@@ -889,7 +1027,7 @@ def main(argv=None):
|
||||
ap.add_argument("--check", metavar="PATH", help="re-derive and compare a rendered index")
|
||||
ap.add_argument("--outside", action="store_true",
|
||||
help="report ids the prose corpus names that the queue never mentions")
|
||||
ap.add_argument("--selftest", action="store_true", help="fixture corpus, 58 cells")
|
||||
ap.add_argument("--selftest", action="store_true", help="fixture corpus, 73 cells")
|
||||
args = ap.parse_args(argv)
|
||||
if args.selftest:
|
||||
return selftest()
|
||||
|
||||
@@ -25,7 +25,7 @@ manifest_audit.py 17 cells
|
||||
path_claim_check.py 31 cells
|
||||
queue_append_audit.py 25 cells
|
||||
queue_claim_check.py 24 fixtures
|
||||
queue_index.py 58 cells
|
||||
queue_index.py 73 cells
|
||||
reply_check.py 49 cells
|
||||
selftest_sweep.py 44 cells
|
||||
served_drift_check.py 25 cells
|
||||
|
||||
|
Reference in new issue
Block a user