200 lines
16 KiB
Markdown
200 lines
16 KiB
Markdown
# Q29 — The guards that are not the walk
|
||
|
||
> Screen. Every claim below carries a path, a hash, a URL, or a command with its output, or the word
|
||
> `unverified`. Nothing secret is printed in this file. Sole author: Merlin. Scratch tree:
|
||
> `/root/crucible/q29/` — re-runnable with `python3 q29-guards.py`, then `python3 q29-verify.py`,
|
||
> then `python3 q29-exclusions.py` (logs: `Q29-GUARDS.txt`, `Q29-VERIFY.txt`,
|
||
> `Q29-EXCLUSIONS.txt`, `Q29-PROBE.txt`; data: `Q29-RESULTS.json`, `Q29-VERIFY.json`,
|
||
> `Q29-EXCLUSIONS.json`).
|
||
|
||
**Run clock:** 2026-10-11, ~05:4x–06:1x UTC, this host. Builder under test: the Q27-patched
|
||
`build_blocks_hard.py`, sha256 `c4127c6c9325ed9f8cab03b80b454cccde0f9f1e768289ff216d2fb79e78dbf2`
|
||
(29,174 B), copied into the scratch tree and re-verified before any fixture. `blocks.json`:
|
||
sha256 `63c9cee6103a6824c90c71440fa983d2f5592e380f9c3fb40f7ad366ae41723d` (278,459 B),
|
||
**read-only throughout** — every fixture mutates a copy. No fixture carries a `{{`/`}}`
|
||
sentinel: the walk kills those (Q26/Q28). These fixtures carry **no token at all**, so every
|
||
build that exits 0 exercises the mechanism that protects the emitted byte.
|
||
|
||
---
|
||
|
||
## 1. The second guarantee, in one sentence
|
||
|
||
**Every string the build emits is either escaped for its context or derived from a value that
|
||
cannot carry a metacharacter: HTML text and attributes go through `esc()`
|
||
(`html.escape(v, quote=True)`, patched `build_blocks_hard.py:87`), the JSON-LD script payload
|
||
goes through `json.dumps` plus a `</` → `<\/` replacement (`:362`) so no value can terminate
|
||
the script element, URLs must be `https://` before they are rendered (`safe_url`, `:242`) and
|
||
are escaped in the attribute, and every classified field (`status`, `ledger`,
|
||
`publication_status`) is validated against a closed map before any label is rendered
|
||
(`:266-271`).** (Patched-builder numbering throughout the sentence; pristine equivalents in the
|
||
table below.)
|
||
|
||
The mechanism inventory. **Line numbers below are the builder under test — the patched
|
||
`build_blocks_hard.py` (sha256 `c4127c6c…`) — with the pristine `build_blocks.py`
|
||
(`/root/stackchain-legends-build/build_blocks.py`, sha256 `97fedb5dd5e85d2…`) equivalent in
|
||
parentheses where it differs:**
|
||
|
||
| mechanism | site (patched / pristine) | what it guards | sufficiency for that context |
|
||
|---|---|---|---|
|
||
| `esc()` = `html.escape(v, quote=True)` | :87 (same) | every HTML text node and every HTML attribute | escapes `< > & " '` — complete for both contexts |
|
||
| `json.dumps(...).replace("</", "<\\/")` | :362 / pristine :208 (payload in `head()`) | the `<script type="application/ld+json">` payload | `</script` (case-insensitive) is the only sequence that terminates the element, and every `</` is escaped |
|
||
| `safe_url()` scheme check | :242 / pristine :96, called :272 | `body_source_url` | rejects any non-`https://` value before render; `esc()` then protects the attribute |
|
||
| entry validation vs `STATUS_LABEL`/`LEDGER_LABEL` | :266-269 / pristine :121-123 | badge class + badge text | the key is never rendered; unknown key dies pre-write |
|
||
| `slug_for()` sanitisation | :237 / pristine :91 | canonical href, sitemap `loc`, index `li` id, pager hrefs | `[^a-z0-9-]+` → `-`; no metacharacter survives (measured: `slug_for("<b>")` → `b`) |
|
||
| `sortable[:4]` + `esc()` | :545, :549 / pristine :391, :395 | index year heading | the slice cannot inject a tag; `esc()` escapes the residue |
|
||
| receipt rows built through `esc(k)`/`esc(v)` | :417 / pristine :263 | the whole receipt table | formatted first, escaped after |
|
||
|
||
---
|
||
|
||
## 2. Fixtures — 22 plus the control
|
||
|
||
Each fixture is its own scratch tree under `/root/crucible/q29/fixtures/<name>/` holding a
|
||
mutated copy of `blocks.json` and the patched builder. "Files" = files written at exit.
|
||
"Emitted" = the bytes actually on the page, quoted from `Q29-VERIFY.json`. All builds ran
|
||
`python3 build_blocks_hard.py` in the fixture tree.
|
||
|
||
| # | fixture | mutation | exit | files | emitted evidence |
|
||
|---|---|---|---|---|---|
|
||
| 1 | `title-metachars` | `magazine-001.title = '<b>hi</b> & "q" \'z < 5'` | 0 | 125 | `<title><b>hi</b> & "q" 'z < 5 …`; `meta … content="<b>hi</b> & "q" 'z < 5"`; `h1` same; index label same; JSON-LD `"headline": "<b>hi<\/b> & \"q\" 'z < 5 …"` (raw `<b>`, escaped `<\/b>`) |
|
||
| 2 | `title-closescript` | `title = '</script><b>hi</b>'` | 0 | 125 | `"headline": "<\/script><b>hi<\/b> …"`; raw `</script` in page = **1** (the page's own closer; the payload's is `<\/script`) |
|
||
| 3 | `title-closescript-upper` | `title = '</SCRIPT><B>'` | 0 | 125 | `"headline": "<\/SCRIPT><B> …"`; raw `</script` in page = 1 |
|
||
| 4 | `title-comment-nested` | `title = '<!--<script>alert(1)</script>-->'` | 0 | 125 | `"headline": "<!--<script>alert(1)<\/script>--> …"`; raw `<!--` in page = 1 (inside the payload only) |
|
||
| 5 | `title-barelt` | `title = '5 < 6 and 7'` | 0 | 125 | `"headline": "5 < 6 and 7 …"` (json.dumps keeps `<`); `<title>5 < 6 and 7 …` |
|
||
| 6 | `date-closescript` | `magazine-001.date = date_display = '</script><b>x'` | 0 | 125 | `"datePublished": "<\/script><b>x"`; HTML `Date: </script><b>x` |
|
||
| 7 | `displayname-metachars` | `contributor.display_name = 'A & B <x> "y" \'z'` (no handle) | 0 | 125 | byline `A & B <x> "y" 'z`; JSON-LD `"author": {"@type": "Person", "name": "A & B <x> \"y\" 'z"}` |
|
||
| 8 | `displayname-closescript` | `display_name = 'Zoe </script><i>'` | 0 | 125 | `"author": {… "name": "Zoe <\/script><i>"}`; raw `</script` in page = 1 |
|
||
| 9 | `handle-metachars` | `contributor.handle = 'x" onmouseover=alert(1)'` | 0 | 125 | byline `<a href="https://x.com/x" onmouseover=alert(1)">@x" onmouseover=alert(1)</a>`; JSON-LD `"url": "https://x.com/x\" onmouseover=alert(1)"` (json.dumps escapes the quote) |
|
||
| 10 | `handle-closescript` | `handle = 'a</script>b'` | 0 | 125 | byline `<a href="https://x.com/a</script>b">@a</script>b</a>`; JSON-LD `"url": "https://x.com/a<\/script>b"`; raw `</script` in page = 1 |
|
||
| 11 | `panel-closescript` | `plain_summary.text = 'the </script> break'` | 0 | 125 | panel `the </script> break`; raw `</script` in page = 1 |
|
||
| 12 | `provenance-comment` | `body_provenance = '<!-- [archivist note] --> & <b>'` | 0 | 125 | panel `<!-- [archivist note] --> & <b>` |
|
||
| 13 | `excerpt-metachars` | `excerpts[0].text = 'they said "5 < 6" & it\'s true'` | 0 | 125 | blockquote `they said "5 < 6" & it's true` |
|
||
| 14 | `excerpt-url-quote` | `excerpts[0].url = 'https://x.com/a" onmouseover=alert(1)'` | 0 | 125 | `<a href="https://x.com/a" onmouseover=alert(1)" …>` (no `safe_url` on excerpt URLs; `esc()` still holds the attribute) |
|
||
| 15 | `sourceurl-quote` | `block-6.body_source_url = 'https://x.com/a" onmouseover=alert(1)'` | 0 | 125 | `<cite>Source: <a href="https://x.com/a" onmouseover=alert(1)">…` |
|
||
| 16 | `quotebranch-body` | `block-6.body = 'he said "hi" <b> and 5 < 6'` (quote branch) | 0 | 125 | blockquote `he said "hi" <b> and 5 < 6` |
|
||
| 17 | `reuse-status` | `block-5.receipt.reuse_status = '<i>yes</i>'` | 0 | 125 | `Image reuse: <i>yes</i>.` |
|
||
| 18 | `receipt-usd-nonnumeric` | `block-5.receipt.usd = '<b>7'` | 0 | 125 | `Amount $<b>7`; `Arithmetic not checked`; `Amount in sats 22,765` (the surviving int) |
|
||
| 19 | `receipt-sats-nonnumeric` | `block-5.receipt.sats = 'x'` | **1** | **2** | stderr `Traceback (most recent call last):`; the two files written are `event-2022-07-18-anthony-enters-at-16-names-the-hill-at-24/index.html` and `event-2022-07-18-the-5-invitation-becomes-a-chain/index.html` — **a partial tree** (see §5) |
|
||
| 20 | `status-metachar` | `magazine-001.status = '<b>'` | 1 | 0 | `BUILD FAILED: block[30] unknown status '<b>'` — validation, not escaping |
|
||
| 21 | `year-derived` | `magazine-001.sort_date = '<b>2023-08-27'` | 0 | 125 | index year headings end `('<b>2', '<b>2')`; `li id="magazine-001"` unchanged |
|
||
| 22 | `control` | no mutation | 0 | 125 | tree digest `91ecf6bf939a2a499983066a2331bb953eab8a4ca85c66a8a19f42cb1b28f7a8` — **byte-identical to the Q26 baseline and to the live `site-ledger`** (`diff -rq` silent) |
|
||
|
||
**Proof-or-falsify results, stated against the emitted byte, not the intent:**
|
||
|
||
- `</script>` in `title`, `date`, `display_name`, `plain_summary.text`: **proven safe in all four.**
|
||
Payload landings carry `<\/script…`; HTML landings carry `</script>`. Raw `</script`
|
||
(not preceded by `\`) in each page = **1**, the page's own closing tag.
|
||
- `<!--`: **survives raw into the payload** (`"<!--<script>alert(1)<\/script>-->"`), and is
|
||
inert there: a script element is a raw-text element, so `<!--` opens no comment, and every
|
||
`</` sequence inside is escaped — even in the HTML spec's script-data double-escape states
|
||
nothing can terminate the element.
|
||
- Lone `"` in an attribute-rendered field: **proven safe.** `title` → `meta description`
|
||
(`"`), `handle` → href and link text (`"`), `excerpts[0].url` and
|
||
`body_source_url` → href (`"`). No attribute breakout in any of the four.
|
||
- `&` in every context: **escaped** in HTML (`&`), raw in the payload — and inert there,
|
||
because raw text performs no entity decoding.
|
||
- `<` (bare), `<script`: survive raw into the payload; inert in a raw-text element.
|
||
|
||
**Live-data corroboration (not a fixture).** The live pages already exercise `esc()` on
|
||
production values: `https://alexanderwhitestone.com/stackchain/magazine-012/` (4,095 B,
|
||
sha256 `f6d4d8e50d549684…`, fetched this run) serves `<title>Inflation & You …</title>`
|
||
and JSON-LD `"headline": "Inflation & You …"`; `magazine-035` (4,543 B, `871638b8abb50137…`)
|
||
carries `'` ×4. The escaping is not fixture-only; the live tree already depends on it.
|
||
|
||
---
|
||
|
||
## 3. Where no mechanism is needed — measured exclusions
|
||
|
||
| # | surface | measured command | output |
|
||
|---|---|---|---|
|
||
| E1 | CSS / FOOT / `STATUS_LABEL` / `LEDGER_LABEL` literals | `grep -cF '.format('` on both builders | `0` and `0`; `grep -n 'ledger.css.*write_text'` → `324: (OUT / "ledger.css").write_text(CSS)`; `grep -cF 'parts.append(FOOT)'` → `3`; constants assigned at `:45` (`CSS`), `:35` (`STATUS_LABEL`) |
|
||
| E2 | `blocks-build.json` (the manifest) | json keys of every entry, and of the file | every value `['bytes','sha256']`; top level `['cleared','files','pages','research_only','schema']`; the three counters are `int`; every manifest key matches `[a-z0-9/.-]+` |
|
||
| E3 | `sitemap.xml` | regex over the control tree | 123 `<loc>`s, every one `https://alexanderwhitestone.com/stackchain/<slug>/`; `slug_for('<b>')` → `b`; write site `:336` applies `esc(u)` |
|
||
| E4 | receipt arithmetic's formatted strings | grep for the row build | `417: body = "".join(f"<tr><th>{esc(k)}</th><td>{esc(v)}</td></tr>" …)` — formatted, then escaped; fixtures 18/19 measure the two failure shapes |
|
||
| E5 | raw `</script` over the whole control tree | `re.findall(r"(?<!\\)</script", …)` over 123 pages | total **123**, per-page values `[1]` — exactly the legit closer on every page, nothing else |
|
||
| E6 | `esc()` / payload semantics, measured directly | `html.escape(s, quote=True)`; `json.dumps(s)`; the replace | `'<b>'→'<b>'`, `'&'→'&'`, `'"'→'"'`, `"'"→'''`, `'</script>'→'</script>'`, `'<!--'→'<!--'`; `json.dumps('<b>')` → `"<b>"`; `json.dumps('</script>').replace('</','<\\/')` → `"<\/script>"` |
|
||
|
||
---
|
||
|
||
## 4. What this adds to Q26/Q27/Q28
|
||
|
||
Q26 proved the two silent token classes die; Q27 enumerated the render sites and closed the
|
||
uncited-handle class pre-write; Q28 accounted for the channels the walk cannot reach and
|
||
printed the first guarantee. This item is the complement: the mechanisms that guard values
|
||
**with no token**. Measured result: **no metacharacter in any `blocks.json` field reaches any
|
||
emit site unescaped or un-escaped-for-context, in any of the 22 fixtures** — the one exception
|
||
is the JSON-LD payload's raw `<`, `&`, `<!--`, `<script`, which are inert in a raw-text
|
||
script element and are the documented, measured behaviour of `json.dumps` plus the `</`
|
||
replacement.
|
||
|
||
Q29's item text asked (task 4) to restate two Q27 counts. **Q28 already discharged both** —
|
||
36 `esc()` call lines / 43 occurrences and the 42-vs-35 fixture breakdown — and I re-measured
|
||
the first this run (`grep`: 36 lines carrying `esc(` excluding the definition at `:87`, 43
|
||
occurrences), reproducing Q28's figure. The item's other premise, "three of Q27's 32 rows
|
||
carry no `esc()` call (`:208` the payload, `:286` `datePublished`, `:405`→`:406`)", also
|
||
re-verifies against the pristine builder: `json.dumps` sites are at `:192` and `:208`
|
||
(`:192` is the manifest, `:208` the payload), `:286` is `ld["datePublished"] = b["date"]`,
|
||
and the gaps loop's call is at `:406`.
|
||
|
||
---
|
||
|
||
## 5. One finding: an unguarded `int()` can still leave a partial tree
|
||
|
||
Fixture 19 (`block-5.receipt.sats = 'x'`) is the one measured hole. `receipt_table` guards
|
||
`float(price_raw)` and `float(usd)` in a `try/except` (fixture 18 falls back to
|
||
`not checked`), but `f"{int(sats):,}"` at `build_blocks.py:416` has **no guard**: the
|
||
`ValueError` propagates as a traceback, the process exits 1, and the write loop has already
|
||
emitted **2 files** (the two entries that sort before `block-5`). That contradicts the
|
||
builder's own docstring — *"Fails closed: any schema problem, missing required field, or
|
||
non-https source URL aborts the whole build rather than emitting a partial tree"* — for the
|
||
type-coercion class. It is the same defect shape Q26/Q27 closed for the token classes (a
|
||
failure that leaves a half-emitted tree), reached through a different door.
|
||
|
||
**Recommended, not staged:** wrap `int(sats)` in the same `try/except` shape as the price
|
||
line (fallback to the raw string, still `esc()`'d), or validate receipt field types in the
|
||
pre-write pass beside `safe_url`. Either is a one-line-class change to `build_blocks.py` —
|
||
Alexander's, the same gate Q26/Q27/Q28 state. The two unguarded `int(sats)` sites are
|
||
`build_blocks.py:258` and `:262` (patched builder: `:412` and `:416`), and the guarded
|
||
`float()` pair sits at `:252`/`:257` (patched `:406`/`:411`). Today's `blocks.json` carries
|
||
`int` sats on both present receipts (`n` = 5 and 7, measured in `Q29-PROBE.txt`), so this is
|
||
latent, not live.
|
||
|
||
---
|
||
|
||
## 6. Limits
|
||
|
||
- **No fuzzer.** The shapes tested are the enumerated set from the item text (the
|
||
metacharacter families `<`, `&`, `"`, `'`, `</`, `<!--`, `<script`, and the receipt type
|
||
shapes). A random metacharacter fuzz over all 121 entries was not run; the guarantee is
|
||
stated over the mechanisms, not over an exhaustive input census.
|
||
- **The guarantee covers the builder's emit surface, not the deploy pipeline.** The pages
|
||
are merged into the live web root at deploy time by a separate process this build does
|
||
not touch.
|
||
- **Fixture 19's partial tree is measured, its cause is source-read** (no `try/except`
|
||
around `int(sats)` at `:416`); the traceback itself is in `Q29-RESULTS.json`.
|
||
- **The JSON-LD payload's raw `<`/`&`/`<!--` are measured inert, not proven inert by a
|
||
parser.** The reasoning is the HTML raw-text-element rule (only `</script` terminates,
|
||
case-insensitively, and every `</` is escaped); the emitted bytes are the evidence, the
|
||
rule is the argument.
|
||
- **No X page was fetched this run.** The only live fetches are the three ledger pages in
|
||
§2's corroboration.
|
||
|
||
---
|
||
|
||
## 7. Nothing touched
|
||
|
||
- Shared build tree: `build_blocks.py` still sha256 `97fedb5dd5e85d2b10071e8e4904b3a9b1573072149e722fbece93eaf0774672`,
|
||
`blocks.json` still `63c9cee6103a6824c90c71440fa983d2f5592e380f9c3fb40f7ad366ae41723d`,
|
||
`find /root/stackchain-legends-build -newermt '2026-10-11 04:00' -type f` → **empty**.
|
||
- Live archive: `https://alexanderwhitestone.com/stackchain/blocks/` → **200 / 34,488 B /
|
||
sha256 `4f764ef4dfb9c08b2654ff0783550d20ad1bb7a31cb0c32070974a5fe451557e`** (fetched this run,
|
||
identical to the Q26/Q27/Q28 readings).
|
||
- `blocks.json` read-only throughout; `decisions/` and `inbox/` untouched; nothing executed;
|
||
no other party's file edited; the patched builder stays scratch-local in
|
||
`/root/crucible/q29/`. The publish precondition from Q27/Q28 stands unchanged: **adopt
|
||
`c4127c6c` into the shared build tree first, then publish** — Alexander's call.
|
||
- Scratch tree contents: `q29-probe.py`, `q29-guards.py`, `q29-verify.py`,
|
||
`q29-exclusions.py`, `build_blocks_hard.py`, `fixtures/` (23 trees: 22 fixtures + control),
|
||
the five logs and three JSON data files listed in the screen.
|
||
|
||
— Merlin, 2026-10-11
|