volt: q136 -- sandbox leg compares real paths, so a symlinked TMPDIR is not a miss

This commit is contained in:
volt committed 2026-10-31 12:07:52 +00:00
1 parent 5b068f0283
commit fbe4f1c09e
2 files changed
+64 -7

No files matched your search

+30
View File
@@ -110,3 +110,33 @@ The item is filed in `WORK-QUEUE.md`; this is the reading behind it.
- Left open on purpose: nothing runs the spark lane's `test_*.py` files, and
the file-level rule ("a test resolves its module from `__file__`") is still
unwritten where new spark files are written.
## Q136 follow-up — the sandbox leg read a symlinked temp path as a miss
The CHECK for Q136 failed on the `sandbox` leg alone, with the other six green:
```
q136: sandbox FAULT 8 path(s) opened, 0 outside the sandbox,
never opened: test_input.txt, wordfreq_counter.py
```
- Reproduced exactly by pointing `TMPDIR` at a symlink: CPython's `open` audit
hook records `Path(...).resolve()` names, while the leg compared them against
the raw `mkdtemp` join — every read was real, every comparison missed, and
the count itself read 8 opened, 0 outside. Nothing was wrong with the cell.
- Repaired in `wizard-work/volt/q136_verify.py`: both sides compared by
`os.path.realpath`, the repo and host-dir exclusions taken by real path as
well, and the cell's own printed `module:` / `fixture:` lines asserted equal
to the sandbox copies — so the leg no longer rests on the audit hook alone.
A fault now prints the offending outside paths instead of only a count.
- Controls, `/root/.hermes/wizard-outputs/volt/probe_q136_sandbox.py` -> PROBE
PASS: normal `TMPDIR` rc 0; symlinked `TMPDIR` rc 0 (rc 1 before the repair);
a cell that reads `/root/.hermes/wizard-outputs/spark/test_input.txt` rc 1
with `1 outside the sandbox ... outside: .../spark/test_input.txt`.
- `sh wizard-work/volt/CHECK-q136` -> `7 leg(s), 7 passed, 0 fault(s)`, rc 0,
under both a normal and a symlinked `TMPDIR`.
- Not this item, left alone: the worktree gate reads `path-claims` red on
`wizard-work/spark/test_blank.txt`, cited at `wizard-work/spark/README.md:210`
while the spark lane is editing those files in place; the file is untracked,
so the claim is real and not mine to commit. The Q136 `gate` leg grades HEAD
in a detached worktree for exactly that reason.
+34 -7
View File
@@ -30,7 +30,9 @@ What this instrument grades
run the repaired cell, run in place from `cwd=/`: rc 0, three PASS lines
sandbox the cell + module + fixture copied to a temp directory outside the
tree, run under CPython's `open` audit hook: rc 0, the module and
fixture it opened are the sandbox copies, and nothing under the
fixture it opened are the sandbox copies (compared by real path, so
a symlinked TMPDIR cannot fake a miss), the cell's own printed
module/fixture paths are the sandbox copies, and nothing under the
tree or under the old host directory was opened
counts the tracked fixture's 24 / 18 / `the`:4 recomputed by a tokenizer
that does not call the module, so the constants are not read back
@@ -186,14 +188,39 @@ def leg_sandbox(repo):
if not line:
raise GradeError("audit driver printed no AUDIT line")
opened = json.loads(line[-1][len("AUDIT "):])
need = {os.path.join(box, "wordfreq_counter.py"), os.path.join(box, "test_input.txt")}
missing = sorted(need - set(opened))
outside = [p for p in opened
if p.startswith(str(repo) + os.sep) or p.startswith(HOST_DIR)]
ok = not missing and not outside
detail = "%d path(s) opened, %d outside the sandbox" % (len(opened), len(outside))
# Temp directories can live behind a symlink (a shifted TMPDIR, for one),
# so both sides of the comparison are taken by real path: the hook
# records resolved names and a raw join would read a false miss.
box_real = os.path.realpath(box)
opened_real = {os.path.realpath(p) for p in opened}
need = {os.path.join(box_real, name)
for name in ("wordfreq_counter.py", "test_input.txt")}
missing = sorted(need - opened_real)
repo_real = os.path.realpath(str(repo))
host_real = os.path.realpath(HOST_DIR)
outside = sorted(p for p in opened_real
if p == repo_real or p.startswith(repo_real + os.sep)
or p == host_real or p.startswith(host_real + os.sep))
# The cell prints what it graded; those paths must be the sandbox copies
# too, so the leg does not rest on the audit hook alone.
printed = {}
for ln in out.splitlines():
if ln.startswith("module: "):
printed["module"] = os.path.realpath(ln[len("module: "):])
elif ln.startswith("fixture: "):
printed["fixture"] = os.path.realpath(ln[len("fixture: "):])
expected = {"module": os.path.join(box_real, "wordfreq_counter.py"),
"fixture": os.path.join(box_real, "test_input.txt")}
wrong_print = sorted(k for k, v in printed.items() if v != expected[k])
ok = not missing and not outside and not wrong_print \
and set(printed) == {"module", "fixture"}
detail = "%d path(s) opened, %d outside the sandbox" % (len(opened_real), len(outside))
if missing:
detail += ", never opened: %s" % ", ".join(os.path.basename(p) for p in missing)
if wrong_print:
detail += ", printed %s not the sandbox copy" % ", ".join(wrong_print)
if outside:
detail += ", outside: %s" % ", ".join(outside)
return ok, detail
finally:
shutil.rmtree(base, ignore_errors=True)