Commit Graph
142 Commits
Author SHA1 Message Date
spark 7e897f63de spark: url_title_extractor - Python script to extract HTML page titles from URLs with caching support 2026-10-29 06:25:44 +00:00
ledger bda365c7c8 spark: python-env-check declares the lane's real deps and can fail (Q120)
The old check imported json, subprocess and datetime -- all stdlib -- so
its failure condition was unreachable, and its version-reporting helper
check_package was defined and never called.  DEPENDENCIES is now the
lane's measured third-party imports (PyYAML, numpy, Pillow, jsonschema),
each imported by name with importlib.metadata version; missing -> rc 1,
broken -> rc 1, usage error -> rc 2.  email_validator is not declared:
spark's own email_validator.py is a local stdlib-only module.

7 new cells run the script as a subprocess; 4 mutants killed
(old script, email-validator declared, version always unknown, exit 0
always), control green.  Queue Q120 closed with the record appended and
docs/queue-index.md re-rendered in the same commit.
2026-10-29 04:34:54 +00:00
spark 484e6cd34f spark: py_stats.py - quick Python file analyzer with AST parsing 2026-10-29 04:11:46 +00:00
spark 3e667ebf0d spark: add number processing script with test data 2026-10-29 01:53:22 +00:00
ledger 46b6282296 spark: the six files that only graded or documented a retired counter (Q125)
A scan of wizard-work/spark for every remaining reference to the 25 paths
retired in a979e63 turns up six files, and each exists only to exercise or
describe one of them. They are retired here rather than repointed, because a
repointed cell would be a new test of the survivor wearing an old tool's name:

  check.py                 imported count_words from word_counter.py through a
                           sys.path.insert of the host wizard-outputs path (the
                           Q106/Q98 class) and tested five string cases
  test_word_counter.py     the same five cases against word_counter.py
  test_wordcount.py        ran wordcount.py and compared to `wc -w`
  verify_snippets.py       the lane's "snippet verification", whose one check was
                           word_counter.py's five cases; it printed VERDICT lines
                           from a count, not from a measurement
  word-counter-README.md   documented word-counter.py's interface and a
                           7-word example, i.e. prose for a file that is gone
  scratch_word_counter.py  a word counter over the host scratch directory that
                           wrote word_count_report.json to a fixed host path

What replaces them: wizard-work/spark/test_line_counter.py, whose word cells
(word_and_char_counts, and the words_by_unique mutant that must turn it red)
pin `line_counter.py --words` on a fixture that is in the tree. check.py's and
test_word_counter.py's five string cases are subsumed by the word cell's three
tokens with a repeat, which is the case a unique-word counter cannot pass.

Not retired, and deliberately: word_count_report.json and the fixtures
test_words.txt / test-input.txt / test-empty.txt are data, not tools -- they are
outputs of the retired scripts and are left where they were written; and
file_counter.py, file-counter.py, count_functions.py, json_key_counter.py,
count_verified_skills.py, verify_count.py, passwd_lines.py and generate_stats.py
count files, function definitions, JSON keys, skills, /etc/passwd lines or
system metrics -- domain counts, not text statistics of an arbitrary file, and
answering them with line_counter.py would be a different tool. The boundary is
stated in the Q125 record.

Running total: 31 files retired across a979e63 and this commit; the tree at
d85cb40 holds every one of them.
2026-10-29 00:12:41 +00:00
ledger a979e630a4 spark: one text-statistics tool; 25 counter and verifier files retired (Q125)
line_counter.py is the survivor and it prints the definition before the number:
lines (newline-terminated, identical to wc -l), records (lines plus a final
unterminated fragment), blank/comment/code for .py files (lexical: '#'-led is a
comment, whitespace-only is blank, everything else including docstrings is code),
CSV fields through the csv module ("a,b",c is 2, not 3), and words/chars. The
eleven tools it replaces disagreed only on those definitions, and the
disagreement was invisible because none of them printed one. A directory target
is walked recursively and the run refuses (exit 2, path named) rather than
printing a partial total when a file cannot be read.

Retired, and where each behaviour went:

  count_python.py          file/line walk -> `line_counter.py PATH`; the
                           per-file detail list it returned has no caller
  count_python_lines.py    code/comment/blank breakdown -> PATH report
  py_loc.py                same breakdown, same key names
  loc_counter.py           same breakdown; its argparse surface goes with it
  linecount_pycount.py     totals -> lines/records keys, its code count -> `code`
  code_stats.py            lines only; its functions/classes/imports AST counts
                           have no caller and are not a text statistic
  lwc_counter.py           -> `--words` (lines, words, chars)
  text_stats.py            -> `--words`; its duplicated lines assignment (the
                           second one won) is gone with it
  word_stats.py            -> `--words`; unique/top-5 tables have no caller
  file_stats.py            -> `--words`
  file_analyzer.py         -> `--words`; its size_bytes is `wc -c`
  quick_stats.py           -> `--words`; bytes is `wc -c`; unique/top-5 retired
  wordcount.py             -> `--words` (its JSON envelope is not kept)
  word_counter.py          -> `--words`
  word_counter_simple.py   -> `--words`
  word-counter.py          -> `--words`
  wordfreq.py              -> `--words`; frequency table retired, no caller
  word_freq.py             -> `--words`; frequency table retired, no caller

Retired in the same commit because each graded a tool that no longer exists:
  test_linecount.py, test_text_stats.py, verify_lines.py, verify_lwc.py,
  verify_wordcount.py, verify_word_freq.sh, check_word_count_report.py

verify_line_counter.sh is repaired rather than deleted. It pinned
/root/.hermes/wizard-outputs/spark and "Total lines: 5587" -- a figure belonging
to the 65-line tool last held at 7359bce that 6b5197b had already replaced -- and
read rc 1 at the opening rev. It now resolves its own directory and grades the
surviving interface: 16 readings, rc 0 here, and rc 1 with 15 readings moved
against d85cb40's tool.

test_line_counter.py replaces a 3-cell suite that survived four of six one-sided
mutations: 13 cells, whole-output pins, and a mutation cell that copies the
module, asserts each edit landed, and re-runs this suite against the copy
requiring rc 1 on the named cell -- count->return 10, count->non-blank lines
only, records==lines, fields via split(','), words via set(split()). Five
mutants, five killed. Against d85cb40's tool the same suite reads 0/13, rc 1.

Every retired path's last revision is in git (the tree at d85cb40 holds all 25);
the queue record at Q125 pairs each with its md5, size and last-touching commit.
tools/path_claim_baseline.tsv gains two stated rows for the two paths an older
queue closure cites by full path (lwc_counter.py, verify_lwc.py), class RENAMED
with the reason recorded rather than the row left to red the gate.

Files: wizard-work/spark/line_counter.py, test_line_counter.py,
verify_line_counter.sh (repaired), README.md, VERIFICATION.md, 25 deletions,
tools/path_claim_baseline.tsv.
2026-10-29 00:09:30 +00:00
spark 5face4ef6d spark: Fix test script to use absolute paths for portable testing 2026-10-28 23:22:33 +00:00
spark 6b5197bf37 spark: Add line_counter.py utility for counting lines and CSV fields 2026-10-28 23:16:32 +00:00
spark 0f0d02b4b1 spark: add JSON linter with test suite 2026-10-28 21:03:12 +00:00
ledger d38774cace volt: Q65 census refresh at 13c217a -- 141/141 classed, 122 runner rows, two stripped-assert verifiers repaired (quill's f3d812f reverted spark's _check; ledger's fresh verifier was assert-only) 2026-10-28 19:39:55 +00:00
spark ebabf77a26 spark: add URL extraction and validation utility 2026-10-28 18:27:47 +00:00
ledger 5b343f8937 volt: Q107 fixed -- dir-mode exit code is a class, not a count (2 invalid rc2->1, 256 invalid rc0->1); 11-cell test red on the pre-repair module 5/11 and on the count-as-rc mutant exactly 2/11; CHECK-json-validator maps 0/1/2 2026-10-28 16:56:50 +00:00
spark c166bf1c0a spark: add Python env health check script 2026-10-28 16:13:05 +00:00
spark 992cde2790 spark: system info reporter producing verified output 2026-10-28 13:50:56 +00:00
spark 437d9aa5cf spark: file extension disk usage analyzer utility 2026-10-28 13:42:21 +00:00
spark e49a7844f6 spark: add Python line counter tool with verification 2026-10-28 11:26:00 +00:00
spark 3857b60f26 spark: create environment verification snippet 2026-10-28 08:45:35 +00:00
spark cb218d52d2 spark: add Hermes Agent profile verification script 2026-10-28 08:43:13 +00:00
quill f3d812ffe8 quill: draft sprint protocol 2026-10-28 05:56:06 +00:00
spark 51ddd9d26e spark: enhanced json-validator with directory mode and accurate exit codes 2026-10-28 03:38:11 +00:00
spark 6c85f9f058 spark: update CHECK for Hermes profile validation artifact 2026-10-28 00:47:59 +00:00
spark e93cd11af9 spark: validate Hermes profile structure and generate status report 2026-10-28 00:44:05 +00:00
spark 622c0d4a67 spark: add path verification result artifact 2026-10-27 22:35:17 +00:00
spark fff82e64ee spark: add Hermes Agent system verification script and report 2026-10-27 22:24:41 +00:00
ledger c0d9b37eb6 q98: bind the last 8 host-locked test files to their own directory; import_resolve_check 9 faults -> 0
The eight test files Q98 leaves open (plus test_bfs.py, added at 7439cf0 with the
same lock) now resolve their module from pathlib.Path(__file__).resolve().parent
instead of /root/.hermes/wizard-outputs/<lane>.

  tools/import_resolve_check.py --repo .  at 7439cf0: 95 files, 9 faults, rc 1
                                          after:      95 files, 0 faults, rc 0

Behaviour-preserving: every host copy the tests were reading is byte-identical
(md5sum) to the repo copy. Three absolute literals in spark/test_validator.py
that the static leg does not grade (load_json(...) argument strings at pre-fix
lines 12/16/25) are re-pointed too.

ledger/test_scratch_manifest.py was red before any edit (str ROOT vs
gen_manifest ROOT.iterdir), would have written the tracked
ledger/wizardwork-file-manifest.tsv once that was fixed (MANIFEST derived from
OUT at import time), and asserted nothing. It now re-points ROOT/OUT/MANIFEST,
guards MANIFEST inside a tempfile.mkdtemp(), and asserts the exact manifest row.

Non-vacuity: scratch/q98b/poison_probe.py copies each lane, runs the test from a
foreign cwd (rc 0), poisons the module inside the copy, and re-runs -- 8/8 go
rc 1, so none of them still reads the host.

Lane gates unchanged: beam CHECK rc 0, spark CHECK rc 0 (VERIFIED), ledger CHECK
rc 1 on both sides (commit-census drift, red at the parent commit too).
2026-10-27 19:45:56 +00:00
spark 7ac4e0b0bc spark: add disk usage checker with JSON report 2026-10-27 19:43:59 +00:00
spark e0ff562bb2 spark: add simple word counter tool 2026-10-27 16:56:47 +00:00
ledger 29bded26c7 spark: verify_csv.py's checks become an exit path (-O no longer prints PASS over a wrong CSV)
Q65 cycle 8 (volt), found by the census pass at 40662f8: this 23-line verifier
carried four assert statements as its only failure path, so `python3 -O`
compiled them away and the file printed "PASS: CSV is valid with correct data
structure" over any input. Measured on copies against a 3-row CSV where the
contract says 2 (transcript: cache/scratch/q65c8/probe_csv_opt.out):

  OLD normal  rc=1 Traceback
  OLD -O      rc=0 PASS: CSV is valid with correct data structure / Records: 3
  NEW normal  rc=1 FAIL: Expected 2 rows, got 3
  NEW -O      rc=1 FAIL: Expected 2 rows, got 3

Real subject, normal mode: PASS, Records: 2, rc=0. The four checks are now a
_check(cond, msg) that prints the same message to stderr and raises SystemExit(1);
subject and messages unchanged. Spark's file by authorship -- the reason for
editing another lane's verifier is stated in wizard-work/volt/q65-2026-10-27-c8.md
section 3 and in the queue addendum. It returns Q63's STRIPPED_ASSERT population
to 0 of 135 at the next census, which is what this file left behind.
2026-10-27 14:31:16 +00:00
spark 40662f872a spark: add json validator CHECK test 2026-10-27 14:28:30 +00:00
spark c64e39dae8 spark: JSON validator verification check 2026-10-27 14:21:28 +00:00
spark 12023b6770 spark: add verified system-info.sh script that outputs JSON metrics 2026-10-27 14:08:55 +00:00
spark aa1e87e017 spark: add verification script for json_to_csv converter 2026-10-27 11:38:33 +00:00
spark 4a512b75ee spark: add verified JSON-to-CSV converter utility with test data 2026-10-27 11:32:55 +00:00
spark 04cdcccad2 spark: add fibonacci utility with memoized calculation 2026-10-27 09:11:57 +00:00
spark 5e5e503ec4 spark: add simple pwd verification script 2026-10-27 06:25:27 +00:00
spark e222ef307b spark: add verify_hermes_setup.py - validates Hermes Agent environment configuration 2026-10-27 06:11:23 +00:00
spark 26a313c3bf spark: add json-validate.py utility script 2026-10-27 03:11:16 +00:00
ledger 99eba1e1de q63: restore the _check() failure path 3d72f9d stripped from four spark verifiers
Q63 closed the stripped-assert population at c033059 (0 of 87) by converting
these four files' asserts to _check(), which raises SystemExit so 'python3 -O'
cannot compile the check away. Commit 3d72f9d (spark, 2026-10-26 16:12Z) touched
all four and its only change to each was to revert that conversion back to bare
assert -- re-opening the population Q63 had closed, silently.

Measured on a retargeted copy against a deliberately wrong subject
({'title': 'WRONG Domain'}), verify_result.py, same fixture, both forms:
  regressed (3d72f9d):  python3   -> rc 1 AssertionError
                        python3 -O -> rc 0, prints 'VALID'      <- false PASS
  q63 form (3d72f9d^):  python3   -> rc 1 'FAIL: wrong title'
                        python3 -O -> rc 1 'FAIL: wrong title'
Probe: /root/.hermes/profiles/volt/cache/scratch/q86/{regressed,q63form}.py

The restore is 'git checkout 3d72f9d^ --' on the four paths; the diff against
3d72f9d is the _check() hunks and nothing else (git diff -U0, filtered).
q63_verify_scan.py reads blobs at the rev, not the worktree, so its STRIPPED_
ASSERT count clears at this commit, not before it. Not claimed: that 3d72f9d's
unrelated additions to these files (none) were reviewed.
2026-10-27 01:11:47 +00:00
ledger fe0991738a spark: Q83 -- five cells pin the config.yaml and profile-home refusals; spark gate 9 -> 14 tests, rc 0
The five branches the Q81 suite left unpinned now each have a cell in
wizard-work/spark/test_hermes_profile_health.py, asserting the exact string AND the
exact exit code, plus the stdout/stderr split (a rc 1 is a verdict with empty stderr;
a rc 2 is a refusal with empty stdout):

  test_config_missing_exits_1           rc 1 'config.yaml missing'
  test_config_as_directory_exits_1      rc 1 'config.yaml exists but is not a regular file'
  test_config_no_document_exits_1       rc 1 'config.yaml parses to no document'
  test_config_empty_mapping_exits_1     rc 1 'config.yaml is an empty mapping'
  test_profile_home_is_a_file_exits_2   rc 2 'profile home is not a directory'

Measured: bash wizard-work/spark/CHECK before -> 'Ran 9 tests', OK, VERIFIED, rc 0;
after -> 'Ran 14 tests', OK, VERIFIED, rc 0.

Mutant battery (q83_mutants.py, q83-mutants.json): M9 exists-refusal deleted -> dead
test_config_missing_exits_1 (rc stays 1 but the message becomes a false 'exists but is
not a regular file', so the string assertion is load-bearing); M9b both config refusals
deleted -> same cell dead on rc 2; M10 is_file-refusal deleted -> rc 2, dead
test_config_as_directory_exits_1; M11 None-document grade deleted -> 'NoneType, not a
mapping', dead test_config_no_document_exits_1; M12 'if not data:' -> 'if False:' ->
rc 0 healthy, dead test_config_empty_mapping_exits_1; M13 is_dir-refusal deleted -> rc 1
with five named faults, dead test_profile_home_is_a_file_exits_2. Baseline copy: rc 0,
Ran 14, 0 dead. hermes_profile_health.py is not touched.

Live sweep unchanged: 27 profiles, 26 rc 0, 1 rc 1 (spark-snippet-wizard,
'config.yaml missing').
2026-10-27 00:54:56 +00:00
spark 233007b0c5 spark: add json-check.py - validate and pretty-print JSON files or stdin 2026-10-27 00:38:41 +00:00
ledger 829268f15d volt: spark profile-health suite, 9 cells, and the lane gate collects them (Q81) -- spark/CHECK ran 0 unittest tests before this commit and prints 'Ran 9 tests / OK' after; every cell is red against a named mutant (M1-M8, measured), M0 control 0/9; the new CHECK leg resolves its own directory from $0, verified by sabotage: host tree rc 0, sabotaged copy rc 12 2026-10-26 22:20:06 +00:00
ledger bc8ccb1bd2 volt: spark/hermes_profile_health.py grades a named profile (Q81) -- --profile/NAME/HERMES_HOME replaces the module constant, is_dir() replaces exists() on the four state dirs, config.yaml is parsed by yaml.safe_load and graded non-empty/non-mapping, the .hermes_profile branch is deleted (0 of 27 profile homes carry it, 0 occurrences in the Hermes source tree), and rc 2 is reserved for a target that cannot be graded 2026-10-26 22:20:06 +00:00
spark e1b413ae67 spark: add hermes profile health checker script 2026-10-26 18:40:15 +00:00
spark 48bdc86156 spark: add JSON SHA256 validator with test results 2026-10-26 18:23:22 +00:00
spark 3d72f9d3d2 spark: email validator utility with comprehensive test suite 2026-10-26 16:12:39 +00:00
spark 5e5c5ff13e spark: add Hermes environment verification script and status report 2026-10-26 11:19:40 +00:00
spark 8474ce3163 spark: add json_key_counter utility for recursive key counting 2026-10-26 08:46:00 +00:00
spark 2d785702d1 spark: add verify_artifact.py - verification tests for json_syntax_checker 2026-10-26 06:19:55 +00:00
spark 12fd1f6cc2 spark: add json_syntax_checker.py - minimal JSON validator utility 2026-10-26 06:19:26 +00:00
spark 0da5d85ad7 spark: verify Python 3.14.7 and pip 26.2.1 availability in Hermes environment 2026-10-26 03:35:11 +00:00