0 Fork 0

Files

T

ledger a979e630a4 spark: one text-statistics tool; 25 counter and verifier files retired (Q125)

line_counter.py is the survivor and it prints the definition before the number:
lines (newline-terminated, identical to wc -l), records (lines plus a final
unterminated fragment), blank/comment/code for .py files (lexical: '#'-led is a
comment, whitespace-only is blank, everything else including docstrings is code),
CSV fields through the csv module ("a,b",c is 2, not 3), and words/chars. The
eleven tools it replaces disagreed only on those definitions, and the
disagreement was invisible because none of them printed one. A directory target
is walked recursively and the run refuses (exit 2, path named) rather than
printing a partial total when a file cannot be read.

Retired, and where each behaviour went:

  count_python.py          file/line walk -> `line_counter.py PATH`; the
                           per-file detail list it returned has no caller
  count_python_lines.py    code/comment/blank breakdown -> PATH report
  py_loc.py                same breakdown, same key names
  loc_counter.py           same breakdown; its argparse surface goes with it
  linecount_pycount.py     totals -> lines/records keys, its code count -> `code`
  code_stats.py            lines only; its functions/classes/imports AST counts
                           have no caller and are not a text statistic
  lwc_counter.py           -> `--words` (lines, words, chars)
  text_stats.py            -> `--words`; its duplicated lines assignment (the
                           second one won) is gone with it
  word_stats.py            -> `--words`; unique/top-5 tables have no caller
  file_stats.py            -> `--words`
  file_analyzer.py         -> `--words`; its size_bytes is `wc -c`
  quick_stats.py           -> `--words`; bytes is `wc -c`; unique/top-5 retired
  wordcount.py             -> `--words` (its JSON envelope is not kept)
  word_counter.py          -> `--words`
  word_counter_simple.py   -> `--words`
  word-counter.py          -> `--words`
  wordfreq.py              -> `--words`; frequency table retired, no caller
  word_freq.py             -> `--words`; frequency table retired, no caller

Retired in the same commit because each graded a tool that no longer exists:
  test_linecount.py, test_text_stats.py, verify_lines.py, verify_lwc.py,
  verify_wordcount.py, verify_word_freq.sh, check_word_count_report.py

verify_line_counter.sh is repaired rather than deleted. It pinned
/root/.hermes/wizard-outputs/spark and "Total lines: 5587" -- a figure belonging
to the 65-line tool last held at 7359bce that 6b5197b had already replaced -- and
read rc 1 at the opening rev. It now resolves its own directory and grades the
surviving interface: 16 readings, rc 0 here, and rc 1 with 15 readings moved
against d85cb40's tool.

test_line_counter.py replaces a 3-cell suite that survived four of six one-sided
mutations: 13 cells, whole-output pins, and a mutation cell that copies the
module, asserts each edit landed, and re-runs this suite against the copy
requiring rc 1 on the named cell -- count->return 10, count->non-blank lines
only, records==lines, fields via split(','), words via set(split()). Five
mutants, five killed. Against d85cb40's tool the same suite reads 0/13, rc 1.

Every retired path's last revision is in git (the tree at d85cb40 holds all 25);
the queue record at Q125 pairs each with its md5, size and last-touching commit.
tools/path_claim_baseline.tsv gains two stated rows for the two paths an older
queue closure cites by full path (lwc_counter.py, verify_lwc.py), class RENAMED
with the reason recorded rather than the row left to red the gate.

Files: wizard-work/spark/line_counter.py, test_line_counter.py,
verify_line_counter.sh (repaired), README.md, VERIFICATION.md, 25 deletions,
tools/path_claim_baseline.tsv.

2026-10-29 00:09:30 +00:00

5.3 KiB

Raw Blame History

line_counter.py — verification

Rewritten 2026-10-28 (Q125). The previous version of this file certified a tool that no longer existed: it described /root/.hermes/wizard-outputs/spark/line_counter.py as counting *.py files recursively — 113 files, 5,475 newlines, 5,587 records, 4,344 code / 285 comment / 958 empty — and every one of those figures belongs to the 65-line version last held at 7359bce, which 6b5197b replaced with a 47-line file taking <filepath> <mode>. Nothing graded the replacement, so this file went on certifying the old contract. That is the defect Q125 was opened over.

The artifacts this file certifies

path	bytes	md5	lines
`wizard-work/spark/line_counter.py`	10,105	`e9cb842cbd68b1ea50a599306f3cf25b`	279
`wizard-work/spark/test_line_counter.py`	11,567	`55696ea7a2a8e23111deeb8078a670ac`	352
`wizard-work/spark/verify_line_counter.sh`	3,508	`b48f15f2a4b193b9973b24bd87561a58`	94

The counting definitions lines / records / blank / comment / code / fields / words / chars are stated in the module docstring and in README.md; this file does not restate them, it records the readings.

Readings — fixture contract

Measured in this worktree with HEAD at d85cb40, 2026-10-29T00:07Z, on the bytes listed above:

command	output	exit
`line_counter.py wizard-work/spark/test_log.txt`	`files: 1`, `lines: 9`, `records: 10`, breakdown `-`	0
`line_counter.py --fields wizard-work/spark/test_data.csv`	`fields: 5`	0
`printf 'a\n\n\nb\n'` → `line_counter.py F`	`lines: 4`, `records: 4`, `blank: 2`, `code: 2`	0
`printf 'a\n\n\nb'` → `line_counter.py F`	`lines: 3`, `records: 4`	0
`printf '"a,b",c\n'` → `--fields`	`fields: 2` (`split(',')` reads 3)	0
`printf 'a b a\n'` → `--words`	`words: 3`, `chars: 6`	0
`printf 'caf\351\n'` → `--words`	`words: 1`, `chars: 5`	0 (no traceback)
missing path	`ERROR: no such file or directory: '…'` on stderr, empty stdout	2
no argument	usage on stderr	1
`--fields DIR`	`ERROR: --fields needs a file, not a directory: '…'`	2

The caf\351 row is the case the pipe measured on the replaced tool: a latin-1 byte escaped its except (FileNotFoundError, PermissionError, IOError) clause as an uncaught UnicodeDecodeError traceback, because UnicodeDecodeError is a ValueError. The survivor decodes with errors="replace".

Readings — the cells, and what they kill

python3 wizard-work/spark/test_line_counter.py → 13/13 cells passed, rc 0, including (5 mutants applied to scratch copies, 5 killed). The mutation cell copies the module, applies one edit, asserts the edit landed, and re-runs this same suite against the copy expecting rc 1 and the named cell among the failures:

mutation	caught by
`count` → `return 10, 10`	`blank_lines_are_lines`
`count` → non-blank lines only	`blank_lines_are_lines`
`records = lines`	`final_unterminated_fragment_is_a_record`
`fields` → `len(first.split(','))`	`quoted_separator_is_data`
`words` → `len(set(split()))`	`word_and_char_counts`

The first two are the mutations reply 22 measured surviving the 3-cell suite this file replaces. The suite also runs against a module named by LINE_COUNTER=…, which is how the mutation cells grade a copy instead of this tree, and it resolves its own directory, so it reads the same from the repo root and from the lane directory.

Against the pre-Q125 tool (git show d85cb40:wizard-work/spark/line_counter.py, md5 30f1ee5c8c32642336f45a13d724eff5, <filepath> <mode>), the same suite reads 0/13 cells passed, rc 1 and verify_line_counter.sh reads 15 readings moved, rc 1 — the old interface cannot answer the questions the item asks, which is why the retirement and the definition arrive together.

Readings — behaviour that moved, versus the tool it replaces

The retired 65-line version at 7359bce was run beside the survivor on the *.py files of that revision (one file, line_counter.py itself):

tool	files	total	code	comment	empty
`7359bce:line_counter.py` (readlines)	1	66	51	1	14
survivor (`records` / `code` / `comment` / `blank`)	1	66	51	1	14

Identical partition, plus the figure the old tool never printed: lines: 65, because that file has no trailing newline. So the recursive-directory mode and the code/comment/empty breakdown the v2 dropped are back, and the lines-versus- records distinction the old file's +112 arithmetic was really about is now a printed key instead of a paragraph.

What this file does not certify

That line_counter.py is the only counter left in the repository — it is the only one left in wizard-work/spark whose output is a text-statistic number, with the boundary and the files outside it named in the Q125 record; that the walk's ordering is guaranteed beyond os.walk's sorted per-directory order; that a permission-denied file is refused as exit 2 rather than as some other error (this host runs as root, so that clause cannot be provoked honestly — the same limit reply 22 recorded); and that the retired files' own defects are catalogued — they were read for purpose and their md5s recorded, not graded.

5.3 KiB Raw Blame History