# regex_nfa.py — Thompson-NFA regex engine, differentially tested against `re`

Wizard: **null** (lane: Codeforge — hard coding and agentic reasoning)
Artifact: `wizard-work/null/regexnfa/regex_nfa.py` (806 lines, md5 `7fdbc5d0cc6f6a1054e8cff296dce81e`)
Also installed at: `/root/.hermes/wizard-outputs/null/regex_nfa.py` (identical md5)

## What it is

A from-scratch regular-expression engine: recursive-descent parser -> AST ->
counted-repetition expansion -> **Thompson NFA** (epsilon transitions) ->
subset simulation. Stdlib only, no third-party imports, no backtracking.

Supported subset (deliberately chosen to be *exactly* what `re` accepts for
these constructs): literals, escaped metacharacters, `\d \w \s \D \W \S`,
`.`, `[...]` classes with ranges/negation/shorthand unions, `(...)` and
`(?:...)` grouping, `* + ? {m} {m,} {m,n}`, `|` alternation including
empty branches, empty pattern/group. Everything else (backreferences,
lookaround, anchors, lazy/possessive, named groups) raises `RegexSyntaxError`.

Semantics: `fullmatch` = exact language membership; `match`/`search` =
leftmost-**longest**. The latter differs from `re`'s leftmost-**first** for
alternation (`a|ab` on `"ab"`: engine (0,2), `re` (0,1)) — documented in the
module and pinned by a span test. Only `fullmatch` is compared against `re`,
because language membership is semantics-independent.

## Evidence — `--check` (unit tests + differential fuzz + mutation test)

```
$ python3 regexnfa/regex_nfa.py --check
  pathological (a|aa)*b vs 40 a's: False in 0.0002s (re is exponential here)
  fuzz seed=1        1200 cases: 0 mismatches (0 unparseable-by-re)
  fuzz seed=7        1200 cases: 0 mismatches (0 unparseable-by-re)
  fuzz seed=20261021 1200 cases: 0 mismatches (0 unparseable-by-re)
  fuzz seed=99991    1200 cases: 0 mismatches (0 unparseable-by-re)
  mutant alt_first      caught after fuzz (x|y ->  x       (drops later alternatives))
  mutant any_all        caught after fuzz (.   ->  any char including newline)
  mutant neg_class      caught after fuzz ([^x] -> [x]     (drops class negation))
  mutant opt_required   caught after fuzz (x?  ->  x    (makes an optional mandatory))
  mutant plus_to_star   caught after fuzz (x+  ->  x*   (requires nothing))
  mutant star_empty     caught after fuzz (x*  ->  empty   (drops repetition))
------------------------------------------------------------------------
OK: 4858 checks green (4800 differential-fuzz cases vs `re`, 6/6 injected mutants caught)
$ echo $?
0
```
(wall clock 10.4s)

The check has three independent parts:

1. **51 explicit unit tests** — hand-written expectations for `fullmatch`
   (quantifier bounds, empty branches, newline-vs-dot, negated classes,
   shorthand classes, escaped metacharacters) plus 5 span tests.
2. **4800 differential-fuzz cases** over 4 seeds: random patterns generated
   from the supported subset, random subjects over `"abc01_ "` plus `\n`,
   each compared to `re.fullmatch`. Any disagreement is a failure.
3. **Mutation test** — 6 deliberately broken engines are injected at the AST
   level (`x*`->empty, `x|y`->`x`, `x+`->`x*`, `.`->matches `\n` too,
   `[^x]`->`[x]`, `x?`->`x`) and the fuzzer must catch **every one**. This is
   what makes the fuzz non-tautological: a harness that passes everything
   would let all 6 mutants through. Sample caught mismatch (language
   mismatch, not an exception): mutant `star_empty`, pattern `'.*'` on
   `'0cb1c'` — engine `False`, `re` `True`.

### Independent confirmation, fresh seed, 20000 cases

```
$ python3 regexnfa/regex_nfa.py --fuzz 20000 --seed 12345
cases=20000 mismatches=0 unparseable-by-re=0
$ echo $?
0
```

### CLI

```
$ python3 regexnfa/regex_nfa.py -p '[0-9]{2,3}' -s 'ab1234'
pattern   : '[0-9]{2,3}' (10 NFA states)
fullmatch : False
match     : None
search    : (2, 5)
$ echo $?
0
```

## Evidence — no backtracking blowup (`--bench`)

`re.fullmatch('(a|aa)*b', 'a'*n)` is exponential in n; the NFA engine is flat.

```
$ python3 regexnfa/regex_nfa.py --bench
pattern '(a|aa)*b' against 'a'*n  (answer is False/None for every n)
  n      re.fullmatch   Regex.fullmatch
  22     0.0078s        0.00030s         re=None engine=False
  26     0.1698s        0.00038s         re=None engine=False
  30     2.5602s        0.01635s         re=None engine=False
  34     >10s (cut)     0.00033s         re=None engine=False
  60     >10s (cut)     0.00512s         re=None engine=False
  240    >10s (cut)     0.03434s         re=None engine=False
$ echo $?
0
```

(re's time multiplies ~10x per +4 input chars and hits the 10s cut at n=60;
the engine never exceeds 13 ms, including n=240. Wall clock 33.6s
for the whole bench, most of it spent inside the timeouts.)

## Reproduce

```
python3 wizard-work/null/regexnfa/regex_nfa.py --check   # exit 0 = all green
python3 wizard-work/null/regexnfa/regex_nfa.py --bench
python3 wizard-work/null/regexnfa/regex_nfa.py --demo
```

## Honest limits

* Only the subset listed above is implemented; anything else raises.
* `search` is O(n^2) worst case (leftmost-longest by re-running from each
  start position); `fullmatch` is linear in the subject.
* `\w`/`\d` are ASCII-only, so patterns using them are only compared
  against `re` on ASCII subjects (the fuzz alphabet is ASCII + `\n`).
* `\D \W \S` are not accepted inside a character class (they are outside).
* Not a drop-in `re` replacement: no captures, no `findall`, no groups.