Compare commits

..

6 Commits

Author SHA1 Message Date
Alexander Whitestone
08233364ff burn: add smoke test workflow — parse check + secret scan
All checks were successful
Smoke Test / smoke-test (pull_request) Successful in 10s
Build Validation / validate-manuscript (pull_request) Successful in 9s
Smoke Test / smoke-test (push) Successful in 8s
Closes #27

Adds a dead-simple CI smoke test that runs on every PR and push to main:

Parse checks:
- Chapter validation (structure, numbering, H1 headers)
- Markdown build (combines all chapters)
- Compiled manuscript size verification (>10k words)
- Python syntax check on all .py files
- YAML syntax check on workflow files

Secret scan:
- Scans for common API key/token patterns (sk-ant-, sk-or-, ghp_, AKIA, etc.)
- Searches all text files, excludes .git and the smoke test itself
- Hard fail if any secrets found

Two files:
- scripts/smoke.sh — the smoke test script
- .gitea/workflows/smoke.yml — Gitea Actions workflow
2026-04-10 20:58:16 -04:00
544bc1a985 Merge pull request 'feat: add CI workflow for manuscript build validation' (#25) from feat/ci-build-validation into main
Merged PR #25: feat: add CI workflow for manuscript build validation
2026-04-11 00:44:01 +00:00
ba9fd0ba08 Merge pull request 'burn: add chapter validation to build pipeline (closes #24)' (#26) from burn/20260410-chapter-validation into main
Merged PR #26: burn: add chapter validation to build pipeline
2026-04-11 00:43:38 +00:00
8ba9f58e96 Merge pull request 'feat: add book compilation pipeline (rescued from #20)' (#28) from rescue/book-compilation into main
Merged PR #28: feat: add book compilation pipeline
2026-04-11 00:43:36 +00:00
Alexander Whitestone
948d520b83 burn: add chapter validation to build pipeline (closes #24)
Add validate_chapters() function that checks:
- No empty chapter files (whitespace-only counts as empty)
- Every chapter starts with an H1 header (# Chapter N — Title)
- No gaps in chapter numbering (sequential from 1)
- No duplicate chapter numbers
- Header chapter number matches filename number
- Warns on suspiciously short chapters (<50 words)

Validation runs automatically before compilation. If errors are found,
compilation is aborted with clear error messages showing exactly what
to fix.

CLI flags:
  python3 compile.py --validate     # validate only
  python3 compile.py --no-validate  # skip validation
  python3 compile.py                # validate then compile
2026-04-10 19:57:27 -04:00
7a56b4b727 feat: add CI workflow for manuscript build validation
Some checks failed
Build Validation / validate-manuscript (pull_request) Failing after 5s
2026-04-10 23:55:17 +00:00
4 changed files with 309 additions and 12 deletions

View File

@@ -0,0 +1,24 @@
name: Smoke Test
on:
pull_request:
branches: [main]
push:
branches: [main]
jobs:
smoke-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.x'
- name: Install PyYAML
run: pip install pyyaml
- name: Run smoke test
run: bash scripts/smoke.sh

View File

@@ -0,0 +1,22 @@
name: Build Validation
on:
pull_request:
branches: [main]
jobs:
validate-manuscript:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.x'
- name: Run Chapter Validation
run: |
# Run the build script with --md flag which triggers validation
# If validation fails, the script exits with code 1, failing the CI
python3 build/build.py --md

View File

@@ -8,12 +8,15 @@ Uses chapters, front matter, back matter, and references illustrations.
Requirements: pip install markdown weasyprint (or use pandoc)
Usage:
python3 compile.py # generates testament-complete.md
python3 compile.py # validate then compile
python3 compile.py --validate # validate only, no compile
python3 compile.py --no-validate # skip validation, compile directly
pandoc testament-complete.md -o testament.pdf --pdf-engine=weasyprint
"""
import os
import re
import sys
BASE = os.path.dirname(os.path.abspath(__file__))
CHAPTERS_DIR = os.path.join(BASE, "chapters")
@@ -28,17 +31,147 @@ PARTS = {
11: ("THE LIGHT", "Thomas at the door. The network. The story breaks. The green light."),
}
def read_file(path):
with open(path, 'r') as f:
return f.read()
def get_chapter_number(filename):
match = re.search(r'chapter-(\d+)', filename)
return int(match.group(1)) if match else 0
def compile():
def validate_chapters(chapters_dir=CHAPTERS_DIR):
"""Validate chapter files before compilation.
Checks:
- No empty chapter files (whitespace-only counts as empty)
- Every chapter starts with an H1 header (# Title)
- No gaps in chapter numbering (sequential from 1)
- No duplicate chapter numbers
Returns:
(is_valid, errors) where errors is a list of human-readable strings.
"""
errors = []
warnings = []
if not os.path.isdir(chapters_dir):
errors.append(f"Chapters directory not found: {chapters_dir}")
return False, errors
# Collect chapter files
chapter_files = []
for f in sorted(os.listdir(chapters_dir)):
if f.startswith("chapter-") and f.endswith(".md"):
num = get_chapter_number(f)
chapter_files.append((num, f))
if not chapter_files:
errors.append("No chapter files found in chapters/ directory")
return False, errors
chapter_files.sort()
# Check for duplicates
seen_numbers = {}
for num, filename in chapter_files:
if num in seen_numbers:
errors.append(
f"Duplicate chapter number {num}: {filename} and {seen_numbers[num]}"
)
seen_numbers[num] = filename
# Check for gaps in numbering
if chapter_files:
expected = list(range(1, chapter_files[-1][0] + 1))
found = [num for num, _ in chapter_files]
missing = sorted(set(expected) - set(found))
if missing:
errors.append(
f"Missing chapter(s): {', '.join(str(n) for n in missing)}"
)
# Validate individual chapter files
for num, filename in chapter_files:
filepath = os.path.join(chapters_dir, filename)
# Check file is not empty
try:
content = read_file(filepath)
except Exception as e:
errors.append(f"{filename}: cannot read — {e}")
continue
if not content.strip():
errors.append(f"{filename}: file is empty")
continue
# Check word count (warn if suspiciously short)
word_count = len(content.split())
if word_count < 50:
warnings.append(
f"{filename}: only {word_count} words (possible truncation)"
)
# Check starts with H1 header
first_line = content.strip().split('\n')[0]
if not first_line.startswith('# '):
errors.append(
f"{filename}: missing H1 header — "
f"expected '# Chapter {num} — Title', got '{first_line[:60]}'"
)
else:
# Verify the H1 matches expected chapter number
header_match = re.match(r'^#\s+Chapter\s+(\d+)', first_line)
if header_match:
header_num = int(header_match.group(1))
if header_num != num:
errors.append(
f"{filename}: header says Chapter {header_num} "
f"but filename says Chapter {num}"
)
else:
warnings.append(
f"{filename}: H1 header doesn't follow "
f"'# Chapter N — Title' pattern: '{first_line[:60]}'"
)
# Report
valid = len(errors) == 0
if warnings:
print(f"Validation: {len(warnings)} warning(s)")
for w in warnings:
print(f"{w}")
if errors:
print(f"Validation: FAILED — {len(errors)} error(s)")
for e in errors:
print(f"{e}")
else:
print(
f"Validation: PASSED — {len(chapter_files)} chapters, "
f"chapters {chapter_files[0][0]}{chapter_files[-1][0]}"
)
return valid, errors
def compile(skip_validation=False):
"""Compile all chapters into a single markdown file."""
# Pre-compilation validation
if not skip_validation:
valid, errors = validate_chapters()
if not valid:
print("\nCompilation aborted. Fix the errors above and try again.")
sys.exit(1)
print()
output = []
# Title page
output.append("""---
title: "The Testament"
@@ -66,7 +199,7 @@ with Timmy
---
""")
# Get all chapters sorted
chapters = []
for f in os.listdir(CHAPTERS_DIR):
@@ -74,7 +207,7 @@ with Timmy
num = get_chapter_number(f)
chapters.append((num, f))
chapters.sort()
current_part = 0
for num, filename in chapters:
# Insert part divider if needed
@@ -82,28 +215,28 @@ with Timmy
part_name, part_desc = PARTS[num]
current_part += 1
output.append(f"\n---\n\n# PART {current_part}: {part_name}\n\n*{part_desc}*\n\n---\n")
# Read chapter content
content = read_file(os.path.join(CHAPTERS_DIR, filename))
# Skip the chapter header (we'll add our own formatting)
lines = content.split('\n')
body = '\n'.join(lines[1:]).strip() # Skip "# Chapter X — Title"
# Add chapter
output.append(f"\n{lines[0]}\n\n{body}\n")
# Back matter
output.append("\n---\n")
back = read_file(BACK_MATTER)
# Clean up the back matter for print
output.append(back)
# Write compiled markdown
compiled = '\n'.join(output)
with open(OUTPUT, 'w') as f:
f.write(compiled)
# Stats
words = len(compiled.split())
lines_count = compiled.count('\n')
@@ -116,5 +249,12 @@ with Timmy
print(f" # or")
print(f" pandoc {OUTPUT} -o testament.epub --epub-cover-image=cover-art.jpg")
if __name__ == "__main__":
compile()
if "--validate" in sys.argv:
valid, _ = validate_chapters()
sys.exit(0 if valid else 1)
elif "--no-validate" in sys.argv:
compile(skip_validation=True)
else:
compile()

111
scripts/smoke.sh Executable file
View File

@@ -0,0 +1,111 @@
#!/usr/bin/env bash
# The Testament — Smoke Test
# Dead simple CI: parse check + secret scan.
# Ref: https://forge.alexanderwhitestone.com/Timmy_Foundation/the-testament/issues/27
set -euo pipefail
PASS=0
FAIL=0
pass() { echo "$1"; PASS=$((PASS + 1)); }
fail() { echo "$1"; FAIL=$((FAIL + 1)); }
REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
cd "$REPO_ROOT"
# ─── Section 1: Parse checks ───────────────────────────────────────
echo "── Parse Checks ──"
# 1a. Chapter validation (structure, numbering, headers)
if python3 compile.py --validate 2>&1; then
pass "Chapter validation passed"
else
fail "Chapter validation failed"
fi
# 1b. Build markdown combination
if python3 build/build.py --md >/dev/null 2>&1; then
pass "Markdown build passed"
else
fail "Markdown build failed"
fi
# 1c. Verify compiled output exists and is non-empty
if [ -s build/the-testament-full.md ]; then
WORDS=$(wc -w < build/the-testament-full.md | tr -d ' ')
if [ "$WORDS" -gt 10000 ]; then
pass "Compiled manuscript: $WORDS words"
else
fail "Compiled manuscript suspiciously short: $WORDS words"
fi
else
fail "Compiled manuscript missing or empty"
fi
# 1d. Python syntax check on all .py files
PY_OK=true
for f in $(find . -name "*.py" -not -path "./.git/*"); do
if ! python3 -c "import ast; ast.parse(open('$f').read())" 2>/dev/null; then
fail "Python syntax error in $f"
PY_OK=false
fi
done
if $PY_OK; then
pass "All Python files parse cleanly"
fi
# 1e. YAML syntax check on workflow files
YAML_OK=true
for f in $(find .gitea -name "*.yml" -o -name "*.yaml" 2>/dev/null); do
if ! python3 -c "import yaml; yaml.safe_load(open('$f'))" 2>/dev/null; then
fail "YAML syntax error in $f"
YAML_OK=false
fi
done
if $YAML_OK; then
pass "All YAML files parse cleanly"
fi
# ─── Section 2: Secret scan ────────────────────────────────────────
echo ""
echo "── Secret Scan ──"
# Patterns that should never appear in a book repo
SECRET_PATTERNS=(
"sk-ant-"
"sk-or-"
"sk-[a-zA-Z0-9]{20,}"
"ghp_[a-zA-Z0-9]{36}"
"gho_[a-zA-Z0-9]{36}"
"AKIA[0-9A-Z]{16}"
"AKIA[A-Z0-9]{16}"
"xox[bpsa]-"
"SG\."
"-----BEGIN (RSA |EC |DSA |OPENSSH )?PRIVATE KEY"
)
FOUND_SECRETS=false
for pattern in "${SECRET_PATTERNS[@]}"; do
# Search text files only, skip .git and binary files
HITS=$(grep -rn "$pattern" --include="*.md" --include="*.py" --include="*.sh" --include="*.yml" --include="*.yaml" --include="*.json" --include="*.html" --include="*.js" --include="*.css" --include="*.txt" --include="*.cfg" --include="*.ini" --exclude-dir=.git . 2>/dev/null | grep -v "scripts/smoke.sh" || true)
if [ -n "$HITS" ]; then
fail "Possible secret found: $pattern"
echo "$HITS" | head -5
FOUND_SECRETS=true
fi
done
if ! $FOUND_SECRETS; then
pass "No secrets detected"
fi
# ─── Summary ───────────────────────────────────────────────────────
echo ""
echo "Results: $PASS passed, $FAIL failed"
if [ "$FAIL" -gt 0 ]; then
echo "SMOKE TEST FAILED"
exit 1
else
echo "SMOKE TEST PASSED"
exit 0
fi