- Added comprehensive local model fine-tuning guide
- Created benchmarking script for inference performance
- Added training data collection script for merged PRs
- Documented current stack (Ollama + llama.cpp + Hermes 4)
- Provided quantization options and best practices
- Included troubleshooting and monitoring guidance
Addresses issue #486 recommendations:
✓ Documented local model stack for reproducibility
✓ Created benchmarking tools for inference latency
✓ Provided training data collection pipeline
✓ Documented quantization options for faster inference
✓ Included fine-tuning pipeline documentation
- Added extract_meaning_kernels.py for processing PDF diagrams
- Extracts text using OCR (Tesseract) when available
- Analyzes diagram structure (type, dimensions, orientation)
- Generates structured meaning kernels with metadata
- Outputs JSON (machine-readable) and Markdown (human-readable)
- Includes test pipeline and documentation
- Supports single files and batch processing
Pipeline components:
- DiagramProcessor: Main processing engine
- MeaningKernel: Structured kernel representation
- PDF to image conversion
- OCR text extraction
- Structure analysis
- Kernel generation with confidence scoring
Acceptance criteria met:
✓ Processes academic PDF diagrams
✓ Extracts structured text meaning kernels
✓ Generates machine-readable JSON output
✓ Includes human-readable reports
✓ Supports batch processing
✓ Provides confidence scoring
