timmy-config/docs/glitch-detection.md

# 3D World Glitch Detection — Matrix Scanner

**Reference:** timmy-config#491  
**Label:** gemma-4-multimodal  
**Version:** 0.2.0  

## Overview

The Matrix Glitch Detector scans 3D web worlds for visual artifacts and rendering anomalies. It uses browser automation to capture screenshots from multiple camera angles, then sends them to a vision AI model for analysis against a library of known glitch patterns.

## Detected Glitch Categories

| Category | Severity | Description |
|---|---|---|
| Floating Assets | HIGH | Objects not grounded — hovering above surfaces |
| Z-Fighting | MEDIUM | Coplanar surfaces flickering/competing for depth |
| Missing Textures | CRITICAL | Placeholder colors (magenta, checkerboard) |
| Clipping | HIGH | Geometry passing through other objects |
| Broken Normals | MEDIUM | Inside-out or incorrectly lit surfaces |
| Shadow Artifacts | LOW | Detached, mismatched, or acne shadows |
| LOD Popping | LOW | Abrupt level-of-detail transitions |
| Lightmap Errors | MEDIUM | Dark splotches, light leaks, baking failures |
| Water/Reflection | MEDIUM | Incorrect environment reflections |
| Skybox Seam | LOW | Visible seams at cubemap face edges |
| Shader Failure | HIGH | Materials failing to render correctly |
| UV Mapping Error | MEDIUM | Incorrect texture coordinate mapping |
| Frustum Culling | INFO | Objects incorrectly culled from view |
| Texture Placeholder | CRITICAL | Fallback magenta/checkerboard textures |

## Installation

No external dependencies required — pure Python 3.10+.

```bash
# Clone the repo
git clone https://forge.alexanderwhitestone.com/Timmy_Foundation/timmy-config.git
cd timmy-config
```

## Usage

### Basic Scan

```bash
python bin/matrix_glitch_detector.py https://matrix.example.com/world/alpha
```

### Multi-Angle Scan

```bash
python bin/matrix_glitch_detector.py https://matrix.example.com/world/alpha \
    --angles 8 \
    --output glitch_report.json
```

### Demo Mode

```bash
python bin/matrix_glitch_detector.py --demo
```

### Options

| Flag | Default | Description |
|---|---|---|
| `url` | (required) | URL of the 3D world to scan |
| `--angles N` | 4 | Number of camera angles to capture |
| `--output PATH` | stdout | Output file for JSON report |
| `--min-severity` | info | Minimum severity: info/low/medium/high/critical |
| `--demo` | off | Run with simulated detections |
| `--verbose` | off | Enable verbose output |

## Report Format

The JSON report includes:

```json
{
  "scan_id": "uuid",
  "url": "https://...",
  "timestamp": "ISO-8601",
  "total_screenshots": 4,
  "angles_captured": ["front", "right", "back", "left"],
  "glitches": [
    {
      "id": "short-uuid",
      "category": "floating_assets",
      "name": "Floating Chair",
      "description": "Office chair floating 0.3m above floor",
      "severity": "high",
      "confidence": 0.87,
      "location_x": 35.2,
      "location_y": 62.1,
      "screenshot_index": 0,
      "screenshot_angle": "front",
      "timestamp": "ISO-8601"
    }
  ],
  "summary": {
    "total_glitches": 4,
    "by_severity": {"critical": 1, "high": 2, "medium": 1},
    "by_category": {"floating_assets": 1, "missing_textures": 1, ...},
    "highest_severity": "critical",
    "clean_screenshots": 0
  },
  "metadata": {
    "detector_version": "0.2.0",
    "pattern_count": 15,
    "reference": "timmy-config#491"
  }
}
```

## Vision AI Integration

The detector supports any OpenAI-compatible vision API. Set these environment variables:

```bash
export VISION_API_KEY="***"
export VISION_API_BASE="https://api.openai.com/v1"  # optional
export VISION_MODEL="gpt-4o"  # optional, default: gpt-4o
```

For browser-based capture with `browser_vision`:

```bash
export BROWSER_VISION_SCRIPT="/path/to/browser_vision.py"
```

## Glitch Patterns

Pattern definitions live in `bin/glitch_patterns.py`. Each pattern includes:

- **category** — Enum matching the glitch type
- **detection_prompts** — Instructions for the vision model
- **visual_indicators** — What to look for in screenshots
- **confidence_threshold** — Minimum confidence to report

### Pattern Definitions (Updated 2026-04-15)

| Category | Name | Severity | Keywords |
|---|---|---|---|
| Floating Assets | Floating Object | HIGH | float, ground, support, shadow |
| Z-Fighting | Z-Fighting Flicker | MEDIUM | flicker, shimmer, overlap |
| Missing Textures | Missing Texture | CRITICAL | magenta, checkerboard, placeholder |
| Clipping | Geometry Clipping | HIGH | clipping, intersect, pass through |
| Broken Normals | Broken Normals | MEDIUM | dark, inverted, lighting |
| Shadow Artifacts | Shadow Artifact | LOW | shadow acne, detached, peter panning |
| LOD | LOD Transition Pop | LOW | pop-in, mipmap, lod |
| Lightmap | Lightmap Error | MEDIUM | dark patch, light leak, bake |
| Water | Water Reflection | MEDIUM | reflection, water, mirror |
| Skybox | Skybox Seam | LOW | seam, sky edge |
| Shader | Shader Failure | HIGH | shader, material, render |
| UV | UV Mapping Error | MEDIUM | uv, texture coord, mapping |
| Frustum | Frustum Culling | INFO | cull, frustum, clip |

### Adding Custom Patterns

```python
from glitch_patterns import GlitchPattern, GlitchCategory, GlitchSeverity

custom = GlitchPattern(
    category=GlitchCategory.FLOATING_ASSETS,
    name="Custom Glitch",
    description="Your description",
    severity=GlitchSeverity.MEDIUM,
    detection_prompts=["Look for..."],
    visual_indicators=["indicator 1", "indicator 2"],
)
```

## Testing

```bash
python -m pytest tests/test_glitch_detector.py -v
# or
python tests/test_glitch_detector.py
```

## Architecture

```
bin/
  matrix_glitch_detector.py  — Main CLI entry point
  glitch_patterns.py         — Pattern definitions and prompt builder
tests/
  test_glitch_detector.py    — Unit and integration tests
docs/
  glitch-detection.md        — This documentation
```

## Validation Against Real Scenes (2026-04-15)

Tested against three Three.js benchmark scenes:

| Scene | Patterns Detected | False Positives | Notes |
|---|---|---|---|
| Animation Blending | shadow_map_artifact (0.4) | 0 | Minor shadow Peter Panning accepted |
| Unreal Bloom | bloom_overflow (0.85) | 0 | Correctly identified threshold=0 issue |
| Shadow Map | shadow_map_artifact (0.4) | 0 | Minor bias confirmed |

**Result:** 2 of 6 target patterns correctly detected; remaining 4 patterns (shader_failure, texture_placeholder, uv_mapping_error, frustum_culling) correctly returned no false positives.

## Limitations

- Browser automation requires a headless browser environment
- Vision AI analysis depends on model availability and API limits
- Placeholder screenshots are generated when browser capture is unavailable
- Detection accuracy varies by scene complexity and lighting conditions
- Some patterns require specific scene content to trigger