docs: add visual evidence for scene description generator #689
Some checks failed
Architecture Lint / Linter Tests (pull_request) Successful in 26s
Smoke Test / smoke (pull_request) Failing after 24s
Validate Config / YAML Lint (pull_request) Failing after 22s
Validate Config / JSON Validate (pull_request) Successful in 20s
Validate Config / Python Syntax & Import Check (pull_request) Failing after 1m29s
Validate Config / Shell Script Lint (pull_request) Failing after 46s
Validate Config / Cron Syntax Check (pull_request) Successful in 15s
Validate Config / Deploy Script Dry Run (pull_request) Successful in 15s
Validate Config / Playbook Schema Validation (pull_request) Successful in 30s
PR Checklist / pr-checklist (pull_request) Failing after 10m10s
Architecture Lint / Lint Repository (pull_request) Has been cancelled
Validate Config / Python Test Suite (pull_request) Has been cancelled
Some checks failed
Architecture Lint / Linter Tests (pull_request) Successful in 26s
Smoke Test / smoke (pull_request) Failing after 24s
Validate Config / YAML Lint (pull_request) Failing after 22s
Validate Config / JSON Validate (pull_request) Successful in 20s
Validate Config / Python Syntax & Import Check (pull_request) Failing after 1m29s
Validate Config / Shell Script Lint (pull_request) Failing after 46s
Validate Config / Cron Syntax Check (pull_request) Successful in 15s
Validate Config / Deploy Script Dry Run (pull_request) Successful in 15s
Validate Config / Playbook Schema Validation (pull_request) Successful in 30s
PR Checklist / pr-checklist (pull_request) Failing after 10m10s
Architecture Lint / Lint Repository (pull_request) Has been cancelled
Validate Config / Python Test Suite (pull_request) Has been cancelled
This commit is contained in:
74
docs/visual-evidence-689.md
Normal file
74
docs/visual-evidence-689.md
Normal file
@@ -0,0 +1,74 @@
|
||||
# Visual Evidence — Gemma 4 Multimodal Scene Description Generator
|
||||
|
||||
## Test Image: Coffee Beans (Macro Photo)
|
||||
|
||||
### Gemma 4 Vision Analysis (via Ollama)
|
||||
|
||||
**Model:** gemma4:latest (8B, Q4_K_M)
|
||||
**Input:** sample_photo.jpg (46KB JPEG)
|
||||
|
||||
**Structured Output (JSONL):**
|
||||
```json
|
||||
{
|
||||
"mood": "dark",
|
||||
"colors": ["dark brown", "espresso", "black"],
|
||||
"composition": "close-up",
|
||||
"camera": "static",
|
||||
"lighting": "soft",
|
||||
"description": "An extreme close-up shot captures a dense pile of roasted coffee beans. The beans are a uniform, deep dark brown and appear slightly oily, filling the entire frame. The focus emphasizes the rich texture and individual shapes of the beans."
|
||||
}
|
||||
```
|
||||
|
||||
### Hermes Vision Analysis (Cross-Validation)
|
||||
|
||||
**Scene ID:** COFFEE_MACRO_001
|
||||
**Mood:** Warm, aromatic, and comforting
|
||||
**Dominant Colors:** Deep umber, burnt sienna, espresso black, mahogany
|
||||
**Composition:** Full-frame fill, centrally weighted
|
||||
**Camera:** High-angle, close-up (Macro)
|
||||
**Lighting:** Soft, diffused top-lighting
|
||||
|
||||
## Test Image: Abstract Geometric Composition
|
||||
|
||||
### Gemma 4 Vision Analysis
|
||||
|
||||
**Input:** scene1.jpg (10KB, PIL-generated)
|
||||
|
||||
**Structured Output (JSONL):**
|
||||
```json
|
||||
{
|
||||
"mood": "energetic",
|
||||
"colors": ["deep blue", "yellow", "coral"],
|
||||
"composition": "wide-shot",
|
||||
"camera": "static",
|
||||
"lighting": "artificial",
|
||||
"description": "This is an abstract graphic composition set against a solid, deep blue background. A bright yellow square is placed in the upper left quadrant, while a large, solid coral-colored circle occupies the lower right quadrant. The geometric shapes create a high-contrast, minimalist visual balance."
|
||||
}
|
||||
```
|
||||
|
||||
## Verification Summary
|
||||
|
||||
| Test | Status | Details |
|
||||
|------|--------|---------|
|
||||
| Model detection | ✅ PASS | `gemma4:latest` auto-detected |
|
||||
| Image scanning | ✅ PASS | 2 images found recursively |
|
||||
| Vision analysis | ✅ PASS | Both images described accurately |
|
||||
| JSON parsing | ✅ PASS | Structured output with all fields |
|
||||
| Training format | ✅ PASS | JSONL with source, model, timestamp |
|
||||
| ShareGPT format | ⚠️ PARTIAL | Works but needs retry on rate limit |
|
||||
|
||||
## Running the Generator
|
||||
|
||||
```bash
|
||||
# Check model availability
|
||||
python scripts/generate_scene_descriptions.py --check-model
|
||||
|
||||
# Generate scene descriptions from assets
|
||||
python scripts/generate_scene_descriptions.py --input ./assets --output training-data/scene-descriptions-auto.jsonl
|
||||
|
||||
# Limit to 10 files with specific model
|
||||
python scripts/generate_scene_descriptions.py --input ./assets --model gemma4:latest --limit 10
|
||||
|
||||
# ShareGPT format for training pipeline
|
||||
python scripts/generate_scene_descriptions.py --input ./assets --format sharegpt
|
||||
```
|
||||
Reference in New Issue
Block a user