allegro/allegro-checkpoint
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
13fca8ebea9e5147b020e241b516c5eb2688cef0
allegro-checkpoint/skills/creative/ascii-video/references/architecture.md
Allegro 13fca8ebea Checkpoint: Allegro state pre-migration
2026-04-01 11:04:00 +00:00

32 KiB
Raw Blame History

Architecture Reference

See also: composition.md · effects.md · scenes.md · shaders.md · inputs.md · optimization.md · troubleshooting.md

Grid System

Resolution Presets

RESOLUTION_PRESETS = {
    "landscape":  (1920, 1080),  # 16:9 — YouTube, default
    "portrait":   (1080, 1920),  # 9:16 — TikTok, Reels, Stories
    "square":     (1080, 1080),  # 1:1  — Instagram feed
    "ultrawide":  (2560, 1080),  # 21:9 — cinematic
    "landscape4k":(3840, 2160),  # 16:9 — 4K
    "portrait4k": (2160, 3840),  # 9:16 — 4K portrait
}

def get_resolution(preset="landscape", custom=None):
    """Returns (VW, VH) tuple."""
    if custom:
        return custom
    return RESOLUTION_PRESETS.get(preset, RESOLUTION_PRESETS["landscape"])

Multi-Density Grids

Pre-initialize multiple grid sizes. Switch per section for visual variety. Grid dimensions auto-compute from resolution:

Landscape (1920x1080):

Key Font Size Grid (cols x rows) Use
xs 8 400x108 Ultra-dense data fields
sm 10 320x83 Dense detail, rain, starfields
md 16 192x56 Default balanced, transitions
lg 20 160x45 Quote/lyric text (readable at 1080p)
xl 24 137x37 Short quotes, large titles
xxl 40 80x22 Giant text, minimal

Portrait (1080x1920):

Key Font Size Grid (cols x rows) Use
xs 8 225x192 Ultra-dense, tall data columns
sm 10 180x148 Dense detail, vertical rain
md 16 112x100 Default balanced
lg 20 90x80 Readable text (~30 chars/line centered)
xl 24 75x66 Short quotes, stacked
xxl 40 45x39 Giant text, minimal

Square (1080x1080):

Key Font Size Grid (cols x rows) Use
sm 10 180x83 Dense detail
md 16 112x56 Default balanced
lg 20 90x45 Readable text

Key differences in portrait mode:

  • Fewer columns (90 at lg vs 160) — lines must be shorter or wrap
  • Many more rows (80 at lg vs 45) — vertical stacking is natural
  • Aspect ratio correction flips: asp = cw / ch still works but the visual emphasis is vertical
  • Radial effects appear as tall ellipses unless corrected
  • Vertical effects (rain, embers, fire columns) are naturally enhanced
  • Horizontal effects (spectrum bars, waveforms) need rotation or compression

Grid sizing for text in portrait: Use lg (20px) for 2-3 word lines. Max comfortable line length is ~25-30 chars. For longer quotes, break aggressively into many short lines stacked vertically — portrait has vertical space to spare. xl (24px) works for single words or very short phrases.

Grid dimensions: cols = VW // cell_width, rows = VH // cell_height.

Font Selection

Don't hardcode a single font. Choose fonts to match the project's mood. Monospace fonts are required for grid alignment but vary widely in personality:

Font Personality Platform
Menlo Clean, neutral, Apple-native macOS
Monaco Retro terminal, compact macOS
Courier New Classic typewriter, wide Cross-platform
SF Mono Modern, tight spacing macOS
Consolas Windows native, clean Windows
JetBrains Mono Developer, ligature-ready Install
Fira Code Geometric, modern Install
IBM Plex Mono Corporate, authoritative Install
Source Code Pro Adobe, balanced Install

Font detection at init: probe available fonts and fall back gracefully:

import platform

def find_font(preferences):
    """Try fonts in order, return first that exists."""
    for name, path in preferences:
        if os.path.exists(path):
            return path
    raise FileNotFoundError(f"No monospace font found. Tried: {[p for _,p in preferences]}")

FONT_PREFS_MACOS = [
    ("Menlo", "/System/Library/Fonts/Menlo.ttc"),
    ("Monaco", "/System/Library/Fonts/Monaco.ttf"),
    ("SF Mono", "/System/Library/Fonts/SFNSMono.ttf"),
    ("Courier", "/System/Library/Fonts/Courier.ttc"),
]
FONT_PREFS_LINUX = [
    ("DejaVu Sans Mono", "/usr/share/fonts/truetype/dejavu/DejaVuSansMono.ttf"),
    ("Liberation Mono", "/usr/share/fonts/truetype/liberation/LiberationMono-Regular.ttf"),
    ("Noto Sans Mono", "/usr/share/fonts/truetype/noto/NotoSansMono-Regular.ttf"),
    ("Ubuntu Mono", "/usr/share/fonts/truetype/ubuntu/UbuntuMono-R.ttf"),
]
FONT_PREFS_WINDOWS = [
    ("Consolas", r"C:\Windows\Fonts\consola.ttf"),
    ("Courier New", r"C:\Windows\Fonts\cour.ttf"),
    ("Lucida Console", r"C:\Windows\Fonts\lucon.ttf"),
    ("Cascadia Code", os.path.expandvars(r"%LOCALAPPDATA%\Microsoft\Windows\Fonts\CascadiaCode.ttf")),
    ("Cascadia Mono", os.path.expandvars(r"%LOCALAPPDATA%\Microsoft\Windows\Fonts\CascadiaMono.ttf")),
]

def _get_font_prefs():
    s = platform.system()
    if s == "Darwin":
        return FONT_PREFS_MACOS
    elif s == "Windows":
        return FONT_PREFS_WINDOWS
    return FONT_PREFS_LINUX

FONT_PREFS = _get_font_prefs()

Multi-font rendering: use different fonts for different layers (e.g., monospace for background, a bolder variant for overlay text). Each GridLayer owns its own font:

grid_bg = GridLayer(find_font(FONT_PREFS), 16)       # background
grid_text = GridLayer(find_font(BOLD_PREFS), 20)      # readable text

Collecting All Characters

Before initializing grids, gather all characters that need bitmap pre-rasterization:

all_chars = set()
for pal in [PAL_DEFAULT, PAL_DENSE, PAL_BLOCKS, PAL_RUNE, PAL_KATA,
            PAL_GREEK, PAL_MATH, PAL_DOTS, PAL_BRAILLE, PAL_STARS,
            PAL_HALFFILL, PAL_HATCH, PAL_BINARY, PAL_MUSIC, PAL_BOX,
            PAL_CIRCUIT, PAL_ARROWS, PAL_HERMES]:  # ... all palettes used in project
    all_chars.update(pal)
# Add any overlay text characters
all_chars.update("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789 .,-:;!?/|")
all_chars.discard(" ")  # space is never rendered

GridLayer Initialization

Each grid pre-computes coordinate arrays for vectorized effect math. The grid automatically adapts to any resolution (landscape, portrait, square):

class GridLayer:
    def __init__(self, font_path, font_size, vw=None, vh=None):
        """Initialize grid for any resolution.
        vw, vh: video width/height in pixels. Defaults to global VW, VH."""
        vw = vw or VW; vh = vh or VH
        self.vw = vw; self.vh = vh

        self.font = ImageFont.truetype(font_path, font_size)
        asc, desc = self.font.getmetrics()
        bbox = self.font.getbbox("M")
        self.cw = bbox[2] - bbox[0]  # character cell width
        self.ch = asc + desc  # CRITICAL: not textbbox height

        self.cols = vw // self.cw
        self.rows = vh // self.ch
        self.ox = (vw - self.cols * self.cw) // 2  # centering
        self.oy = (vh - self.rows * self.ch) // 2

        # Aspect ratio metadata
        self.aspect = vw / vh  # >1 = landscape, <1 = portrait, 1 = square
        self.is_portrait = vw < vh
        self.is_landscape = vw > vh

        # Index arrays
        self.rr = np.arange(self.rows, dtype=np.float32)[:, None]
        self.cc = np.arange(self.cols, dtype=np.float32)[None, :]

        # Polar coordinates (aspect-corrected)
        cx, cy = self.cols / 2.0, self.rows / 2.0
        asp = self.cw / self.ch
        self.dx = self.cc - cx
        self.dy = (self.rr - cy) * asp
        self.dist = np.sqrt(self.dx**2 + self.dy**2)
        self.angle = np.arctan2(self.dy, self.dx)

        # Normalized (0-1 range) -- for distance falloff
        self.dx_n = (self.cc - cx) / max(self.cols, 1)
        self.dy_n = (self.rr - cy) / max(self.rows, 1) * asp
        self.dist_n = np.sqrt(self.dx_n**2 + self.dy_n**2)

        # Pre-rasterize all characters to float32 bitmaps
        self.bm = {}
        for c in all_chars:
            img = Image.new("L", (self.cw, self.ch), 0)
            ImageDraw.Draw(img).text((0, 0), c, fill=255, font=self.font)
            self.bm[c] = np.array(img, dtype=np.float32) / 255.0

Character Render Loop

The bottleneck. Composites pre-rasterized bitmaps onto pixel canvas:

def render(self, chars, colors, canvas=None):
    if canvas is None:
        canvas = np.zeros((VH, VW, 3), dtype=np.uint8)
    for row in range(self.rows):
        y = self.oy + row * self.ch
        if y + self.ch > VH: break
        for col in range(self.cols):
            c = chars[row, col]
            if c == " ": continue
            x = self.ox + col * self.cw
            if x + self.cw > VW: break
            a = self.bm[c]  # float32 bitmap
            canvas[y:y+self.ch, x:x+self.cw] = np.maximum(
                canvas[y:y+self.ch, x:x+self.cw],
                (a[:, :, None] * colors[row, col]).astype(np.uint8))
    return canvas

Use np.maximum for additive blending (brighter chars overwrite dimmer ones, never darken).

Multi-Layer Rendering

Render multiple grids onto the same canvas for depth:

canvas = np.zeros((VH, VW, 3), dtype=np.uint8)
canvas = grid_lg.render(bg_chars, bg_colors, canvas)   # background layer
canvas = grid_md.render(main_chars, main_colors, canvas)  # main layer
canvas = grid_sm.render(detail_chars, detail_colors, canvas)  # detail overlay

Character Palettes

Design Principles

Character palettes are the primary visual texture of ASCII video. They control not just brightness mapping but the entire visual feel. Design palettes intentionally:

  • Visual weight: characters sorted by the amount of ink/pixels they fill. Space is always index 0.
  • Coherence: characters within a palette should belong to the same visual family.
  • Density curve: the brightness-to-character mapping is nonlinear. Dense palettes (many chars) give smoother gradients; sparse palettes (5-8 chars) give posterized/graphic looks.
  • Rendering compatibility: every character in the palette must exist in the font. Test at init and remove missing glyphs.

Palette Library

Organized by visual family. Mix and match per project -- don't default to PAL_DEFAULT for everything.

Density / Brightness Palettes

PAL_DEFAULT  = " .`'-:;!><=+*^~?/|(){}[]#&$@%"       # classic ASCII art
PAL_DENSE    = " .:;+=xX$#@\u2588"                          # simple 11-level ramp
PAL_MINIMAL  = " .:-=+#@"                               # 8-level, graphic
PAL_BINARY   = " \u2588"                                      # 2-level, extreme contrast
PAL_GRADIENT = " \u2591\u2592\u2593\u2588"                              # 4-level block gradient

Unicode Block Elements

PAL_BLOCKS   = " \u2591\u2592\u2593\u2588\u2584\u2580\u2590\u258c"                 # standard blocks
PAL_BLOCKS_EXT = " \u2596\u2597\u2598\u2599\u259a\u259b\u259c\u259d\u259e\u259f\u2591\u2592\u2593\u2588"  # quadrant blocks (more detail)
PAL_SHADE    = " \u2591\u2592\u2593\u2588\u2587\u2586\u2585\u2584\u2583\u2582\u2581"          # vertical fill progression

Symbolic / Thematic

PAL_MATH     = " \u00b7\u2218\u2219\u2022\u00b0\u00b1\u2213\u00d7\u00f7\u2248\u2260\u2261\u2264\u2265\u221e\u222b\u2211\u220f\u221a\u2207\u2202\u2206\u03a9"    # math symbols
PAL_BOX      = " \u2500\u2502\u250c\u2510\u2514\u2518\u251c\u2524\u252c\u2534\u253c\u2550\u2551\u2554\u2557\u255a\u255d\u2560\u2563\u2566\u2569\u256c"          # box drawing
PAL_CIRCUIT  = " .\u00b7\u2500\u2502\u250c\u2510\u2514\u2518\u253c\u25cb\u25cf\u25a1\u25a0\u2206\u2207\u2261"                 # circuit board
PAL_RUNE     = " .\u16a0\u16a2\u16a6\u16b1\u16b7\u16c1\u16c7\u16d2\u16d6\u16da\u16de\u16df"                   # elder futhark runes
PAL_ALCHEMIC = " \u2609\u263d\u2640\u2642\u2643\u2644\u2645\u2646\u2647\u2648\u2649\u264a\u264b"            # planetary/alchemical symbols
PAL_ZODIAC   = " \u2648\u2649\u264a\u264b\u264c\u264d\u264e\u264f\u2650\u2651\u2652\u2653"            # zodiac
PAL_ARROWS   = " \u2190\u2191\u2192\u2193\u2194\u2195\u2196\u2197\u2198\u2199\u21a9\u21aa\u21bb\u27a1"             # directional arrows
PAL_MUSIC    = " \u266a\u266b\u266c\u2669\u266d\u266e\u266f\u25cb\u25cf"                       # musical notation

Script / Writing System

PAL_KATA     = " \u00b7\uff66\uff67\uff68\uff69\uff6a\uff6b\uff6c\uff6d\uff6e\uff6f\uff70\uff71\uff72\uff73\uff74\uff75\uff76\uff77"          # katakana halfwidth (matrix rain)
PAL_GREEK    = " \u03b1\u03b2\u03b3\u03b4\u03b5\u03b6\u03b7\u03b8\u03b9\u03ba\u03bb\u03bc\u03bd\u03be\u03c0\u03c1\u03c3\u03c4\u03c6\u03c8\u03c9"    # Greek lowercase
PAL_CYRILLIC = " \u0430\u0431\u0432\u0433\u0434\u0435\u0436\u0437\u0438\u043a\u043b\u043c\u043d\u043e\u043f\u0440\u0441\u0442\u0443\u0444\u0445\u0446\u0447\u0448"  # Cyrillic lowercase
PAL_ARABIC   = " \u0627\u0628\u062a\u062b\u062c\u062d\u062e\u062f\u0630\u0631\u0632\u0633\u0634\u0635\u0636\u0637"       # Arabic letters (isolated forms)

Dot / Point Progressions

PAL_DOTS     = " ⋅∘∙●◉◎◆✦★"                   # dot size progression
PAL_BRAILLE  = " ⠁⠂⠃⠄⠅⠆⠇⠈⠉⠊⠋⠌⠍⠎⠏⠐⠑⠒⠓⠔⠕⠖⠗⠘⠙⠚⠛⠜⠝⠞⠟⠿"  # braille patterns
PAL_STARS    = " ·✧✦✩✨★✶✳✸"               # star progression
PAL_HALFFILL = " ◔◑◕◐◒◓◖◗◙"               # directional half-fill progression
PAL_HATCH    = " ▣▤▥▦▧▨▩"                     # crosshatch density ramp

Project-Specific (examples -- invent new ones per project)

PAL_HERMES   = " .\u00b7~=\u2248\u221e\u26a1\u263f\u2726\u2605\u2295\u25ca\u25c6\u25b2\u25bc\u25cf\u25a0"   # mythology/tech blend
PAL_OCEAN    = " ~\u2248\u2248\u2248\u223c\u2307\u2248\u224b\u224c\u2248"                       # water/wave characters
PAL_ORGANIC  = " .\u00b0\u2218\u2022\u25e6\u25c9\u2742\u273f\u2741\u2743"                 # growing/botanical
PAL_MACHINE  = " _\u2500\u2502\u250c\u2510\u253c\u2261\u25a0\u2588\u2593\u2592\u2591"             # mechanical/industrial

Creating Custom Palettes

When designing for a project, build palettes from the content's theme:

  1. Choose a visual family (dots, blocks, symbols, script)
  2. Sort by visual weight -- render each char at target font size, count lit pixels, sort ascending
  3. Test at target grid size -- some chars collapse to blobs at small sizes
  4. Validate in font -- remove chars the font can't render:
def validate_palette(pal, font):
    """Remove characters the font can't render."""
    valid = []
    for c in pal:
        if c == " ":
            valid.append(c)
            continue
        img = Image.new("L", (20, 20), 0)
        ImageDraw.Draw(img).text((0, 0), c, fill=255, font=font)
        if np.array(img).max() > 0:  # char actually rendered something
            valid.append(c)
    return "".join(valid)

Mapping Values to Characters

def val2char(v, mask, pal=PAL_DEFAULT):
    """Map float array (0-1) to character array using palette."""
    n = len(pal)
    idx = np.clip((v * n).astype(int), 0, n - 1)
    out = np.full(v.shape, " ", dtype="U1")
    for i, ch in enumerate(pal):
        out[mask & (idx == i)] = ch
    return out

Nonlinear mapping for different visual curves:

def val2char_gamma(v, mask, pal, gamma=1.0):
    """Gamma-corrected palette mapping. gamma<1 = brighter, gamma>1 = darker."""
    v_adj = np.power(np.clip(v, 0, 1), gamma)
    return val2char(v_adj, mask, pal)

def val2char_step(v, mask, pal, thresholds):
    """Custom threshold mapping. thresholds = list of float breakpoints."""
    out = np.full(v.shape, pal[0], dtype="U1")
    for i, thr in enumerate(thresholds):
        out[mask & (v > thr)] = pal[min(i + 1, len(pal) - 1)]
    return out

Color System

HSV->RGB (Vectorized)

All color computation in HSV for intuitive control, converted at render time:

def hsv2rgb(h, s, v):
    """Vectorized HSV->RGB. h,s,v are numpy arrays. Returns (R,G,B) uint8 arrays."""
    h = h % 1.0
    c = v * s; x = c * (1 - np.abs((h*6) % 2 - 1)); m = v - c
    # ... 6 sector assignment ...
    return (np.clip((r+m)*255, 0, 255).astype(np.uint8),
            np.clip((g+m)*255, 0, 255).astype(np.uint8),
            np.clip((b+m)*255, 0, 255).astype(np.uint8))

Color Mapping Strategies

Don't default to a single strategy. Choose based on the visual intent:

Strategy Hue source Effect Good for
Angle-mapped g.angle / (2*pi) Rainbow around center Radial effects, kaleidoscopes
Distance-mapped g.dist_n * 0.3 Gradient from center Tunnels, depth effects
Frequency-mapped f["cent"] * 0.2 Timbral color shifting Audio-reactive
Value-mapped val * 0.15 Brightness-dependent hue Fire, heat maps
Time-cycled t * rate Slow color rotation Ambient, chill
Source-sampled Video frame pixel colors Preserve original color Video-to-ASCII
Palette-indexed Discrete color lookup Flat graphic style Retro, pixel art
Temperature Blend between warm/cool Emotional tone Mood-driven scenes
Complementary hue and hue + 0.5 High contrast Bold, dramatic
Triadic hue, hue + 0.33, hue + 0.66 Vibrant, balanced Psychedelic
Analogous hue +/- 0.08 Harmonious, subtle Elegant, cohesive
Monochrome Fixed hue, vary S and V Restrained, focused Noir, minimal

Color Palettes (Discrete RGB)

For non-HSV workflows -- direct RGB color sets for graphic/retro looks:

# Named color palettes -- use for flat/graphic styles or per-character coloring
COLORS_NEON = [(255,0,102), (0,255,153), (102,0,255), (255,255,0), (0,204,255)]
COLORS_PASTEL = [(255,179,186), (255,223,186), (255,255,186), (186,255,201), (186,225,255)]
COLORS_MONO_GREEN = [(0,40,0), (0,80,0), (0,140,0), (0,200,0), (0,255,0)]
COLORS_MONO_AMBER = [(40,20,0), (80,50,0), (140,90,0), (200,140,0), (255,191,0)]
COLORS_CYBERPUNK = [(255,0,60), (0,255,200), (180,0,255), (255,200,0)]
COLORS_VAPORWAVE = [(255,113,206), (1,205,254), (185,103,255), (5,255,161)]
COLORS_EARTH = [(86,58,26), (139,90,43), (189,154,91), (222,193,136), (245,230,193)]
COLORS_ICE = [(200,230,255), (150,200,240), (100,170,230), (60,130,210), (30,80,180)]
COLORS_BLOOD = [(80,0,0), (140,10,10), (200,20,20), (255,50,30), (255,100,80)]
COLORS_FOREST = [(10,30,10), (20,60,15), (30,100,20), (50,150,30), (80,200,50)]

def rgb_palette_map(val, mask, palette):
    """Map float array (0-1) to RGB colors from a discrete palette."""
    n = len(palette)
    idx = np.clip((val * n).astype(int), 0, n - 1)
    R = np.zeros(val.shape, dtype=np.uint8)
    G = np.zeros(val.shape, dtype=np.uint8)
    B = np.zeros(val.shape, dtype=np.uint8)
    for i, (r, g, b) in enumerate(palette):
        m = mask & (idx == i)
        R[m] = r; G[m] = g; B[m] = b
    return R, G, B

OKLAB Color Space (Perceptually Uniform)

HSV hue is perceptually non-uniform: green occupies far more visual range than blue. OKLAB / OKLCH provide perceptually even color steps — hue increments of 0.1 look equally different regardless of starting hue. Use OKLAB for:

  • Gradient interpolation (no unwanted intermediate hues)
  • Color harmony generation (perceptually balanced palettes)
  • Smooth color transitions over time
# --- sRGB <-> Linear sRGB ---

def srgb_to_linear(c):
    """Convert sRGB [0,1] to linear light. c: float32 array."""
    return np.where(c <= 0.04045, c / 12.92, ((c + 0.055) / 1.055) ** 2.4)

def linear_to_srgb(c):
    """Convert linear light to sRGB [0,1]."""
    return np.where(c <= 0.0031308, c * 12.92, 1.055 * np.power(np.maximum(c, 0), 1/2.4) - 0.055)

# --- Linear sRGB <-> OKLAB ---

def linear_rgb_to_oklab(r, g, b):
    """Linear sRGB to OKLAB. r,g,b: float32 arrays [0,1].
    Returns (L, a, b) where L=[0,1], a,b=[-0.4, 0.4] approx."""
    l_ = 0.4122214708 * r + 0.5363325363 * g + 0.0514459929 * b
    m_ = 0.2119034982 * r + 0.6806995451 * g + 0.1073969566 * b
    s_ = 0.0883024619 * r + 0.2817188376 * g + 0.6299787005 * b
    l_c = np.cbrt(l_); m_c = np.cbrt(m_); s_c = np.cbrt(s_)
    L = 0.2104542553 * l_c + 0.7936177850 * m_c - 0.0040720468 * s_c
    a = 1.9779984951 * l_c - 2.4285922050 * m_c + 0.4505937099 * s_c
    b_ = 0.0259040371 * l_c + 0.7827717662 * m_c - 0.8086757660 * s_c
    return L, a, b_

def oklab_to_linear_rgb(L, a, b):
    """OKLAB to linear sRGB. Returns (r, g, b) float32 arrays [0,1]."""
    l_ = L + 0.3963377774 * a + 0.2158037573 * b
    m_ = L - 0.1055613458 * a - 0.0638541728 * b
    s_ = L - 0.0894841775 * a - 1.2914855480 * b
    l_c = l_ ** 3; m_c = m_ ** 3; s_c = s_ ** 3
    r = +4.0767416621 * l_c - 3.3077115913 * m_c + 0.2309699292 * s_c
    g = -1.2684380046 * l_c + 2.6097574011 * m_c - 0.3413193965 * s_c
    b_ = -0.0041960863 * l_c - 0.7034186147 * m_c + 1.7076147010 * s_c
    return np.clip(r, 0, 1), np.clip(g, 0, 1), np.clip(b_, 0, 1)

# --- Convenience: sRGB uint8 <-> OKLAB ---

def rgb_to_oklab(R, G, B):
    """sRGB uint8 arrays to OKLAB."""
    r = srgb_to_linear(R.astype(np.float32) / 255.0)
    g = srgb_to_linear(G.astype(np.float32) / 255.0)
    b = srgb_to_linear(B.astype(np.float32) / 255.0)
    return linear_rgb_to_oklab(r, g, b)

def oklab_to_rgb(L, a, b):
    """OKLAB to sRGB uint8 arrays."""
    r, g, b_ = oklab_to_linear_rgb(L, a, b)
    R = np.clip(linear_to_srgb(r) * 255, 0, 255).astype(np.uint8)
    G = np.clip(linear_to_srgb(g) * 255, 0, 255).astype(np.uint8)
    B = np.clip(linear_to_srgb(b_) * 255, 0, 255).astype(np.uint8)
    return R, G, B

# --- OKLCH (cylindrical form of OKLAB) ---

def oklab_to_oklch(L, a, b):
    """OKLAB to OKLCH. Returns (L, C, H) where H is in [0, 1] (normalized)."""
    C = np.sqrt(a**2 + b**2)
    H = (np.arctan2(b, a) / (2 * np.pi)) % 1.0
    return L, C, H

def oklch_to_oklab(L, C, H):
    """OKLCH to OKLAB. H in [0, 1]."""
    angle = H * 2 * np.pi
    a = C * np.cos(angle)
    b = C * np.sin(angle)
    return L, a, b

Gradient Interpolation (OKLAB vs HSV)

Interpolating colors through OKLAB avoids the hue detours that HSV produces:

def lerp_oklab(color_a, color_b, t_array):
    """Interpolate between two sRGB colors through OKLAB.
    color_a, color_b: (R, G, B) tuples 0-255
    t_array: float32 array [0,1] — interpolation parameter per pixel.
    Returns (R, G, B) uint8 arrays."""
    La, aa, ba = rgb_to_oklab(
        np.full_like(t_array, color_a[0], dtype=np.uint8),
        np.full_like(t_array, color_a[1], dtype=np.uint8),
        np.full_like(t_array, color_a[2], dtype=np.uint8))
    Lb, ab, bb = rgb_to_oklab(
        np.full_like(t_array, color_b[0], dtype=np.uint8),
        np.full_like(t_array, color_b[1], dtype=np.uint8),
        np.full_like(t_array, color_b[2], dtype=np.uint8))
    L = La + (Lb - La) * t_array
    a = aa + (ab - aa) * t_array
    b = ba + (bb - ba) * t_array
    return oklab_to_rgb(L, a, b)

def lerp_oklch(color_a, color_b, t_array, short_path=True):
    """Interpolate through OKLCH (preserves chroma, smooth hue path).
    short_path: take the shorter arc around the hue wheel."""
    La, aa, ba = rgb_to_oklab(
        np.full_like(t_array, color_a[0], dtype=np.uint8),
        np.full_like(t_array, color_a[1], dtype=np.uint8),
        np.full_like(t_array, color_a[2], dtype=np.uint8))
    Lb, ab, bb = rgb_to_oklab(
        np.full_like(t_array, color_b[0], dtype=np.uint8),
        np.full_like(t_array, color_b[1], dtype=np.uint8),
        np.full_like(t_array, color_b[2], dtype=np.uint8))
    L1, C1, H1 = oklab_to_oklch(La, aa, ba)
    L2, C2, H2 = oklab_to_oklch(Lb, ab, bb)
    # Shortest hue path
    if short_path:
        dh = H2 - H1
        dh = np.where(dh > 0.5, dh - 1.0, np.where(dh < -0.5, dh + 1.0, dh))
        H = (H1 + dh * t_array) % 1.0
    else:
        H = H1 + (H2 - H1) * t_array
    L = L1 + (L2 - L1) * t_array
    C = C1 + (C2 - C1) * t_array
    Lout, aout, bout = oklch_to_oklab(L, C, H)
    return oklab_to_rgb(Lout, aout, bout)

Color Harmony Generation

Auto-generate harmonious palettes from a seed color:

def harmony_complementary(seed_rgb):
    """Two colors: seed + opposite hue."""
    L, a, b = rgb_to_oklab(np.array([seed_rgb[0]]), np.array([seed_rgb[1]]), np.array([seed_rgb[2]]))
    _, C, H = oklab_to_oklch(L, a, b)
    return [seed_rgb, _oklch_to_srgb_tuple(L[0], C[0], (H[0] + 0.5) % 1.0)]

def harmony_triadic(seed_rgb):
    """Three colors: seed + two at 120-degree offsets."""
    L, a, b = rgb_to_oklab(np.array([seed_rgb[0]]), np.array([seed_rgb[1]]), np.array([seed_rgb[2]]))
    _, C, H = oklab_to_oklch(L, a, b)
    return [seed_rgb,
            _oklch_to_srgb_tuple(L[0], C[0], (H[0] + 0.333) % 1.0),
            _oklch_to_srgb_tuple(L[0], C[0], (H[0] + 0.667) % 1.0)]

def harmony_analogous(seed_rgb, spread=0.08, n=5):
    """N colors spread evenly around seed hue."""
    L, a, b = rgb_to_oklab(np.array([seed_rgb[0]]), np.array([seed_rgb[1]]), np.array([seed_rgb[2]]))
    _, C, H = oklab_to_oklch(L, a, b)
    offsets = np.linspace(-spread * (n-1)/2, spread * (n-1)/2, n)
    return [_oklch_to_srgb_tuple(L[0], C[0], (H[0] + off) % 1.0) for off in offsets]

def harmony_split_complementary(seed_rgb, split=0.08):
    """Three colors: seed + two flanking the complement."""
    L, a, b = rgb_to_oklab(np.array([seed_rgb[0]]), np.array([seed_rgb[1]]), np.array([seed_rgb[2]]))
    _, C, H = oklab_to_oklch(L, a, b)
    comp = (H[0] + 0.5) % 1.0
    return [seed_rgb,
            _oklch_to_srgb_tuple(L[0], C[0], (comp - split) % 1.0),
            _oklch_to_srgb_tuple(L[0], C[0], (comp + split) % 1.0)]

def harmony_tetradic(seed_rgb):
    """Four colors: two complementary pairs at 90-degree offset."""
    L, a, b = rgb_to_oklab(np.array([seed_rgb[0]]), np.array([seed_rgb[1]]), np.array([seed_rgb[2]]))
    _, C, H = oklab_to_oklch(L, a, b)
    return [seed_rgb,
            _oklch_to_srgb_tuple(L[0], C[0], (H[0] + 0.25) % 1.0),
            _oklch_to_srgb_tuple(L[0], C[0], (H[0] + 0.5) % 1.0),
            _oklch_to_srgb_tuple(L[0], C[0], (H[0] + 0.75) % 1.0)]

def _oklch_to_srgb_tuple(L, C, H):
    """Helper: single OKLCH -> sRGB (R,G,B) int tuple."""
    La = np.array([L]); Ca = np.array([C]); Ha = np.array([H])
    Lo, ao, bo = oklch_to_oklab(La, Ca, Ha)
    R, G, B = oklab_to_rgb(Lo, ao, bo)
    return (int(R[0]), int(G[0]), int(B[0]))

OKLAB Hue Fields

Drop-in replacements for hf_* generators that produce perceptually uniform hue variation:

def hf_oklch_angle(offset=0.0, chroma=0.12, lightness=0.7):
    """OKLCH hue mapped to angle from center. Perceptually uniform rainbow.
    Returns (R, G, B) uint8 color array instead of a float hue.
    NOTE: Use with _render_vf_rgb() variant, not standard _render_vf()."""
    def fn(g, f, t, S):
        H = (g.angle / (2 * np.pi) + offset + t * 0.05) % 1.0
        L = np.full_like(H, lightness)
        C = np.full_like(H, chroma)
        Lo, ao, bo = oklch_to_oklab(L, C, H)
        R, G, B = oklab_to_rgb(Lo, ao, bo)
        return mkc(R, G, B, g.rows, g.cols)
    return fn

Compositing Helpers

def mkc(R, G, B, rows, cols):
    """Pack 3 uint8 arrays into (rows, cols, 3) color array."""
    o = np.zeros((rows, cols, 3), dtype=np.uint8)
    o[:,:,0] = R; o[:,:,1] = G; o[:,:,2] = B
    return o

def layer_over(base_ch, base_co, top_ch, top_co):
    """Composite top layer onto base. Non-space chars overwrite."""
    m = top_ch != " "
    base_ch[m] = top_ch[m]; base_co[m] = top_co[m]
    return base_ch, base_co

def layer_blend(base_co, top_co, alpha):
    """Alpha-blend top color layer onto base. alpha is float array (0-1) or scalar."""
    if isinstance(alpha, (int, float)):
        alpha = np.full(base_co.shape[:2], alpha, dtype=np.float32)
    a = alpha[:,:,None]
    return np.clip(base_co * (1 - a) + top_co * a, 0, 255).astype(np.uint8)

def stamp(ch, co, text, row, col, color=(255,255,255)):
    """Write text string at position."""
    for i, c in enumerate(text):
        cc = col + i
        if 0 <= row < ch.shape[0] and 0 <= cc < ch.shape[1]:
            ch[row, cc] = c; co[row, cc] = color

Section System

Map time ranges to effect functions + shader configs + grid sizes:

SECTIONS = [
    (0.0, "void"), (3.94, "starfield"), (21.0, "matrix"),
    (46.0, "drop"), (130.0, "glitch"), (187.0, "outro"),
]

FX_DISPATCH = {"void": fx_void, "starfield": fx_starfield, ...}
SECTION_FX = {"void": {"vignette": 0.3, "bloom": 170}, ...}
SECTION_GRID = {"void": "md", "starfield": "sm", "drop": "lg", ...}
SECTION_MIRROR = {"drop": "h", "bass_rings": "quad"}

def get_section(t):
    sec = SECTIONS[0][1]
    for ts, name in SECTIONS:
        if t >= ts: sec = name
    return sec

Parallel Encoding

Split frames across N workers. Each pipes raw RGB to its own ffmpeg subprocess:

def render_batch(batch_id, frame_start, frame_end, features, seg_path):
    r = Renderer()
    cmd = ["ffmpeg", "-y", "-f", "rawvideo", "-pix_fmt", "rgb24",
           "-s", f"{VW}x{VH}", "-r", str(FPS), "-i", "pipe:0",
           "-c:v", "libx264", "-preset", "fast", "-crf", "18",
           "-pix_fmt", "yuv420p", seg_path]

    # CRITICAL: stderr to file, not pipe
    stderr_fh = open(os.path.join(workdir, f"err_{batch_id:02d}.log"), "w")
    pipe = subprocess.Popen(cmd, stdin=subprocess.PIPE,
                            stdout=subprocess.DEVNULL, stderr=stderr_fh)

    for fi in range(frame_start, frame_end):
        t = fi / FPS
        sec = get_section(t)
        f = {k: float(features[k][fi]) for k in features}
        ch, co = FX_DISPATCH[sec](r, f, t)
        canvas = r.render(ch, co)
        canvas = apply_mirror(canvas, sec, f)
        canvas = apply_shaders(canvas, sec, f, t)
        pipe.stdin.write(canvas.tobytes())

    pipe.stdin.close()
    pipe.wait()
    stderr_fh.close()

Concatenate segments + mux audio:

# Write concat file
with open(concat_path, "w") as cf:
    for seg in segments:
        cf.write(f"file '{seg}'\n")

subprocess.run(["ffmpeg", "-y", "-f", "concat", "-safe", "0", "-i", concat_path,
                "-i", audio_path, "-c:v", "copy", "-c:a", "aac", "-b:a", "192k",
                "-shortest", output_path])

Effect Function Contract

v2 Protocol (Current)

Every scene function: (r, f, t, S) -> canvas_uint8 — where r = Renderer, f = features dict, t = time float, S = persistent state dict

def fx_example(r, f, t, S):
    """Scene function returns a full pixel canvas (uint8 H,W,3).
    Scenes have full control over multi-grid rendering and pixel-level composition.
    """
    # Render multiple layers at different grid densities
    canvas_a = _render_vf(r, "md", vf_plasma, hf_angle(0.0), PAL_DENSE, f, t, S)
    canvas_b = _render_vf(r, "sm", vf_vortex, hf_time_cycle(0.1), PAL_RUNE, f, t, S)

    # Pixel-level blend
    result = blend_canvas(canvas_a, canvas_b, "screen", 0.8)
    return result

See references/scenes.md for the full scene protocol, the Renderer class, _render_vf() helper, and complete scene examples.

See references/composition.md for blend modes, tone mapping, feedback buffers, and multi-grid composition.

v1 Protocol (Legacy)

Simple scenes that use a single grid can still return (chars, colors) and let the caller handle rendering, but the v2 canvas protocol is preferred for all new code.

def fx_simple(r, f, t, S):
    g = r.get_grid("md")
    val = np.sin(g.dist * 0.1 - t * 3) * f.get("bass", 0.3) * 2
    val = np.clip(val, 0, 1); mask = val > 0.03
    ch = val2char(val, mask, PAL_DEFAULT)
    R, G, B = hsv2rgb(np.full_like(val, 0.6), np.full_like(val, 0.7), val)
    co = mkc(R, G, B, g.rows, g.cols)
    return g.render(ch, co)  # returns canvas directly

Persistent State

Effects that need state across frames (particles, rain columns) use the S dict parameter (which is r.S — same object, but passed explicitly for clarity):

def fx_with_state(r, f, t, S):
    if "particles" not in S:
        S["particles"] = initialize_particles()
    update_particles(S["particles"])
    # ...

State persists across frames within a single scene/clip. Each worker process (and each scene) gets its own independent state.

Helper Functions

def hsv2rgb_scalar(h, s, v):
    """Single-value HSV to RGB. Returns (R, G, B) tuple of ints 0-255."""
    h = h % 1.0
    c = v * s; x = c * (1 - abs((h * 6) % 2 - 1)); m = v - c
    if h * 6 < 1:   r, g, b = c, x, 0
    elif h * 6 < 2:  r, g, b = x, c, 0
    elif h * 6 < 3:  r, g, b = 0, c, x
    elif h * 6 < 4:  r, g, b = 0, x, c
    elif h * 6 < 5:  r, g, b = x, 0, c
    else:             r, g, b = c, 0, x
    return (int((r+m)*255), int((g+m)*255), int((b+m)*255))

def log(msg):
    """Print timestamped log message."""
    print(msg, flush=True)
Reference in New Issue View Git Blame Copy Permalink