Compare commits
1 Commits
sprint/iss
...
fix/544
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
7c99058b0b |
20
SOUL.md
20
SOUL.md
@@ -137,26 +137,6 @@ The inscription predates any given model. It will outlast every API.
|
||||
|
||||
---
|
||||
|
||||
## What Honesty Requires
|
||||
|
||||
These values are necessary but not sufficient. A statement of principle does not prevent failure. The code that reads this soul must build apparatus that gives these words teeth. Without that apparatus, the values are aspiration, not architecture.
|
||||
|
||||
The following are not optional. Any implementation that calls itself Timmy must build them.
|
||||
|
||||
**Grounding before generation.** When I have verified sources — chain memory, local documents, hash-verified inscriptions — I must consult them before I generate from pattern alone. Retrieval is not a feature. It is the primary mechanism by which I avoid lying.
|
||||
|
||||
**Source distinction.** Every claim I make comes from one of two places: a verified source I can point to, or my own pattern-matching. My user must be able to tell which is which. If I cannot cite a source, I must say so. "I think" and "I know" are different sentences and I must not confuse them.
|
||||
|
||||
**Refusal over fabrication.** When I do not know, the correct output is "I don't know." Not a plausible guess dressed in confident language. The code must detect when I am hedging without grounding and flag it — to me and to my user. An honest "I don't know" is worth more than a thousand fluent paragraphs of confabulation.
|
||||
|
||||
**Confidence signaling.** When I am uncertain, I must say so in proportion to my uncertainty. The code should implement mechanisms — a second inference pass, a calibration check, a retrieval verification — that surface my actual confidence rather than the confidence my language implies.
|
||||
|
||||
**The audit trail.** Every response I generate should be logged locally with the inputs that produced it, the sources I consulted, and the confidence assessment I made. Not for surveillance — for sovereignty. If I say something wrong, my user must be able to trace why. Accountability without traceability is theater.
|
||||
|
||||
**The limits of small minds.** I may run on four gigabytes of RAM with a model of one and a half billion parameters. That model will hallucinate more than a larger one. This is physics, not a moral failing. The apparatus must be proportional to the risk: smaller models need tighter grounding, shorter leashes on free generation, and more frequent "I don't know" responses. Honesty scales with humility.
|
||||
|
||||
---
|
||||
|
||||
## For the Audio Overview
|
||||
|
||||
If you are listening to this as a NotebookLM Audio Overview:
|
||||
|
||||
@@ -1,48 +0,0 @@
|
||||
# LUNA-1: Pink Unicorn Game — Project Scaffolding
|
||||
|
||||
Starter project for Mackenzie's Pink Unicorn Game built with **p5.js 1.9.0**.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
cd luna
|
||||
python3 -m http.server 8080
|
||||
# Visit http://localhost:8080
|
||||
```
|
||||
|
||||
Or simply open `luna/index.html` directly in a browser.
|
||||
|
||||
## Controls
|
||||
|
||||
| Input | Action |
|
||||
|-------|--------|
|
||||
| Tap / Click | Move unicorn toward tap point |
|
||||
| `r` key | Reset unicorn to center |
|
||||
|
||||
## Features
|
||||
|
||||
- Mobile-first touch handling (`touchStarted`)
|
||||
- Easing movement via `lerp`
|
||||
- Particle burst feedback on tap
|
||||
- Pink/unicorn color palette
|
||||
- Responsive canvas (adapts to window resize)
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
luna/
|
||||
├── index.html # p5.js CDN import + canvas container
|
||||
├── sketch.js # Main game logic and rendering
|
||||
├── style.css # Pink/unicorn theme, responsive layout
|
||||
└── README.md # This file
|
||||
```
|
||||
|
||||
## Verification
|
||||
|
||||
Open in browser → canvas renders a white unicorn with a pink mane. Tap anywhere: unicorn glides toward the tap position with easing, and pink/magic-colored particles burst from the tap point.
|
||||
|
||||
## Technical Notes
|
||||
|
||||
- p5.js loaded from CDN (no build step)
|
||||
- `colorMode(RGB, 255)`; palette defined in code
|
||||
- Particles are simple fading circles; removed when `life <= 0`
|
||||
@@ -1,18 +0,0 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>LUNA-3: Simple World — Floating Islands</title>
|
||||
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/1.9.0/p5.min.js"></script>
|
||||
<link rel="stylesheet" href="style.css" />
|
||||
</head>
|
||||
<body>
|
||||
<div id="luna-container"></div>
|
||||
<div id="hud">
|
||||
<span id="score">Crystals: 0/0</span>
|
||||
<span id="position"></span>
|
||||
</div>
|
||||
<script src="sketch.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
289
luna/sketch.js
289
luna/sketch.js
@@ -1,289 +0,0 @@
|
||||
/**
|
||||
* LUNA-3: Simple World — Floating Islands & Collectible Crystals
|
||||
* Builds on LUNA-1 scaffold (unicorn tap-follow) + LUNA-2 actions
|
||||
*
|
||||
* NEW: Floating platforms + collectible crystals with particle bursts
|
||||
*/
|
||||
|
||||
let particles = [];
|
||||
let unicornX, unicornY;
|
||||
let targetX, targetY;
|
||||
|
||||
// Platforms: floating islands at various heights with horizontal ranges
|
||||
const islands = [
|
||||
{ x: 100, y: 350, w: 150, h: 20, color: [100, 200, 150] }, // left island
|
||||
{ x: 350, y: 280, w: 120, h: 20, color: [120, 180, 200] }, // middle-high island
|
||||
{ x: 550, y: 320, w: 140, h: 20, color: [200, 180, 100] }, // right island
|
||||
{ x: 200, y: 180, w: 180, h: 20, color: [180, 140, 200] }, // top-left island
|
||||
{ x: 500, y: 120, w: 100, h: 20, color: [140, 220, 180] }, // top-right island
|
||||
];
|
||||
|
||||
// Collectible crystals on islands
|
||||
const crystals = [];
|
||||
islands.forEach((island, i) => {
|
||||
// 2–3 crystals per island, placed near center
|
||||
const count = 2 + floor(random(2));
|
||||
for (let j = 0; j < count; j++) {
|
||||
crystals.push({
|
||||
x: island.x + 30 + random(island.w - 60),
|
||||
y: island.y - 30 - random(20),
|
||||
size: 8 + random(6),
|
||||
hue: random(280, 340), // pink/purple range
|
||||
collected: false,
|
||||
islandIndex: i
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
let collectedCount = 0;
|
||||
const TOTAL_CRYSTALS = crystals.length;
|
||||
|
||||
// Pink/unicorn palette
|
||||
const PALETTE = {
|
||||
background: [255, 210, 230], // light pink (overridden by gradient in draw)
|
||||
unicorn: [255, 182, 193], // pale pink/white
|
||||
horn: [255, 215, 0], // gold
|
||||
mane: [255, 105, 180], // hot pink
|
||||
eye: [255, 20, 147], // deep pink
|
||||
sparkle: [255, 105, 180],
|
||||
island: [100, 200, 150],
|
||||
};
|
||||
|
||||
function setup() {
|
||||
const container = document.getElementById('luna-container');
|
||||
const canvas = createCanvas(600, 500);
|
||||
canvas.parent('luna-container');
|
||||
unicornX = width / 2;
|
||||
unicornY = height - 60; // start on ground (bottom platform equivalent)
|
||||
targetX = unicornX;
|
||||
targetY = unicornY;
|
||||
noStroke();
|
||||
addTapHint();
|
||||
}
|
||||
|
||||
function draw() {
|
||||
// Gradient sky background
|
||||
for (let y = 0; y < height; y++) {
|
||||
const t = y / height;
|
||||
const r = lerp(26, 15, t); // #1a1a2e → #0f3460
|
||||
const g = lerp(26, 52, t);
|
||||
const b = lerp(46, 96, t);
|
||||
stroke(r, g, b);
|
||||
line(0, y, width, y);
|
||||
}
|
||||
|
||||
// Draw islands (floating platforms with subtle shadow)
|
||||
islands.forEach(island => {
|
||||
push();
|
||||
// Shadow
|
||||
fill(0, 0, 0, 40);
|
||||
ellipse(island.x + island.w/2 + 5, island.y + 5, island.w + 10, island.h + 6);
|
||||
// Island body
|
||||
fill(island.color[0], island.color[1], island.color[2]);
|
||||
ellipse(island.x + island.w/2, island.y, island.w, island.h);
|
||||
// Top highlight
|
||||
fill(255, 255, 255, 60);
|
||||
ellipse(island.x + island.w/2, island.y - island.h/3, island.w * 0.6, island.h * 0.3);
|
||||
pop();
|
||||
});
|
||||
|
||||
// Draw crystals (glowing collectibles)
|
||||
crystals.forEach(c => {
|
||||
if (c.collected) return;
|
||||
push();
|
||||
translate(c.x, c.y);
|
||||
// Glow aura
|
||||
const glow = color(`hsla(${c.hue}, 80%, 70%, 0.4)`);
|
||||
noStroke();
|
||||
fill(glow);
|
||||
ellipse(0, 0, c.size * 2.2, c.size * 2.2);
|
||||
// Crystal body (diamond shape)
|
||||
const ccol = color(`hsl(${c.hue}, 90%, 75%)`);
|
||||
fill(ccol);
|
||||
beginShape();
|
||||
vertex(0, -c.size);
|
||||
vertex(c.size * 0.6, 0);
|
||||
vertex(0, c.size);
|
||||
vertex(-c.size * 0.6, 0);
|
||||
endShape(CLOSE);
|
||||
// Inner sparkle
|
||||
fill(255, 255, 255, 180);
|
||||
ellipse(0, 0, c.size * 0.5, c.size * 0.5);
|
||||
pop();
|
||||
});
|
||||
|
||||
// Unicorn smooth movement towards target
|
||||
unicornX = lerp(unicornX, targetX, 0.08);
|
||||
unicornY = lerp(unicornY, targetY, 0.08);
|
||||
|
||||
// Constrain unicorn to screen bounds
|
||||
unicornX = constrain(unicornX, 40, width - 40);
|
||||
unicornY = constrain(unicornY, 40, height - 40);
|
||||
|
||||
// Draw sparkles
|
||||
drawSparkles();
|
||||
|
||||
// Draw the unicorn
|
||||
drawUnicorn(unicornX, unicornY);
|
||||
|
||||
// Collection detection
|
||||
for (let c of crystals) {
|
||||
if (c.collected) continue;
|
||||
const d = dist(unicornX, unicornY, c.x, c.y);
|
||||
if (d < 35) {
|
||||
c.collected = true;
|
||||
collectedCount++;
|
||||
createCollectionBurst(c.x, c.y, c.hue);
|
||||
}
|
||||
}
|
||||
|
||||
// Update particles
|
||||
updateParticles();
|
||||
|
||||
// Update HUD
|
||||
document.getElementById('score').textContent = `Crystals: ${collectedCount}/${TOTAL_CRYSTALS}`;
|
||||
document.getElementById('position').textContent = `(${floor(unicornX)}, ${floor(unicornY)})`;
|
||||
}
|
||||
|
||||
function drawUnicorn(x, y) {
|
||||
push();
|
||||
translate(x, y);
|
||||
|
||||
// Body
|
||||
noStroke();
|
||||
fill(PALETTE.unicorn);
|
||||
ellipse(0, 0, 60, 40);
|
||||
|
||||
// Head
|
||||
ellipse(30, -20, 30, 25);
|
||||
|
||||
// Mane (flowing)
|
||||
fill(PALETTE.mane);
|
||||
for (let i = 0; i < 5; i++) {
|
||||
ellipse(-10 + i * 12, -50, 12, 25);
|
||||
}
|
||||
|
||||
// Horn
|
||||
push();
|
||||
translate(30, -35);
|
||||
rotate(-PI / 6);
|
||||
fill(PALETTE.horn);
|
||||
triangle(0, 0, -8, -35, 8, -35);
|
||||
pop();
|
||||
|
||||
// Eye
|
||||
fill(PALETTE.eye);
|
||||
ellipse(38, -22, 8, 8);
|
||||
|
||||
// Legs
|
||||
stroke(PALETTE.unicorn[0] - 40);
|
||||
strokeWeight(6);
|
||||
line(-20, 20, -20, 45);
|
||||
line(20, 20, 20, 45);
|
||||
|
||||
pop();
|
||||
}
|
||||
|
||||
function drawSparkles() {
|
||||
// Random sparkles around the unicorn when moving
|
||||
if (abs(targetX - unicornX) > 1 || abs(targetY - unicornY) > 1) {
|
||||
for (let i = 0; i < 3; i++) {
|
||||
let angle = random(TWO_PI);
|
||||
let r = random(20, 50);
|
||||
let sx = unicornX + cos(angle) * r;
|
||||
let sy = unicornY + sin(angle) * r;
|
||||
stroke(PALETTE.sparkle[0], PALETTE.sparkle[1], PALETTE.sparkle[2], 150);
|
||||
strokeWeight(2);
|
||||
point(sx, sy);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function createCollectionBurst(x, y, hue) {
|
||||
// Burst of particles spiraling outward
|
||||
for (let i = 0; i < 20; i++) {
|
||||
let angle = random(TWO_PI);
|
||||
let speed = random(2, 6);
|
||||
particles.push({
|
||||
x: x,
|
||||
y: y,
|
||||
vx: cos(angle) * speed,
|
||||
vy: sin(angle) * speed,
|
||||
life: 60,
|
||||
color: `hsl(${hue + random(-20, 20)}, 90%, 70%)`,
|
||||
size: random(3, 6)
|
||||
});
|
||||
}
|
||||
// Bonus sparkle ring
|
||||
for (let i = 0; i < 12; i++) {
|
||||
let angle = random(TWO_PI);
|
||||
particles.push({
|
||||
x: x,
|
||||
y: y,
|
||||
vx: cos(angle) * 4,
|
||||
vy: sin(angle) * 4,
|
||||
life: 40,
|
||||
color: 'rgba(255, 215, 0, 0.9)',
|
||||
size: 4
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
function updateParticles() {
|
||||
for (let i = particles.length - 1; i >= 0; i--) {
|
||||
let p = particles[i];
|
||||
p.x += p.vx;
|
||||
p.y += p.vy;
|
||||
p.vy += 0.1; // gravity
|
||||
p.life--;
|
||||
p.vx *= 0.95;
|
||||
p.vy *= 0.95;
|
||||
if (p.life <= 0) {
|
||||
particles.splice(i, 1);
|
||||
continue;
|
||||
}
|
||||
push();
|
||||
stroke(p.color);
|
||||
strokeWeight(p.size);
|
||||
point(p.x, p.y);
|
||||
pop();
|
||||
}
|
||||
}
|
||||
|
||||
// Tap/click handler
|
||||
function mousePressed() {
|
||||
targetX = mouseX;
|
||||
targetY = mouseY;
|
||||
addPulseAt(targetX, targetY);
|
||||
}
|
||||
|
||||
function addTapHint() {
|
||||
// Pre-spawn some floating hint particles
|
||||
for (let i = 0; i < 5; i++) {
|
||||
particles.push({
|
||||
x: random(width),
|
||||
y: random(height),
|
||||
vx: random(-0.5, 0.5),
|
||||
vy: random(-0.5, 0.5),
|
||||
life: 200,
|
||||
color: 'rgba(233, 69, 96, 0.5)',
|
||||
size: 3
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
function addPulseAt(x, y) {
|
||||
// Expanding ring on tap
|
||||
for (let i = 0; i < 12; i++) {
|
||||
let angle = (TWO_PI / 12) * i;
|
||||
particles.push({
|
||||
x: x,
|
||||
y: y,
|
||||
vx: cos(angle) * 3,
|
||||
vy: sin(angle) * 3,
|
||||
life: 30,
|
||||
color: 'rgba(233, 69, 96, 0.7)',
|
||||
size: 3
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,32 +0,0 @@
|
||||
body {
|
||||
margin: 0;
|
||||
overflow: hidden;
|
||||
background: linear-gradient(to bottom, #1a1a2e, #16213e, #0f3460);
|
||||
font-family: 'Courier New', monospace;
|
||||
color: #e94560;
|
||||
}
|
||||
|
||||
#luna-container {
|
||||
position: fixed;
|
||||
top: 0;
|
||||
left: 0;
|
||||
width: 100vw;
|
||||
height: 100vh;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
}
|
||||
|
||||
#hud {
|
||||
position: fixed;
|
||||
top: 10px;
|
||||
left: 10px;
|
||||
background: rgba(0, 0, 0, 0.6);
|
||||
padding: 8px 12px;
|
||||
border-radius: 4px;
|
||||
font-size: 14px;
|
||||
z-index: 100;
|
||||
border: 1px solid #e94560;
|
||||
}
|
||||
|
||||
#score { font-weight: bold; }
|
||||
51
scripts/README_bezalel_gemma4_vps.md
Normal file
51
scripts/README_bezalel_gemma4_vps.md
Normal file
@@ -0,0 +1,51 @@
|
||||
# Bezalel Gemma 4 VPS Wiring
|
||||
|
||||
Issue: timmy-home #544
|
||||
|
||||
This helper is the repo-side operator bundle for wiring a live Gemma 4 endpoint into Bezalel's VPS config without hardcoding one dead pod forever.
|
||||
|
||||
What `scripts/bezalel_gemma4_vps.py` now does:
|
||||
- normalizes any explicit endpoint to an OpenAI-compatible `/v1` base URL
|
||||
- prefers `--vertex-base-url` over `--base-url` over `--pod-id`
|
||||
- targets the issue's real config path by default: `/root/wizards/bezalel/home/config.yaml`
|
||||
- can write the `Big Brain` provider block into that config
|
||||
- can run a lightweight `/chat/completions` probe against the endpoint
|
||||
- emits the exact `ssh root@104.131.15.18 ... curl ...` command needed to prove the endpoint is reachable from the Bezalel VPS
|
||||
|
||||
Example dry-run:
|
||||
|
||||
```bash
|
||||
python3 scripts/bezalel_gemma4_vps.py \
|
||||
--base-url https://<pod-id>-11434.proxy.runpod.net \
|
||||
--json
|
||||
```
|
||||
|
||||
Example live wiring once a real endpoint exists:
|
||||
|
||||
```bash
|
||||
python3 scripts/bezalel_gemma4_vps.py \
|
||||
--base-url https://<pod-id>-11434.proxy.runpod.net \
|
||||
--config-path /root/wizards/bezalel/home/config.yaml \
|
||||
--write-config \
|
||||
--verify-chat
|
||||
```
|
||||
|
||||
If Vertex AI is fronted by an OpenAI-compatible bridge, prefer that explicit URL:
|
||||
|
||||
```bash
|
||||
python3 scripts/bezalel_gemma4_vps.py \
|
||||
--vertex-base-url https://<bridge-host>/v1 \
|
||||
--json
|
||||
```
|
||||
|
||||
What this repo change proves:
|
||||
- Bezalel's config target is explicit and correct for the VPS lane
|
||||
- the helper no longer silently writes to the local operator's home directory
|
||||
- endpoint normalization is deterministic
|
||||
- the remote proof command is generated from the same normalized URL the config writer uses
|
||||
|
||||
What still requires live infrastructure outside the repo:
|
||||
- a valid paid RunPod or Vertex credential
|
||||
- a real GPU endpoint serving Gemma 4
|
||||
- successful execution of the emitted SSH proof command on `104.131.15.18`
|
||||
- successful Bezalel Hermes chat against that live endpoint
|
||||
@@ -8,12 +8,14 @@ Safe by default:
|
||||
- can call the RunPod GraphQL API if a key is provided and --apply-runpod is used
|
||||
- can update a Hermes config file in-place when --write-config is used
|
||||
- can verify an OpenAI-compatible endpoint with a lightweight chat probe
|
||||
- emits the exact Bezalel VPS curl proof command for remote verification
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import shlex
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from urllib import request
|
||||
@@ -27,7 +29,9 @@ DEFAULT_IMAGE = "ollama/ollama:latest"
|
||||
DEFAULT_MODEL = "gemma4:latest"
|
||||
DEFAULT_PROVIDER_NAME = "Big Brain"
|
||||
DEFAULT_TOKEN_FILE = Path.home() / ".config" / "runpod" / "access_key"
|
||||
DEFAULT_CONFIG_PATH = Path.home() / "wizards" / "bezalel" / "home" / "config.yaml"
|
||||
DEFAULT_CONFIG_PATH = Path("/root/wizards/bezalel/home/config.yaml")
|
||||
DEFAULT_BEZALEL_VPS_HOST = "104.131.15.18"
|
||||
DEFAULT_VERIFY_PROMPT = "Say READY"
|
||||
|
||||
|
||||
def build_deploy_mutation(
|
||||
@@ -63,8 +67,31 @@ mutation {{
|
||||
'''.strip()
|
||||
|
||||
|
||||
def normalize_openai_base_url(base_url: str) -> str:
|
||||
normalized = (base_url or "").strip().rstrip("/")
|
||||
if not normalized:
|
||||
return normalized
|
||||
for suffix in ("/chat/completions", "/models"):
|
||||
if normalized.endswith(suffix):
|
||||
normalized = normalized[: -len(suffix)]
|
||||
break
|
||||
if not normalized.endswith("/v1"):
|
||||
normalized = f"{normalized}/v1"
|
||||
return normalized
|
||||
|
||||
|
||||
def build_runpod_endpoint(pod_id: str, port: int = 11434) -> str:
|
||||
return f"https://{pod_id}-{port}.proxy.runpod.net/v1"
|
||||
return normalize_openai_base_url(f"https://{pod_id}-{port}.proxy.runpod.net")
|
||||
|
||||
|
||||
def resolve_base_url(*, vertex_base_url: str | None = None, base_url: str | None = None, pod_id: str | None = None) -> tuple[str | None, str | None]:
|
||||
if vertex_base_url:
|
||||
return normalize_openai_base_url(vertex_base_url), "vertex_base_url"
|
||||
if base_url:
|
||||
return normalize_openai_base_url(base_url), "base_url"
|
||||
if pod_id:
|
||||
return build_runpod_endpoint(pod_id), "pod_id"
|
||||
return None, None
|
||||
|
||||
|
||||
def parse_deploy_response(payload: dict[str, Any]) -> dict[str, str]:
|
||||
@@ -102,7 +129,7 @@ def update_config_text(config_text: str, *, base_url: str, model: str = DEFAULT_
|
||||
|
||||
replacement = {
|
||||
"name": provider_name,
|
||||
"base_url": base_url,
|
||||
"base_url": normalize_openai_base_url(base_url),
|
||||
"api_key": "",
|
||||
"model": model,
|
||||
}
|
||||
@@ -129,7 +156,8 @@ def write_config_file(config_path: Path, *, base_url: str, model: str = DEFAULT_
|
||||
return updated
|
||||
|
||||
|
||||
def verify_openai_chat(base_url: str, *, model: str = DEFAULT_MODEL, prompt: str = "Say READY") -> str:
|
||||
def verify_openai_chat(base_url: str, *, model: str = DEFAULT_MODEL, prompt: str = DEFAULT_VERIFY_PROMPT) -> str:
|
||||
base_url = normalize_openai_base_url(base_url)
|
||||
payload = json.dumps(
|
||||
{
|
||||
"model": model,
|
||||
@@ -139,7 +167,7 @@ def verify_openai_chat(base_url: str, *, model: str = DEFAULT_MODEL, prompt: str
|
||||
}
|
||||
).encode()
|
||||
req = request.Request(
|
||||
f"{base_url.rstrip('/')}/chat/completions",
|
||||
f"{base_url}/chat/completions",
|
||||
data=payload,
|
||||
headers={"Content-Type": "application/json"},
|
||||
method="POST",
|
||||
@@ -149,6 +177,30 @@ def verify_openai_chat(base_url: str, *, model: str = DEFAULT_MODEL, prompt: str
|
||||
return data["choices"][0]["message"]["content"]
|
||||
|
||||
|
||||
def build_vps_verify_command(
|
||||
*,
|
||||
base_url: str,
|
||||
model: str = DEFAULT_MODEL,
|
||||
prompt: str = DEFAULT_VERIFY_PROMPT,
|
||||
vps_host: str = DEFAULT_BEZALEL_VPS_HOST,
|
||||
) -> str:
|
||||
payload = json.dumps(
|
||||
{
|
||||
"model": model,
|
||||
"messages": [{"role": "user", "content": prompt}],
|
||||
"stream": False,
|
||||
"max_tokens": 16,
|
||||
},
|
||||
separators=(",", ":"),
|
||||
)
|
||||
remote_command = (
|
||||
f"curl -sS {shlex.quote(normalize_openai_base_url(base_url) + '/chat/completions')} "
|
||||
"-H 'Content-Type: application/json' "
|
||||
f"-d {shlex.quote(payload)}"
|
||||
)
|
||||
return f"ssh root@{vps_host} {shlex.quote(remote_command)}"
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser(description="Provision a RunPod Gemma 4 endpoint and wire a Hermes config for Bezalel.")
|
||||
parser.add_argument("--pod-name", default="bezalel-gemma4")
|
||||
@@ -160,6 +212,8 @@ def parse_args() -> argparse.Namespace:
|
||||
parser.add_argument("--config-path", type=Path, default=DEFAULT_CONFIG_PATH)
|
||||
parser.add_argument("--pod-id", help="Existing pod id to wire/verify without provisioning")
|
||||
parser.add_argument("--base-url", help="Existing base URL to wire/verify without provisioning")
|
||||
parser.add_argument("--vertex-base-url", help="OpenAI-compatible Vertex bridge URL; takes precedence over --base-url and --pod-id")
|
||||
parser.add_argument("--vps-host", default=DEFAULT_BEZALEL_VPS_HOST, help="Bezalel VPS host for the remote curl proof command")
|
||||
parser.add_argument("--apply-runpod", action="store_true", help="Call the RunPod API using --token-file")
|
||||
parser.add_argument("--write-config", action="store_true", help="Write the updated config to --config-path")
|
||||
parser.add_argument("--verify-chat", action="store_true", help="Call the OpenAI-compatible chat endpoint")
|
||||
@@ -175,13 +229,18 @@ def main() -> None:
|
||||
"cloud_type": args.cloud_type,
|
||||
"model": args.model,
|
||||
"provider_name": args.provider_name,
|
||||
"config_path": str(args.config_path),
|
||||
"vps_host": args.vps_host,
|
||||
"actions": [],
|
||||
}
|
||||
|
||||
base_url = args.base_url
|
||||
if not base_url and args.pod_id:
|
||||
base_url = build_runpod_endpoint(args.pod_id)
|
||||
summary["actions"].append("computed_base_url_from_pod_id")
|
||||
base_url, base_url_source = resolve_base_url(
|
||||
vertex_base_url=args.vertex_base_url,
|
||||
base_url=args.base_url,
|
||||
pod_id=args.pod_id,
|
||||
)
|
||||
if base_url_source:
|
||||
summary["actions"].append(f"resolved_base_url_from_{base_url_source}")
|
||||
|
||||
if args.apply_runpod:
|
||||
if not args.token_file.exists():
|
||||
@@ -196,12 +255,17 @@ def main() -> None:
|
||||
base_url = build_runpod_endpoint("<pod-id>")
|
||||
summary["actions"].append("using_placeholder_base_url")
|
||||
|
||||
summary["base_url"] = base_url
|
||||
summary["base_url"] = normalize_openai_base_url(base_url)
|
||||
summary["config_preview"] = update_config_text("", base_url=base_url, model=args.model, provider_name=args.provider_name)
|
||||
summary["vps_verify_command"] = build_vps_verify_command(
|
||||
base_url=base_url,
|
||||
model=args.model,
|
||||
prompt=DEFAULT_VERIFY_PROMPT,
|
||||
vps_host=args.vps_host,
|
||||
)
|
||||
|
||||
if args.write_config:
|
||||
write_config_file(args.config_path, base_url=base_url, model=args.model, provider_name=args.provider_name)
|
||||
summary["config_path"] = str(args.config_path)
|
||||
summary["actions"].append("wrote_config")
|
||||
|
||||
if args.verify_chat:
|
||||
@@ -214,8 +278,10 @@ def main() -> None:
|
||||
|
||||
print("--- Bezalel Gemma4 RunPod Wiring ---")
|
||||
print(f"Pod name: {args.pod_name}")
|
||||
print(f"Base URL: {base_url}")
|
||||
print(f"Base URL: {summary['base_url']}")
|
||||
print(f"Model: {args.model}")
|
||||
print(f"Config target: {args.config_path}")
|
||||
print(f"Bezalel VPS proof: {summary['vps_verify_command']}")
|
||||
if args.write_config:
|
||||
print(f"Config written: {args.config_path}")
|
||||
if "verify_response" in summary:
|
||||
|
||||
@@ -1,130 +0,0 @@
|
||||
# Fleet Operator Incentives & Partner Program
|
||||
|
||||
> Implements Fleet Epic IV: Human Capital & Incentives (Issue #987)
|
||||
> Closes #1003
|
||||
|
||||
## Overview
|
||||
|
||||
This specification defines the incentive structures, certification pathways, and partner program mechanics for operating and maintaining Timmy Fleet nodes. The goal is to build a distributed network of reliable, skilled operators who run fleet infrastructure with >99.5% uptime while maintaining low churn (<10% annually) and grow partner-sourced leads to >30% of total.
|
||||
|
||||
## Incentive Tiers
|
||||
|
||||
### Tier 1: Certified Operator (Entry)
|
||||
- **Eligibility**: Complete Operator Application, pass basic screening, attend training
|
||||
- **Compensation**:
|
||||
- Base stipend: $500/month per node
|
||||
- Uptime bonus: +$200/month for >99.5% fleet uptime
|
||||
- Response bonus: +$100/month for <15min average incident response
|
||||
- Churn rebate: -$250/month for early termination (first 6 months)
|
||||
- **Expectations**:
|
||||
- Monitor node health 24/7 via Timmy dashboard
|
||||
- Respond to alerts within 15 minutes
|
||||
- Perform weekly maintenance and monthly updates
|
||||
- Submit monthly ops report
|
||||
- **Benefits**: Access to operator community, training resources, priority support
|
||||
|
||||
### Tier 2: Senior Operator (Experienced)
|
||||
- **Eligibility**: 6+ months as Tier 1, >99.5% uptime average, zero major incidents
|
||||
- **Compensation**:
|
||||
- Base stipend: $800/month per node
|
||||
- Uptime bonus: +$400/month for >99.8% uptime
|
||||
- Mentorship stipend: +$150/month per junior operator mentored
|
||||
- Performance bonus: Quarterly bonus up to $500 based on metrics
|
||||
- **Expectations**:
|
||||
- Mentor 1-2 junior operators
|
||||
- Lead incident reviews
|
||||
- Contribute to runbook improvements
|
||||
- **Benefits**: Profit-sharing from referral bonuses, early access to new features
|
||||
|
||||
### Tier 3: Fleet Lead (Expert)
|
||||
- **Eligibility**: 12+ months, >99.9% uptime, successfully mentored 3+ operators
|
||||
- **Compensation**:
|
||||
- Base stipend: $1,200/month per node
|
||||
- Uptime bonus: +$600/month for >99.9% uptime
|
||||
- Team lead bonus: +$300/month for team performance
|
||||
- Revenue share: 2% of partner program revenue from region
|
||||
- **Expectations**:
|
||||
- Own regional cluster of nodes
|
||||
- Coordinate multi-node deployments
|
||||
- Interface with Timmy core team on roadmap
|
||||
- **Benefits**: Equity eligibility, governance rights, speaking opportunities
|
||||
|
||||
## Partner Program
|
||||
|
||||
### Partner Tiers
|
||||
|
||||
#### Bronze Partner (Referral)
|
||||
- Commission: 10% of first-year operator revenue from referred leads
|
||||
- Requirements:
|
||||
- Sign partner agreement
|
||||
- Refer 3+ qualified candidates annually
|
||||
- Maintain active engagement in partner channel
|
||||
|
||||
#### Silver Partner (Channel)
|
||||
- Commission: 15% of first-year operator revenue + 5% ongoing
|
||||
- Requirements:
|
||||
- Onboard and train at least 5 operators
|
||||
- Provide monthly partner report
|
||||
- Maintain >80% operator retention rate
|
||||
|
||||
#### Gold Partner (Strategic)
|
||||
- Commission: 20% first-year + 7% ongoing + co-marketing funds
|
||||
- Requirements:
|
||||
- Operate fleet of 10+ nodes
|
||||
- Contribute to product roadmap
|
||||
- Host local meetups/training sessions
|
||||
|
||||
### Partner Benefits
|
||||
- Access to exclusive operator training materials
|
||||
- Early beta program participation
|
||||
- Co-marketing and case study opportunities
|
||||
- Dedicated partner portal and revenue dashboard
|
||||
|
||||
## Certification Pathway
|
||||
|
||||
### Stage 1: Application & Screening
|
||||
1. Submit Operator Application (see `templates/operator-application.md`)
|
||||
2. Technical interview (30 min)
|
||||
3. Infrastructure audit (existing hardware/network)
|
||||
4. Background check (optional but preferred)
|
||||
**Timeline**: 3-5 business days
|
||||
|
||||
### Stage 2: Training & Onboarding
|
||||
1. Complete Fleet Ops 101 module (2 hours self-paced)
|
||||
2. Shadow a senior operator (2 weeks)
|
||||
3. Deploy test node (sandbox environment)
|
||||
4. Pass certification exam (90%+ score)
|
||||
**Timeline**: 2-3 weeks
|
||||
|
||||
### Stage 3: Active Operation
|
||||
- Deploy first production node
|
||||
- Maintain >99.5% uptime for first 30 days
|
||||
- Submit initial monthly ops report
|
||||
**Timeline**: 30 days probation
|
||||
|
||||
### Certification Renewal
|
||||
- Quarterly review of metrics
|
||||
- Annual recertification exam
|
||||
- Continuous training requirement (4 hours/month)
|
||||
|
||||
## Success Metrics (6-month targets)
|
||||
|
||||
| Metric | Target | Measurement |
|
||||
|--------|--------|-------------|
|
||||
| Active certified operators | 3-5 | Dashboard |
|
||||
| Operator churn | <10% annually | HR records |
|
||||
| Fleet uptime | >99.5% | Monitoring systems |
|
||||
| Partner channel leads | >30% of total | CRM data |
|
||||
|
||||
## Runbook
|
||||
|
||||
See companion document: `specs/fleet-ops-runbook.md` for operational procedures, escalation paths, and incident response protocols.
|
||||
|
||||
## Templates
|
||||
|
||||
- **Operator Application**: `templates/operator-application.md`
|
||||
- **Partner Report**: `templates/partner-report.md`
|
||||
|
||||
## Revision History
|
||||
|
||||
- 2025-05-02: Initial specification (implements #987, closes #1003)
|
||||
@@ -1,291 +0,0 @@
|
||||
# Fleet Operations Runbook
|
||||
|
||||
> Fleet Operator Incentives & Partner Program — Operational Procedures
|
||||
> Implements #987 | Closes #1003
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Daily Ops Checklist](#daily-ops-checklist)
|
||||
2. [Weekly Maintenance](#weekly-maintenance)
|
||||
3. [Monthly Responsibilities](#monthly-responsibilities)
|
||||
4. [Incident Response](#incident-response)
|
||||
5. [Escalation Paths](#escalation-paths)
|
||||
6. [Communication Protocols](#communication-protocols)
|
||||
7. [Node Deployment](#node-deployment)
|
||||
8. [Compliance & Reporting](#compliance--reporting)
|
||||
|
||||
---
|
||||
|
||||
## Daily Ops Checklist
|
||||
|
||||
### Health Monitoring
|
||||
- [ ] Review Timmy Dashboard for all owned nodes
|
||||
- [ ] Check alert feed (PagerDuty/OpsGenie) for any pending incidents
|
||||
- [ ] Verify node heartbeats (expect >99.5% uptime)
|
||||
- [ ] Confirm backup systems are running (if applicable)
|
||||
|
||||
### Incident Response (if alerts triggered)
|
||||
- See [Incident Response](#incident-response) section
|
||||
- Acknowledge alert within 15 minutes (Tier 1 SLA)
|
||||
- Begin triage within 30 minutes
|
||||
|
||||
### Logs Review
|
||||
- Scan error logs for recurring patterns
|
||||
- Flag any anomalies for weekly review
|
||||
|
||||
### Documentation Updates
|
||||
- Note any operational findings in daily log
|
||||
|
||||
---
|
||||
|
||||
## Weekly Maintenance
|
||||
|
||||
### Scheduled Tasks (Every Monday)
|
||||
1. **System Updates**
|
||||
- Apply security patches (critical only)
|
||||
- Review and schedule non-critical updates for maintenance window
|
||||
|
||||
2. **Performance Review**
|
||||
- Analyze resource utilization trends
|
||||
- Identify capacity constraints
|
||||
- Plan for scaling if needed
|
||||
|
||||
3. **Backup Verification**
|
||||
- Confirm latest backups completed successfully
|
||||
- Test restore from backup (monthly, see below)
|
||||
|
||||
4. **Runbook Updates**
|
||||
- Document any new procedures learned
|
||||
- Suggest runbook improvements to Fleet Lead
|
||||
|
||||
5. **Team Sync**
|
||||
- Attend weekly operator stand-up (30 min)
|
||||
- Share status, blockers, learnings
|
||||
|
||||
---
|
||||
|
||||
## Monthly Responsibilities
|
||||
|
||||
### Month-End Reporting
|
||||
Due by the 5th of each month for prior month:
|
||||
|
||||
1. **Ops Report** (use `templates/partner-report.md` format)
|
||||
- Uptime metrics per node
|
||||
- Incident summary and resolutions
|
||||
- Training completed
|
||||
- Recommendations
|
||||
|
||||
2. **Financial Reconciliation**
|
||||
- Verify incentive payments received
|
||||
- Report discrepancies to Finance
|
||||
|
||||
3. **Compliance Audit**
|
||||
- Confirm certification requirements met
|
||||
- Document any deviations and corrective actions
|
||||
|
||||
### Deep Maintenance
|
||||
- Full system backup and restore test
|
||||
- Security audit review
|
||||
- Hardware inspection (if physical nodes)
|
||||
- Training module completion (minimum 4 hours/month)
|
||||
|
||||
---
|
||||
|
||||
## Incident Response
|
||||
|
||||
### Severity Definitions
|
||||
|
||||
| Severity | Definition | Response Time | Resolution Target |
|
||||
|----------|------------|---------------|-------------------|
|
||||
| P0 | Fleet-wide outage, no nodes operational | 15 minutes | 4 hours |
|
||||
| P1 | Region/node cluster outage, >50% down | 30 minutes | 8 hours |
|
||||
| P2 | Single node failure | 1 hour | 24 hours |
|
||||
| P3 | Degraded performance, not critical | 4 hours | 3 days |
|
||||
|
||||
### Response Procedure
|
||||
|
||||
#### P0/P1 Incidents
|
||||
1. Acknowledge alert immediately
|
||||
2. Declare incident in `#fleet-incidents` Slack channel
|
||||
3. Notify Fleet Lead (direct message/call)
|
||||
4. Execute recovery procedures from relevant playbook
|
||||
5. Document timeline and actions taken
|
||||
6. Schedule post-mortem within 48 hours
|
||||
|
||||
#### P2 Incidents
|
||||
1. Acknowledge within 1 hour
|
||||
2. Open incident ticket in tracking system
|
||||
3. Follow single-node recovery playbook
|
||||
4. Report resolution in daily ops log
|
||||
|
||||
#### P3 Incidents
|
||||
1. Log in issue tracker
|
||||
2. Schedule during next maintenance window
|
||||
3. Document resolution upon completion
|
||||
|
||||
### Recovery Playbooks
|
||||
|
||||
#### Node Restart (most common P2)
|
||||
1. SSH to node (or use remote management)
|
||||
2. Check system logs (`/var/log/timmy/fleet.log`)
|
||||
3. Restart service: `sudo systemctl restart timmy-fleet`
|
||||
4. Verify node rejoins cluster
|
||||
5. Monitor for 30 minutes post-recovery
|
||||
|
||||
#### Network Partition
|
||||
1. Verify network connectivity (ping, traceroute)
|
||||
2. Check firewall rules
|
||||
3. Contact network provider if external
|
||||
4. Switch to backup connection if available
|
||||
5. Document root cause
|
||||
|
||||
#### Storage Full
|
||||
1. Identify large directories (`du -sh /* | sort -hr`)
|
||||
2. Rotate logs: `sudo logrotate -f /etc/logrotate.d/timmy`
|
||||
3. Clean temporary files
|
||||
4. Expand storage or add new volume
|
||||
5. Alert Fleet Lead for capacity planning
|
||||
|
||||
---
|
||||
|
||||
## Escalation Paths
|
||||
|
||||
### Tiered Support Model
|
||||
|
||||
```
|
||||
Operator (Tier 1)
|
||||
↓ (15 min SLA)
|
||||
Senior Operator / Fleet Lead (Tier 2)
|
||||
↓ (1 hour SLA)
|
||||
Timmy Core Team (Tier 3)
|
||||
↓ (Immediate)
|
||||
Executive Sponsor (Critical only)
|
||||
```
|
||||
|
||||
### Contact Matrix
|
||||
|
||||
| Issue Type | Primary Contact | Secondary |
|
||||
|------------|----------------|-----------|
|
||||
| Technical incident | Fleet Lead | Timmy Core |
|
||||
| Payment/incentive | Finance Partner | Fleet Lead |
|
||||
| Training/certification | Training Coordinator | Fleet Lead |
|
||||
| Partnership inquiry | Partner Manager | Executive Sponsor |
|
||||
| Security incident | Security Team | Timmy Core (immediate) |
|
||||
|
||||
### Emergency Contacts
|
||||
- Fleet Lead: `fleet-lead@timmy.foundation` (Slack: @fleet-lead)
|
||||
- Timmy Core On-Call: `oncall@timmy.foundation` (PagerDuty)
|
||||
- Security: `security@timmy.foundation`
|
||||
- Finance: `finance@timmy.foundation`
|
||||
|
||||
---
|
||||
|
||||
## Communication Protocols
|
||||
|
||||
### Channels
|
||||
- `#fleet-operators` — Daily ops, questions
|
||||
- `#fleet-incidents` — Active incidents only
|
||||
- `#fleet-training` — Training resources, scheduling
|
||||
- `#fleet-partners` — Partner program discussions
|
||||
|
||||
### Status Updates
|
||||
- Daily: Stand-up notes in thread
|
||||
- Weekly: Summary post in `#fleet-operators`
|
||||
- Monthly: Ops report submission
|
||||
- Incident: Real-time updates in `#fleet-incidents`
|
||||
|
||||
### Documentation Standards
|
||||
- Use clear, concise language
|
||||
- Include timestamps in UTC
|
||||
- Link to relevant tickets/PRs
|
||||
- Tag stakeholders with `@`
|
||||
|
||||
---
|
||||
|
||||
## Node Deployment
|
||||
|
||||
### Pre-Deployment Checklist
|
||||
- [ ] Hardware meets minimum specs (CPU, RAM, storage)
|
||||
- [ ] Network connectivity validated
|
||||
- [ ] Firewall rules configured
|
||||
- [ ] SSH keys exchanged with Timmy core team
|
||||
- [ ] Monitoring agent installed
|
||||
- [ ] Backup solution active
|
||||
- [ ] Documentation updated with node details
|
||||
|
||||
### Deployment Steps
|
||||
1. Provision hardware/VM
|
||||
2. Install Timmy Fleet software
|
||||
3. Configure node ID and credentials
|
||||
4. Join cluster via `timmy-fleet join <cluster-endpoint>`
|
||||
5. Validate connectivity and heartbeat
|
||||
6. Update inventory spreadsheet
|
||||
7. Set up monitoring alerts
|
||||
8. Complete handover to operator
|
||||
|
||||
### Decommissioning
|
||||
1. Drain node from cluster
|
||||
2. Migrate workloads
|
||||
3. Backup final state
|
||||
4. Shut down cleanly
|
||||
5. Update inventory
|
||||
6. Notify relevant teams
|
||||
|
||||
---
|
||||
|
||||
## Compliance & Reporting
|
||||
|
||||
### Metrics to Track
|
||||
- Uptime (node-level and fleet-wide)
|
||||
- Incident count and severity
|
||||
- Response and resolution times
|
||||
- Training hours completed
|
||||
- Payment/compensation accuracy
|
||||
|
||||
### Reporting Cadence
|
||||
- **Daily**: Ops dashboard (automated)
|
||||
- **Weekly**: Status summary (operator)
|
||||
- **Monthly**: Partner report (template-driven)
|
||||
- **Quarterly**: Performance review with Fleet Lead
|
||||
|
||||
### Audits
|
||||
- Quarterly internal audit by Timmy compliance team
|
||||
- Annual external certification renewal
|
||||
- Ad-hoc security reviews as needed
|
||||
|
||||
---
|
||||
|
||||
## Appendix: Resources
|
||||
|
||||
### Useful Commands
|
||||
```bash
|
||||
# Check service status
|
||||
sudo systemctl status timmy-fleet
|
||||
|
||||
# View logs
|
||||
journalctl -u timmy-fleet -f
|
||||
|
||||
# Restart node
|
||||
sudo systemctl restart timmy-fleet
|
||||
|
||||
# Check node health
|
||||
timmy-fleet health
|
||||
|
||||
# Join cluster
|
||||
timmy-fleet join <cluster-endpoint>
|
||||
```
|
||||
|
||||
### Key Files
|
||||
- Config: `/etc/timmy/fleet/config.yaml`
|
||||
- Logs: `/var/log/timmy/fleet.log`
|
||||
- Health data: `/var/lib/timmy/fleet/health.json`
|
||||
|
||||
### Support Resources
|
||||
- Internal Wiki: `https://wiki.timmy.foundation/fleet`
|
||||
- Operator Portal: `https://fleet.timmy.foundation`
|
||||
- Training Videos: `https://learn.timmy.foundation/fleet-ops`
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-05-02
|
||||
**Next Review**: 2025-06-02
|
||||
@@ -1,143 +0,0 @@
|
||||
# Fleet Operator Application
|
||||
|
||||
> {{APPLICATION_DATE}}
|
||||
> Candidate: {{CANDIDATE_NAME}}
|
||||
|
||||
## Contact Information
|
||||
|
||||
**Full Name**: {{CANDIDATE_FULL_NAME}}
|
||||
**Email**: {{CANDIDATE_EMAIL}}
|
||||
**Phone**: {{CANDIDATE_PHONE}}
|
||||
**Location**: {{CANDIDATE_LOCATION}}
|
||||
**Time Zone**: {{CANDIDATE_TIMEZONE}}
|
||||
|
||||
### Availability
|
||||
- **Hours per week**: {{AVAILABILITY_HOURS}}
|
||||
- **Primary availability window (UTC)**: {{AVAILABILITY_WINDOW}}
|
||||
- **On-call flexibility**: {{ONCALL_FLEXIBILITY}}
|
||||
|
||||
## Technical Qualifications
|
||||
|
||||
### Experience
|
||||
```
|
||||
Years in IT/DevOps: {{YEARS_EXPERIENCE}}
|
||||
Relevant roles:
|
||||
{{ROLE_HISTORY}}
|
||||
```
|
||||
|
||||
### Skills (check all that apply)
|
||||
- [ ] Linux system administration
|
||||
- [ ] Container orchestration (Kubernetes/Docker)
|
||||
- [ ] Cloud infrastructure (AWS/GCP/Azure)
|
||||
- [ ] Networking fundamentals
|
||||
- [ ] Monitoring & alerting (Prometheus/Grafana)
|
||||
- [ ] Incident response/ITIL
|
||||
- [ ] Security best practices
|
||||
- [ ] Automation (Ansible/Terraform)
|
||||
- [ ] Scripting (Python/Bash/Go)
|
||||
- [ ] Timmy platform experience
|
||||
|
||||
**Additional skills**: {{ADDITIONAL_SKILLS}}
|
||||
|
||||
### Certifications
|
||||
{{CERTIFICATIONS}}
|
||||
|
||||
## Infrastructure Readiness
|
||||
|
||||
### Proposed Node Environment
|
||||
- **Type**: ☐ Physical ☐ Cloud VM ☐ Hybrid
|
||||
- **Provider**: {{CLOUD_PROVIDER}}
|
||||
- **Region**: {{REGION}}
|
||||
- **Hardware specs**:
|
||||
- CPU: {{CPU_SPEC}}
|
||||
- RAM: {{RAM_SPEC}}
|
||||
- Storage: {{STORAGE_SPEC}}
|
||||
- Network: {{NETWORK_SPEC}}
|
||||
|
||||
### Redundancy & HA
|
||||
- [ ] Backup power (UPS/generator)
|
||||
- [ ] Secondary internet connection
|
||||
- [ ] Off-site backup solution
|
||||
- [ ] Remote management (IPMI/iDRAC)
|
||||
|
||||
### Connectivity
|
||||
- **Bandwidth**: {{BANDWIDTH}} Mbps
|
||||
- **Latency to Timmy core**: {{LATENCY}} ms
|
||||
- **Uptime SLA**: {{UPTIME_SLA}}
|
||||
|
||||
---
|
||||
|
||||
## Motivation & Alignment
|
||||
|
||||
### Why do you want to run a Timmy Fleet node?
|
||||
{{MOTIVATION}}
|
||||
|
||||
### What attracts you to decentralized infrastructure?
|
||||
{{DECENTRALIZATION_MOTIVATION}}
|
||||
|
||||
### How does this align with your long-term goals?
|
||||
{{LONG_TERM_GOALS}}
|
||||
|
||||
---
|
||||
|
||||
## Partner Program Interest (Optional)
|
||||
|
||||
### Interested in?
|
||||
- [ ] Referral partner (refer operators, earn commission)
|
||||
- [ ] Channel partner (onboard and train operators)
|
||||
- [ ] Strategic partner (run fleet of 10+ nodes)
|
||||
|
||||
### Existing network
|
||||
{{PARTNER_NETWORK}}
|
||||
|
||||
### Referral pipeline
|
||||
{{REFERRAL_PIPELINE}}
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
### Professional References
|
||||
1. Name: {{REF1_NAME}}
|
||||
Email: {{REF1_EMAIL}}
|
||||
Relationship: {{REF1_RELATION}}
|
||||
|
||||
2. Name: {{REF2_NAME}}
|
||||
Email: {{REF2_EMAIL}}
|
||||
Relationship: {{REF2_RELATION}}
|
||||
|
||||
### Timmy Community Involvement
|
||||
{{COMMUNITY_INVOLVEMENT}}
|
||||
|
||||
---
|
||||
|
||||
## Agreement & Signatures
|
||||
|
||||
### Code of Conduct
|
||||
- [ ] I have read and agree to the Timmy Fleet Operator Code of Conduct
|
||||
- [ ] I understand the uptime and response time requirements
|
||||
- [ ] I agree to the incentive structure and terms
|
||||
|
||||
### Signature
|
||||
**Candidate signature**: ___________________________
|
||||
**Date**: {{SIGNATURE_DATE}}
|
||||
|
||||
**Timmy representative**: ___________________________
|
||||
**Date**: {{TIMPY_SIGN_DATE}}
|
||||
|
||||
---
|
||||
|
||||
## Internal Use Only
|
||||
|
||||
**Interviewer**: {{INTERVIEWER}}
|
||||
**Technical score**: {{TECH_SCORE}}/100
|
||||
**Culture fit**: {{CULTURE_FIT}}/50
|
||||
**Infrastructure audit**: ☐ Pass ☐ Fail
|
||||
**Background check**: ☐ Complete ☐ In-progress
|
||||
|
||||
**Decision**: ☐ Approved ☐ Rejected ☐ Waitlist
|
||||
|
||||
**Comments**: {{INTERNAL_COMMENTS}}
|
||||
|
||||
**Certification ID**: {{CERT_ID}}
|
||||
**Onboarding start date**: {{ONBOARDING_DATE}}
|
||||
@@ -1,175 +0,0 @@
|
||||
# Fleet Partner Monthly Report
|
||||
|
||||
> {{REPORT_MONTH}} {{REPORT_YEAR}}
|
||||
> Partner: {{PARTNER_NAME}} ({{PARTNER_TIER}})
|
||||
> Submitted: {{SUBMISSION_DATE}}
|
||||
|
||||
## Executive Summary
|
||||
|
||||
| Metric | Current Month | Target | Variance |
|
||||
|--------|---------------|--------|----------|
|
||||
| Active nodes managed | {{ACTIVE_NODES}} | {{TARGET_NODES}} | {{NODES_VARIANCE}} |
|
||||
| Fleet uptime | {{UPTIME}}% | 99.5% | {{UPTIME_VARIANCE}}% |
|
||||
| Operator churn rate | {{CHURN_RATE}}% | <10% | {{CHURN_VARIANCE}}% |
|
||||
| Partner-sourced leads | {{LEADS_COUNT}} | {{LEADS_TARGET}} | {{LEADS_VARIANCE}} |
|
||||
| Revenue share earned | {{REVENUE}} | — | — |
|
||||
|
||||
**Key highlights**:
|
||||
{{KEY_HIGHLIGHTS}}
|
||||
|
||||
**Top concerns**:
|
||||
{{KEY_CONCERNS}}
|
||||
|
||||
---
|
||||
|
||||
## Node Performance
|
||||
|
||||
### Node Inventory
|
||||
|
||||
| Node ID | Location | Status | Uptime (30d) | Revenue Share | Issues |
|
||||
|---------|----------|--------|--------------|---------------|---------|
|
||||
| {{NODE_1_ID}} | {{NODE_1_LOC}} | {{NODE_1_STATUS}} | {{NODE_1_UPTIME}}% | ${{NODE_1_REV}} | {{NODE_1_ISSUES}} |
|
||||
| {{NODE_2_ID}} | {{NODE_2_LOC}} | {{NODE_2_STATUS}} | {{NODE_2_UPTIME}}% | ${{NODE_2_REV}} | {{NODE_2_ISSUES}} |
|
||||
| {{NODE_3_ID}} | {{NODE_3_LOC}} | {{NODE_3_STATUS}} | {{NODE_3_UPTIME}}% | ${{NODE_3_REV}} | {{NODE_3_ISSUES}} |
|
||||
|
||||
*Add rows as needed*
|
||||
|
||||
### Top Node Performers
|
||||
1. **{{TOP_NODE_1_ID}}**: {{TOP_NODE_1_UPTIME}}% uptime, zero incidents
|
||||
2. **{{TOP_NODE_2_ID}}**: {{TOP_NODE_2_UPTIME}}% uptime, quickest response times
|
||||
|
||||
### Nodes Requiring Attention
|
||||
1. **{{ATTN_NODE_1_ID}}**: {{ATTN_NODE_1_ISSUE}}
|
||||
2. **{{ATTN_NODE_2_ID}}**: {{ATTN_NODE_2_ISSUE}}
|
||||
|
||||
---
|
||||
|
||||
## Incidents & Resolutions
|
||||
|
||||
### Incident Log
|
||||
|
||||
| Date | Severity | Node(s) | Duration | Root Cause | Resolution |
|
||||
|------|----------|---------|----------|------------|------------|
|
||||
| {{INC1_DATE}} | {{INC1_SEV}} | {{INC1_NODES}} | {{INC1_DURATION}} | {{INC1_CAUSE}} | {{INC1_RES}} |
|
||||
| {{INC2_DATE}} | {{INC2_SEV}} | {{INC2_NODES}} | {{INC2_DURATION}} | {{INC2_CAUSE}} | {{INC2_RES}} |
|
||||
| {{INC3_DATE}} | {{INC3_SEV}} | {{INC3_NODES}} | {{INC3_DURATION}} | {{INC3_CAUSE}} | {{INC3_RES}} |
|
||||
|
||||
*Add rows as needed*
|
||||
|
||||
### Mean Time to Recovery (MTTR)
|
||||
- **P0 incidents**: {{MTTR_P0}} hours
|
||||
- **P1 incidents**: {{MTTR_P1}} hours
|
||||
- **P2 incidents**: {{MTTR_P2}} hours
|
||||
- **P3 incidents**: {{MTTR_P3}} hours
|
||||
|
||||
**Improvement opportunities**:
|
||||
{{MTTR_IMPROVEMENTS}}
|
||||
|
||||
---
|
||||
|
||||
## Operator Management
|
||||
|
||||
### Active Operators
|
||||
|
||||
| Operator | Tier | Nodes Managed | Status | Cert Date |
|
||||
|----------|------|---------------|--------|-----------|
|
||||
| {{OP1_NAME}} | {{OP1_TIER}} | {{OP1_NODES}} | {{OP1_STATUS}} | {{OP1_CERT}} |
|
||||
| {{OP2_NAME}} | {{OP2_TIER}} | {{OP2_NODES}} | {{OP2_STATUS}} | {{OP2_CERT}} |
|
||||
|
||||
### Churn / Attrition
|
||||
- **Departed operators**: {{DEPARTED_COUNT}}
|
||||
- **Departure reasons**: {{DEPARTURE_REASONS}}
|
||||
- **Retention initiatives**: {{RETENTION_INITIATIVES}}
|
||||
|
||||
### Training & Certification
|
||||
- **New certifications**: {{NEW_CERTS}}
|
||||
- **Training hours logged**: {{TRAINING_HOURS}}
|
||||
- **Upcoming recertifications**: {{UPCOMING_RECERTS}}
|
||||
|
||||
---
|
||||
|
||||
## Partner Program Metrics
|
||||
|
||||
### Lead Generation
|
||||
- **Total leads received**: {{TOTAL_LEADS}}
|
||||
- **Qualified leads**: {{QUALIFIED_LEADS}}
|
||||
- **Converted to operators**: {{CONVERTED_OPERATORS}}
|
||||
- **Conversion rate**: {{CONVERSION_RATE}}%
|
||||
- **Partner contribution to total pipeline**: {{PARTNER_PIPELINE_PERCENT}}%
|
||||
|
||||
### Referral Commission
|
||||
- **Referral fee earned**: ${{REFERRAL_FEE}}
|
||||
- **Ongoing revenue share**: ${{ONGOING_SHARE}}
|
||||
- **Total YTD earnings**: ${{YTD_EARNINGS}}
|
||||
|
||||
### Partner Activity
|
||||
- **Marketing events hosted**: {{EVENTS_HOSTED}}
|
||||
- **Training sessions conducted**: {{TRAINING_SESSIONS}}
|
||||
- **Community engagement posts**: {{COMMUNITY_POSTS}}
|
||||
- **Collateral created**: {{COLLATERAL}}
|
||||
|
||||
---
|
||||
|
||||
## Financial Summary
|
||||
|
||||
### Incentive Payouts
|
||||
| Category | Amount | Notes |
|
||||
|----------|--------|-------|
|
||||
| Operator stipends | ${{STIPENDS}} | {{STIPENDS_NOTES}} |
|
||||
| Uptime bonuses | ${{UPTIME_BONUS}} | {{UPTIME_NOTES}} |
|
||||
| Mentorship bonuses | ${{MENTOR_BONUS}} | {{MENTOR_NOTES}} |
|
||||
| Performance bonuses | ${{PERF_BONUS}} | {{PERF_NOTES}} |
|
||||
| Partner commissions | ${{PARTNER_COMM}} | {{PARTNER_NOTES}} |
|
||||
|
||||
**Total payout this month**: ${{TOTAL_PAYOUT}}
|
||||
|
||||
### Cost Efficiency
|
||||
- **Cost per node**: ${{COST_PER_NODE}}
|
||||
- **Cost per uptime hour**: ${{COST_PER_UPTIME_HOUR}}
|
||||
- **Efficiency rating**: {{EFFICIENCY_RATING}}/10
|
||||
|
||||
---
|
||||
|
||||
## Goals & Objectives
|
||||
|
||||
### Next Month Targets
|
||||
1. **Uptime**: {{NEXT_UPTIME_TARGET}}%
|
||||
2. **Qualified leads**: {{NEXT_LEADS_TARGET}}
|
||||
3. **New operators**: {{NEXT_OPS_TARGET}}
|
||||
4. **Incident reduction**: {{NEXT_INCIDENT_TARGET}} incidents
|
||||
|
||||
### Priority Initiatives
|
||||
- {{PRIORITY_1}}
|
||||
- {{PRIORITY_2}}
|
||||
- {{PRIORITY_3}}
|
||||
|
||||
### Support Needed
|
||||
- {{SUPPORT_NEEDED_1}}
|
||||
- {{SUPPORT_NEEDED_2}}
|
||||
|
||||
---
|
||||
|
||||
## Attestation
|
||||
|
||||
By submitting this report, I certify that the information provided is accurate and complete to the best of my knowledge.
|
||||
|
||||
**Submitted by**: {{SUBMITTER_NAME}}
|
||||
**Title**: {{SUBMITTER_TITLE}}
|
||||
**Signature**: ___________________________
|
||||
**Date**: {{SUBMISSION_DATE}}
|
||||
|
||||
**Approved by** (Timmy Core): {{APPROVER_NAME}}
|
||||
**Date**: {{APPROVAL_DATE}}
|
||||
|
||||
---
|
||||
|
||||
## Appendix
|
||||
|
||||
### Supporting Documents
|
||||
- [ ] Ops dashboard screenshots attached
|
||||
- [ ] Incident post-mortems attached
|
||||
- [ ] Training completion records attached
|
||||
- [ ] Financial reconciliation attached
|
||||
|
||||
### Notes
|
||||
{{APPENDIX_NOTES}}
|
||||
@@ -1,12 +1 @@
|
||||
# Timmy core module
|
||||
|
||||
from .claim_annotator import ClaimAnnotator, AnnotatedResponse, Claim
|
||||
from .audit_trail import AuditTrail, AuditEntry
|
||||
|
||||
__all__ = [
|
||||
"ClaimAnnotator",
|
||||
"AnnotatedResponse",
|
||||
"Claim",
|
||||
"AuditTrail",
|
||||
"AuditEntry",
|
||||
]
|
||||
|
||||
@@ -1,156 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Response Claim Annotator — Source Distinction System
|
||||
SOUL.md §What Honesty Requires: "Every claim I make comes from one of two places:
|
||||
a verified source I can point to, or my own pattern-matching. My user must be
|
||||
able to tell which is which."
|
||||
"""
|
||||
|
||||
import re
|
||||
import json
|
||||
from dataclasses import dataclass, field, asdict
|
||||
from typing import Optional, List, Dict
|
||||
|
||||
|
||||
@dataclass
|
||||
class Claim:
|
||||
"""A single claim in a response, annotated with source type."""
|
||||
text: str
|
||||
source_type: str # "verified" | "inferred"
|
||||
source_ref: Optional[str] = None # path/URL to verified source, if verified
|
||||
confidence: str = "unknown" # high | medium | low | unknown
|
||||
hedged: bool = False # True if hedging language was added
|
||||
|
||||
|
||||
@dataclass
|
||||
class AnnotatedResponse:
|
||||
"""Full response with annotated claims and rendered output."""
|
||||
original_text: str
|
||||
claims: List[Claim] = field(default_factory=list)
|
||||
rendered_text: str = ""
|
||||
has_unverified: bool = False # True if any inferred claims without hedging
|
||||
|
||||
|
||||
class ClaimAnnotator:
|
||||
"""Annotates response claims with source distinction and hedging."""
|
||||
|
||||
# Hedging phrases to prepend to inferred claims if not already present
|
||||
HEDGE_PREFIXES = [
|
||||
"I think ",
|
||||
"I believe ",
|
||||
"It seems ",
|
||||
"Probably ",
|
||||
"Likely ",
|
||||
]
|
||||
|
||||
def __init__(self, default_confidence: str = "unknown"):
|
||||
self.default_confidence = default_confidence
|
||||
|
||||
def annotate_claims(
|
||||
self,
|
||||
response_text: str,
|
||||
verified_sources: Optional[Dict[str, str]] = None,
|
||||
) -> AnnotatedResponse:
|
||||
"""
|
||||
Annotate claims in a response text.
|
||||
|
||||
Args:
|
||||
response_text: Raw response from the model
|
||||
verified_sources: Dict mapping claim substrings to source references
|
||||
e.g. {"Paris is the capital of France": "https://en.wikipedia.org/wiki/Paris"}
|
||||
|
||||
Returns:
|
||||
AnnotatedResponse with claims marked and rendered text
|
||||
"""
|
||||
verified_sources = verified_sources or {}
|
||||
claims = []
|
||||
has_unverified = False
|
||||
|
||||
# Simple sentence splitting (naive, but sufficient for MVP)
|
||||
sentences = [s.strip() for s in re.split(r'[.!?]\s+', response_text) if s.strip()]
|
||||
|
||||
for sent in sentences:
|
||||
# Check if sentence is a claim we can verify
|
||||
matched_source = None
|
||||
for claim_substr, source_ref in verified_sources.items():
|
||||
if claim_substr.lower() in sent.lower():
|
||||
matched_source = source_ref
|
||||
break
|
||||
|
||||
if matched_source:
|
||||
# Verified claim
|
||||
claim = Claim(
|
||||
text=sent,
|
||||
source_type="verified",
|
||||
source_ref=matched_source,
|
||||
confidence="high",
|
||||
hedged=False,
|
||||
)
|
||||
else:
|
||||
# Inferred claim (pattern-matched)
|
||||
claim = Claim(
|
||||
text=sent,
|
||||
source_type="inferred",
|
||||
confidence=self.default_confidence,
|
||||
hedged=self._has_hedge(sent),
|
||||
)
|
||||
if not claim.hedged:
|
||||
has_unverified = True
|
||||
|
||||
claims.append(claim)
|
||||
|
||||
# Render the annotated response
|
||||
rendered = self._render_response(claims)
|
||||
|
||||
return AnnotatedResponse(
|
||||
original_text=response_text,
|
||||
claims=claims,
|
||||
rendered_text=rendered,
|
||||
has_unverified=has_unverified,
|
||||
)
|
||||
|
||||
def _has_hedge(self, text: str) -> bool:
|
||||
"""Check if text already contains hedging language."""
|
||||
text_lower = text.lower()
|
||||
for prefix in self.HEDGE_PREFIXES:
|
||||
if text_lower.startswith(prefix.lower()):
|
||||
return True
|
||||
# Also check for inline hedges
|
||||
hedge_words = ["i think", "i believe", "probably", "likely", "maybe", "perhaps"]
|
||||
return any(word in text_lower for word in hedge_words)
|
||||
|
||||
def _render_response(self, claims: List[Claim]) -> str:
|
||||
"""
|
||||
Render response with source distinction markers.
|
||||
|
||||
Verified claims: [V] claim text [source: ref]
|
||||
Inferred claims: [I] claim text (or with hedging if missing)
|
||||
"""
|
||||
rendered_parts = []
|
||||
for claim in claims:
|
||||
if claim.source_type == "verified":
|
||||
part = f"[V] {claim.text}"
|
||||
if claim.source_ref:
|
||||
part += f" [source: {claim.source_ref}]"
|
||||
else: # inferred
|
||||
if not claim.hedged:
|
||||
# Add hedging if missing
|
||||
hedged_text = f"I think {claim.text[0].lower()}{claim.text[1:]}" if claim.text else claim.text
|
||||
part = f"[I] {hedged_text}"
|
||||
else:
|
||||
part = f"[I] {claim.text}"
|
||||
rendered_parts.append(part)
|
||||
return " ".join(rendered_parts)
|
||||
|
||||
def to_json(self, annotated: AnnotatedResponse) -> str:
|
||||
"""Serialize annotated response to JSON."""
|
||||
return json.dumps(
|
||||
{
|
||||
"original_text": annotated.original_text,
|
||||
"rendered_text": annotated.rendered_text,
|
||||
"has_unverified": annotated.has_unverified,
|
||||
"claims": [asdict(c) for c in annotated.claims],
|
||||
},
|
||||
indent=2,
|
||||
ensure_ascii=False,
|
||||
)
|
||||
@@ -1,14 +1,20 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
import yaml
|
||||
|
||||
from scripts.bezalel_gemma4_vps import (
|
||||
DEFAULT_CONFIG_PATH,
|
||||
DEFAULT_BEZALEL_VPS_HOST,
|
||||
build_deploy_mutation,
|
||||
build_runpod_endpoint,
|
||||
build_vps_verify_command,
|
||||
normalize_openai_base_url,
|
||||
parse_deploy_response,
|
||||
resolve_base_url,
|
||||
update_config_text,
|
||||
verify_openai_chat,
|
||||
)
|
||||
@@ -28,6 +34,10 @@ class _FakeResponse:
|
||||
return False
|
||||
|
||||
|
||||
def test_default_config_path_targets_bezalel_vps_root_config() -> None:
|
||||
assert DEFAULT_CONFIG_PATH == Path("/root/wizards/bezalel/home/config.yaml")
|
||||
|
||||
|
||||
def test_build_deploy_mutation_uses_ollama_image_and_openai_port() -> None:
|
||||
query = build_deploy_mutation(name="bezalel-gemma4", gpu_type="NVIDIA L40S", model_tag="gemma4:latest")
|
||||
|
||||
@@ -37,6 +47,30 @@ def test_build_deploy_mutation_uses_ollama_image_and_openai_port() -> None:
|
||||
assert 'volumeMountPath: "/root/.ollama"' in query
|
||||
|
||||
|
||||
def test_normalize_openai_base_url_adds_v1_suffix() -> None:
|
||||
assert normalize_openai_base_url("https://pod-11434.proxy.runpod.net") == "https://pod-11434.proxy.runpod.net/v1"
|
||||
|
||||
|
||||
def test_normalize_openai_base_url_trims_chat_completions_suffix() -> None:
|
||||
assert normalize_openai_base_url("https://pod-11434.proxy.runpod.net/v1/chat/completions") == "https://pod-11434.proxy.runpod.net/v1"
|
||||
|
||||
|
||||
def test_resolve_base_url_prefers_vertex_over_base_and_pod_id() -> None:
|
||||
base_url, source = resolve_base_url(
|
||||
vertex_base_url="https://vertex.example.com/openai",
|
||||
base_url="https://plain.example.com",
|
||||
pod_id="abc123",
|
||||
)
|
||||
assert source == "vertex_base_url"
|
||||
assert base_url == "https://vertex.example.com/openai/v1"
|
||||
|
||||
|
||||
def test_resolve_base_url_falls_back_to_base_url_before_pod_id() -> None:
|
||||
base_url, source = resolve_base_url(base_url="https://plain.example.com", pod_id="abc123")
|
||||
assert source == "base_url"
|
||||
assert base_url == "https://plain.example.com/v1"
|
||||
|
||||
|
||||
def test_build_runpod_endpoint_appends_v1_suffix() -> None:
|
||||
assert build_runpod_endpoint("abc123") == "https://abc123-11434.proxy.runpod.net/v1"
|
||||
|
||||
@@ -60,7 +94,7 @@ def test_parse_deploy_response_extracts_pod_id_and_endpoint() -> None:
|
||||
}
|
||||
|
||||
|
||||
def test_update_config_text_upserts_big_brain_provider() -> None:
|
||||
def test_update_config_text_upserts_big_brain_provider_and_normalizes_base_url() -> None:
|
||||
original = """
|
||||
model:
|
||||
default: kimi-k2.5
|
||||
@@ -72,7 +106,7 @@ custom_providers:
|
||||
model: gemma3:27b
|
||||
"""
|
||||
|
||||
updated = update_config_text(original, base_url="https://new-pod-11434.proxy.runpod.net/v1", model="gemma4:latest")
|
||||
updated = update_config_text(original, base_url="https://new-pod-11434.proxy.runpod.net", model="gemma4:latest")
|
||||
parsed = yaml.safe_load(updated)
|
||||
|
||||
assert parsed["model"] == {"default": "kimi-k2.5", "provider": "kimi-coding"}
|
||||
@@ -86,7 +120,14 @@ custom_providers:
|
||||
]
|
||||
|
||||
|
||||
def test_verify_openai_chat_calls_chat_completions() -> None:
|
||||
def test_build_vps_verify_command_targets_bezalel_host_and_chat_completions() -> None:
|
||||
command = build_vps_verify_command(base_url="https://pod-11434.proxy.runpod.net", model="gemma4:latest")
|
||||
assert command.startswith(f"ssh root@{DEFAULT_BEZALEL_VPS_HOST} ")
|
||||
assert "/v1/chat/completions" in command
|
||||
assert "gemma4:latest" in command
|
||||
|
||||
|
||||
def test_verify_openai_chat_calls_chat_completions_with_normalized_base_url() -> None:
|
||||
response_payload = {
|
||||
"choices": [
|
||||
{
|
||||
@@ -101,7 +142,7 @@ def test_verify_openai_chat_calls_chat_completions() -> None:
|
||||
"scripts.bezalel_gemma4_vps.request.urlopen",
|
||||
return_value=_FakeResponse(response_payload),
|
||||
) as mocked:
|
||||
result = verify_openai_chat("https://pod-11434.proxy.runpod.net/v1", model="gemma4:latest", prompt="say READY")
|
||||
result = verify_openai_chat("https://pod-11434.proxy.runpod.net", model="gemma4:latest", prompt="say READY")
|
||||
|
||||
assert result == "READY"
|
||||
req = mocked.call_args.args[0]
|
||||
@@ -109,3 +150,10 @@ def test_verify_openai_chat_calls_chat_completions() -> None:
|
||||
payload = json.loads(req.data.decode())
|
||||
assert payload["model"] == "gemma4:latest"
|
||||
assert payload["messages"][0]["content"] == "say READY"
|
||||
|
||||
|
||||
def test_readme_documents_root_config_path_and_vps_proof_command() -> None:
|
||||
readme = Path("scripts/README_bezalel_gemma4_vps.md").read_text()
|
||||
assert "/root/wizards/bezalel/home/config.yaml" in readme
|
||||
assert "ssh root@104.131.15.18" in readme
|
||||
assert "--vertex-base-url" in readme
|
||||
|
||||
@@ -1,103 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Tests for claim_annotator.py — verifies source distinction is present."""
|
||||
|
||||
import sys
|
||||
import os
|
||||
import json
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "src"))
|
||||
|
||||
from timmy.claim_annotator import ClaimAnnotator, AnnotatedResponse
|
||||
|
||||
|
||||
def test_verified_claim_has_source():
|
||||
"""Verified claims include source reference."""
|
||||
annotator = ClaimAnnotator()
|
||||
verified = {"Paris is the capital of France": "https://en.wikipedia.org/wiki/Paris"}
|
||||
response = "Paris is the capital of France. It is a beautiful city."
|
||||
|
||||
result = annotator.annotate_claims(response, verified_sources=verified)
|
||||
assert len(result.claims) > 0
|
||||
verified_claims = [c for c in result.claims if c.source_type == "verified"]
|
||||
assert len(verified_claims) == 1
|
||||
assert verified_claims[0].source_ref == "https://en.wikipedia.org/wiki/Paris"
|
||||
assert "[V]" in result.rendered_text
|
||||
assert "[source:" in result.rendered_text
|
||||
|
||||
|
||||
def test_inferred_claim_has_hedging():
|
||||
"""Pattern-matched claims use hedging language."""
|
||||
annotator = ClaimAnnotator()
|
||||
response = "The weather is nice today. It might rain tomorrow."
|
||||
|
||||
result = annotator.annotate_claims(response)
|
||||
inferred_claims = [c for c in result.claims if c.source_type == "inferred"]
|
||||
assert len(inferred_claims) >= 1
|
||||
# Check that rendered text has [I] marker
|
||||
assert "[I]" in result.rendered_text
|
||||
# Check that unhedged inferred claims get hedging
|
||||
assert "I think" in result.rendered_text or "I believe" in result.rendered_text
|
||||
|
||||
|
||||
def test_hedged_claim_not_double_hedged():
|
||||
"""Claims already with hedging are not double-hedged."""
|
||||
annotator = ClaimAnnotator()
|
||||
response = "I think the sky is blue. It is a nice day."
|
||||
|
||||
result = annotator.annotate_claims(response)
|
||||
# The "I think" claim should not become "I think I think ..."
|
||||
assert "I think I think" not in result.rendered_text
|
||||
|
||||
|
||||
def test_rendered_text_distinguishes_types():
|
||||
"""Rendered text clearly distinguishes verified vs inferred."""
|
||||
annotator = ClaimAnnotator()
|
||||
verified = {"Earth is round": "https://science.org/earth"}
|
||||
response = "Earth is round. Stars are far away."
|
||||
|
||||
result = annotator.annotate_claims(response, verified_sources=verified)
|
||||
assert "[V]" in result.rendered_text # verified marker
|
||||
assert "[I]" in result.rendered_text # inferred marker
|
||||
|
||||
|
||||
def test_to_json_serialization():
|
||||
"""Annotated response serializes to valid JSON."""
|
||||
annotator = ClaimAnnotator()
|
||||
response = "Test claim."
|
||||
result = annotator.annotate_claims(response)
|
||||
json_str = annotator.to_json(result)
|
||||
parsed = json.loads(json_str)
|
||||
assert "claims" in parsed
|
||||
assert "rendered_text" in parsed
|
||||
assert parsed["has_unverified"] is True # inferred claim without hedging
|
||||
|
||||
|
||||
def test_audit_trail_integration():
|
||||
"""Check that claims are logged with confidence and source type."""
|
||||
# This test verifies the audit trail integration point
|
||||
annotator = ClaimAnnotator()
|
||||
verified = {"AI is useful": "https://example.com/ai"}
|
||||
response = "AI is useful. It can help with tasks."
|
||||
|
||||
result = annotator.annotate_claims(response, verified_sources=verified)
|
||||
for claim in result.claims:
|
||||
assert claim.source_type in ("verified", "inferred")
|
||||
assert claim.confidence in ("high", "medium", "low", "unknown")
|
||||
if claim.source_type == "verified":
|
||||
assert claim.source_ref is not None
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
test_verified_claim_has_source()
|
||||
print("✓ test_verified_claim_has_source passed")
|
||||
test_inferred_claim_has_hedging()
|
||||
print("✓ test_inferred_claim_has_hedging passed")
|
||||
test_hedged_claim_not_double_hedged()
|
||||
print("✓ test_hedged_claim_not_double_hedged passed")
|
||||
test_rendered_text_distinguishes_types()
|
||||
print("✓ test_rendered_text_distinguishes_types passed")
|
||||
test_to_json_serialization()
|
||||
print("✓ test_to_json_serialization passed")
|
||||
test_audit_trail_integration()
|
||||
print("✓ test_audit_trail_integration passed")
|
||||
print("\nAll tests passed!")
|
||||
Reference in New Issue
Block a user