feat: add pokemon-player skill with battle-tested gameplay tips
Comprehensive skill for playing Pokemon Red/Blue via the pokemon-agent package (NousResearch/pokemon-agent). Includes: - Full startup procedure (uv venv, server, localhost.run dashboard tunnel) - Save/load lifecycle and naming conventions - Gameplay loop with emphasis on frequent vision checks - Hard-learned navigation tips: - Use vision every 2-4 steps (RAM state is blind to obstacles) - Wait 2-3 seconds after door/stair warps for map transitions - Sidestep after exiting buildings to avoid re-entering - Hold B to speed Gen 1's slow text scrolling - Ledges are one-way — use vision to find gaps - Battle strategy, type chart, Gen 1 quirks - Memory conventions with PKM: prefix - Progression milestones through all 8 gyms + Elite Four
This commit is contained in:
teknium1
215
skills/gaming/pokemon-player/SKILL.md
Normal file
215
skills/gaming/pokemon-player/SKILL.md
Normal file
@@ -0,0 +1,215 @@
|
||||
---
|
||||
name: pokemon-player
|
||||
description: Play Pokemon games autonomously via headless emulation. Starts a game server, reads structured game state from RAM, makes strategic decisions, and sends button inputs — all from the terminal.
|
||||
tags: [gaming, pokemon, emulator, pyboy, gameplay, gameboy]
|
||||
---
|
||||
# Pokemon Player
|
||||
|
||||
Play Pokemon games via headless emulation using the `pokemon-agent` package.
|
||||
|
||||
## When to Use
|
||||
- User says "play pokemon", "start pokemon", "pokemon game"
|
||||
- User asks about Pokemon Red, Blue, Yellow, FireRed, etc.
|
||||
- User wants to watch an AI play Pokemon
|
||||
- User references a ROM file (.gb, .gbc, .gba)
|
||||
|
||||
## Startup Procedure
|
||||
|
||||
### 1. First-time setup (clone, venv, install)
|
||||
The repo is NousResearch/pokemon-agent on GitHub. Clone it, then
|
||||
set up a Python 3.10+ virtual environment. Use uv (preferred for speed)
|
||||
to create the venv and install the package in editable mode with the
|
||||
pyboy extra. If uv is not available, fall back to python3 -m venv + pip.
|
||||
|
||||
On this machine it is already set up at /home/teknium/pokemon-agent
|
||||
with a venv ready — just cd there and source .venv/bin/activate.
|
||||
|
||||
You also need a ROM file. Ask the user for theirs. On this machine
|
||||
one exists at roms/pokemon_red.gb inside that directory.
|
||||
NEVER download or provide ROM files — always ask the user.
|
||||
|
||||
### 2. Start the game server
|
||||
From inside the pokemon-agent directory with the venv activated, run
|
||||
pokemon-agent serve with --rom pointing to the ROM and --port 9876.
|
||||
Run it in the background with &.
|
||||
To resume from a saved game, add --load-state with the save name.
|
||||
Wait 4 seconds for startup, then verify with GET /health.
|
||||
|
||||
### 3. Set up live dashboard for user to watch
|
||||
Use an SSH reverse tunnel via localhost.run so the user can view
|
||||
the dashboard in their browser. Connect with ssh, forwarding local
|
||||
port 9876 to remote port 80 on nokey@localhost.run. Redirect output
|
||||
to a log file, wait 10 seconds, then grep the log for the .lhr.life
|
||||
URL. Give the user the URL with /dashboard/ appended.
|
||||
The tunnel URL changes each time — give the user the new one if restarted.
|
||||
|
||||
## Save and Load
|
||||
|
||||
### When to save
|
||||
- Every 15-20 turns of gameplay
|
||||
- ALWAYS before gym battles, rival encounters, or risky fights
|
||||
- Before entering a new town or dungeon
|
||||
- Before any action you are unsure about
|
||||
|
||||
### How to save
|
||||
POST /save with a descriptive name. Good examples:
|
||||
before_brock, route1_start, mt_moon_entrance, got_cut
|
||||
|
||||
### How to load
|
||||
POST /load with the save name.
|
||||
|
||||
### List available saves
|
||||
GET /saves returns all saved states.
|
||||
|
||||
### Loading on server startup
|
||||
Use --load-state flag when starting the server to auto-load a save.
|
||||
This is faster than loading via the API after startup.
|
||||
|
||||
## The Gameplay Loop
|
||||
|
||||
### Step 1: OBSERVE — check state AND take a screenshot
|
||||
GET /state for position, HP, battle, dialog.
|
||||
GET /screenshot and save to /tmp/pokemon.png, then use vision_analyze.
|
||||
Always do BOTH — RAM state gives numbers, vision gives spatial awareness.
|
||||
|
||||
### Step 2: ORIENT
|
||||
- Dialog/text on screen → advance it
|
||||
- In battle → fight or run
|
||||
- Party hurt → head to Pokemon Center
|
||||
- Near objective → navigate carefully
|
||||
|
||||
### Step 3: DECIDE
|
||||
Priority: dialog > battle > heal > story objective > training > explore
|
||||
|
||||
### Step 4: ACT — move 2-4 steps max, then re-check
|
||||
POST /action with a SHORT action list (2-4 actions, not 10-15).
|
||||
|
||||
### Step 5: VERIFY — screenshot after every move sequence
|
||||
Take a screenshot and use vision_analyze to confirm you moved where
|
||||
intended. This is the MOST IMPORTANT step. Without vision you WILL get lost.
|
||||
|
||||
### Step 6: RECORD progress to memory with PKM: prefix
|
||||
|
||||
### Step 7: SAVE periodically
|
||||
|
||||
## Action Reference
|
||||
- press_a — confirm, talk, select
|
||||
- press_b — cancel, close menu
|
||||
- press_start — open game menu
|
||||
- walk_up/down/left/right — move one tile
|
||||
- hold_b_N — hold B for N frames (use for speeding through text)
|
||||
- wait_60 — wait about 1 second (60 frames)
|
||||
- a_until_dialog_end — press A repeatedly until dialog clears
|
||||
|
||||
## Critical Tips from Experience
|
||||
|
||||
### USE VISION CONSTANTLY
|
||||
- Take a screenshot every 2-4 movement steps
|
||||
- The RAM state tells you position and HP but NOT what is around you
|
||||
- Ledges, fences, signs, building doors, NPCs — only visible via screenshot
|
||||
- Ask the vision model specific questions: "what is one tile north of me?"
|
||||
- When stuck, always screenshot before trying random directions
|
||||
|
||||
### Warp Transitions Need Extra Wait Time
|
||||
When walking through a door or stairs, the screen fades to black during
|
||||
the map transition. You MUST wait for it to complete. Add 2-3 wait_60
|
||||
actions after any door/stair warp. Without waiting, the position reads
|
||||
as stale and you will think you are still in the old map.
|
||||
|
||||
### Building Exit Trap
|
||||
When you exit a building, you appear directly IN FRONT of the door.
|
||||
If you walk north, you go right back inside. ALWAYS sidestep first
|
||||
by walking left or right 2 tiles, then proceed in your intended direction.
|
||||
|
||||
### Dialog Handling
|
||||
Gen 1 text scrolls slowly letter-by-letter. To speed through dialog,
|
||||
hold B for 120 frames then press A. Repeat as needed. Holding B makes
|
||||
text display at max speed. Then press A to advance to the next line.
|
||||
The a_until_dialog_end action checks the RAM dialog flag, but this flag
|
||||
does not catch ALL text states. If dialog seems stuck, use the manual
|
||||
hold_b + press_a pattern instead and verify via screenshot.
|
||||
|
||||
### Ledges Are One-Way
|
||||
Ledges (small cliff edges) can only be jumped DOWN (south), never climbed
|
||||
UP (north). If blocked by a ledge going north, you must go left or right
|
||||
to find the gap around it. Use vision to identify which direction the
|
||||
gap is. Ask the vision model explicitly.
|
||||
|
||||
### Navigation Strategy
|
||||
- Move 2-4 steps at a time, then screenshot to check position
|
||||
- When entering a new area, screenshot immediately to orient
|
||||
- Ask the vision model "which direction to [destination]?"
|
||||
- If stuck for 3+ attempts, screenshot and re-evaluate completely
|
||||
- Do not spam 10-15 movements — you will overshoot or get stuck
|
||||
|
||||
### Running from Wild Battles
|
||||
On the battle menu, RUN is bottom-right. To reach it from the default
|
||||
cursor position (FIGHT, top-left): press down then right to move cursor
|
||||
to RUN, then press A. Wrap with hold_b to speed through text/animations.
|
||||
|
||||
### Battling (FIGHT)
|
||||
On the battle menu FIGHT is top-left (default cursor position).
|
||||
Press A to enter move selection, A again to use the first move.
|
||||
Then hold B to speed through attack animations and text.
|
||||
|
||||
## Battle Strategy
|
||||
|
||||
### Decision Tree
|
||||
1. Want to catch? → Weaken then throw Poke Ball
|
||||
2. Wild you don't need? → RUN
|
||||
3. Type advantage? → Use super-effective move
|
||||
4. No advantage? → Use strongest STAB move
|
||||
5. Low HP? → Switch or use Potion
|
||||
|
||||
### Gen 1 Type Chart (key matchups)
|
||||
- Water beats Fire, Ground, Rock
|
||||
- Fire beats Grass, Bug, Ice
|
||||
- Grass beats Water, Ground, Rock
|
||||
- Electric beats Water, Flying
|
||||
- Ground beats Fire, Electric, Rock, Poison
|
||||
- Psychic beats Fighting, Poison (dominant in Gen 1!)
|
||||
|
||||
### Gen 1 Quirks
|
||||
- Special stat = both offense AND defense for special moves
|
||||
- Psychic type is overpowered (Ghost moves bugged)
|
||||
- Critical hits based on Speed stat
|
||||
- Wrap/Bind prevent opponent from acting
|
||||
- Focus Energy bug: REDUCES crit rate instead of raising it
|
||||
|
||||
## Memory Conventions
|
||||
| Prefix | Purpose | Example |
|
||||
|--------|---------|---------|
|
||||
| PKM:OBJECTIVE | Current goal | Get Parcel from Viridian Mart |
|
||||
| PKM:MAP | Navigation knowledge | Viridian: mart is northeast |
|
||||
| PKM:STRATEGY | Battle/team plans | Need Grass type before Misty |
|
||||
| PKM:PROGRESS | Milestone tracker | Beat rival, heading to Viridian |
|
||||
| PKM:STUCK | Stuck situations | Ledge at y=28 go right to bypass |
|
||||
| PKM:TEAM | Team notes | Squirtle Lv6, Tackle + Tail Whip |
|
||||
|
||||
## Progression Milestones
|
||||
- Choose starter
|
||||
- Deliver Parcel from Viridian Mart, receive Pokedex
|
||||
- Boulder Badge — Brock (Rock) → use Water/Grass
|
||||
- Cascade Badge — Misty (Water) → use Grass/Electric
|
||||
- Thunder Badge — Lt. Surge (Electric) → use Ground
|
||||
- Rainbow Badge — Erika (Grass) → use Fire/Ice/Flying
|
||||
- Soul Badge — Koga (Poison) → use Ground/Psychic
|
||||
- Marsh Badge — Sabrina (Psychic) → hardest gym
|
||||
- Volcano Badge — Blaine (Fire) → use Water/Ground
|
||||
- Earth Badge — Giovanni (Ground) → use Water/Grass/Ice
|
||||
- Elite Four → Champion!
|
||||
|
||||
## Stopping Play
|
||||
1. Save the game with a descriptive name via POST /save
|
||||
2. Update memory with PKM:PROGRESS
|
||||
3. Tell user: "Game saved as [name]! Say 'play pokemon' to resume."
|
||||
4. Kill the server and tunnel background processes
|
||||
|
||||
## Pitfalls
|
||||
- NEVER download or provide ROM files
|
||||
- Do NOT send more than 4-5 actions without checking vision
|
||||
- Always sidestep after exiting buildings before going north
|
||||
- Always add wait_60 x2-3 after door/stair warps
|
||||
- Dialog detection via RAM is unreliable — verify with screenshots
|
||||
- Save BEFORE risky encounters
|
||||
- The tunnel URL changes each time you restart it
|
||||
Reference in New Issue
Block a user