Skip to content
NoSrc
Go back

Reading Pokémon Pinball's Internals to Build an RL Environment

Updated:

Most of my reverse engineering writing is about reconstructing a binary from nothing. This project is the opposite case, and I think it is worth writing up precisely because of that difference. Here I did not start from a stripped binary. I started from a community disassembly of Pokémon Pinball, contributed to it, and then built tooling on top of it: a memory-mapped game wrapper for the PyBoy emulator that exposes the game’s internal state as a clean Python interface, and eventually a reinforcement learning environment on top of that wrapper.

The interesting skill here is not “reverse engineer a black box”. The hard reconstruction work on this game was largely done by the pret project, a community effort to produce a byte-accurate, buildable disassembly of the original ROM. What this project is actually about is translation: taking ground-truth knowledge of a game’s internals and turning it into instrumentation and control. That is a large part of what practical tooling work looks like once someone has mapped a target, and it is a different muscle from the initial reconstruction.

Table of contents

Open Table of contents

Scope and sources

This work was done against my own legally acquired copy of the Pokémon Pinball ROM, and no ROM, game assets, or copyrighted material are distributed here or in any of the repositories. The pret disassembly is a research and preservation project that describes the game’s code without shipping the game itself; you bring your own ROM to build it. The game wrapper lives in the PyBoy emulator, and the gym environment and training code are my own separate repositories. For the Game Boy hardware itself, the memory map, banking, and how the CPU and I/O fit together, I leaned heavily on the excellent Pan Docs.

Starting from the disassembly

Before writing any wrapper code, I spent time in the pret disassembly itself and landed several commits there. Contributing to the disassembly was not a side quest. It was how I built the map I would later use. Naming a RAM location, labeling a routine, or documenting a state flag in the disassembly is the same act of understanding that later lets you read that value out of a running game with confidence. By the time I started the wrapper, I was not guessing at addresses. I was reading them off work I had already helped verify.

Those commits are also the part of this project I’m most confident pointing people to, since they’re merged changes in an established disassembly project rather than something you only have my word for.

Building the wrapper

PyBoy is a Game Boy emulator with a plugin system for “game wrappers”, classes that expose a specific game’s state to an AI or a script. The Pokémon Pinball wrapper is the bridge between the disassembly’s ground truth and a usable programmatic interface. It does three distinct kinds of work: reading state, hooking events at code addresses, and patching code. Each maps onto a technique that shows up in binary analysis tooling well beyond this game.

Reading state out of memory

The simplest layer is direct memory reads at fixed addresses. The wrapper keeps a table of RAM locations pulled from the disassembly and reads them every tick: ball position and velocity, current stage, current map, ball type, multiplier, ball-saver time remaining, and so on.

ADDR_BALL_X = 0xD4B3
ADDR_BALL_Y = 0xD4B5
ADDR_BALL_X_VELOCITY = 0xD4BB
ADDR_BALL_Y_VELOCITY = 0xD4BD

Not every value is a plain integer. The score, for example, is stored in binary-coded decimal, so reading it means pulling the raw bytes, decoding the BCD, and scaling:

self.score = bcd_to_dec(
    int.from_bytes(self.pyboy.memory[ADDR_SCORE : ADDR_SCORE + SCORE_BYTE_WIDTH], "little"),
    byte_width=SCORE_BYTE_WIDTH,
) * 10

The Pokédex is another example. Rather than a single counter, the game keeps a per-species table, so “how many unique Pokémon has the player caught” means iterating 151 entries and checking each flag’s value. Knowing that a caught flag reads as a specific value, distinct from merely seen, is exactly the kind of detail that comes from the disassembly and would be painful to infer by poking at memory blind.

Hooking events at code addresses

The more interesting layer is event tracking. Polling memory every frame tells you the current state, but it is a bad way to count discrete events. If you want to know how many times a Pokémon was caught, or an evolution succeeded, or a bonus stage was completed, watching a counter in RAM is fragile and easy to miscount.

Instead, the wrapper registers hooks at specific code addresses taken from the disassembly. PyBoy lets you attach a Python callback to a bank and offset, so the callback fires the moment execution reaches that instruction. I hooked the routines that the game itself runs when these events happen, and incremented my own counters from the callback:

def pokemon_caught(context):
    context.pokemon_caught_in_session += 1

self.pyboy.hook_register(
    BANK_OFFSET_ADD_CAUGHT_POKEMON_TO_PARTY[0],
    BANK_OFFSET_ADD_CAUGHT_POKEMON_TO_PARTY[1],
    pokemon_caught,
    self,
)

There are dozens of these: catches, sightings, evolution successes and failures, bonus-stage visits and completions for each of the five bonus stages, map-change attempts and successes, ball upgrades, extra balls, and the Pikachu saver. This is instrumentation at code addresses rather than at data. It is conceptually the same thing as setting a breakpoint with a callback in a debugger, done here declaratively and at scale, and it is only possible because the disassembly told me exactly which routine corresponds to each event.

Patching the ROM to force state

The third layer is the one that moves from observation to control. For an RL environment you do not just want to watch the game, you want to put it into specific situations on demand, quickly and repeatably.

To force the starting stage, the wrapper patches the game’s code directly. The original routine looks up the starting stage from a table based on a selected field index. I NOP out that lookup and write in a single instruction that loads the stage I want:

# no-op out the original stage-lookup instructions, then insert:
# ld a, stage.value
self.pyboy.memory[bank, addr]     = 0b00111110   # LD A, n8 opcode
self.pyboy.memory[bank, addr + 1] = stage.value

A similar patch repoints the pause button. enable_evolve_hack overwrites the bank and 16-bit address of the method the pause button calls, so pressing pause instead jumps to the evolution-start routine. That is rewiring a UI action to a different code path by editing the call target in memory, which is a small, concrete instance of the same idea behind larger binary patches.

Rounding this out, start_catch_mode writes the handful of RAM values that put the game into catch mode with a chosen Pokémon and a set timer, and start_game drives the intro with frame-precise input, synchronizing on a specific background tile value before it sends button presses so the timing stays deterministic across runs.

Taken together, the wrapper reads state, counts events at code addresses, and patches code to force state. All three are grounded in the disassembly, and all three are the useful half of reverse engineering: not just understanding a target, but building something that drives it.

The reinforcement learning environment

The wrapper existed to serve a goal: training an agent to play Pokémon Pinball. On top of it I built a Gymnasium environment and a training setup using Stable Baselines 3, vectorized across many parallel emulator instances, with WandB logging, checkpointing, hyperparameter sweeps, and several reward-shaping modes (a basic score-difference reward, a catch-focused reward, and a comprehensive multi-objective reward that also credits evolutions, stage completions, ball upgrades, and survival). The event hooks in the wrapper are what make the richer reward modes possible, since each shaped reward is reading a counter that a code hook maintains.

Where it stalled

The environment and the training infrastructure worked. The agent could learn narrow behavior, catching a single Pokémon under a catch-focused reward, which was enough to confirm the whole pipeline was sound end to end: emulator, memory mapping, event hooks, gym interface, reward shaping, and training loop all connected and learning.

Where it stalled was in generalizing beyond that to competent full play, and the reason is that I hit the edge of my own RL knowledge. The environment was solid, but I did not yet know enough about reinforcement learning to make the right calls on the parts that matter for a game like this: how to represent the observation, how to shape rewards for a long, multi-objective task, and what model was appropriate for a fast, reactive game with sparse high-level goals. I could tell the agent was not learning the longer-horizon behavior; I could not yet tell which of those choices was holding it back. The environment is finished and reusable, and the agent that fully exploits it is waiting on me getting deeper into RL.

What I took from it

The lasting value of this project was not the RL result, which is incomplete. It was learning how to turn a disassembly into working instrumentation and control. Reading structured state out of memory, decoding non-obvious encodings, counting events by hooking the exact routines that produce them, and patching code to force a target into a chosen state are all directly transferable to analysis tooling well beyond a Game Boy game. And contributing to the disassembly first, rather than treating it as a black box to consume, is what made the rest of it trustworthy. When the addresses came from work I had helped verify, I could build on them without second-guessing every read.


Share this post on:

Next Post
Reversing TeamSpeak 3's Recording Notifications