Game Boy Emulator: Writing the Z80 Disassembler

What is a Game Boy Cartridge ROM?

When you slot a cartridge into the back of the Game Boy it – somehow – boots up, and starts the game. Game Boy cartridges differ quite a bit depending on the game it was made for, the era it was created in, and the game developer who made it.

They all have some form of storage for the game’s code. Some of the larger games have more than one chip, and therefore need a memory bank controller in the cartridge, as the Game Boy only had a 16-bit address bus. The games could then switch between the chips, as needed. Later generations featured everything from camera attachments to accelerometers. Each of these features would in turn would simply write to dedicated areas of memory which the Game Boy could in turn read and the game’s code make use of. Simple, but effective.

Some also featured some sort of main memory, to store things like high scores and save games, and a small battery to keep a charge to said chip, to prevent data loss.

Laid out in full, the size of the cartridge’s effective storage ranged from 32 KiB to several MiB.

So, that’s a cartridge. A ROM – ROM being Read-Only Memory – is a catch-all term used in emulator circles to describe a clone of a cartridge, floppy disk, CD-ROM – anything, really – laid out in a format that emulator writers have agreed on over time. For simpler things it’s a 1:1 mapping. One byte in a chip somewhere; one byte in a file on your PC. Game Boy cartridges mostly work that way, which is good news for us.

To start with, and for quite a while actually, we won’t worry too much about complex memory bank switching and instead focus on games that don’t have any of those. They are easily identifiable in one of two ways: the size is 32 KiB exactly and the other we’ll talk about later when we look at how to read out cartridge ROM metadata.

Cartridge ROMs are byte-accurate ROM images of the cartridge’s chips

So a cartridge ROM, then, is just a series of bytes lifted from one or more chips in a physical cartridge. And that’s exactly the representation we want, as it’s easy to reason about.

Reading a Cartridge ROM’s Metadata

$ xxd -l 8 test.zip
00000000: 504b 0304 1400 0000           PK......
^^^^^^^^
Offset    ^^^^^^^^^^^^^^^^^^^
          Hexadecimal representation    ^^^^^^^^
                                        Textual/Byte representation

$ xxd -s $((0x100)) -l $((0x0150 - 0x100)) snake.gb
00000100: 00c3 5001 ceed 6666 cc0d 000b 0373 0083  ..P...ff.....s..
00000110: 000c 000d 0008 111f 8889 000e dccc 6ee6  ..............n.
00000120: dddd d999 bbbb 6763 6e0e eccc dddc 999f  ......gcn.......
00000130: bbb9 333e 5976 6172 2773 2047 4220 536e  ..3>Yvar's GB Sn
00000140: 616b 6580 0000 0000 0000 0100 2d42 dec7  ake.........-B..

>>> data = bytes.fromhex('AB CD')
>>> data
b'\xab\xcd'

>>> little = int.from_bytes(data, 'little')
>>> big = int.from_bytes(data, 'big')
>>> (little, big)
(52651, 43981)
>>> (hex(little), hex(big))
('0xcdab', '0xabcd')

>>> int.to_bytes(0xCD, length=1, byteorder='little')
b'\xcd'

>>> int.to_bytes(0xCD, length=1, byteorder='big')
b'\xcd'

>>> int.to_bytes(0xCD, length=2, byteorder='big')
b'\x00\xcd'

>>> int.to_bytes(0xCD, length=2, byteorder='little')
b'\xcd\x00'

>>> b'Inspired Python'
b'Inspired Python'

>>> b'\xf0\x9f\x90\x8d'
b'\xf0\x9f\x90\x8d'

>>> b'\xf0\x9f\x90\x8d'.decode('utf-8')
'🐍'

>>> struct.unpack_from('I', b'\xf0\x9f\x90\x8d')
(2375065584,)

>>> struct.unpack_from('>BB5sH', b'\x01\x02HELLO\x03\x04')
(1, 2, b'HELLO', 772)

FIELDS = [
  (None, "="), # "Native" endian.
  (None, 'xxxx'), # 0x100-0x103 (entrypoint)
  (None, '48x'), # 0x104-0x133 (nintendo logo)
  ("title", '15s'), # 0x134-0x142 (cartridge title) (0x143 is shared with the cgb flag)
  ("cgb", 'B'), # 0x143 (cgb flag)
  ("new_licensee_code", 'H'), # 0x144-0x145 (new licensee code)
  ("sgb", 'B'), # 0x146 (sgb `flag)
  ("cartridge_type", 'B'), # 0x147 (cartridge type)
  ("rom_size", 'B'), # 0x148 (ROM size)
  ("ram_size", 'B'), # 0x149 (RAM size)
  ("destination_code", 'B'), # 0x14A (destination code)
  ("old_licensee_code", 'B'), # 0x14B (old licensee code)
  ("mask_rom_version", 'B'), # 0x14C (mask rom version)
  ("header_checksum", 'B'), # 0x14D (header checksum)
  ("global_checksum", 'H'), # 0x14E-0x14F (global checksum)
]

import sys
import hypothesis.strategies as st
from hypothesis import given

HEADER_START = 0x100
HEADER_END = 0x14F
# Header size as measured from the last element to the first + 1
HEADER_SIZE = (HEADER_END - HEADER_START) + 1

@given(data=st.binary(min_size=HEADER_SIZE + HEADER_START,
                      max_size=HEADER_SIZE + HEADER_START))
def test_read_cartridge_metadata_smoketest(data):
    def read(offset, count=1):
        return data[offset: offset + count + 1]

    metadata = read_cartridge_metadata(data)
    assert metadata.title == read(0x134, 14)
    checksum = read(0x14E, 2)
    # The checksum is in _big endian_ -- so we need to tell Python to
    # read it back in properly!
    assert metadata.global_checksum == int.from_bytes(checksum, sys.byteorder)

CARTRIDGE_HEADER = "".join(format_type for _, format_type in FIELDS)

CartridgeMetadata = namedtuple(
    "CartridgeMetadata",
    [field_name for field_name, _ in FIELDS if field_name is not None],
)

def read_cartridge_metadata(buffer, offset: int = 0x100):
    """
    Unpacks the cartridge metadata from `buffer` at `offset` and
    returns a `CartridgeMetadata` object.
    """
    data = struct.unpack_from(CARTRIDGE_HEADER, buffer, offset=offset)
    return CartridgeMetadata._make(data)

>>> p = Path('snake.gb')
>>> read_cartridge_metadata(p.read_bytes())
CartridgeMetadata(
    title=b"Yvar's GB Snake",
    cgb=128,
    new_licensee_code=0,
    sgb=0,
    cartridge_type=0,
    rom_size=0,
    ram_size=0,
    destination_code=1,
    old_licensee_code=0,
    mask_rom_version=45,
    header_checksum=66,
    global_checksum=51166,
)

The Z80 Instruction Decoder and Disassembler

With a decent understanding of how the representation of a sequence of bytes depends on the context, let us now turn our attention to the disassembler.

One salient point before I proceed. The CPU emulator does not actually need a disassembler at all; but you will. The CPU only cares about decoding instructions from the byte stream, and it does not care about displaying them for humans to read on a screen. But, good debugging and instrumentation facilities is paramount to a successful emulator project. And the best place to start is with the disassembler (and decoder) as you’ll want to understand the instructions the CPU is about to emulate, and why.

In Game Boy Emulator Introduction we parsed the opcodes file and there was an optional task to pretty print the opcodes also. We’ll need those parsed dictionaries of opcodes for this next step. I opted for dataclasses; they look like a little bit like this:

Instruction(
    opcode=0x0,
    immediate=True,
    operands=[],
    cycles=[4],
    bytes=1,
    mnemonic="NOP",
    comment="",
)

We need two dictionaries of instructions. One for the prefix instructions, and another for the regular instructions. There are two because it is not possible to represent all the different instructions with just a single byte. The prefixed instructions are thus, well, prefixed with 0xCB to indicate to the CPU that the byte following that one is the prefixed instruction.

So CB 26 has the mnemonic of SLA (HL). You can see a list of the CPU Instruction Sets on pandoc and, of course, in your parsed dictionaries. I also recommend you keep the Game Boy CPU Manual on hand as it has more detailed explanations of the instructions.

So now that we have a list of opcodes it’s a case of mapping a stream of bytes to their opcode equivalents. There are, however, a couple of snags that make it infeasible to use the struct approach we used above:

The byte lengths of the instructions are not fixed

Each instruction size varies from one to two bytes. All prefixed instructions are by their nature two bytes long.

Opcodes are variadic

Some opcodes have operands, and others do not. 0x0 (NOP) has no operands, for instance. But CB 26 has one. Some also reference a special memory location, further lengthening the amount of bytes to read.

The offset you read from is unknown

Maybe you’re reading from 0x0, or perhaps another offset.

The stream is potentially infinite

This is not the case when we disassemble a cartridge ROM (it has a fixed size), but it could happen once our emulator starts executing instructions, and we’d have no easy way of knowing, either .

So it’s much easier to take what we’ve learned and go about reading the data in one byte at a time, using the parsed opcodes as a guide for what we need to read.

So the goal is roughly:

1. Given an address (think index in an array of bytes) and our parsed opcodes, read one byte and increment address by 1

2. If the byte equals 0xCB, use the prefix instructions opcode lookup table, and increment the address by 1.

3. Get the instruction from the opcode lookup table

4. Loop over the instruction’s operands and:

   1. If the operand has bytes > 0, read that many bytes and increment the address by the same and store it as the value of the operand.

   2. If it bytes is None field, then the operand is not data value and a fixed operand, so store that instead in name.

5. At this point you’ll have an instruction and associated operands, if any. Return the address and the instruction.

6. Ensure that any value you read is converted to your system’s byteorder. Use sys.byteorder

The point of the exercise is to translate strings of bytes into the equivalent high-level instruction that both the CPU and us, the developers, can comprehend. Because the byte length varies depending on the opcode, we cannot simply chunk the stream into packets of instructions to parse.

Let’s start with a test for the NOP instruction:

@pytest.fixture
def make_decoder(request):
    def make(data: bytes, address: int = 0):
        opcode_file = Path(request.config.rootdir) / "etc/opcodes.json"
        return Decoder.create(opcode_file=opcode_file, data=data, address=address)
    return make

def test_decoder_nop_instruction(make_decoder):
    decoder = make_decoder(data=bytes.fromhex("00"))
    new_address, instruction = decoder.decode(0x0)
    assert new_address == 0x1
    assert instruction == Instruction(
        opcode=0x0,
        immediate=True,
        operands=[],
        cycles=[4],
        bytes=1,
        mnemonic="NOP",
        comment="",
    )

Here I’m using a pytest factory fixture to generate the Decoder object that’ll do all the heavy lifting. The test, then, generates a decoder with a bytestring \x00. Next, I ask the decoder to decode address 0x0 (which is of course the first and only byte in our bytestring) and assert that the instruction matches the one I got from my parsed opcodes file, and that the address returned by the decoder reflects the new position: 0x1.

Now for the decoder. Let’s start with the constructor and the skeleton of the class.

@dataclass
class Decoder:

    data: bytes
    address: int
    prefixed_instructions: dict
    instructions: dict

    @classmethod
    def create(cls, opcode_file: Path, data: bytes, address: int = 0):
        # Loads the opcodes from the opcode file
        prefixed, regular = load_opcodes(opcode_file)
        return cls(
            prefixed_instructions=prefixed,
            instructions=regular,
            data=data,
            address=address,
        )

The Decoder requires data to decode. Later we’ll replace the generic concept of “data” with the emulator’s memory banks. For now, a generic bytestring is a decent stand-in.

There’s also an address that we encapsulate so we can later query the last position it had. Not needed just yet, but useful to have around. Finally there are two dictionaries containing the parsed opcodes.

The create classmethod is a factory that reads in the opcode file and calls load_opcodes (not shown) that parses the JSON opcodes file. It also takes two other parameters to seed the Decoder with data and a starting address.

And now for the meat of the class. The decoder method itself.

import sys

@dataclass
class Decoder:

    # ... Decoder continued ...

    def read(self, address: int, count: int = 1):
        """
        Reads `count` bytes starting from `address`.
        """
        if 0 <= address + count <= len(self.data):
            v = self.data[address : address + count]
            return int.from_bytes(v, sys.byteorder)
        else:
            raise IndexError(f'{address=}+{count=} is out of range')

    def decode(self, address: int):
        """
        Decodes the instruction at `address`.
        """
        opcode = None
        decoded_instruction = None
        opcode = self.read(address)
        address += 1
        # 0xCB is a special prefix instruction. Read from
        # prefixed_instructions instead and increment address.
        if opcode == 0xCB:
            opcode = self.read(address)
            address += 1
            instruction = self.prefixed_instructions[opcode]
        else:
            instruction = self.instructions[opcode]
        new_operands = []
        for operand in instruction.operands:
            if operand.bytes is not None:
                value = self.read(address, operand.bytes)
                address += operand.bytes
                new_operands.append(operand.copy(value))
            else:
                # No bytes; that means it's not a memory address
                new_operands.append(operand)
        decoded_instruction = instruction.copy(operands=new_operands)
        return address, decoded_instruction

I think the read method speaks for itself. If we attempt to read beyond the bounds of the bytestring, raise an IndexError, otherwise return count number of bytes from address.

The decode method follows the algorithm I laid out above. We read one byte at a time, remembering to increment address when we do, and if there are operands associated with the matching instruction, we read an additional operand.bytes (again incrementing address) and store it in operand.value. If operand.bytes is None we instead just store the operand as-is.

Both dictionaries of instructions contain instances of the Instruction dataclasses that I defined in Instruction and Operand Dataclasses. The only thing to note is the copy methods that return an identical copy of the Instruction or Operand instances, but with the value (for Operand) or operands (for Instruction) swapped out.

I also added a couple of pretty printers to both the Operand and Instruction classes:

@dataclass
class Operand:

    # ... etc ...

    def print(self):
        if self.adjust is None:
            adjust = ""
        else:
            adjust = self.adjust
        if self.value is not None:
            if self.bytes is not None:
                val = hex(self.value)
            else:
                val = self.value
            v = val
        else:
            v = self.name
        v = v + adjust
        if self.immediate:
            return v
        return f'({v})'

@dataclass
class Instruction:

    # ... etc ...

    def print(self):
        ops = ', '.join(op.print() for op in self.operands)
        s = f"{self.mnemonic:<8} {ops}"
        if self.comment:
            s = s + f" ; {self.comment:<10}"
        return s

The printer code is self-explanatory. The goal is to format an instruction (and any operands) to look like hand-written assembly code. There’s a style to it, and you can see it’s more or less the same in all the Game Boy and Z80 Assembly language manuals.

With a pretty printer and working decoder we’re almost done:

>>> dec = Decoder.create(opcode_file=opcode_file, data=Path('bin/snake.gb').read_bytes(), address=0)
>>> _, instruction = dec.decode(0x201)
>>> instruction
Instruction(opcode=224, immediate=False, operands=[
    Operand(immediate=False, name='a8', bytes=1, value=139, adjust=None),
    Operand(immediate=True, name='A', bytes=None, value=None, adjust=None)
    ], cycles=[12], bytes=2, mnemonic='LDH', comment='')
>>> instruction.print()
'LDH      (0x8b), A'

Generalizing this to a function capable of disassembling an arbitrary length of bytes is now easy:

def disassemble(decoder: Decoder, address: int, count: int):
    for _ in range(count):
        try:
            new_address, instruction = decoder.decode(address)
            pp = instruction.print()
            print(f'{address:>04X} {pp}')
            address = new_address
        except IndexError as e:
            print('ERROR - {e!s}')
            break

Which, when run with the offset of 0x150 (which happens to be entrypoint for snake.gb):

>>> disassemble(dec, 0x150, 16)
0150 NOP
0151 DI
0152 LD       SP, 0xfffe
0155 LD       B, 0x80
0157 LD       C, 0x0
0159 LDH      A, (0x44)
015B CP       0x90
015D JR       NZ, 0xfa
015F DEC      C
0160 LD       A, C
0161 LDH      (0x42), A
0163 DEC      B
0164 JR       NZ, 0xf3
0166 XOR      A
0167 LDH      (0x40), A
0169 LD       A, 0x0

And that’s it. A working disassembler. Advanced ones like Ghidra and IDA Pro come with a battery of additional features like figuring out call graphs, where functions begin and end, and so much more. But this is enough for us to begin to understand what our future emulator CPU is executing.

We’re now ready to tackle the next part of the equation: writing the framework that will make up our CPU; the CPU registers (and what they are); and a crash course on Z80 assembly language to get us started.

Summary

Representation is a matter of interpretation

Big and little endian is one thing to be aware of. Another is that a consecutive series of bits and bytes can mean different things. And we’ve only scratched the surface. Later on the concept of signed and unsigned numbers and how to represent them rears its head.

Disassemblers are key to CPU emulation

If you’ve never done systems programming before, then the thought of writing a disassembler may seem difficult or challenging: and they definitely can be, if you have to reverse engineer the opcodes and operands! We’ve been given a big leg up because someone has carefully transcribed the opcodes and operands into parseable JSON. Without it, we’d have to do that tedious manual work first.

But even though pretty-printed disassembly is useful to us, the developers, the CPU still needs to go through what is known as a “Fetch-Decode-Execute” cycle. We’ve simplified the fetching, for now, as it does not read from memory yet. But the decoder is complete and it’ll serve as a keystone in the emulator going forward.