FiiO X1 FTL reversed: the map was in the OOB bytes I never dumped

· 9 min read
Contents

Rockbox's hwstub tools have a Lua function for NAND flash called read_page, and it does exactly what it says. You give it a page and a length, it gives you that many bytes of the page. I gave it 2,048, the size of a page's data area, 65,536 times, and it never once complained. Good tool. The complaint is about the length.

The flash belongs to a FiiO X1, a second-hand music player whose 128 MB NAND chip I copied in full over USB in Part 6, using hwstub, Rockbox's debug stub, and that function. Each page on this chip is 2,112 bytes: 2,048 of data and 64 of spare area, the OOB ("out of band") bytes where flash keeps error-correction codes and bookkeeping. Part 16 carved a SQLite database out of that copy, found pages from different versions of the file side by side in it, and blamed the flash translation layer (FTL), the firmware code that decides which physical block holds which logical one. "The FTL did it" is the flash-storage version of "works on my machine". This part reads the X1's FTL in the firmware to see what it actually does, and finds out where it keeps its map.

The read chain: four functions#

In Ghidra, the disassembler I read the firmware with, I named four functions in this session. From the top:

blockdev_raw_read(handle, sec_off, nsect, buf)     @0x800180c0
  └─ ftl_read_sectors(view, buf, sec_off, nsect)   @0x8002915c
       └─ ftl_read_logical_block(view, buf, L, …)  @0x80026ac0
            └─ nand_read_page_ecc(zone, page, buf)  @0x8002ba4c

The firmware reaches storage through numbered block-device handles. Handle 0x19 is the z: system volume, where the firmware's own IHFS filesystem keeps fonts and resources; handle 0x18 is the y: user-data volume, home of the media database usrlocal_media.db. Both go through the FTL. Everything else goes to the SD card driver. The word "raw" in blockdev_raw_read was the first thing I distrusted, and it earned it: for the two flash volumes it isn't raw at all.

ftl_read_sectors splits a sector offset into a logical block number and an offset inside the block. ftl_read_logical_block does the lookup that matters. It reads a 32-bit entry for the logical block from a map in RAM. An entry of zero means the block was never written, and the read returns a buffer full of 0xCC. Otherwise the top byte selects a NAND die, always 0 on the X1's single chip, and the low 24 bits are the physical block to read the pages from. The map is the whole game.

Where the map comes from: the OOB#

The map is not stored anywhere as a table. ftl_mount_build_map at 0x80028094 rebuilds it every time the firmware mounts a volume: it walks the physical blocks and reads the spare area of each block's first page.

OOB byteSizeField
OOB[1]1 bytebad/erased marker (0xFF = good)
OOB[spare_size - 5]1 byteversion
OOB[spare_size - 4]4 bytes, u32 little-endianlogical block tag

The tag says what the physical block means. 0xFFFFFFFF is an erased, free block. 0x38746262 spells "bbt8" in little-endian ASCII and marks the FTL's own bad-block table. Anything else is a logical block number in the low 24 bits and a die in the top 8, and the mount writes map[logical] = (die << 24) | physical, the exact format the read path consumes.

There is no copy of this map in the data area, no table to fall back on. If the spare bytes are gone, so is the answer to where anything lives.

Newest wins: a 16-bit compare on a one-byte version#

When the FTL rewrites a logical block, it writes a fresh physical block and queues the old one for erasure. If the erase never happens, two physical blocks claim the same logical number, and the mount has to pick one. It picks by the version byte. This is the comparison as Ghidra decompiles it, from the branch for a logical block that already has a map entry: FUN_8002ae24 reads the spare area of the block the map points to, and bVar1 is the candidate's version byte:

iVar4 = FUN_8002ae24(uVar10 >> 0x18,(uVar10 & 0xffffff) * (uint)_DAT_804f9728,
                     iVar3);
if (iVar4 == 0) {
  iVar4 = (uint)*(byte *)(iVar3 + _DAT_804f9730 + -5) - (uint)bVar1;
  if (iVar4 < 0) {
    iVar4 = iVar4 + 0x10000;
  }
  if (iVar4 < 0x8001) {
    uVar10 = uVar17 << 0x18 | uVar11 & 0xffffff;
  }
  else {
    uVar10 = *(uint *)(iVar12 + local_78[2]);
    *(uint *)(iVar12 + local_78[2]) = uVar17 << 0x18 | uVar11 & 0xffffff;
  }
  *(undefined1 *)
   (*(int *)(&DAT_804f9590 + (uVar10 >> 0x18) * 0x14) + (uVar10 & 0xffffff)) =
       0;
  iVar12 = FUN_80027354(uVar10 >> 0x18,uVar10 & 0xffffff,1,1);

The mapped block's version minus the candidate's, wrapped into 16 bits; past 0x8000, the candidate takes the map entry. Either way the loser goes to FUN_80027354, the erase queue. The same arithmetic, a few lines earlier, settles duplicate copies of the FTL's own bbt8 blocks. Both operands are bytes. With values from 0 to 255, the difference is negative only when the candidate is larger, so the larger version always wins. The arithmetic that would let a sequence number roll over is 16-bit, on a version that is 8 bits, so the one case it was written for never comes up: a version that wraps from 255 to 0 loses to the copy it replaced. Whether the write side ever lets that happen, I haven't read.

Part 16's carve, revisited#

The database in Part 16 starts 100 KB into block 859, so its sixty 1 KB pages, read in physical order, run on into block 860, and the pages there don't agree with each other: the schema table's last leaf, page 58, is empty, and page 60 holds schema rows from another version of the file. With the FTL in hand, the likely story is plain: the logical blocks that hold those pages were rewritten at some point, the current copies went to other physical blocks, and a carve in physical order reads stale ones. The map would have said where the current copies are.

That is the likely story, and I can't check it. The map is built from the spare bytes.

The 134,217,728-byte dump#

The dump from Part 6 is this big:

134,217,728 bytes = 1024 blocks × 64 pages × 2048 bytes

I distrust a file size that comes out as a round product. It proves the arithmetic worked, not that everything is in the file, and this one is exactly the right size for the data area and exactly the wrong size for the chip. The dump script asks for JZ.nand.read_page(0, 2, page, 2, 16, 2048) and never calls read_oob. On every one of 65,536 pages, the 64 spare bytes were left on the chip: every logical tag, every version byte, every bad-block marker. Everything the mount needs to rebuild the map sat in bytes 2,048 to 2,111 of each page, and the dump stopped at 2,047.

Part 6 knew. It says, in so many words, that nobody needed the OOB that night. Seven and a half hours of copying, and the part I needed was the 3 percent I didn't ask for.

Two routes to the logical volumes#

Both need the player, and neither has run yet.

Route 1: dump again, with the spare area. dump_nand_oob.sh adds a read_oob call after each page's 2,048 data bytes, for 2,112 bytes a page and 138,412,032 bytes in all. ftl_reassemble.py then reads the first-page tags of every block, applies the newest-wins rule, builds the map and writes out each volume, with unmapped blocks as 0xCC the way the firmware fills them. It has run on a small synthetic image built by hand, and on the existing dump, where its audit mode sees there is no spare area and refuses to produce anything. It has not seen real OOB data.

Route 2: let the player do it. The same sweep of the firmware turned up an orphaned dumper, FUN_800027c4: it reads handle 0x19 with blockdev_raw_read and writes the result to a:\dump.fw on the SD card in 10,240-byte chunks, and a companion function copies y:\usrlocal_media.db to a:\dump2.db through the file layer. Nothing in the image calls either one, at least according to the cross-reference sweep, which is the kind of claim I re-check before I build on it. Handle 0x19 goes through ftl_read_sectors, so dump.fw would be the z: system volume already put in logical order by the player's own FTL, with the map it built from the spare bytes at mount. The dumper and the copier each have a three-instruction entry thunk that loads the progress callback FUN_8000299c and jumps in, 0x80002990 for the dumper and 0x800027b8 for the copier, and nothing calls those either. Patching calls to them into a reachable menu action or button handler would make it usable: flash the patched firmware, insert a card, trigger it.

RouteYieldsNeeds
OOB dump + ftl_reassemble.pyboth volumes, offlinea second full dump over USB
patched self-dumperz: volume image + media database on SDpatched firmware, a run on the player

Route 2 is shorter and gives less: the system volume and the media database file, nothing else from user data. Route 1 gives everything and costs another night of USB.

What I have now#

The read path, the map format, the tags in the spare area and the newest-wins rule are decoded and named in Ghidra, and ftl_reassemble.py implements them. Two details wait on a real OOB dump: which physical blocks belong to which volume, and whether the tag offsets line up with the ECC bytes the hardware writes into the same spare area. The data is still not recovered.

The paths of the old library from Part 16 live in the y: volume, and a clean usrlocal_media.db needs one of the two routes. Until then I have the recovered six-track database and a lesson Part 6 had already written down without taking it: when dumping flash, read the margins.

Filed under: reverse-engineering, mips, embedded, fiio, hardware, nand, ftl

Sources#