One rebuild moved gpt-oss off ROCm. Two models stayed

Part 6 settled it by depth. gpt-oss-120b at 64k of context: ROCm prefilled 324 tok/s, Vulkan 131. By 127k the gap was 7×. The serving table put gpt-oss on ROCm, under a rule that fits in four words: ROCm for long context.

On the night of 15 October the queue built llama.cpp master for Vulkan, 132 commits past the host build behind parts 5 to 7, and ran both builds back to back. gpt-oss-120b at 64k, old build: 114.7 tok/s. New build: 355.9.

That is 3.1× from a rebuild, and it puts Vulkan above ROCm at a depth where part 6 had ROCm 2.5× ahead. The ROCm row behind the rule was measured on 12 October. The rule was less than four days old.

One caveat up front, because the rest of the post leans on it: this is a single pass per build. Each row is llama-bench's mean of three repetitions, but the A/B itself ran once. The speed screens that ran eight days later re-measure the new build out to 127k on gpt-oss and add two more models; they close the post.

The A/B: two builds, two models, one evening#

The old side is host llama.cpp Vulkan at commit 79e2e74eb, build 11538: the b11538 of parts 5 to 7. The new side is master at b76b13379, build 11670, which the queue's build job produced in the 90 seconds before the test started. Same kernel, same boot, same Mesa RADV driver; the only thing swapped is the binary directory. Both sides ran this:

choom -n 1000 -- llama-bench -m <model> -fa 1 -lm none \
  -p 512 -n 128 -d 0,32768,65536 -r 3 -o jsonl

Two models: gpt-oss-120b MXFP4, the model part 6 moved to ROCm, and Qwen3-Coder-30B-A3B at Q8_0, a MoE with full attention on all 48 layers. pp512 / tg128 in tokens per second:

The shape is the whole story. At an empty context the builds are level: gpt-oss prefill +2.5 %, Coder-30B −0.8 %. At 32k master prefills 54 % and 59 % faster. At 64k it is 3.1× on gpt-oss and 3.6× on Coder-30B. Decode barely notices at any depth: under 1.5 % on gpt-oss, under 0.5 % on Coder-30B.

The wall clock agrees. llama-bench's 64k prefill row fills the context three times and measures a 512-token batch on top of each fill. On Coder-30B that row took 7 min 43 s on the old build and 3 min 31 s on master. That is 2.2×, not 3.6×, because every fill climbs from an empty context through the shallow depths where the two builds are close; the 3.6× is the rate at the top.

Master Vulkan against ROCm at depth#

The ROCm side is part 6's own run: kyuz0's rocm-10.0 container, llama.cpp b11541, measured on 12 October. gpt-oss-120b, pp512 / tg128:

The master row is a mean of three llama-bench repetitions. The ROCm row is a single repetition from part 6's screen.

Different days, so the prefill margin is the number to hold loosely. But ROCm's 64k row is one of the steadiest in the series. Part 10 ran it on four boots on 14 October and got 324.7, 322.4, 338.3 and 324.0; the 338.3 was the boot with the IOMMU switched off entirely. Master's 355.9 clears all of them. At 32k, a ROCm hipBLASLt A/B on the afternoon of 15 October measured 497 and 511. Master's 547 clears both.

Decode was never the argument. Part 6 already had Vulkan ahead on gpt-oss decode at every depth, 34.1 against 27.9 at 64k. ROCm won this model on prefill alone, and prefill is what moved.

The power draw barely did. On the 64k prefill row the APU package averaged 106 W on the old build and 110 W on master, with the GPU busy about 99 % of the time on both. Master got through the same row in 127 s instead of 225 s: about 14 kJ for the row against 24.

What changed in 132 commits#

I didn't bisect. Between 79e2e74eb and b76b13379 there are 132 commits, 15 of them with "vulkan" in the subject line. One of them describes this result before it happened.

llama.cpp #27703, "vulkan : copy strided f16 KV once on AMD so reads spread across memory channels", merged on the morning of 15 October as b11647. Its argument: consecutive KV rows of one attention head sit n_kv_heads × head_dim × 2 bytes apart, and for typical models that stride is an even multiple of the 256-byte channel-interleave block, so the reads land on a subset of the memory channels. The fix copies each layer's f16 KV once into a scratch buffer, so the reads run sequentially. It is gated to AMD.

The PR's table puts Qwen3-Coder-30B (4 KV heads × 128) at a 1024-byte stride: 4 channels engaged of 16. gpt-oss-120b has 8 KV heads × 64 by its model config, the same 1024 bytes. The author's numbers on a Radeon 8060S, Coder-30B at Q6: pp512 at 64k from 72.4 to 186.5, depth 0 unchanged, and "TG is measured unaffected". Mine, Coder-30B at Q8: 53.9 to 192.5, depth 0 within 1 %, decode flat.

Flat at depth 0, growing with depth, decode untouched: that is what a fix to how KV is read should look like, and #27703 is the Vulkan commit in the range that claims one. It's the obvious suspect, not a proven cause. The same stride doesn't promise the same gain, either: the PR's own Qwen3.6-35B also sits at 1024 bytes and gained nothing at 64k, and the PR doesn't say why.

It also reframes part 6. That post put the collapse on Vulkan's flash-attention kernel scaling badly with KV length, and part 7 measured it on MiniMax-M2.7: FLASH_ATTN_EXT from about 19 TFLOPS at 512 tokens of KV to 0.66 at 31k. If #27703 is the cause, a good share of that slope was memory channels sitting idle. MiniMax, with 8 KV heads × 128, has a 2048-byte stride: 2 channels of 16 by the same table. It hasn't run on master yet.

What the rebuild didn't fix#

Mistral-Small-4. Part 7 traced its 3.4 tok/s decode to one op per layer that ran on the CPU, and measured the fix from an open pull request, #30105, at 40.1. Master decodes it at 3.6 tok/s at depth 0 and 3.3 at 32k. #30105 is still open. Mistral stays on the PR build.

Decode. In the A/B, nothing moved beyond 1.5 %. Where Vulkan already decoded faster, it still does.

127k. The A/B stopped at 64k. Part 6's 7× was measured at 127k, and on the old build Vulkan prefill was still falling steeply there. The screen at the end of this post answers it.

How far one pass can carry this#

The old side is the noisy row. gpt-oss on b11538 at 64k is the row part 10 called the one that won't sit still. Across ten logged runs from 10 to 15 October, over boots and IOMMU modes, it landed between 105.1 and 139.7. The 114.7 here sits inside that range. Against the whole spread, master's gain is 2.5× to 3.4×. Master's own three repetitions at 64k came in at 345.9, 361.2 and 360.7.

Nothing else ran. The A/B started a second after the build job finished, with load still decaying from the compile. During the two 64k gpt-oss rows the 1-minute load averaged 1.0 and 0.3, CPU use under 1 %. The one download that had been running in the queue's network lane was stopped as the build started.

Two models. Of the rest of part 6's ROCm row, Qwen3.5-122B and Step-3.7-Flash were the next two to run on master. They are in the update below.

Update, 24 October: the three screens#

The screens ran on 24 October on the same master binary, b76b13379, with the same screen script as part 6's rows: llama-bench, pp512 / tg128, flash attention on, f16 KV, one repetition per depth. The ROCm rows are part 6's, measured on 12 October, also one repetition each.

gpt-oss-120b first, prefill in tok/s:

It holds at 127k: 218.9 against ROCm's 190.1, 15 % ahead, and about 9× the old build at the depth where part 6 measured the 7× gap. The 64k row, 368.0, lands within 4 % of the A/B's 355.9. The old-build row is the 10 October screen; its 139.7 at 64k is the top of the noise range described above, against the A/B's 114.7. Decode at 127k is 25.2 on both Vulkan builds and 18.9 on ROCm.

The other two models didn't follow it past ROCm. Prefill in tok/s:

The old Step screen was stopped after its 64k row on 11 October; at 12 tok/s the 128k row would have run for hours.

On Qwen3.5-122B, master changes nothing out to 64k, 1 to 3 % below the old build, and gains only from 128k on: 1.7× there, 2.5× at 256k. ROCm stays ahead at every depth from 32k, by 15 % at 32k and 44 % at 256k.

On Step-3.7-Flash the rebuild is the bigger story. The old build collapsed by 32k, far more steeply than gpt-oss did; master prefills 7.4× faster at 32k and 15× at 64k. ROCm stays 18 to 26 % ahead of it at every depth from 32k.

Decode stayed with Vulkan on both. Step decodes 23.3 against ROCm's 20.5 at 32k and 10.8 against 10.4 at 256k; Qwen3.5 decodes 3 to 7 % faster on master at every depth.

Why gpt-oss moved past ROCm and these two didn't is not something three screens can say. They don't isolate #27703 or any other commit in the range.

The serving table after the screens:

"ROCm for long context" was a fair summary of two builds on 12 October. It was never a fact about the chip. For gpt-oss it was a fact about where llama.cpp's Vulkan backend stood that week, and, if #27703 is what moved it, part of where it stood was a stride. For the other two it still holds, by a narrower margin from 128k on than part 6 printed.

Filed under: amd, linux, local-llm, benchmarks

Read next#

• Vulkan or ROCm on Strix Halo? Ask at which depth — part 6: the depth sweep and serving table this A/B overturns for gpt-oss.
• 128k context is the floor, and prefill is the wall — part 5: where the Vulkan prefill collapse was first measured, on the same b11538 build.
• Mistral-Small-4 ran one matmul on the CPU, and said nothing — part 7: the decode bug master still has, and the flash-attention slope this post reframes.
• IOMMU off buys a 128 GB iGPU 5 %, and costs the NPU — part 10: the boot-to-boot spread on the exact row this A/B moved.

Sources#

• llama.cpp #27703: "vulkan : copy strided f16 KV once on AMD so reads spread across memory channels" — the stride argument, the channel table and the author's Radeon 8060S numbers; merged 15 October 2026 (b11647).
• llama.cpp compare 79e2e74eb...b76b13379 — the 132 commits between the two builds.
• llama.cpp #30105: "vulkan : honor FP32 source precision in MUL_MAT_ID" — the Mistral-Small-4 decode fix; still open on 16 October.
• openai/gpt-oss-120b config.json — 8 KV heads, head_dim 64.
• Qwen/Qwen3-Coder-30B-A3B-Instruct config.json — 4 KV heads, head_dim 128.