Vulkan or ROCm on Strix Halo? Ask at which depth

Four models, two backends, eight pairs of numbers. On 9 October, the day the A9 Mega first booted on real hardware, a quick llama-bench pass put Vulkan ahead on all eight. gpt-oss-120b: 1077 tok/s prefill against ROCm's 675, 55.5 tok/s generation against 51.0. Clean sweep (part 2 has the full table). The note in the project file said so in one line, and the one line read like a decision.

Here is the same model, same box, three days later, with 127k tokens already in the context:

Prefill is 7.0× faster on the backend that lost every row. The first table wasn't wrong. It answered a question nobody running an agent asks.

Why the depth-0 verdict was the wrong question#

The 9 October bench measured pp512 and tg128 at depth 0: 512 prompt tokens into an empty context. An agent never works at depth 0. Part 5 set 128k as the floor for agent work and found prefill to be the wall, all on stock llama.cpp Vulkan. The open question from there: is the wall the box, or the backend?

The ROCm on 9 October was also a different one: Fedora's packaged llama.cpp b9840. Every number below uses kyuz0's rocm-10.0 container (llama.cpp b11541) for ROCm, and either the same image's Vulkan build or host Vulkan b11538, which ran within 3 % of each other. Both reference models are llama-bench pp512/tg128 at depth 0, 32k, 64k and the edge of 128k. Each cell is one run.

Prefill and decode by depth: two models, two backends#

pp512 / tg128 in tokens per second:

gpt-oss stops at 127k (130,048 tokens), Flash-Next runs to 131,072.

At depth 0 Vulkan prefills both models about 25 % faster. By 32k ROCm prefills faster on both. Somewhere in between, the lines cross. Where exactly, I don't know: nothing was measured between 0 and 32k.

Decode never crosses on these two. Vulkan generates faster at every depth, by 33 % on gpt-oss at 127k and 50 % on Flash-Next at 128k.

The ROCm prefill lead at the deep end is very different per model: 7.0× on gpt-oss-120b, 1.5× on Flash-Next. That ratio is the clue.

A note for readers of part 5: its gpt-oss 127k figure was 24.6 tok/s on host Vulkan b11538; the 27 above is kyuz0's Vulkan build. Two builds, two numbers, not one measurement quoted twice.

Why full attention decides it#

gpt-oss-120b does full attention in every other layer; the rest use a short sliding window. Flash-Next does it in a minority and carries a fixed-size linear-attention state in the rest (part 5 read the layout from its GGUF). Full-attention layers get more expensive with every token already in the context, so the backend whose attention kernel scales worse with KV length loses more on the model that leans on them.

On Vulkan that kernel scales badly, at least on the one model profiled so far. The per-op logger (GGML_VK_PERF_LOGGER=1) on another model, MiniMax-M2.7, showed FLASH_ATTN_EXT running at ~19 TFLOPS with 512 tokens of KV and 0.66 TFLOPS at 31k. The next part takes that apart.

The models Vulkan couldn't finish#

The two reference models only lean. Others fall over. On Vulkan, the speed screens saw prefill collapse well before 128k on four models:

• Step-3.7-Flash: 334 tok/s at d0, 31 at 32k, 12 at 64k. The 128k row was killed with an estimated 3–4 hours still to go.
• Laguna-XS: 1563 at d0, 82 at 32k, 25.5 at 64k: the fastest start of the four. Its 128k row was killed too.
• Laguna-S: 533 at d0, 9.7 at 128k, 4.4 at 256k.
• MiniMax-M2.7: 304 at d0, 11.4 at 32k in a diagnostic run.

The pattern across the screens: hybrid linear-attention models held up, full-attention models collapsed. Then the same screens went through ROCm. pp512 / tg128 at ~128k:

Step-3.7 on ROCm runs to 256k at 101 tok/s prefill, with decode about 10 % below Vulkan's at the depths both reached. A model that wouldn't finish a 128k row became a 256k model by changing a container image.

Laguna-S is the twist. ROCm prefill beats Vulkan by 6.8× at 32k and 34× at 256k, and decode goes the other way: 6.7 tok/s at 128k against Vulkan's 15.5, less than half. It's usable on ROCm to about 64k. MiniMax is worse: ROCm fixes its prefill and its decode collapses, so it's out on both backends.

So the pattern after the re-screens: ROCm always fixes long-context prefill. What it does to decode depends on the model: it holds up for Step-3.7, Qwen3.5, Ornith and Coder-Next, it halves for Laguna-S, it collapses for MiniMax. "Vulkan decodes faster at every depth" was true of the two reference models and nothing more.

Ornith and Coder-Next split differently again. Vulkan prefills both about 30 % faster at depth 0. Ornith is even at 32k and ROCm leads from 64k; Coder-Next is even at 64k and ROCm leads from 128k. At 128k that lead is small, 14 % and 10 %, and decode is level by then. At 256k ROCm prefills Ornith 2.6× faster, 227 against 88.

Power: the faster backend is also the cooler one#

At depth 0 Vulkan also drew less power: part 2 re-cut the 9 October telemetry to busy samples and found Vulkan drawing 17–30 W less on every model. At long context that reverses. The telemetry logger samples power and temperatures every 2 seconds; a script cuts that log into one window per llama-bench row. On Ornith during long-context prefill:

At equal or better speed, ROCm draws less and runs about 10 °C cooler. Energy per context fill follows: 7.6 Wh on Vulkan against 5.9 Wh on ROCm to fill 128k, and 31.6 against 18.2 Wh at 256k.

Decode at 128k was measured once per backend: one fill, then tg2048, one 66-second window each. Vulkan made 31.2 tok/s at 83.6 W, 2.68 J per token. ROCm made 30.8 tok/s at 72.0 W, 2.34 J per token. One run each, so treat it as a direction rather than a constant. On this model, at this depth, ROCm costs less energy in both phases.

Part 2's figures and these don't contradict each other. They measure different depths and different models, and part 2's ROCm was the old packaged build (llama.cpp b9840).

The #25992 patch changed nothing measurable here#

llama.cpp #25992 reports wrong, cross-request output from llama-server -np 4 --kv-unified on gfx1151 under HIP, bisected to c7d87229, and was closed on 9 October as stale, not fixed. kyuz0's image carries a host-buffer patch for it, yet llama.cpp's official unpatched b11541 ROCm 10.0 binaries in the same image also passed the greedy output check 10/10 on gpt-oss-120b and Flash-Next, byte-identical to both Vulkan builds across 11 prompts (4 of them concurrent), at speeds within ±1.5 %. What ROCm does cost is concurrency: four simultaneous ~6k-token needle requests on gpt-oss took 59 s each, against 42 s on Vulkan.

The ubatch trap: a flag that helps where nobody works#

The highest public "pp" figures for gpt-oss-120b on this chip, in the strix-halo-llm-speeds list, are 1.40–1.46k, well above the 1177 above. Every one of them is pp2048 with -ub 2048 or a service build's larger ubatch. Part of the gap is the prompt size: pp512 cannot show a bigger ubatch, because a 512-token prompt never fills a 2048-token batch. So gpt-oss-120b went through pp2048 at four depths on Vulkan, default ubatch against -ub 2048:

-ub 2048 is a 31 % win at zero context, which is where those public figures were measured. At 64k it more than halves prefill. It helps exactly where no agent works and hurts everywhere one does. On Flash-Next it is 5–21 % slower at every depth. Decode doesn't notice on either model.

Which is also why a public pp number can't be set against ours: different prompt size, different ubatch, no stated depth. A prefill number without its depth and ubatch says nothing useful about agent work.

The serving table (as of 12 October)#

One engine per model, because a session can't move between engines. The KV cache lives in the server process: switching means a reload and a full re-prefill. With prompt caching, an agent turn only prefills the new tool output, but it does so at whatever depth the conversation has reached. So the backend has to suit the depth the model works at, not the depth it benchmarks at.

The Vulkan row looks like it breaks the rule, since ROCm prefills Ornith faster from 64k and costs less energy at 128k. The reason is where an agent spends its time. In the pilot agent run on Ornith, decode took 93 % of the wall time with the context reaching 99k. Vulkan decodes Ornith and Coder-Next 13–17 % faster at depth 0 and stays ahead or level all the way up, while ROCm's prefill lead at 128k is 10–14 %. It only becomes 2.4–2.6× at 256k, so those two move to ROCm for sessions that long. Qwen3.5-122B goes the other way: ROCm prefills it faster from 32k on, 2.1× at 128k, with decode about equal. The models whose Vulkan prefill collapses outright, Step-3.7, gpt-oss-120b and Laguna-S, go to ROCm too. DeepSeek-V4-Flash wasn't screened on ROCm and is slow on Vulkan too (48 / 7.3 at 128k).

Flash-Next sits outside both llama.cpp backends because neither comes close to a purpose-built engine on it. Mistral-Small-4 needed a pull request, not a backend.

The 9 October table is still correct. Every row in it was measured at a depth an agent leaves after its first turn.

Filed under: amd, linux, local-llm, benchmarks

Read next#

• 128k context is the floor, and prefill is the wall — part 5: the Vulkan-only depth sweep this part reruns on ROCm, and the ring timeout on the way there.
• 120 GiB for the GPU, set in the kernel, not the BIOS — part 2: the memory setup that makes 256k rows possible, and the depth-0 power numbers this part turns around.

Sources#

• huppiflupp/strix-halo-llm-speeds — public Strix Halo llama.cpp speed list (gpt-oss-120b pp rows)
• llama.cpp #25992: "Eval bug: server -np 4 --kv-unified returns other requests' responses verbatim on integrated HIP GPU (gfx1151) — bisected to c7d87229" — the wrong-output report on gfx1151; closed 9 October as not planned after going stale.