128k context is the floor, and prefill is the wall

The arithmetic that sells a 128 GB box goes like this. Weights are 63 GB. The KV cache for a long context is a few more. Add it up, it's under 128, so it fits. And if it fits, it runs.

The first half holds up. Qwen3.8-Flash-Next at UD-IQ4_XS is an 87.24 GiB file. With llama.cpp's lazy mode (-lzm on) it ran a 256k-token context on the A9 Mega, and GTT, the GPU's share of system memory, never went past 69.75 GiB during that job. Most of the file's per-layer table stays out of GPU memory and gets read on demand. There was room to spare.

The second half is where the box spends its afternoons.

Why 128k is the floor for agent work#

A coding agent's prompt is a system prompt, a tool schema, a dozen files and the conversation so far. It grows every turn. The study plan for this box sets 128k as the minimum for agent work, runs every screen at 131072, and treats 256k as the target. gpt-oss-120b's own GGUF says context_length = 131072, so for that model the floor and the ceiling are the same number.

The question is what 128k costs.

Prefill vs depth: five clean curves#

Every number below is llama-bench on the A9, llama.cpp build 11538 (79e2e74eb), Vulkan, flash attention on, f16 K cache. At each depth -d, llama-bench first fills that many tokens of context, then times 512 more prompt tokens (pp512) and 128 generated ones (tg128). So each figure is a marginal rate: how fast the next tokens go when that much is already there. All runs were on an otherwise idle box. Part 4 is about the run that wasn't.

pp512 / tg128 in tokens per second:

¹ 131072 for every model except the two gpt-oss variants, which stop at 130048: that depth plus the 512 timed tokens lands just inside their 131072 context. So gpt-oss runs to 127k depth, the edge of its 128k context. Every cell is a single run.

gpt-oss-120b's prefill falls 48.2× from an empty context to 127k. It keeps 2.1 % of its starting speed. Flash-Next falls 4.90× to 128k and 7.29× to 256k.

Flash-Next's prefill is not flat, though. It loses 86 % of its speed by 256k. What holds up is generation: tg keeps 80 % of its d0 speed at 128k and 67 % at 256k. gpt-oss keeps 46 % at 127k.

The time to fill a context: 29 minutes#

24.6 tok/s sounds like a slow number. The row timestamps make it a slow afternoon. Each prefill row starts with filling the context, so the gap between one row's stamp and the next is mostly fill time:

The 512 timed tokens account for 21 seconds of gpt-oss's 29 minutes. The rest is one cold 127k prompt being read. At 64k the two models are within ten seconds of each other. Past it, gpt-oss's cost per token keeps climbing steeply and Flash-Next's climbs far more slowly.

The box isn't idling through any of that. Over Flash-Next's whole job, telemetry logged a 91.0 W mean package power with the GPU 98.9 % busy. It's working flat out to read a prompt.

For an agent this is the number that matters. A prefix the server still has cached isn't processed again; an edit near the top of a 120k conversation throws most of that cache away.

The crossover between 32k and 64k#

At an empty context gpt-oss prefills 2.29× faster than Flash-Next. At 32k the lead is down to 1.25×. By 64k Flash-Next is ahead, 180.63 against 139.67, and at ~128k it's 4.30× faster: 105.75 against 24.62. (That last comparison pits Flash-Next at 131072 against gpt-oss at 130048. The 1k difference is not where a 4× gap comes from.)

The model files suggest why. Flash-Next's GGUF declares general.architecture = qwen4exp, a set of ssm.* keys, and full_attention_interval = 4: only one layer in four does full attention, and the others carry a fixed-size state instead of attending over the whole context. gpt-oss-120b's file has no ssm keys and a sliding_window = 128. Read from metadata rather than a profile, that matches the curves.

Generation doesn't cross. At ~128k gpt-oss still produces 25.23 tok/s against Flash-Next's 21.23. Which model "wins at 128k" depends on whether the turn is mostly reading or mostly writing.

The other three curves#

The ISTA Coder prune runs at full Flash-Next speed. Its file is 62 % of the bytes (58.40 GB against 93.67 GB), and every row is within 5 % of the full model's, faster on some and slower on others. The study plan describes it as pruned of experts; I haven't checked how many.

gpt-oss heretic-v2 is about 30 % slower at d0, then catches up on prefill. At an empty context it trails the base model by 32.7 % on prefill and 29.1 % on generation, with a file 3.1 % larger. At 127k their prefill is within 2 % (24.14 against 24.62), while generation is still 16 % behind. I don't have an explanation for that.

DeepSeek-V4-Flash is the slowest everywhere. 4.05 tok/s of generation at 256k, and filling that 256k took it 1 h 29 min, going by the same row stamps.

What the kernel logged during the longest prefills#

During the clean sweeps, the kernel said something three times. From journalctl -k, every amdgpu line between 02:30 and 04:05:

Oct 10 03:14:38 a9-mega kernel: amdgpu 0000:c6:00.0: Fence fallback timer expired on ring comp_1.1.0
Oct 10 03:19:45 a9-mega kernel: amdgpu 0000:c6:00.0: Fence fallback timer expired on ring comp_1.2.0
Oct 10 03:44:38 a9-mega kernel: amdgpu 0000:c6:00.0: Fence fallback timer expired on ring comp_1.1.0

The first two fall inside Flash-Next's 256k prefill row, the third inside gpt-oss's 127k row. Both jobs exited 0 with every row written. All three came in the deepest prefill rows of those two jobs. Three events are a correlation, too few to call a threshold.

The warning comes from amdgpu_fence_fallback() in amdgpu_fence.c:

/**
 * amdgpu_fence_fallback - fallback for hardware interrupts
 *
 * @t: timer context used to obtain the pointer to ring structure
 *
 * Checks for fence activity.
 */
static void amdgpu_fence_fallback(struct timer_list *t)
{
	struct amdgpu_ring *ring = timer_container_of(ring, t,
						      fence_drv.fallback_timer);

	if (amdgpu_fence_process(ring))
		dev_warn(ring->adev->dev,
			 "Fence fallback timer expired on ring %s\n",
			 ring->name);
}

amdgpu_fence_process() returns true if a fence was processed. So the line prints only when the timer found finished work: the timer, not the interrupt, was what noticed the work had finished. A missed interrupt is the natural reading of "fallback for hardware interrupts". It's an inference from a doc comment, not something the log proves.

Laguna at 64k: a ring timeout, three times#

The next job in the queue was Laguna-S-2.1 at UD-Q5_K_XL, and it did not get away with a warning. Its prefill matches Flash-Next's at an empty context (525.08 against 518.26 tok/s) and collapses by 32k, 63.10 against 305.80. At 04:08 the kernel logged two more fallback lines. At 04:19:11 came a ring timeout and a compute queue reset. Part 4 has that log and the queue's three retries. All three attempts died in the 64k row. This is the third, from the per-run kernel capture the speed screen now writes beside each result:

Oct 10 04:50:16 a9-mega kernel: amdgpu 0000:c6:00.0: Dumping IP State
Oct 10 04:50:16 a9-mega kernel: amdgpu 0000:c6:00.0: Dumping IP State Completed
Oct 10 04:50:16 a9-mega kernel: amdgpu 0000:c6:00.0: [drm] AMDGPU device coredump file has been created
Oct 10 04:50:16 a9-mega kernel: amdgpu 0000:c6:00.0: [drm] Check your /sys/class/drm/card1/device/devcoredump/data
Oct 10 04:50:16 a9-mega kernel: amdgpu 0000:c6:00.0: ring comp_1.2.0 timeout, signaled seq=319016, emitted seq=319018
Oct 10 04:50:16 a9-mega kernel: amdgpu 0000:c6:00.0:  Process llama-bench pid 14459 thread llama-bench pid 14459
Oct 10 04:50:16 a9-mega kernel: amdgpu 0000:c6:00.0: Starting comp_1.2.0 ring reset
Oct 10 04:50:16 a9-mega kernel: amdgpu 0000:c6:00.0: reset compute queue (1:2:0)
Oct 10 04:50:16 a9-mega kernel: amdgpu 0000:c6:00.0: Ring comp_1.2.0 reset succeeded
Oct 10 04:50:16 a9-mega kernel: amdgpu 0000:c6:00.0: [drm] device wedged, but no recovery needed

Two fences emitted, not signalled, and the driver ran out of patience. The whole sequence carries one timestamp, so line order is logging order and says nothing about cause. From the day's first fallback warning at 03:14:38 to the first reset at 04:19:11 was about an hour, across three models' jobs.

The lockup_timeout workaround in llama.cpp #25664#

The same vocabulary shows up in llama.cpp issue #25664, "Eval bug: vk::DeviceLostError on linux-7.x kernels (RADV_STRIXHALO)", still open. A dmesg from a DeepSeek-V4-Flash run in its comments has reset compute queue, Ring comp_1.1.0 reset succeeded, device wedged, but no recovery needed and a fence fallback line on comp_1.2.0. Our box logged the same lines, though in the issue's log the fallback warning sits between resets rather than before them.

The issue body lists its suspects in bold:

CONFOUNDING FACTOR: compute lockup_timeout default for AMD driver changed from 60s to 2s.

WORKAROUND: restore linux 6.x behavior with cmdline: amdgpu.lockup_timeout=10000,60000,10000,10000

That 60 s → 2 s change is what the issue reports. I haven't checked it against the kernel. The A9 runs kernel 7.2.9 and had no lockup_timeout on its command line. The thread also has a counter-report: a 395 box on kernel 7.2.0 with the parameter unset, Qwen3.8-27B at 38k, 75k and 113k tokens, no ring timeout. So the trigger depends on the model and the stack, not just on running 7.x. (An earlier DeepSeek-specific explanation in the thread, a missing Vulkan op, predates the commit that added that op. Our build contains it.)

The hypothesis for Laguna: its long-context prefill submits GPU work that runs longer than the compute timeout allows. That's a hypothesis, and fix-16 is the test.

fix-16: the 64k row, before and after#

fix-16 adds the issue's workaround to the kernel command line, which brings the compute timeout to 60 s. The box rebooted into it at 10:54:40, and Laguna went back on the queue. The 64k row completed in that rerun, completed again after the first power cut, and both times a power cut took the results with it (part 4 has that story). The rows below come from the third completion, in the runs after the box booted at 16:38. The before rows are attempt three's partial file, same build, same flags:

The 64k prefill row is stamped 16:51:39 and the tg row 17:11:21. Filling 64k took Laguna 19 min 42 s, against about three and a half minutes for gpt-oss or Flash-Next. Attempt three had died at 04:50:16, about a dozen minutes after finishing its 32k rows, well short of that.

The speeds match where both runs have rows. The longer timeout made nothing faster; the row got to finish.

And the warning didn't go away. Over the whole run the kernel logged exactly two amdgpu lines, both fallback warnings: one two minutes into the 64k prefill, one thirteen minutes into the 256k fill:

Oct 10 16:53:31 a9-mega kernel: amdgpu 0000:c6:00.0: Fence fallback timer expired on ring comp_1.0.1
Oct 10 19:04:57 a9-mega kernel: amdgpu 0000:c6:00.0: Fence fallback timer expired on ring comp_1.0.1

No timeout line follows either, no reset, no coredump. Before the fix the 64k row died three times out of three; after it, it completed three times out of three. That supports "the workaround lets this row complete on this box" without proving the 2 s default was the cause.

The rest of the screen finished at 02:42 the next morning, rc 0. By the same row stamps, filling 128k took Laguna 1 h 39 min and filling 256k took 7 h 51 min, for 4.40 tok/s of prefill and 10.78 of generation at the end. The row completes. Nobody would wait for it.

What 128 GB buys for long context#

Room: an 87 GiB model, a 256k context, and change left over. Not a fast first read. A cold 127k prompt cost gpt-oss half an hour, and Flash-Next eleven minutes, still long enough to make coffee.

One caveat on the absolute numbers: this is stock llama.cpp. Gufo, an MIT-licensed engine built for this chip, claims 1,702.93 pp and 65.25 tg on Flash-Next at Q4_K_XL in its repo description, as fetched on 10 October. That's the project's own claim, on a different quant, with no stated depth, so it doesn't compare to any cell above.

What survives a faster engine is the shape. Generation degrades gently, prefill doesn't, and an agent's cost is set by how often it makes the box reread what it already read. 128k is the floor because the prompt needs it. Prefill is the wall because every cache miss pays to climb it again.

Filed under: amd, linux, kernel, local-llm, benchmarks

Read next#

• The download that contaminated the benchmark — part 4: why these sweeps were rerun on an idle box, and the queue that caught Laguna's first crash.
• 120 GiB for the GPU, set in the kernel, not the BIOS — part 2: the TTM settings that give the GPU the 120 GiB these runs draw from.

Sources#

• llama.cpp #25664: "Eval bug: vk::DeviceLostError on linux-7.x kernels (RADV_STRIXHALO)" — the lockup_timeout workaround, the reported 60 s → 2 s change, a matching dmesg, and the 7.2.0 counter-report.
• drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c, torvalds/linux master — amdgpu_fence_fallback() and amdgpu_fence_process(), where the warning comes from.
• gufo-org/gufo — the project's own Flash-Next speed claim.