On a laptop iGPU, memory bandwidth picked the model

Assumes you've run a local model behind a coding agent. The 3090 Agent Bench ran the same kind of models on a desktop GPU; this series moves them to a laptop.

I ran local models as coding agents on a laptop: a Ryzen 7 PRO 6850U from 2022, with the models running on its integrated Radeon 680M GPU through llama.cpp. The first request opencode, the coding-agent client I drove them with, sent in a session was 11,000 to 12,400 tokens long, and the model had to read all of it before it wrote a word. A dense 27-billion-parameter model, one that runs every weight for every token, took 299 seconds, about five minutes, to read it. Three mixture-of-experts models of 35 to 40 billion parameters, which route each token through a few expert blocks so that only 2.5 to 3 billion parameters are active, took 77 to 78 seconds. None of them had opened a file of the project yet. Five minutes, and the agent hadn't done a single thing I could blame it for. That part would have to wait as well.

The question behind this short series: which open-weights model can do real multi-step coding-agent work on a 680M-class iGPU, unattended, and how good is the code it leaves behind? Laptop GPUs have taught me to read their spec sheets the way I read hotel photos, so I measured instead. This part is the speed half. Part 2 has what seven models built and how it was graded.

The laptop: a GPU with no memory of its own#

llama.cpp drives the 680M through its Vulkan backend. The 680M has no memory of its own. On the boot every run in this series used, the kernel reported 0.5 GiB carved out of system RAM for it and let it map up to 15.3 GiB more through GTT (the graphics translation table, which lets the GPU use ordinary system memory): about 15.8 GiB in all, on the same LPDDR5 bus the CPU uses. A Core Ultra CPU lost to a 2022 laptop iGPU has the first llama-bench tables from this machine; it ran with a bigger BIOS carve-out (4 GiB), so its pool size doesn't apply here.

Two consequences shape everything below. The model, its KV cache (the attention state kept for every token in the context) and everything else the laptop runs share one pool of RAM. And every token the model generates has to stream its weights through that one bus.

A dense 27B generated 5.7 tokens a second#

llama-bench measures two speeds. Prompt processing (pp) is how fast the model reads input, here a 2,048-token prompt. Token generation (tg) is how fast it writes, here 64 tokens. All rows ran llama.cpp build b11960 from the official Vulkan container image, one repetition each, with flash attention (a fused attention kernel that avoids writing the full attention matrix to memory) and an 8-bit KV cache (q8_0):

* Tiel ran with the experts of 8 layers on the CPU, because its 16.9 GiB file is bigger than the GPU pool; more on that below.

** Swift's row is the spread over four micro-batch sizes, one run each; see the knobs below.

The quant names are llama.cpp's quantization formats: the digit is roughly the bits stored per weight, so these files hold the models at about 3 to 5 bits instead of 16. Nail is a fine-tune of Qwen3.6-35B-A3B and Swift-1.5 one of Qwen3.8-27B; Flint is a 40B model with about 2.5B active (part 22 of the 3090 bench lists it as 40B-A2.5B).

Swift-1.5, the dense model, has the smallest file in the table and is the slowest by far. A dense model runs every weight for every token, so each generated token reads the whole 11.3 GiB file. At 5.7 tokens a second that is roughly 64 GiB of weights read per second, before counting the reads of the KV cache. A token that needs to read 11 GiB can't go faster than the bus can deliver 11 GiB.

Smallest file, slowest model. Swift-1.5, I'd love to say you surprised me, but you were only doing arithmetic, and the arithmetic was sitting in your file size the whole time.

The other four are mixture-of-experts (MoE) models. "35B-A3B" means 35 billion parameters in total and about 3 billion active per token: a router inside each layer picks a few small expert blocks, and only those get read. The whole file still has to sit in memory, but a token touches roughly a tenth of it. On the 3090, active parameters set the speed band; on a laptop that shares its RAM bus with everything, the same rule decides whether a model is usable at all, though being a MoE alone wasn't enough (part 2's GLM-4.7-Flash generated at under 7 tokens a second).

Knobs that did nothing for the dense model#

Before giving up on the dense 27B, I tried the usual knobs, in the spirit of a man jiggling a key in a lock: hopeful, methodical and arguing with physics. None moved it past about 6 tokens a second generating or about 47 reading:

• Micro-batch size, the number of tokens llama.cpp pushes through the GPU per step while reading a prompt. Sizes of 256, 512, 1,024 and 2,048 read the 2,048-token prompt at 46.2, 43.8, 40.1 and 47.4 tokens a second and generated at 5.64 to 5.71, one run each. That sweep is the 40-47 in the table.
• Flash attention, on and off.
• MTP drafting. MTP (multi-token prediction) is an extra layer some models ship that drafts the next few tokens cheaply; the full model then checks the draft in one pass, and every accepted token is a step it didn't have to take. On the 3090 it doubled a dense 27B: Swift-1.5 went from 38.7 to 77.2 tokens a second in one pair of runs (part 22). On the 680M, the check itself wasn't cheap: verifying three drafted tokens cost 1.77 times as long as one plain step. Generation stayed at about 6 tokens a second.

The dense model's number comes from a different file than the one in the earlier 680M post, which measured a 14.4 GiB file of Swift-1.5 at 5.33 tokens a second on an older build. Neither gets near the MoEs.

The MoEs generated four to eight times faster#

Against the dense model's 5.7 tokens a second, the MoEs that sit entirely on the GPU generated 26.9 (Nail), 33.1 (Flint) and 48.1 (LFM2.5): 4.7 to 8.4 times as fast. Reading prompts, they managed 166 to 353 tokens a second against the dense model's 40 to 47.

Their files are bigger than the dense one: 13.5 to 14.3 GiB, against 11.3. File size told me which models fit; it told me nothing about speed. I keep a short list of numbers that look like a speed and aren't, and file size has just joined it.

LFM2.5, the fastest, is also the smallest MoE here (24B total, 2B active). It doesn't appear in part 2, which graded seven other models on the agent tasks.

The first agent request: 299 seconds or 77#

llama-bench speeds are for a short prompt on an empty context. An agent session starts with a long one. I ran two short scripts through the llama.cpp router (build b11803, the one the agent runs used, at a 64k context) on three of the MoEs, one right after the other: a tool-call test with a tiny prompt and 512 generated tokens, then a 12k-token first request. Swift-1.5 ran the same pair of scripts earlier. Nothing happens in an agent session until the model has read all of its first request:

* Swift's generation figure is from the same pair of scripts; the log doesn't say which of the two requests it came from.

Flint's request was about 1,100 tokens shorter than the others. Tiel-Coder read as fast as Nail with part of its experts on the CPU. The dense model read at 41 tokens a second, inside the 40 to 47 of the llama-bench table, and spent 299 seconds, 3.8 to 3.9 times as long as the MoEs, before its first generated token.

Later requests in a session are cheaper to read, because llama.cpp keeps the conversation it has already processed in its cache and only reads what's new. On the 3090, part 8 found the server evaluated 12.5% and 2.8% of the prompt tokens in two agent cells; the rest came from the cache. The first request has no cache to hit.

On the dense model, those 299 seconds are 4% of the two-hour wall limit part 2 uses, gone on the request that loads the instructions: four percent of the budget, spent before the agent has been allowed to be wrong about anything.

In real agent runs: 15.6 to 23 tokens a second, GLM aside#

The table above is llama-bench: 64 tokens generated on an empty context. In the agent runs on the laptop, through opencode at a 64k context, the server logged generation of 15.6 to 23 tokens a second and prompt reading of 87 to 247. (Flint ran those with a larger reply limit than Nail, Tiel and Groot; part 2 explains why.) GLM-4.7-Flash, graded later in part 2, fell well outside that range: it generated only 4.6 to 6.8 tokens a second.

Part of the drop is the context, though the agent runs also differ from the bench rows in models, prompt lengths, how long the laptop had been working and llama.cpp build (b11803 for the agent runs). Every generated token also reads the KV cache, and an agent session fills the context with files, tool output and its own earlier replies. The earlier 680M post measured how depth costs on its own, with LFM2.5-24B under the laptop's low-power profile, on its older build and carve-out: 47.3 tokens a second on an empty context, 30.1 with 32,000 tokens already in the cache and 23.4 at 64,000 (depth table). Part 22 saw the same on the 3090, where long Zig sessions generated 7% to 29% slower than short ones.

The dense model starts at 5.7 on an empty context. Context only takes speed away from there.

Trap: a model split across GPU and CPU failed to load#

The GPU pool is about 15.8 GiB, and it has to hold the weights, the KV cache for a 64k context and llama.cpp's working buffers. Tiel-Coder's 16.9 GiB file is bigger than the pool on its own, so in the llama-bench row the experts of 8 layers stay in system RAM and the CPU computes them.

That placement has a trap of its own. Without one more setting, a model split this way fails at load with this line in the server log:

radv/amdgpu: Not enough memory for command submission.

followed by a lost device (ErrorDeviceLost), and nothing gets served. One line, a full stop, and the device is gone: as resignation letters go, it's admirably short. The fix is llama-server's --no-host option ("bypass host buffer"), no-host = true in a router preset. The manual for serving these models to opencode, written under the earlier 4 GiB carve-out, has the full error chain, the nine launch variants that failed the same way and the preset that works.

What the split costs depends on the setup. Tiel-Coder and Nail are both 35B-A3B models, and both read prompts at 201 tokens a second in the llama-bench table, where Tiel generated 18.5 against Nail's 26.9. In the short generation test on the router, Tiel generated 22.9 against Nail's 23.8. The two also differ in quant (IQ4_XS against IQ3_S), and the two measurements differ in build (b11960 against b11803) and in how many tokens they generate, so neither gap is a clean price for the CPU-side experts.

Trap: the agent's own build shares the same RAM#

On a desktop, the model lives in VRAM and the agent's sandbox lives in system RAM. On this laptop there's one pool. A model with a big file plus a Java build that the agent started in its podman sandbox filled all of the laptop's RAM. The agent did its half of that by doing exactly what the prompt told it to: build its code and run the tests.

Nothing in the model's settings warns about that, because the model isn't the only tenant. The file size counts twice on a shared-memory laptop: once for whether it fits the GPU pool, and again for what's left for the code the agent builds and tests.

What these numbers are not#

• Not one file per model. Quants range from about 3 to 5 bits, and Tiel ran with 8 expert layers on the CPU. The table compares the files I could fit, not the models at equal precision.
• Not agent speed. The llama-bench rows are an empty context, 64 generated tokens and one repetition. The agent-run ranges come from opencode at 64k context over the runs, not a figure per model; GLM-4.7-Flash's 4.6 to 6.8 is its own range.
• Not one setup. The llama-bench table ran b11960; the MoE router tests and the agent runs ran b11803. The bench table and the router's short test don't rank Tiel and Nail the same way, so I don't rank models by a few tokens a second.
• One test per model. The router timings above are single runs of each script, and Flint's first request was shorter.

Every graded model is a MoE#

The four models graded first in part 2 are mixture-of-experts models with 2.5 to 3 billion active parameters: Nail, Tiel-Coder, Flint and Groot (19B-A3B at Q4_K_M, which has no llama-bench row above). The three graded later, Ornith, plain Qwen3.6-35B-A3B and GLM-4.7-Flash, are MoEs too. The dense 27B is not among them: 299 seconds for the first request, and under 6 tokens a second generating. Being a MoE didn't guarantee speed, though: GLM-4.7-Flash, graded as delivered although all three of its runs crashed, generated at 4.6 to 6.8 tokens a second in its agent runs, very slow on this laptop and not competitive here.

Speed says nothing about the code. A fast model only arrives at its mistakes sooner, which on a two-hour clock I'll still count as the better deal. Part 2 has what these models built on three tasks, and what a blind review found in it.

Filed under: local-llm, llm, ai-agents, benchmarks, llama.cpp

Read next#

• Part 2: Twenty-one agent runs on a laptop iGPU, none merge-ready: what the same models built, and how it was graded.
• On one RTX 3090, agent speed followed active parameters: the same rule on a desktop GPU with 24 GB of VRAM.

Sources#

• My own llama-bench runs, first-request timings and agent-run serve logs on the laptop, and its sysfs GPU memory totals (mem_info_vram_total, mem_info_gtt_total) on the boot of 28 October 2026 that every run used (local, not published).
• A Core Ultra CPU lost to a 2022 laptop iGPU: the earlier Swift-1.5 file and the depth sweep cited above.
• Serve local GGUFs to opencode with llama-server router mode: the error chain and the no-host setting.
• Parts 8 and 22 of the 3090 Agent Bench, for the cache share and the MTP and active-parameter figures cited from them.