On one RTX 3090, agent speed followed active parameters

Assumes you've run a local model behind a coding agent. Part 1 introduced the bench; this part and the next one sum it up.

The smallest file on my bench is Ternary Bonsai 2, a 27-billion-parameter model packed into 7.2 GB. Driving a coding agent, it generated 37.9 tokens per second, the median over its three batches of runs. Ornith 1.5, a 35B model in a 19.3 GB file, generated 171.2. The file 2.7 times bigger ran 4.5 times faster.

The bench behind those numbers puts local models on one RTX 3090 (24 GB of VRAM), served mostly by llama.cpp, and has them drive agent clients through coding tasks graded by hidden tests. OpenCode, an open-source coding agent, is the main client. The server logs prompt-reading and generation speed for every request, so every batch of runs comes with its own speed record. This part reads those records for the 28 models with a speed record since 15 October: what set their speed and what barely moved it. Part 23 covers which of them passed.

What the numbers are#

Every number here is a serve-log figure, not a synthetic benchmark. The bench groups runs into cells: one model, one client, one arm (a batch of runs under one set of settings). For each cell, the speeds are token-weighted over every request the server logged:

• Generation (tg): tokens written per second.
• Prompt reading (pp): prompt tokens processed per second. llama.cpp counts only the tokens it actually evaluated, so a prompt prefix served from the cache doesn't count.

The prompts are whatever the agent sent: a short tool result, a conversation tens of thousands of tokens long, everything in between. A model's median below is the median over its cells since 15 October, and those cells mix task tiers, and for some models clients and engines. For an even number of cells, the summary takes the upper of the two middle values.

Dates matter, because the setup changed under the bench twice. Before 15 October, the bench mostly served models without MTP drafting, the speed-up explained below. Since 15 October its intake arms draft with MTP wherever the model has the extra layer for it; a few control arms ran without. From 26 October, a newer llama.cpp build (caa8191 instead of f95b0d9) ran on a card with an unlock module loaded, an open-source kernel module that restores the full rate of the card's half-speed tensor math. Everything in the chart is from 15 October on.

Active parameters set the band#

The chart sorts into three bands, and the band follows how many parameters each token touches.

Mixture-of-experts models with 2 to 4 billion active parameters: 94 to 187 tokens per second. The name says it: "35B-A3B" is 35 billion parameters in total, about 3 billion used per token. LFM2.5 24B-A2B leads at 186.9, then Ornith 1.5 35B-A3B (171.2), Nemotron 3.5 Lightning 30B-A3B (151.2), Gemma 4 26B-A4B (118.2), Qwen3.6 35B-A3B (117.7), Laguna-XS 2.1 (111.4; 33B with 3B active, per its model card in part 14), Groot 19B-A3B (102.7) and Nail 35B-A3B (94.3). Flash-Next on the Strata engine, a 2-bit MoE with 10 of its 512 routed experts per layer active for each token (part 1), sits with them at 95.4.

Dense 27B models with MTP drafting: 56.6 to 74.3. Eight files of the same Qwen3.8-27B architecture, tunes and quants, all in one cluster. Every token runs through all 27 billion parameters.

Dense models with no drafting: 28.2 and 21.7. Granite 4.2 30B and Nanbeige4.3 33B.

Two MoE models on llama.cpp break the pattern, and both kept part of their experts off the card. K2 Horizon 36B-A4B, 4 billion active, generated 24.7, slower than any dense 27B. At 128k context it ran with the experts of 24 layers on the CPU (part 14). Flint 40B-A2.5B ran with the experts of 8 layers in system RAM (part 19) and generated 59.5, below every MoE with 2 to 4 billion active except K2. Flash-Next doesn't fit on the card either, but Strata is built for that: it keeps an expert cache on the GPU and the rest in RAM (part 1), and it still landed in the fast band.

Nemotron 3.5 Bolt 74B-A9B fits the pattern from the other side: 9 billion active, three times the A3B models, and 69.5, inside the dense 27B cluster.

Six models carry no architecture in their names, and nothing I've published states it, so the chart marks them "not stated" rather than guessing from their speed.

File size and quant: a smaller file didn't buy speed#

Two 27B models first. Bonsai's 7.2 GB file on Prism's llama.cpp fork generated 37.7 to 47.8 across its three intake arms. Swift-1.5, a Qwen3.8-27B tune in a 15.5 GB IQ4_XS file, generated 35.6 to 38.7 in the OpenCode arms since 15 October that ran it without drafting (41.3 in one Pi arm). Less than half the bytes, about the same speed, on two different models and engine builds.

Within one model, on one build, the quant moved generation by tens of percent at most. Swift-1.5 and Qwen3.8 also exist as GSQ-RCO IQ3_S files of 12.1 GB, 22% smaller than Swift's IQ4_XS. On the newer build with three MTP drafts, same tasks:

The smaller file came out 4 to 5% slower. One setting differed: the IQ4_XS arms ran the model's default reasoning effort, the IQ3_S arms "medium".

Going the other way cost speed. Swift's 17 GB Q4_K_M file, the one built for open-jet in part 7, ran three Zig arms next to IQ4_XS on the same build: 55.7, 54.0 and 58.7 tokens per second against 69.1, 64.2 and 63.6. IQ4_XS was 8 to 24% faster.

Across models, the spread is 8.6 times, from 186.9 down to 21.7. The quant is a correction to the band, not the band.

MTP doubled the dense 27Bs; acceptance didn't predict by how much#

MTP (multi-token prediction) is an extra layer some models ship: it drafts the next few tokens cheaply, and the full model checks the whole draft in one pass. For a dense model, where every pass touches all the weights, that's the cheapest speed there is.

On the older build, through OpenCode, four dense 27Bs without drafting (the base Qwen3.8-27B and three tunes, one arm each) generated 35.6 to 37.1 tokens per second. On the same build and client with three drafts, their medians are 67.8 to 73.8. The cleanest single pair is two Swift-1.5 arms on the same build: 38.7 without MTP, 77.2 with it. Those are the arms behind part 6, where the bench's own server script had been leaving MTP off.

How much of the draft gets accepted is the number people quote, and it didn't track speed. Across the 83 llama.cpp cells of the 27Bs with MTP, acceptance ran from 0.65 to 0.90. Two of the GSQ files ran a second set of arms with two drafts instead of three, and a different sampler (temperature 0.6 instead of 1.0):

Acceptance went up in all six pairs, by 0.08 to 0.16. Generation went up in one, by 1.8%, and down in five, by 1.1 to 13.4%. Two settings changed at once, so this doesn't isolate the draft length. It does show that higher acceptance and higher speed are separate results. Part 18 swept draft length on its own and found the same thing: acceptance falls as drafts get longer, by construction, and speed peaks early.

Ornith, the fast MoE, makes the point from the other side. Its cells with one draft accepted 0.80 to 0.89 and generated 134.2 to 203.0. One Pi arm on the CLI tools, on the older build, ran it with three drafts: acceptance 0.40, generation 137.3. Pi's one-draft arms gave 178.4 on operator tasks on the same build and 203.0 on the CLI tools on the newer one. On a MoE a full pass is already cheap, so a wasted draft slot costs a larger share of it.

Prompt reading: the MoEs read faster, and it mattered less#

Median prompt reading for the MoEs with 2 to 4 billion active that sit entirely on the card runs from 2,074 (Nail) to 4,475 tokens per second (LFM2.5). The dense 27Bs with MTP read at 841 to 1,139. The MoEs with experts off the card read slower: Flash-Next on Strata 1,378, Flint 1,005 and K2 Horizon, with the experts of 24 layers on the CPU, 528.

That gap matters less than it looks. Part 8 summed llama.cpp's logs for two agent cells: the server evaluated 12.5% and 2.8% of the prompt tokens, and the rest came from the cache. Generation took most of the server's time. An agent resends the conversation each turn, but the server has seen most of it already.

The unlock and the newer build showed up exactly there. Swift-1.5 ran the same tiers before and after 26 October:

Prompt reading rose 15% and 19%; generation stayed flat (-1.9% and +1.0%). The reasoning-effort setting differed here too: "medium" before, the default after.

Long sessions generate slower#

The same model, on the same build, generated slower the longer its tasks ran. Fourteen models ran all three intake tiers through OpenCode with a speed on each: the dev tier (six small CLI tools), the hard tier (feature work in existing Java, Go and Spring Boot code) and Zig (a ledger task in Zig 0.16 with a four-hour limit). Context was the same across a model's three arms except for Muse-Glimmer, whose Zig arm ran at 128k and the other two at 64k. Generation, tokens per second:

The dev arm was the fastest of the three for all 14. Zig was the slowest for 12; Bonsai and K2 Horizon ran Zig 0.2 and 1.9 tokens per second faster than their hard arm. From dev to Zig, generation fell by 7.0% (Bolt) to 29.4% (Flint).

Zig runs are the long ones: for these 14 models, the Zig arms took 393 to 9,136 seconds per run, the dev arms 92 to 789. A longer session means a deeper context on every request, and part 15 measured what depth does on its own: Qwen3.8-27B generated 42.02 tokens per second on an empty context and 22.44 at 100k. I didn't split the agent logs by context depth, so the table shows the effect, not its size per token of context.

Same weights, other engines and clients#

Swift-1.5 ran on four engines, which makes it the best-measured model on the bench and the worst one to quote a single number for:

• llama.cpp with three MTP drafts: 57.0 to 79.3 tokens per second across 24 cells.
• Tabby (ExLlamaV3) with MTP: 84.4 and 85.5, on an EXL3 quant (part 11).
• open-jet's megakernel, no drafting: 105.1, and 101.2 from one log shared by three arms, on open-jet's own Q4_K_M file.
• ninfer with DFlash2 drafting: 128.5 and 137.6, on its own W4A16 weights, with acceptance of 0.50 and 0.41. A different drafter counts acceptance its own way, so those two don't compare with MTP's.

Faster generation didn't become a proportionally faster agent: in part 8, ninfer's agent cells finished 4% sooner than llama.cpp with MTP.

The client moves the number too, though I can't separate it from the workload. On the newer build, Ornith with one draft generated 152.1 tokens per second through Claude Code, 171.2 through OpenCode, 182.4 through omp and 203.0 through Pi. Same server, same weights, same draft setting, but not the same tasks: the OpenCode figure comes from one log that covered all four task tiers (the three above plus open-ended operator tasks), while the omp, Pi and Claude Code figures each come from a single arm of small CLI tasks.

What these numbers are not#

• Not a like-for-like table. Context was 64k for some models and 128k for others, samplers came from each model's card, and the reasoning effort varied. A model whose cells are mostly Zig gets a lower median than one whose cells are mostly short tasks.
• Some cells share a log. Before 15 October, several arms often point at one serve log per model and context: Laguna-XS's seven earliest arms all show 145.7 from the same file. It still happens since then, less often: Ornith's OpenCode log stands for four cells and open-jet's 128k log for three. Counting each of those once would move Swift-1.5's median from 74.3 to 73.8 and leave Ornith's at 171.2.
• One gap: Flash-Next's IQ2_XS file on its newer engine build has results but no speed record in my summary, so it isn't in the chart.

So when a new model shows up for this card, I look at the part of its name after the "A" before I look at the file size, then at whether it fits or has an engine built for the part that doesn't, and then at whether it ships an MTP layer. Part 23 has the half of the question speed can't answer: which of these models passed.

Filed under: local-llm, llm, ai-agents, benchmarks, llama.cpp

Read next#

• Part 23: Most models passed the scripts. The ledgers split them — the results for the same models, task by task.
• MTP acceptance is a side effect, not a target — the controlled draft-length sweep behind the acceptance section.
• 2.2x faster prefill, 1.04x faster agent — why prompt-reading speed buys an agent so little.
• It fits in 16 GB. That was the easy question — how context depth slows generation on the same 27B.

Sources#

• My own bench data (local, not published): per-cell serve-log speeds (prompt reading, generation, MTP acceptance) for every cell with a speed, per-model medians since 15 October, and the weights manifest with file sizes, as of 30 October 2026.
• The earlier parts of this series linked above, for the facts cited from them: expert placement for K2 Horizon (part 14) and Flint (part 19), Flash-Next's expert count (part 1), the engines' weight formats (parts 7, 8, 11) and the cached-prompt share (part 8).