Faster prompts, slower agents: a second engine on the 680M

Sun, 01 Nov 2026 00:00:00 +0000

Assumes you've run a local model behind a coding agent. Part 1 has the laptop and why every model here is a mixture-of-experts; part 2 has how the agent runs were graded.

TL;DR On a laptop's integrated Radeon 680M, UnbiasedStrata, an inference engine built around mixture-of-experts models (each token runs through a few expert blocks, a small slice of the parameters), read a 2,048-token prompt 1.4 to 1.5 times as fast as llama.cpp's Vulkan backend once warm, but only on models whose experts all fit in GPU memory. It generated slower on every model it ran: 52 to 78% of llama.cpp's speed. Three coding-agent runs on the same model file got the same grades on both engines (2, 3 and 3 out of 5) from part 2's blind review, which grades the code under a random id. They took longer on Strata: one ran out the two-hour clock, one died on the context limit. Agents spend most of their time generating, so llama.cpp stays the engine for agents on this laptop.

The README of UnbiasedStrata closes its human-written note with "Use at your own risk. Works on my machine." My machine is a 2022 laptop with a Ryzen 7 PRO 6850U and its integrated Radeon 680M GPU, which has no memory of its own and borrows about 16 GB of the laptop's 30 GB of RAM. It is not the machine the README means. The engine targets an NVIDIA card with CUDA, and the 680M's chip, gfx1035, appears nowhere in the build's list of AMD GPUs, not even among the ones marked "unvalidated". I read the warning, agreed with it, and built the thing anyway.

120 GiB for the GPU, set in the kernel, not the BIOS

Sat, 10 Oct 2026 00:00:00 +0000

TL;DR On a Ryzen AI Max+ 395 with 128 GB, set the BIOS's Dedicated Graphics Memory to its minimum (0.5 G) and raise the TTM limit on the kernel command line: ttm.pages_limit=31457280 ttm.page_pool_size=31457280. The unit is 4 KiB pages, so that is 120 GiB. Linux then sees 124.9 GiB, and the GPU can map up to 120 GiB of it on demand. amdgpu.gttsize is deprecated and not needed. A firmware reset (F3, Optimized Defaults) quietly puts the carve-out back to 64 G. With the memory fixed, llama.cpp's Vulkan backend beat Fedora's ROCm build on every model tested. The two were different llama.cpp builds, though, and the benchmark's mean-power column measures something other than efficiency.

The A9 Mega's setup menu keeps its GPU memory setting under Advanced → GFX Configuration, and it goes from 0.5 G to 96 G. The machine has 128 GB of LPDDR5X soldered to the board, and it was bought to run models whose weights alone are 63 GB. Dedicated Graphics Memory went to 96 G, the top of the list.

Rocm on tsvinc

Faster prompts, slower agents: a second engine on the 680M

120 GiB for the GPU, set in the kernel, not the BIOS