local-llm
- My benchmark read my CLAUDE.md
An audit built to check sampler overrides found my OpenCode bench runs carrying my global ~/.claude/CLAUDE.md and skills: 53,483 characters of prompt.
- The best-reviewed code came from the slowest Flash-Next arm
Stage 1 of the A9 agent study: the slowest Flash-Next arm reviewed best, the 62 GiB Q2_0 passed 6/6 at nearly 6x the rate. A sampler confound splits them.
- Zig needed context, not time
Four 27B local models kept hitting the one-hour wall on a Zig task. Four hours at 64k got 7 of 12 through the core tests. At 128k, 35 of 36 did, cleanly.
- Inventing the work vs promising what isn't there
Checking 71 local-model agent runs' claims against their own command logs: three models invent work, three promise atomicity they never built.
- The fastest model finished first by stopping first
gpt-oss-120b cleared six agent cells in 51 minutes by quitting early on each. Two of those stops shipped Zig that does not compile.
- IOMMU off buys a 128 GB iGPU 5 %, and costs the NPU
On this Strix Halo box the GPU was never behind IOMMU translation, so iommu=pt changed nothing for it. amd_iommu=off bought ~5 % prefill and killed the NPU.
- Read the transcript, not the gate
15 of 17 local models passed the agent smoke gate on the 128 GB box. Eight of those passes hid a Claude Code failure the intake had logged and nobody read.
- The benchmark was talking to the previous server
A Flash-Next-only engine, three modes, and a mode result that was really the previous run's server: how cd && server & made $! lie.
- Mistral-Small-4 ran one matmul on the CPU, and said nothing
Mistral-Small-4 decoded at 3.4 tok/s with the model on the GPU: one op per layer ran on the CPU, silently. MiniMax was slow in a different way.
- Vulkan or ROCm on Strix Halo? Ask at which depth
On an empty context Vulkan won the first-day bench on every model. At 127k, ROCm prefilled gpt-oss-120b 7× faster. The answer depends on depth.