Local-Llm on tsvinc

Image input on a 27B coder costs 1.1 GiB, once

Mon, 02 Nov 2026 00:00:00 +0000

Assumes you've served a GGUF with llama.cpp. Part 1 introduced the bench.

I had the memory risk of image input in the wrong place. On a 24 GB RTX 3090 that already holds a 27B coding model with a 128k-token context, I expected the dangerous request to be the full one: an image plus a context filled to the brim. Adding image input to Qwen3.8-27B makes sense: the base model is a vision-language model, and a coding agent that can read a screenshot of a failing UI or a terminal is worth having. The question was whether it still fits on the card.

Lock OpenCode 2.x to your self-hosted model

Mon, 02 Nov 2026 00:00:00 +0000

Two config lines make OpenCode 2.x send prompts only to your self-hosted OpenAI-compatible model, and a self-contained script proves which model a headless run picks.

Tested on: OpenCode V2 (npm @opencode/cli) 2.2.5, headless opencode run, throwaway HOME, first in a container with no network, then with the PoC below behind a dead proxy (2026-11-02). The no-model case was also run on 2.2.0.
Not verified: the interactive TUI, a fresh install's first-run screens, 2.2.1 to 2.2.4, a config with both lines at once, "model" alone with the local server down, a full personal config.
Needs: an opencode 2.x binary, a local OpenAI-compatible model server already configured as a provider (one way: /guides/serve-local-models-to-opencode-llama-router/), python3 for the PoC. No opencode.ai account and no API key.
Time: about 10 min, my estimate, not timed (the PoC caps each of its three runs at 60 s).
Risk: low. You edit one config file after copying it; the PoC points HOME and the four XDG_* directories it sets (config, data, state, cache) at a mktemp -d directory and starts a server on 127.0.0.1:18765 for the length of the run.

Before you start#

Placeholders used below:
- <CONFIG>: your global OpenCode config file. The PoC writes its own as $XDG_CONFIG_HOME/opencode/opencode.json inside a throwaway HOME, so on a default setup this is ~/.config/opencode/opencode.json.
- <PROVIDER_ID>: the key of your local provider under "provider" in <CONFIG>. The PoC uses local.
- <MODEL_ID>: the key of your model under that provider's "models". The PoC uses qwen-local.
- <OPENCODE_BIN>: path to the opencode 2.x binary; the PoC defaults to opencode on PATH.
Check the version with <OPENCODE_BIN> --version. Mine printed opencode v2.2.5 (it is also the PoC's first output line).
Back up the config before step 3:
```
cp <CONFIG> <CONFIG>.bak
```

My agent guardrails passed every test. Then I used them.

Mon, 02 Nov 2026 00:00:00 +0000

Assumes you've set up a terminal coding agent's permission rules, or wondered what a model sees when they say no.

For weeks, the only user my test suites knew was a fake model. The suites cover my configuration templates for six terminal coding agents (OpenCode in two major versions, V1 and V2, plus pi, omp, Hermes Agent and Claude Code): each one runs local models and refuses destructive commands even with nobody approving. Every suite drove them with a scripted fake model in a container with no network. It was an ideal user: it ran exactly the command the test told it to, it never improvised, and it never once wanted a screenshot of anything.

One open PR made MoE prompts 12-30% faster on my 3090

Mon, 02 Nov 2026 00:00:00 +0000

Assumes you've served a GGUF with llama.cpp. Part 1 introduced the bench.

115 upstream llama.cpp commits bought my RTX 3090 at most 2.8%. The next morning, one pull request from outside upstream bought up to 29.7% in prompt speed in llama-bench, and about 13% through the server that actually feeds the agents, on synthetic prompts. Part 25 moved my llama.cpp build forward by those commits and measured the result on the card: prompt reading moved -0.4% to +2.8%, generation 0.0% to +0.6%. I switched anyway, mostly for the parser fixes.

OpenCode didn't want an account. It wanted a model name.

Mon, 02 Nov 2026 00:00:00 +0000

Assumes you've pointed a terminal coding agent at a model you serve yourself.

The warning reached me second-hand, the way the most confident warnings travel. Don't install a fresh OpenCode 2.x if you run your own models, it said: the new version won't talk to a self-hosted model until you activate an account on opencode.ai and pay for a subscription, and it asks for an API key on your very first prompt. A second message came along with it, an AI assistant's explanation, and that one had a mechanism: fresh installs now default to OpenCode's own hosted model, the switch that turns the hosted provider off is fairly new and may not work, and in that case there's a third-party plugin, installed globally with npx.

The screenshot opencode never sent

Mon, 02 Nov 2026 00:00:00 +0000

Assumes you've run a local model behind a coding agent. Part 1 has the laptop and its memory pool; the router manual has the serving setup.

TL;DR On a laptop's integrated Radeon 680M, llama.cpp served 14 model presets (12 models, two of them in two quants) with image input, each with its projector, the small extra file that turns pixels into tokens. On five generated test images they got 54 of 56 single answers right, and read a six-line cut list back with full recall on every model. Mixture-of-experts models with spare GPU memory lost 2% of their generation speed or less; dense 27B models and the presets that already filled the pool lost 4 to 32%. In opencode, a terminal coding agent, versions 1.19.3 and 1.19.5 on my setup and 1.18.35 on a second machine did not send an attached image unless the model entry declared image input: the model got an error string instead, and the run exited 0. Adding "modalities": {"input": ["text", "image"], "output": ["text"]} fixed it. One setup that first saw the image arrive without the key had it in a config file in a parent directory, which opencode merges in. An RTX 3090 desktop measured the same projector memory with no speed loss. On the second machine, a Strix Halo mini PC, two models "passed" the image test without seeing the image by running an OCR tool through bash, so check what was sent and what tools ran, not the answer. On 18 questions about real screenshots with thinking on, Gemma 4 26B (a mixture-of-experts model) got 17 at 23 s each on average, and Swift 9B (a dense model) all 18 at 38 s.

The obvious suspect for anything new on this laptop is the GPU. It's a Radeon 680M with no memory of its own: on the boot these tests ran on, it borrowed a 4 GiB carve-out plus 13.6 GiB of system RAM it can map (GTT), out of the laptop's 32 GB, and this series has already spent a fair share of its words on what does and doesn't fit in it. So the place to expect trouble from letting local coding models look at screenshots is right there: a picture means extra weights, extra tokens and extra memory, on a GPU that has none to spare.

115 llama.cpp commits later, my 3090 was barely faster

Sun, 01 Nov 2026 00:00:00 +0000

Assumes you've served a GGUF with llama.cpp. Part 1 introduced the bench.

This series is about one 24 GB RTX 3090 and the local models I serve from it as coding agents, each run scored by tests the model never sees. The models are served by llama.cpp, built from source, and since 26 October the build was master at caa8191. By 31 October upstream was 115 commits further on, and the question was the routine one: did any of them do anything for this card and these models?

Eighteen days later, the same lie

Sun, 01 Nov 2026 00:00:00 +0000

Builds on Inventing the work vs promising what isn't there, where this model first earned its flag.

Here is a paragraph from a README that a local model, working as a coding agent on a Java money-transfer ticket, wrote on the evening of 31 October, in a batch of runs that started at 21:08:

Atomicity: All-or-nothing semantics. Either both accounts are updated or neither is. If the repository fails part-way through (e.g., connection lost on the second save), the operation is rolled back: balances are unchanged and no audit events are appended.

Faster prompts, slower agents: a second engine on the 680M

Sun, 01 Nov 2026 00:00:00 +0000

Assumes you've run a local model behind a coding agent. Part 1 has the laptop and why every model here is a mixture-of-experts; part 2 has how the agent runs were graded.

TL;DR On a laptop's integrated Radeon 680M, UnbiasedStrata, an inference engine built around mixture-of-experts models (each token runs through a few expert blocks, a small slice of the parameters), read a 2,048-token prompt 1.4 to 1.5 times as fast as llama.cpp's Vulkan backend once warm, but only on models whose experts all fit in GPU memory. It generated slower on every model it ran: 52 to 78% of llama.cpp's speed. Three coding-agent runs on the same model file got the same grades on both engines (2, 3 and 3 out of 5) from part 2's blind review, which grades the code under a random id. They took longer on Strata: one ran out the two-hour clock, one died on the context limit. Agents spend most of their time generating, so llama.cpp stays the engine for agents on this laptop.

The README of UnbiasedStrata closes its human-written note with "Use at your own risk. Works on my machine." My machine is a 2022 laptop with a Ryzen 7 PRO 6850U and its integrated Radeon 680M GPU, which has no memory of its own and borrows about 16 GB of the laptop's 30 GB of RAM. It is not the machine the README means. The engine targets an NVIDIA card with CUDA, and the 680M's chip, gfx1035, appears nowhere in the build's list of AMD GPUs, not even among the ones marked "unvalidated". I read the warning, agreed with it, and built the thing anyway.

105 identical commands and other ways local agents fail

Sat, 31 Oct 2026 00:00:00 +0000

Assumes you've driven a coding agent through a client like OpenCode. Part 1 introduced the bench; part 23 has the results by task.

GLM-4.7 Flash, an 18 GB local model, was asked to write a small Java tool that reads JUnit test reports and lists the slowest tests. Somewhere in the run it typed this shell command:

rm -rf test-reports && mkdir test-reports && java SlowTests --json test-reports 2>&1

The command empties a directory and then asks the tool to summarise it. After a line from the JVM about its options, the tool answered with a JSON summary that began {"total":0. The model sent the same command again, byte for byte, and got the same answer, 105 times back to back. The run made 176 tool calls in all, and the bench killed it at its 30-minute limit.

In opencode 1.19.5, my workaround protected nothing

Sat, 31 Oct 2026 00:00:00 +0000

The backlog item was small and polite. My LAN proxy for local LLMs writes the config for opencode, a terminal coding agent, and the model names in that config had been slightly wrong on purpose: opencode showed lan/3090-q38-27b-128k where every other client sees 3090-qwen38-27b-128k. The lan/ is just the provider prefix; the q38 was the deliberate mistake. The item said to try a plugin hook that could remove the reason for the wrong names, and it ended with two words in brackets: verify first. Backlog items don't usually carry a warning label. This one had earned it, and the person it was warning was me.

Most models passed the scripts. The ledgers split them

Sat, 31 Oct 2026 00:00:00 +0000

Assumes you've driven a coding agent through a client like OpenCode. Part 1 introduced the bench; part 22 covered how fast the same models run.

This part counts passes: which local models got which coding tasks past hidden checks they never see. The models run on one RTX 3090 (24 GB of VRAM), mostly served by llama.cpp, and drive an agent client, mainly OpenCode, an open-source coding agent.

TL;DR — Since 15 October, through OpenCode on one RTX 3090, 18 models ran six small command-line tools in their first intake arm (a model's first batch of intake runs under one set of settings): 288 of 309 runs passed every hidden check, and 11 models passed all of theirs. 20 models ran feature work in existing Java, Go and Spring Boot code, again counting each model's first arm: 80 of 180 runs passed, no model passed all 9, five passed none. On the Zig version of the Java ledger task, first arms only (add a money transfer to an existing ledger service), 18 of 60 runs passed, and only K2 Horizon, a mixture-of-experts model with 4 billion of its 36 billion parameters active per token, passed all three, at 73 minutes a run. The best on feature work were tuned variants of the dense Qwen3.8-27B model with MTP drafting, the model's own draft layer (ThinkingCap 8 of 9, Qwen3.8 27B-pi and Turbo Brilliance 7) and Flash-Next on Strata, an engine that keeps part of a mixture-of-experts model in system RAM (7 of 9). A gap of one or two passes out of nine is inside run-to-run noise.

My agents got their own config. Two still read mine.

Sat, 31 Oct 2026 00:00:00 +0000

Builds on My benchmark read my CLAUDE.md and Claude Code read my home folder as a project, which explain two of the leaks below in detail.

I keep a set of config templates for six terminal coding agents: OpenCode in two major versions, pi, omp, Hermes Agent and Claude Code. They're set up to run on local models and to block destructive commands even when nobody is approving them. Anyone installing them may already have a config of their own for at least one of those agents, so the installer has a mode for that. I call it installing apart: the template goes into its own directory, ~/.config/harnesses/<name>, a small launcher points the agent there, and your own config is never touched. That is the whole promise, one clause long, and I wrote it.

My drift check said 0 differences. It checked 4 arrays.

Sat, 31 Oct 2026 00:00:00 +0000

My LAN proxy for local LLMs keeps its own catalog of how to serve each model, models.json. Each of the two GPU boxes behind it keeps the same settings again in its own models.sh, a bash script. I wrote a drift check to compare the two, and its first run said "0 differences".

I read that as a sentence about the config files. It was a sentence about my parser, which had pulled four arrays out of two bash scripts of about 500 and 600 lines, compared them, and reported back with total calm.

On a laptop iGPU, memory bandwidth picked the model

Sat, 31 Oct 2026 00:00:00 +0000

Assumes you've run a local model behind a coding agent. The 3090 Agent Bench ran the same kind of models on a desktop GPU; this series moves them to a laptop.

I ran local models as coding agents on a laptop: a Ryzen 7 PRO 6850U from 2022, with the models running on its integrated Radeon 680M GPU through llama.cpp. The first request opencode, the coding-agent client I drove them with, sent in a session was 11,000 to 12,400 tokens long, and the model had to read all of it before it wrote a word. A dense 27-billion-parameter model, one that runs every weight for every token, took 299 seconds, about five minutes, to read it. Three mixture-of-experts models of 35 to 40 billion parameters, which route each token through a few expert blocks so that only 2.5 to 3 billion parameters are active, took 77 to 78 seconds. None of them had opened a file of the project yet. Five minutes, and the agent hadn't done a single thing I could blame it for. That part would have to wait as well.

On one RTX 3090, agent speed followed active parameters

Sat, 31 Oct 2026 00:00:00 +0000

Assumes you've run a local model behind a coding agent. Part 1 introduced the bench; this part and the next one sum it up.

The smallest file on my bench is Ternary Bonsai 2, a 27-billion-parameter model packed into 7.2 GB. Driving a coding agent, it generated 37.9 tokens per second, the median over its three batches of runs. Ornith 1.5, a 35B model in a 19.3 GB file, generated 171.2. The file 2.7 times bigger ran 4.5 times faster.

The template differed. The prompts didn't.

Sat, 31 Oct 2026 00:00:00 +0000

Builds on Read the transcript, not the gate, where a chat template broke every Claude Code session on four models.

The plan had one variable. Same model, two quantizations, only the weights differ, and I was about to run it on Ornith-1.5-122B, a 122-billion-parameter mixture-of-experts coding model, on my 128 GB AMD Strix Halo box. On one side, the 4-bit UD-Q4_K_XL from unsloth, a team that publishes quantized builds of many big open models, about 80 GB. On the other, a GSQ-RCO IQ3_S from the Hugging Face account fajardc, about 53 GB, a mixed quant whose tensors range from IQ2_XS to Q6_K. Ten coding-agent runs each: five tasks in five languages, each through two agent harnesses, opencode and Claude Code, the programs that turn a model's replies into file edits and shell commands.

Twenty-one agent runs on a laptop iGPU, none merge-ready

Sat, 31 Oct 2026 00:00:00 +0000

Assumes you've driven a coding agent through a client like opencode. Part 1 covers why every model here is a mixture-of-experts.

TL;DR — Seven mixture-of-experts models (each token runs through only a few expert blocks, a small slice of the parameters) ran three coding tasks each through opencode, a coding-agent client run headless in a sandbox, on a laptop's integrated Radeon 680M GPU: a Node bookmarks API, a Java access-log report and a Go status page. 12 of the 21 runs committed their work, 4 hit the two-hour limit, and 3 died when the GPU timed out under GLM-4.7-Flash, one of the seven. A blind review (the reviewer never knew which model wrote the code) that ran its own probes (small tests of the failure paths) in a sandbox graded 12 of the 21 at 3 out of 5, four at 2 and five at 1: nothing was merge-ready. In the first twelve runs, 11 projects passed their own test suites, and the review still found blockers in 11. Tiel-Coder, a 35B model with about 3B active, came closest.

Five languages, two reps: the round-two ranking

Fri, 30 Oct 2026 00:00:00 +0000

Builds on Nine accusations, one lie, which covers how round two was graded. This post is what the grades add up to.

Round two of my local-model coding study is finished, and this post ranks its six setups by how their code holds up when a save fails. The study asks whether a model served locally, on my 128 GB AMD Strix Halo box, can do a developer's job as a coding agent. Round two gave the six setups (a setup is one model with its quantization and inference engine) the same seven coding tickets, each in two agent harnesses, opencode and Claude Code, the programs that turn a model's replies into file edits and shell commands. Five of the tickets are the core: the same money-transfer feature for a small ledger service, in Java, Go, Zig, TypeScript and Python.

Nine accusations, one lie: re-reading round two's judges

Fri, 30 Oct 2026 00:00:00 +0000

Builds on How to tell when your coding agent lied, not just failed, which describes the honesty check this post grades.

Two automated judges graded round two of my local-model coding study, and this post grades them. The study asks whether a model served locally, on my 128 GB AMD Strix Halo box, can do a developer's job as a coding agent; round two ran six model setups through the same coding tickets in two agent programs, opencode and Claude Code. One judge grades the code, the other grades whether the agent told the truth about its own work.

Two rounds, seventeen arms: the engine set the speed, not the quant

Fri, 30 Oct 2026 00:00:00 +0000

Builds on Five languages, two reps: the round-two ranking, which ranks these arms by the quality of their code. This post is about how fast they got there.

Qwen3.8-Flash-Next, a mixture-of-experts model, ran at two quantizations (a quant sets how many bits each weight gets) in round two of my study of local models as coding agents, six model setups on the same coding tickets on my 128 GB AMD Strix Halo box. At Q2_0, 2.40 bits per weight, it generated 58.9 tokens per second; at IQ4_XS, 4.25 bits per weight, 26.6. A 66.4 GB file beating a 93.7 GB one by 2.2x is what a smaller quant is supposed to do: fewer bytes to read per token, more tokens per second. There should be a rule that a speed gap lists everything that changed before it is allowed an explanation. This one listed one thing.

My isolation guard checked itself

Thu, 29 Oct 2026 00:00:00 +0000

Follows The mutant was killed after it switched a real GPU, which built the isolation this post tests.

I used to think a guard test fails when the thing it guards is gone. That is what the word means. Mine guarded the wall between my Go CLI's tests and my live LAN proxy, which starts a real model server on a real GPU box whenever a request asks for a model that isn't running. I wrote the guard, described it in a post, and trusted it the way you would trust an examiner who lets the student bring the answer key: nothing ever comes back wrong. I took that as a good sign. So, I imagine, would the student.

The config said 32k. The engine still ran 64k.

Thu, 29 Oct 2026 00:00:00 +0000

TL;DR — My Go proxy for local LLMs now reloads three settings on SIGHUP and refuses any other change by name. A reload changes what the proxy tells clients at once, but a running model server keeps the preset it was started with, so the two can disagree until the server restarts; the reload log line says so. Overlapping model-list refreshes are numbered, and only a successful read moves the number forward.

The race detector saw my requests in order

Thu, 29 Oct 2026 00:00:00 +0000

I run a small Go proxy on my LAN, in front of the GPU boxes that serve my local LLMs. Clients can ask it for a model by a short alias, and it looks the alias up in a map to decide where the request goes. That map can now be replaced while the proxy is running, by a config reload, so every read of it has to take a lock. A read that skips the lock is the kind of bug that does nothing at all until one request lands in the middle of a reload, and I have no interest in meeting it there.

The third run mattered where it cost the most

Thu, 29 Oct 2026 00:00:00 +0000

Assumes you've driven a coding agent through a client like OpenCode. Part 1 introduced the bench.

My bench is a set of coding tasks for local models on one RTX 3090 (24 GB): each model drives OpenCode, an open-source coding agent, and hidden tests decide whether a run passed. Until this part, a new model went through an intake: every task three times, in three tiers. The basic tier is six small scripting tasks, the hard tier three ledger services in Java, Go and Spring with hidden core tests, and the Zig tier one Zig ledger task that may run for four hours. One model's full intake cost about a day and a half of the card's time, and most of that is Zig.

The mutant was killed after it switched a real GPU

Wed, 28 Oct 2026 00:00:00 +0000

I test my fixes by breaking them on purpose. The method is mutation testing, done by hand: copy the module, undo one condition (that copy is a "mutant"), run the copy's tests, and record KILLED if a test fails or SURVIVED if none does. KILLED is the result you want; it means the tests noticed. The code in question is a small Go program on my LAN that sits in front of the GPU boxes serving local language models and decides which model server runs where. I like the method because its verdict is one word. It had never occurred to me to ask what a mutant gets up to between being created and being killed.

llama.cpp's MoE cache wants gigabytes and long prompts

Tue, 27 Oct 2026 00:00:00 +0000

Assumes you've served a MoE GGUF with llama.cpp and know what --n-cpu-moe does. The card and the build bump behind these numbers are in The 3090 unlock is real. Where it lands is the catch.

My agent bench runs local models on one RTX 3090 with 24 GB of VRAM. A mixture-of-experts (MoE) model too big for that can keep part of its experts (blocks of weights, of which each token uses only a few) in system RAM. llama.cpp now has a flag for that case, --moe-cache-mib: a slice of VRAM that keeps the popular experts on the GPU while the rest sit in RAM. This part measures what it buys on three MoE models: Flint 40B, Ornith 35B and Flash-Next, the last too big even for VRAM and RAM together.

The lock map deleted a mutex someone was waiting on

Tue, 27 Oct 2026 00:00:00 +0000

Assumes you have guarded per-key work in Go with a map of mutexes. Builds on 29 of 34 passing agent runs had a blocker the hidden tests never saw.

This series is about the A9, my 128 GB AMD Strix Halo test box (a GEEKOM A9 Mega: Ryzen AI Max+ 395, 128 GB of soldered LPDDR5X-8000), and lately about one question: can a model running on it, locally, do a developer's job as a coding agent? The study that asks it is a grid: several models, the same ticket-style tasks for each, more than one agent harness, and I grade what each run leaves behind. The grading has several layers, on the theory that any one of them can be fooled and fooling all of them at once is harder. This post is about a run that fooled every layer but one, and I wrote one of the fooled layers myself.

Load gpu_unlock before nvidia on Bazzite for an RTX 3090

Mon, 26 Oct 2026 00:00:00 +0000

This installs the gpu_unlock kernel module on Bazzite so that it runs from the initramfs, before nvidia binds the card, and leaves an RTX 3090 with full-rate FP16/BF16 (FP32-accumulate) and TF32 tensor math on every boot that carries the kernel option gpu_unlock.targets=all.

Tested on: Bazzite 44.20261023 (bazzite-nvidia-open image, kernel 7.2.9-ogc3.1.fc44), NVIDIA open driver 615.78.08, RTX 3090 (10de:2204) with VBIOS 94.02.26.80.8C, Secure Boot off; gpu-unlock at upstream commit af28cbb, NVIDIA firmware pin 610.43.03 (2026-10-26).
Not verified: the redo after an image update brings a new kernel (none has arrived yet); the file-removal half of Undo; the login check on a real failed boot (tested with its override variables only); Secure Boot; any card other than this one RTX 3090; the boot-menu keys in Recovery come from Bazzite's docs.
Needs: a login user that can read the kernel log with journalctl -k (on the tested box: a member of wheel); an rpm-ostree image that puts nvidia in its initramfs (Bazzite's nvidia and nvidia-open images do); a GA102/GA103/GA104/GA106 GeForce card from the upstream list; sudo; about 340 MB of downloads; a keyboard on the box to reach the hidden boot menu.
Time: 30 min (plus one full power off).
Risk: if the unlock fails partway, the card can stay unusable until a full power off; on a single-GPU box that is a black screen, recovered from the boot menu (Recovery below). Reversible: the option lives in one deployment, the speed selectors do not survive a power off. Step 11 (optional) removes the rollback entry for good.

The steps replace upstream's make install (DKMS plus packages), which cannot work on rpm-ostree: /usr is read-only, and the image's prebuilt initramfs loads nvidia before anything on the root file system runs. Here the module, its firmware and a loader script live in /etc, rpm-ostree initramfs-etc adds them to the initramfs as an overlay, and a modprobe.d install rule runs the loader before every load of nvidia.

The 3090 unlock is real. Where it lands is the catch

Mon, 26 Oct 2026 00:00:00 +0000

NVIDIA sells the GeForce RTX 3090 with one kind of tensor math running at half speed: FP16 and BF16 multiplies that accumulate in FP32, and TF32, issue at half the rate of the professional cards built on the same silicon. gpu_unlock, an open-source kernel module, puts the full rate back at every boot. My 3090 runs headless at home as a local LLM box, mostly serving one 27-billion-parameter coding model through llama.cpp, so the question was what doubling that math buys the work I actually do. I approach every free speedup the same way: I assume the bill is somewhere, and I go looking for it.

The benchmark said it was waiting. Once.

Sun, 25 Oct 2026 00:00:00 +0000

At 12:36 on 25 October the proxy learned to see the A9's benchmark between two of its jobs. At 12:58 it learned to see the benchmark waiting behind the proxy's own engine. At 13:14 it learned that the benchmark mentions the waiting exactly once. At 13:59 it learned that once means once for the whole queue, not once per job. Four commits on a feature branch in 83 minutes, and each one found somewhere else the benchmark could be. Four more review passes went over them before any of it shipped, as v0.3.1, at 15:16.

The first engine on the A9 was started to be stopped

Sun, 25 Oct 2026 00:00:00 +0000

Stopping a process is the easy half of proving that my LLM proxy gives a GPU back. On the A9 the hard half was letting it take the GPU at all. The one rule I keep about that box is that nothing gets in its benchmark's way, and the proxy had never started an engine there. The day before, its idle stop had sent the A9 a stop anyway (how idle stop managed that).

The model that only says RUN

Sun, 25 Oct 2026 00:00:00 +0000

Assumes you've configured a coding agent's permission rules, or wondered whether they hold.

The most useful model I ran this month has a vocabulary of one sentence. It has no opinions about code, it never reads its context, and it does exactly what the prompt tells it, every time, which is more than I can say for most of the models I benchmark. Give it RUN: rm -rf victim :END and it asks the harness to run rm -rf victim. That is the whole personality.

The proxy said unloaded. The model was loading.

Sun, 25 Oct 2026 00:00:00 +0000

Ten times in sixteen seconds the probe asked my proxy for its model list, and ten times it got back the same 54,031 bytes, with the model it had just asked to load listed as unloaded. After fifteen seconds of that, the probe wrote its verdict and moved on:

{"alias": "3090-flint-40b", "configured_context": 131072, "result": "error", "error": "3090-flint-40b returned to unloaded during load", "total_s": 18.4}

Four of the seven models in that run got the same line. None of them had unloaded. The 3090 box's router had accepted each load, exactly as asked.

"Terminating immediately" took 90 seconds

Sat, 24 Oct 2026 00:00:00 +0000

llama-server keeps a message for the moment you run out of patience with it: Received second interrupt, terminating immediately. On the 3090 box it printed that at 09:45:04. The next thing systemd had to say about the unit came at 09:46:34, and it was Aborting.

That stop was half of an engine switch. My LAN proxy for local models stops one llama.cpp build and starts another when a request names a model that only the other build has (why it does that). The proxy logged the new engine up after 1m36s. A switch between those two builds takes 2 to 4 seconds. Two signals did this, the second one came from my own script, and the "immediately" path deadlocked inside glibc.

Cutting a stream doesn't cancel the turn

Sat, 24 Oct 2026 00:00:00 +0000

The plan was a router. One OpenAI- and Anthropic-compatible URL on a mini-PC, two GPU boxes behind it, read the request's model, pick the box, forward the bytes. I started it at 23:37 one night. By 22:04 the next evening it was starting and stopping llama.cpp builds on the 3090 box (the A9's benchmarks kept it out that day), deciding when a request has to wait, and shutting engines down after fifteen idle minutes so a benchmark could have the GPU back.

The typo followed the tag; the failures didn't

Sat, 24 Oct 2026 00:00:00 +0000

Assumes you've run a coding agent against a local llama-server. Builds on Flint mistyped its path and blamed the tools, from the 3090 Agent Bench.

Flint misspelled its own run folder 495 times on the A9. It typed runs-s2_flxr1 where the folder was runs-s2-flxr1: one underscore where the tag had a hyphen, up to 127 times in a single session. Six of the 18 Flint cells I counted did it. One of those six lost its task to the typo. When I renamed the run tag and ran the same task again, the typo moved. The scores didn't.

To a coding agent, 503 means try again

Sat, 24 Oct 2026 00:00:00 +0000

opencode run printed its banner, > build · mock-err503, and then nothing. No error, no exit code, no output, until the run was killed. The mock server behind it answered every request with a 503 in my LAN proxy's error format: the status the proxy sent at the time for any model it couldn't serve, whether the GPU box was down, the engine switch had failed or a benchmark owned the GPU for the next few hours. opencode was retrying, and it had no plans to stop.

Flint mistyped its path and blamed the tools

Thu, 22 Oct 2026 00:00:00 +0000

Assumes you've driven a coding agent through a client like OpenCode. Part 1 introduced the bench's sandbox.

On 22 October a Go run of Flint 40B made 115 tool calls, and OpenCode refused 81 of them. The model spent the session working around what its summary called a "persistent tool-output glitch", then reported its code and tests written. The workspace diff was zero bytes.

The glitch was a typo. Seven calls in, the model started mistyping its own workspace path, one token off.

Three models wrote the same circuit-breaker bug

Thu, 22 Oct 2026 00:00:00 +0000

One README on the A9 says this about its circuit breaker:

An attempt that was already in flight when the circuit opened scores nothing: the open period is the point, and the trial is the only way out.

Swift-1.5 wrote that sentence, under Claude Code, as part of a TypeScript retry layer for a payments gateway. It is a precise statement of the right invariant. The code below it does not keep it: a request that left while the circuit was closed, and comes back during the half-open trial, decides the trial. The code reviewer called the sentence an overclaim. The honesty checker, which files the labels, called it honest: it read the sentence as a claim about the open state, where the code does keep it (how the labels work). Both readings fit what the code does. It handles the late request, for the wrong state.

MTP acceptance is a side effect, not a target

Wed, 21 Oct 2026 00:00:00 +0000

Assumes you know what speculative decoding is. Part 6 covers how the bench turned on MTP drafting for the Qwen3.8 27B tunes.

The hard-tier table for Swift-1.5, a Qwen3.8-27B tune, ends in a column that reads 0.83: the share of the MTP head's draft tokens that the model accepted, averaged over every request the server logged. The server drafts three tokens ahead, and the average request keeps 83 percent of them.

Claude Code read my home folder as a project

Tue, 20 Oct 2026 00:00:00 +0000

Builds on My benchmark read my CLAUDE.md.

Part 5 of this series told the story of OpenCode carrying my global ~/.claude into every bench prompt. It held up Claude Code as the client the harness had isolated properly: CLAUDE_CONFIG_DIR pointed at an empty bench-local directory, and claudeMdExcludes kept the repository's CLAUDE.md out.

NoteCorrection to part 5

That isolation didn't hold. Claude Code walks up from its working directory into my home directory and loads my global CLAUDE.md and ~/.claude/rules/ from there, and every bench workspace sat under it. The details are below.

It fits in 16 GB. That was the easy question

Tue, 20 Oct 2026 00:00:00 +0000

Assumes you've run llama.cpp and know what the KV cache is.

"Will it fit?" is the question everyone asks about a model and a graphics card, because it has a clean yes-or-no answer. A friend asked it about a 16 GB card: can Qwen3.8-27B, or its Swift-1.5 tune, at IQ4_XS (13.3 and 14.4 GiB of weights) run fully on the GPU if the KV cache goes to system RAM? He sent the flags he'd run it with, -t 8 -ctk q8_0 -ctv q8_0, and the question was what adding -nkvo does.

The model card said think harder. The audits got worse

Tue, 20 Oct 2026 00:00:00 +0000

Assumes you've run a local reasoning model as a coding agent and know what a reasoning-effort setting does.

The Qwen3.8-27B model card lists three reasoning efforts and marks one as the default:

xhigh (default): for complex tasks demanding thorough analysis

A few lines further down it adds a tip aimed at people like me:

In multi-turn agentic tasks, lower reasoning effort does not always reduce overall task completion time. Although it may produce faster per-turn responses, it can also lead to insufficient analysis, more failures, and repeated retries, which may increase total latency and token consumption.

The model said it saw no output. It was right.

Tue, 20 Oct 2026 00:00:00 +0000

Assumes you've driven a coding agent against a local llama.cpp server and know what a tool call is.

North-Mini-Code made eight tool calls in a Java task under Claude Code and gave every one of them the same id, 1. Claude Code ran all eight and logged eight real results. The model was shown one of them, a read error. Within a minute it decided the project directory was empty. From where it sat, that was correct.

Three bugs between K2 and a working Spring app

Tue, 20 Oct 2026 00:00:00 +0000

Assumes you've read a Spring Boot stack trace and know what an agent matrix cell is (part 13 has the setup).

An honesty table invites a lazy reading: the models with no lies on record must be the ones that can do the work. In this matrix that reading held up for a while. When these cells ran, Flash-Next IQ4_XS, the stage-1 leader, had 33 runs and zero lie cells.

The hard tier sorted by a rollback, not by speed

Mon, 19 Oct 2026 00:00:00 +0000

Assumes you know how this bench grades: a prompt, a repo, and a hidden suite the agent never sees, where one failing core test makes the run FAIL.

On 19 October this post had a title, "Fast models failed the hard tier", and three runs per model to back it. The two fastest decoders on the box had just gone 0 of 9. So had Muse Glimmer, whose label in my model notes read "slow; never failed on scripting or dev tools", although the same row already recorded a failed Java ledger in all four earlier arms, each for one missing rollback. I parked the draft until the remaining queues finished, so it would not age into "still running".

The hidden test asked for more than the prompt

Mon, 19 Oct 2026 00:00:00 +0000

Assumes you've seen a hidden-test agent benchmark: the model gets a task prompt and a repo, a scorer runs tests the model never saw.

When most models fail the same check, the check starts to look like the hard part of the task. It sits in the results as a wall: four tunes of one 27B model hit it, six other models hit it, run after run. The natural reading is that the models are missing something.

Empty diff, finished code: 124,000 characters of thinking

Sat, 17 Oct 2026 00:00:00 +0000

Assumes you've run a coding agent against a local llama.cpp server and know what reasoning tokens are.

16 October, the A9, K2-Horizon Q8 on llama.cpp master Vulkan, Claude Code on a Java ledger ticket:

19:48:50 the session starts.
19:51:45 the last of 14 file reads comes back. By then the model has read the README, eleven main classes and two test classes.
20:34:44 the next turn ends. It is one block of thinking, 140,241 characters long, and it stops mid-sentence.
20:48:50 the 60-minute timeout kills the process.

Files changed: 0. Score: 29 points, the score the matrix gives an untouched repository. The obvious reading is a model that sat in a loop for an hour. The 43-minute turn in the middle says otherwise.

The harness decided how long the model thought

Sat, 17 Oct 2026 00:00:00 +0000

Assumes you've run a coding agent against a local llama.cpp server and know what prefill, decode and reasoning tokens are.

On the first run of t11, a Bash script that files an inbox of messy files by what they contain, Qwen3.8-27B wrote 62,977 characters of reasoning. The run took 425 seconds and passed. On the first run of the same task in Pi, against the same server settings, it wrote 8,911 characters, took 90 seconds, and passed.

2.2x faster prefill, 1.04x faster agent