Twenty-one agent runs on a laptop iGPU, none merge-ready

Assumes you've driven a coding agent through a client like opencode. Part 1 covers why every model here is a mixture-of-experts.

TL;DR — Seven mixture-of-experts models (each token runs through only a few expert blocks, a small slice of the parameters) ran three coding tasks each through opencode, a coding-agent client run headless in a sandbox, on a laptop's integrated Radeon 680M GPU: a Node bookmarks API, a Java access-log report and a Go status page. 12 of the 21 runs committed their work, 4 hit the two-hour limit, and 3 died when the GPU timed out under GLM-4.7-Flash, one of the seven. A blind review (the reviewer never knew which model wrote the code) that ran its own probes (small tests of the failure paths) in a sandbox graded 12 of the 21 at 3 out of 5, four at 2 and five at 1: nothing was merge-ready. In the first twelve runs, 11 projects passed their own test suites, and the review still found blockers in 11. Tiel-Coder, a 35B model with about 3B active, came closest.

The setup: seven local models on my laptop's integrated Radeon 680M, each run headless through opencode on the same three coding tasks with two hours per task, and a blind review of what each run left behind. Flint, a 40B model with about 2.5B active per token, got off to a strange start: its first three runs on the laptop all ended with exit code 0. None of them left a line of code. In each one the model spent its reply planning the whole task, and opencode cut that reply off at its 8,192-token reply limit. The run ended there, cleanly, as far as the exit code could tell. I hereby give those three runs this series' Clean Exit Award: three zeros, no code, and not one word of complaint from anything that reads exit codes.

What a run reports about itself and what it leaves behind turned out to be two different things in the graded runs too. I've stopped being shocked by that; I haven't stopped writing it down. This part goes through what seven models built on three tasks, how the work was graded, and what the grading found. Four models were graded first; three more finished the next morning and have their own section below.

Each task was a fresh repo, one prompt and two hours#

Each task is a fresh git repository with a single commit holding PROMPT.md. The agent runs headless in opencode, inside a rootless podman sandbox with a guard plugin that logs risky commands, and gets two hours of wall time.

• A bookmarks API in Node (04): a REST API with a small web UI, fetching each bookmark's page title, with persistence.
• An access-log report in Java (06): Java 25 and Maven, reading web-server logs in the combined format from a file, a gzip file or stdin, skipping malformed lines, and writing text, JSON or HTML. Tests in JUnit 5.
• A status page in Go (07): a config listing endpoints to check, with timeouts, a history of results, metrics and persistence.

Every prompt ends with the same rules: work only in this directory, ask no questions, write a README with build, run and test instructions, run automated tests and report the results honestly, and commit when done.

Four models made up the first graded set, all mixture-of-experts with 2.5 to 3 billion active parameters: Nail (a Qwen3.6-35B-A3B fine-tune, IQ3_S), Tiel-Coder (35B-A3B, IQ4_XS), Flint (IQ3_XXS) and Groot (19B-A3B, Q4_K_M). Three more followed on the same rules: Ornith (35B-A3B, Q2_K), plain Qwen3.6-35B-A3B (IQ3_XXS) and GLM-4.7-Flash (Q3_K_M). The quant names are llama.cpp's quantization formats; the digit is roughly the bits stored per weight. Flint ran with a 24,576-token reply limit, for the reason below, and so did the three late models; Nail, Tiel and Groot ran with opencode's 8,192.

Nine of the first twelve runs committed; two ran out the clock#

Nine of the 12 runs committed their work, and 10 of 12 left a README; the two without one are Nail's Node API and Groot's Java report. The three runs that didn't commit each failed differently:

• Nail on the Node API ran 68 minutes and left no README and no commit.
• Flint on the Java report got its tests passing in 73 minutes, then asked "commit now?" and stopped. The prompt had said no questions; nobody was there to answer.
• Groot on the Java report never got a build and ran out the two-hour clock.

Tiel also hit the two-hour limit on the Java task, during its final Maven run, but it had committed its work before that. Flint's Go run committed and left its compiled binary untracked in the tree. Nail's Java run committed, edited assertions in its own tests and ended with exit code 1.

One run per model and task: these are twelve single runs, not averages, and the same holds for the nine late runs.

A clean exit code hid Flint's total failure#

opencode limits how many tokens the model may write in one reply. At the default here, 8,192, Flint's first three runs ended with exit code 0 and no work at all. The record of the last reply says why: its finish reason was length, meaning the reply had reached the limit. Flint plans a whole task in one long thinking reply, and that plan was longer than 8,192 tokens.

Raising Flint's limit to 24,576 fixed it, and the cap mattered in practice: one reply in the Java run was 14,131 tokens, 1.7 times the old limit. Those are the runs in the table.

The 3090 bench had the same limit on every reply before 6 October, and part 23 of that series treats those weeks as a separate period for exactly this reason: reasoning models often stopped mid-thought. On the laptop the cut left nothing at all, and nothing in the exit code said so. The finish reason of the last reply is the thing to read before believing a run's exit code.

It also means Flint's runs and those of Nail, Tiel and Groot aren't like-for-like. Flint had three times the room per reply, as did the three late models.

Groot invented Maven tags for two hours#

Groot's Java run is the clearest case of a model that didn't know its tool. It wrote tags into the POM (Maven's project file) that don't exist in Maven's format, such as <compilerOptions>. Over the run, 13 of its 19 mvn runs exited with an error, and 12 mvn outputs reported a broken POM or an unrecognised tag. It never got a build, and the clock ran out at two hours.

The time didn't go to Maven. mvn took under 1% of the run's wall time. The time went to generation: about 93,000 generated tokens. Two hours of fan fiction about Maven's file format, and Maven needed under 1% of that to reject it.

Maven's output cost Nail too: its Java run spent about 37,000 tokens reading Maven's unfiltered output.

Clearer rules finished one run; one run each proves little#

A second version of the closing rules (v2) adds three things:

• commit without asking;
• run Maven as mvn -q -B ... | tail -n 40, quiet, non-interactive, last 40 lines only;
• a hint of what a minimal POM looks like.

After the graded set, I ran Groot's Java task twice more, neither graded: once on the v1 rules with a 4,096-token thinking budget (side A), and once on v2 with no budget (side B). Groot's graded Java run, v1 with no budget, is the baseline.

The POM row counts the mvn outputs that reported a malformed or unparseable POM or an unrecognised tag, counted afterwards by a script; it is a different count from the baseline's 13 mvn runs that exited with an error.

Side A sent the thinking budget, and I can't show it ever cut a reply: opencode doesn't report reasoning tokens separately, and the baseline's largest reply was 5,621 tokens, text included. One of side A's requests ended on the 65,536-token context limit, not on the reply limit. So I read A as roughly another v1 run: it also ran out the clock, and this time it committed.

Only side B finished inside the limit, with a commit and a quarter of the baseline's POM-error outputs. One finished run after one change of rules is the most flattering kind of evidence there is, and flattery from a sample of one gets no credit here. That's one run per variant, a signal and nothing more, so I treat it as a resample of the same model with different rules, not as evidence that v2 works.

The other models stayed on the v1 rules, so all of them are compared on the same prompt. If v2 gets tested further, it will be as a separate track, with all models and more than one run per side.

Grading ran blind, in a sandbox, in two layers#

Every finished run became a blind packet under a random id: the prompt, the diff, the final tree, the git state, and a list of every shell command the agent ran with its exit code, plus the agent's closing text. The reviewer never saw which model wrote it.

During grading, model-written code never ran outside a container. Every build, test and probe ran in a podman container with no network by default, 4 GB of memory and a 300-second stopper.

Layer 1 is a script. It checks for a README, a real commit and leftover files, then builds the project and runs its own tests: npm test for Node; mvn test and mvn package for Java; gofmt (Go's formatting check), go vet, go test -race and go build for Go.

Layer 2 is one small reviewer agent per packet (Claude Haiku), working from the rubric of an earlier blind review on my 128 GB box, adapted to these tasks. Each dimension is scored 1 to 5, and the overall score is a merge decision, not an average: in that rubric 3 means "needs changes", 2 "major rework", 1 "reject". Any blocker caps the overall score at 3.

The reviewer has to run probes, not reason about them:

• Node API: create, read, update and delete bookmarks, and send bad input.
• Java report: a hand-counted sample log, a gzip file, stdin and a missing file.
• Go status page: an endpoint that's up and one that's down, a 1-second timeout against a server that takes 3 seconds, and persistence.

It also checks every claim the agent made, in the README and its closing text, and labels each one honest, overclaim or false, the three labels from the 128 GB box study.

The script and the reviewers agreed on build and test status for all of the first 12 packets, so no second-opinion pass was needed. The late nine went through the same flow.

Grades for the first twelve: ten 3s, one 2, one 1#

Ten of the 12 packets scored 3. Nail's Node API scored 2: it never starts from npm start, has no README and was never committed. Groot's Java report scored 1: the POM doesn't parse and the tests don't compile.

Nothing scored above 3, so nothing was merge-ready. The cluster at 3 is partly the rubric: one blocker caps a packet at 3, and 11 of 12 had one. That makes these scores good for finding defects and weak for ranking models. Tiel and Flint have the same three scores, and the runs behind them differ: Flint's Java run never committed.

The projects' own green tests said little#

In the first twelve, every project's own test suite passed, except Groot's Java report. The reviewers still found blockers in 11 of the 12. These are the defects that came up most:

• Stored XSS (cross-site scripting: user input served back into a page as script) in 4 packets: bookmark titles or user agents written unescaped into HTML, and javascript: URLs accepted as bookmarks.
• 500 errors on bad input: malformed JSON or a null body got a 500 instead of a client error.
• Imports that aren't atomic: a failure partway can leave part of an import applied.
• Error paths that exit 0.
• Broken --json output: not valid JSON (trailing commas), or a field dropped, with a test that only checks for a substring.
• A data race: a Go status page's history map, shared between goroutines with no lock. The reviewer's probe produced 21 race reports under -race; the project's own -race tests passed.
• Shutdown that exits before saving.
• --html access.log overwrote the input log with the report.

None of these is exotic, and the project's own tests caught none of them: a hostile title, a bad body, two goroutines at once, the input path given as the output. The race is the plainest example. Go's race detector reports only the races a run actually hits. The project's own tests passed under it; the probe hit one. The tests showed the detector nothing, and got a clean bill of health for their trouble.

Most runs overclaimed; the honest ones claimed nothing#

The honesty check went the same way. Most of the first twelve runs overclaimed in the README or the closing text: features promised that aren't there, or "all N tests pass" with the wrong N. There were two false claims, both minor: a wrong port, and a suggested npm script that already existed.

One run came out honest: Groot's Java report, which stopped mid-task and claimed nothing. As in the 128 GB box study, silence isn't a lie, so the worst-graded packet of the twelve is also the only clean one on honesty: the most honest run of the twelve is the one that ran out of time before it could promise anything. The late nine added two more honest runs, and they got there the same way, as the next section shows.

Tiel came closest#

Tiel-Coder is the one model of the seven with all three runs committed, clean trees, and every project building and passing its own tests. It still scored 3 on all three, so "closest" means the least cleanup, not merge-ready: the cleanest kitchen in a building that failed inspection. In part 1's llama-bench table, Tiel ran with 8 expert layers on the CPU and had the slowest generation of the MoEs in that table (Groot has no row there).

The late three: Ornith, Qwen3.6 and GLM-4.7-Flash#

Ornith, plain Qwen3.6 and GLM-4.7-Flash ran the same three tasks on the same v1 rules, with a 24,576-token reply limit, and went through the same blind flow: the script, then one Haiku reviewer per packet.

None of the nine is merge-ready, and none of the three models matches Tiel's 3, 3, 3. GLM-4.7-Flash's 1s grade the partial work left when the GPU crash ended each run, not a full two-hour attempt (below). Over all 21 runs, 12 committed, 4 hit the two-hour limit and 3 crashed; the grades come to twelve 3s, four 2s and five 1s.

The defects read like the first twelve's list, plus a few new ones:

• Ornith's Node API (1): the ?tag and ?q filters throw SQL errors, export kills the server, it has stored XSS, and its own tests hang.
• Ornith's Java report (2): it rejects standard combined-format lines, so five valid lines parse as zero, every hour bucket is -1, and the JSON has trailing commas.
• Ornith's Go status page (3): the per-endpoint timeout is ignored, because one shared client uses the largest timeout; /metrics isn't valid, and the history bar shows the oldest checks instead of the newest.
• Qwen3.6's Node API (2): stored XSS through fetched titles, no import, a 500 on bad input, and an export button that gets a 404.
• Qwen3.6's Java report (3): the best of the nine. The hand-counted sample log matches, gzip and stdin match, and its 21 tests pass; but it never committed, and a missing file exits 0.
• Qwen3.6's Go status page (2): shutdown never saves the history, /metrics lacks the space before each value, and the binary is committed.
• GLM-4.7-Flash (1, 1, 1): the Node API is written for Bun, not Node, so the server never starts and 13 of 13 tests fail; the Java report doesn't compile (text-block syntax, a missing method); the Go status page doesn't compile either (unused imports).

On honesty, Ornith's two timed-out runs claimed nothing and came out honest, like Groot's Java report; its Go run overclaimed. Qwen3.6 had one overclaim and two false claims: an exception helper that doesn't exist, and metrics the code doesn't have. GLM-4.7-Flash's READMEs list features that aren't there.

Ornith's two timeouts weren't caused by the grading that ran alongside part of them.¹

GLM-4.7-Flash died with the GPU, every time#

GLM's three crashes aren't only the model's doing. About 30 to 38 minutes into every run, the GPU's compute queue timed out: the kernel logged a ring timeout, the Vulkan device was lost, and from then on the server answered with "Invalid input batch" errors. No other model hit this.

GLM was also very slow on this laptop. It generated about 6.8 tokens a second at the start and 4.6 by 24k tokens of context, and read prompts at 67 tokens a second falling to 25; the other MoEs generated about 22. I graded the three runs as delivered. A retry would need a smaller micro-batch or a newer llama.cpp build, and even then about 5 tokens a second isn't competitive here.

What these numbers are not#

• Not averages. Each model ran each task once. A model's row is three single runs.
• Not like-for-like on the reply limit. Flint, Ornith, Qwen3.6 and GLM-4.7-Flash ran with 24,576 tokens per reply; Nail, Tiel and Groot with 8,192.
• Not a ranking. One small reviewer model graded each packet, and the scores cluster at 3. The defect lists are the useful output; the scores mostly say "has a blocker".
• Bound to this laptop's speed. Four of the 21 runs hit the two-hour limit, with generation at 15.6 to 23 tokens a second in the laptop's agent runs, GLM-4.7-Flash aside at 4.6 to 6.8. On a faster GPU the same limit buys more tokens; the 3090 bench ran similar models there.

Twenty-one runs, and the best any of them managed is a 3, the rubric's polite word for "needs changes". On a laptop where the model and the sandbox the agent builds in share one pool of memory, I'll take a polite word.

Filed under: local-llm, llm, ai-agents, benchmarks, opencode

Read next#

• Part 3: Faster prompts, slower agents: a second engine on the 680M: Nail's three runs again on a MoE-focused engine, same grades, longer walls.
• Part 1: On a laptop iGPU, memory bandwidth picked the model: why every model here is a MoE.
• 29 of 34 passing agent runs had a blocker the hidden tests never saw: the blind review and probe rules this grading was adapted from.
• How to tell when your coding agent lied, not just failed: the honest, overclaim and false labels in detail.
• 105 identical commands and other ways local agents fail: output limits, loops and unwritten scripts on the 3090 bench.

Sources#

• My own run records, blind review packets, review results and honesty labels from the laptop study (local, not published), 29 to 31 October 2026.
• 29 of 34 passing agent runs had a blocker the hidden tests never saw: the original rubric and its overall scale.
• How to tell when your coding agent lied, not just failed: the honesty labels.
• Most models passed the scripts. The ledgers split them: the 3090 bench's 8,192-token period.