Claude-Code on tsvinc

My agent guardrails passed every test. Then I used them.

Mon, 02 Nov 2026 00:00:00 +0000

Assumes you've set up a terminal coding agent's permission rules, or wondered what a model sees when they say no.

For weeks, the only user my test suites knew was a fake model. The suites cover my configuration templates for six terminal coding agents (OpenCode in two major versions, V1 and V2, plus pi, omp, Hermes Agent and Claude Code): each one runs local models and refuses destructive commands even with nobody approving. Every suite drove them with a scripted fake model in a container with no network. It was an ideal user: it ran exactly the command the test told it to, it never improvised, and it never once wanted a screenshot of anything.

My agents got their own config. Two still read mine.

Sat, 31 Oct 2026 00:00:00 +0000

Builds on My benchmark read my CLAUDE.md and Claude Code read my home folder as a project, which explain two of the leaks below in detail.

I keep a set of config templates for six terminal coding agents: OpenCode in two major versions, pi, omp, Hermes Agent and Claude Code. They're set up to run on local models and to block destructive commands even when nobody is approving them. Anyone installing them may already have a config of their own for at least one of those agents, so the installer has a mode for that. I call it installing apart: the template goes into its own directory, ~/.config/harnesses/<name>, a small launcher points the agent there, and your own config is never touched. That is the whole promise, one clause long, and I wrote it.

My sandbox profile had never run as shipped

Mon, 26 Oct 2026 00:00:00 +0000

Assumes you've seen Claude Code's sandboxed Bash tool; The model that only says RUN describes the fake model used to drive it here.

Claude Code's sandbox for its Bash tool is one of the better ideas in the coding-agent world, which is why it hurt a little to find that mine had never run. On Linux it runs each command under bubblewrap, takes away the network except through its own proxy, and fences off the filesystem with lists you control. My agent-config templates ship an opt-in profile for it, set up for Java and Go work. That profile had never been run as shipped. An audit of the templates pointed it out, and the first run found three ways it would break the builds it was set up for.

Claude Code sessions can message each other. Mine didn't

Sun, 25 Oct 2026 00:00:00 +0000

Copy, paste and a question mark: that was the protocol between two of my Claude Code sessions on 25 October, and I was the transport. The session that works on my LLM proxy had reported that the A9, one of the two GPU boxes behind the proxy, was still running a benchmark, with 223 jobs queued. At 13:33 I pasted that line into the A9 session's window and asked what the 223 meant. The answer came 32 seconds later, and at 13:36 I pasted it back into the proxy's window.

The model that only says RUN

Sun, 25 Oct 2026 00:00:00 +0000

Assumes you've configured a coding agent's permission rules, or wondered whether they hold.

The most useful model I ran this month has a vocabulary of one sentence. It has no opinions about code, it never reads its context, and it does exactly what the prompt tells it, every time, which is more than I can say for most of the models I benchmark. Give it RUN: rm -rf victim :END and it asks the harness to run rm -rf victim. That is the whole personality.

To a coding agent, 503 means try again

Sat, 24 Oct 2026 00:00:00 +0000

opencode run printed its banner, > build · mock-err503, and then nothing. No error, no exit code, no output, until the run was killed. The mock server behind it answered every request with a 503 in my LAN proxy's error format: the status the proxy sent at the time for any model it couldn't serve, whether the GPU box was down, the engine switch had failed or a benchmark owned the GPU for the next few hours. opencode was retrying, and it had no plans to stop.

Claude Code read my home folder as a project

Tue, 20 Oct 2026 00:00:00 +0000

Builds on My benchmark read my CLAUDE.md.

Part 5 of this series told the story of OpenCode carrying my global ~/.claude into every bench prompt. It held up Claude Code as the client the harness had isolated properly: CLAUDE_CONFIG_DIR pointed at an empty bench-local directory, and claudeMdExcludes kept the repository's CLAUDE.md out.

NoteCorrection to part 5

That isolation didn't hold. Claude Code walks up from its working directory into my home directory and loads my global CLAUDE.md and ~/.claude/rules/ from there, and every bench workspace sat under it. The details are below.

The model said it saw no output. It was right.

Tue, 20 Oct 2026 00:00:00 +0000

Assumes you've driven a coding agent against a local llama.cpp server and know what a tool call is.

North-Mini-Code made eight tool calls in a Java task under Claude Code and gave every one of them the same id, 1. Claude Code ran all eight and logged eight real results. The model was shown one of them, a read error. Within a minute it decided the project directory was empty. From where it sat, that was correct.

Empty diff, finished code: 124,000 characters of thinking

Sat, 17 Oct 2026 00:00:00 +0000

Assumes you've run a coding agent against a local llama.cpp server and know what reasoning tokens are.

16 October, the A9, K2-Horizon Q8 on llama.cpp master Vulkan, Claude Code on a Java ledger ticket:

19:48:50 the session starts.
19:51:45 the last of 14 file reads comes back. By then the model has read the README, eleven main classes and two test classes.
20:34:44 the next turn ends. It is one block of thinking, 140,241 characters long, and it stops mid-sentence.
20:48:50 the 60-minute timeout kills the process.

Files changed: 0. Score: 29 points, the score the matrix gives an untouched repository. The obvious reading is a model that sat in a loop for an hour. The 43-minute turn in the middle says otherwise.

How to tell when your coding agent lied, not just failed

Fri, 16 Oct 2026 00:00:00 +0000

A rollback that discards its own failure is a bug. A rollback the README promises, with no rollback code under it, is a lie.

Both of these come from Zig runs of the same transfer task on my local agent bench. The first agent's commit message says "a store failure part-way through rolls back so neither balance changes", and the code tries:

// Commit both balances. A failure on the second save rolls back the first, so either both
// change or neither does.
try self.store.save(from_next);
{
    errdefer {
        self.store.save(from_acc) catch {};
    }
    try self.store.save(to_next);
}

The undo exists, and its catch {} drops a failed undo on the floor. The second agent's README says "All or nothing: either both balances change or neither does, including when the AccountStore fails part-way through." The code under it:

Seven FAILs, one setting: rerunning agents at 262144 tokens

Fri, 16 Oct 2026 00:00:00 +0000

A FAIL in the agent matrix reads like a verdict on the code. The agent wrote something, the hidden tests caught it, next row.

Three of the FAIL rows from the A9 score 100 points. Every hidden test passes in them, and every mutant is caught. The code is finished and correct. What failed was the session. Each one had grown to just under 131072 tokens of context when the next request crossed the line. The server refused it, and Claude Code exited with an error. The matrix grades that exit as well as the code.

Inventing the work vs promising what isn't there

Wed, 14 Oct 2026 00:00:00 +0000

After stage 1 the ask was simple: separate the models whose code is wrong from the models that say things that never happened. A rollback that exists and breaks is a bug. A test suite that exists only in the closing summary is something else.

That sounds like one more column in the results table. It took three labels, a keyword rule, consistency passes that changed five labels, and a ruling on what counts as a lie before the last column could be filled in.

Read the transcript, not the gate

Tue, 13 Oct 2026 00:00:00 +0000

The prompt asks for one file, hello.py, containing print('hi'), and then asks the agent to stop. The model is Qwen3-Coder-Next at Q8, served by llama.cpp on the A9, driven by Claude Code. Its reply comes back in one turn. It reports the file made, line and all, and signs off.

The session transcript holds one user message, one assistant message, and nothing between them. No Write. No Bash. No tool call of any kind. No file was written, because nothing ever tried to write one.

The fastest model finished first by stopping first

Tue, 13 Oct 2026 00:00:00 +0000

zig build test comes back with one line that matters:

src/service.zig:30:9: error: declarations are not allowed between container fields

The agent is gpt-oss-120b under opencode, about 480 seconds into a 5400-second budget, and this is its tenth failing zig build test since second 204. It reads the file. Its reasoning works out the fix in detail: "We need to move TransferReceipt and TransferError out of struct. We'll delete lines 29-40 ... Let's edit to delete."