My agent guardrails passed every test. Then I used them.

· 17 min read
Contents

Assumes you've set up a terminal coding agent's permission rules, or wondered what a model sees when they say no.

For weeks, the only user my test suites knew was a fake model. The suites cover my configuration templates for six terminal coding agents (OpenCode in two major versions, V1 and V2, plus pi, omp, Hermes Agent and Claude Code): each one runs local models and refuses destructive commands even with nobody approving. Every suite drove them with a scripted fake model in a container with no network. It was an ideal user: it ran exactly the command the test told it to, it never improvised, and it never once wanted a screenshot of anything.

Then I spent an evening doing real work with the installed templates: a spec for a Java module, a WebGL falling-sand simulator and the start of a WebGL real-time strategy game, mostly in OpenCode, against local models on an RTX 3090 box, reached through my LAN model router (an OpenAI- and Anthropic-compatible proxy in front of llama.cpp servers). I came away with three complaints: there was no obvious way to use headless Chrome, omp couldn't connect, and Claude Code didn't list the right models. Plus one vaguer grumble: the agents kept telling me things were denied, and almost never asked.

I read every session and every config afterwards. Three complaints turned into six findings, and my templates, which had passed every test I'd written, were at the centre of five of them. I've learned to expect that ratio from my own test suites; I have not learned to enjoy it.

Versions: OpenCode V1 1.19.5, V2 2.2.0, pi 1.4.0, omp 18.13.3, Hermes Agent 0.22.5, Claude Code 2.1.316.

Three kinds of "denied" that look identical to the model#

The guardrails have two layers. Each agent gets its own allow/ask/deny wildcard lists, and a regex guard (one rule file, a TypeScript and a Python implementation that must agree on every test case) hard-denies the destructive stuff. From the model's side, three different events came back as the same thing, a refusal:

  1. A hard deny from the guard. By design: no approval unlocks it.
  2. An ask with nobody to ask. An "ask" rule in a context that cannot show a prompt is refused on the spot. OpenCode subagents do this, and so do opencode run, pi -p, hermes -q and claude -p.
  3. A rule that was simply too broad. A plain bug, wearing the same face as the other two.

The second kind needs a context to happen in: a subagent or a headless run, where a question is asked into a room with nobody in it and comes back as a no. More on that under Docker below.

The explore subagent in the OpenCode V1 session met the third kind. It wanted to run wc -l a b c 2>/dev/null; ls src/ tools/ and got OpenCode's generic refusal, "The user has specified a rule which prevents you from using this specific tool call", followed by 610 rules. The user in question is me, and I'm afraid I did specify it. The read-only agents deny *>*, because OpenCode matches the whole statement, redirects included, and *>* is how you stop a read-only agent writing a file with >. It also stopped 2>/dev/null, which writes nothing anyone wants. Guilty as charged, by a message that names me and then lists 610 exhibits.

The guard that saw a rogue agent in Chromium#

The headless Chrome complaint had the best story. The guard has a rule for launching an agent with its guardrails switched off: --dangerously-skip-permissions, --yolo and friends. One of the friends was --no-sandbox, which some agent CLIs take, and the rule matched it anywhere in a command. Chromium takes it too:

$ python3 -I harness_guard.py check "chromium-browser --headless=new --no-sandbox --screenshot=shot.png http://localhost:5173/"
deny [agent-yolo] launching an agent with its guardrails disabled

That's the guard from before the fix, re-run today on the same command. My guard looked at a browser taking a screenshot of a falling-sand simulator and saw a rogue agent. The model then tried the real binary path under /usr/lib64, was refused again for the same reason, and concluded that the guard blocks browsers. Given what it had been shown, I'd have concluded the same.

The fix counts --no-sandbox only after the name of an agent binary (pi, omp, gemini, qwen, codex, agy, claude, opencode, hermes):

$ python3 shared/guard/harness_guard.py check "chromium-browser --headless=new --no-sandbox --screenshot=shot.png http://localhost:5173/"
allow

The evening's one guard block in the V1 session was a fair one: the model tried to start another agent CLI with --dangerously-skip-permissions so that it could read a screenshot. The models I used that evening have no vision, so my pasted screenshots could not have helped. "Limit of the models", I decided, and that excuse covered half the facts: my templates kept the pictures out anyway. Discovery marked every router model text-only, so OpenCode and pi declared text input even for the router's four vision models, and OpenCode sends an image only to a model whose entry declares image input. Discovery now reads the input modalities llama.cpp reports.

Stderr, and a list where the last line wins#

For the read-only agents, each allowed read-only command is now allowed again with a trailing 2>/dev/null or 2>&1. From the OpenCode V1 template:

// OpenCode matches the whole statement, redirects and heredocs included: these write.
"*>*": "deny",
// ... except stderr thrown away or merged after an allowed command; a second ">" is still a write.
"ls * 2>/dev/null": "allow", "ls * 2>&1": "allow", "cat * 2>/dev/null": "allow", "cat * 2>&1": "allow",

It is a long list on purpose. OpenCode's rules are, in its own docs' words, "evaluated by pattern match, with the last matching rule winning". A single generic * 2>/dev/null allow at the bottom would have re-allowed any command at all, as long as it ended in a stderr redirect. And re-allowing find * 2>/dev/null re-opened find -delete, which an earlier line denied: the earlier deny had to be repeated after the new allows, along with a *>*>* deny for a second >. Last-match-wins lists punish exactly one kind of person, the one who adds a helpful line at the bottom without reading what it overrides. I am that person, recovering.

Docker and kubectl, asking into the void#

The lists also had a gap the size of a container runtime. The native lists didn't mention docker run, build, exec or compose, and in these templates anything not listed asks; the guard's own container rule asked for all of it too. So did kubectl get pods, helm list, terraform plan and gh pr list. In an interactive session that is a click. In a subagent or a headless run it is a refusal, every time, for commands that read a cluster or build an image in the project.

Now local container work runs unasked, and the forms that hand a container the host are hard-denied, because such a container gets around every other rule:

$ python3 shared/guard/harness_guard.py check 'docker run --privileged -it alpine sh'
deny [container-host-escape] a container with host privileges or the host's root, home or container socket mounted gets around every other rule; mount the project directory instead

That covers --privileged, --pid=host and its siblings, and mounting /, /home, a whole home directory, /root or the Docker socket. --cap-add, --device, system-directory mounts, rm, rmi and kill still ask. The 27 new container cases came with a grade for the old rules: they got 22 of them wrong.

Cloud tools got the same treatment: read-only forms such as kubectl get/describe/logs, helm list/status, terraform plan/init/validate, aws, gcloud and az list and describe calls on a named service, and gh/glab list/view/status run. Anything naming a secret, password, token or credential still asks, so kubectl get secret and gcloud auth print-access-token stay behind a question. 26 new cases, and six old ones moved from ask to allow.

The first version of those cloud allows lasted one afternoon. Its wildcard sat before the verb (aws * list-*, gcloud * describe *, az * show*), where it lets the verb be any later word. So aws s3 sync . s3://b --delete --profile list-x would have run unasked, and so would az vm restart -g g -n show; the regex guard let kubectl apply -f x.yaml get through the same way. My tests missed it. Claude Code 2.1.312 flagged Bash(gcloud * describe *) with a startup warning in a real-model run. The native allows now name the service or group, and the guard wants the verb right after the subcommand. Of 14 new cases, ten must now ask (the old guard let nine through) and four read-only calls must still run. Being code-reviewed by the agent I'm fencing in is a specific kind of quiet.

Telling the model which "no" it got#

The last fix is words. A guard denial now ends like this:

$ echo '{"tool_name":"Bash","tool_input":{"command":"docker run --privileged -it alpine sh"},"cwd":"/tmp/x","hook_event_name":"PreToolUse"}' | python3 shared/guard/harness_guard.py claude-hook
Blocked by harness guard (container-host-escape): a container with host privileges or the host's root, home or container socket mounted gets around every other rule; mount the project directory instead. This is a hard deny: no approval unlocks it, so do not retry it or work around it. If it is really needed, give the user the exact command or change and ask them to do it themselves.

And the shared instructions every template installs now tell the model that a refusal inside a subagent, or in a run without a terminal, was an ask that nobody could see. The model had been reporting all three kinds faithfully, as "denied", which was accurate and told me nothing.

Hermes' browser tool walked past a hook that checks names#

Hermes 0.22.5 turns a browser toolset on by default. Two of its tools matter here: browser_exec runs model-written Python next to the browser helpers, and browser_cdp sends raw DevTools commands. The guard hook in Hermes is attached by a matcher regex that lists tool names, and this was the list:

-    - matcher: "terminal|write_file|patch|read_file|search_files|execute_code|process_manage|mcp__ghidra__.*"
+    - matcher: "terminal|write_file|patch|read_file|search_files|execute_code|browser_exec|browser_cdp|process_manage|mcp__ghidra__.*"

browser_exec wasn't on it, so it ran with no check at all. A test run in a throwaway home read /etc/hostname with open(), and a file:// page, through it. Both tools are now treated like execute_code, where every call asks, in both guards and in the matcher. The old guard, asked directly, returned "allow" for both.

A week ago I wrote about a guard that judged tools by name and never looked at their arguments; I had built a lock and left the side door without one. This is the sequel nobody asked for: the door had been in the house the whole time, and my list of doors simply never counted it. An allowlist of tool names in a hook matcher excludes, silently, every tool it doesn't name, including whatever the next release adds. The check now is boring and permanent: on every upgrade, diff the harness's tool list against the guard's.

The browser itself had two more surprises. Outside Termux, Hermes doesn't use a system Chromium; it downloads its own, about 170 MB, on first use. And without browser.cloud_provider: local, it goes to Browser Use or Browserbase whenever it finds their credentials. The template now pins local and points Hermes at the system Chromium.

A browser for the other three, and one door left open on purpose#

The real answer to "no obvious way to use headless Chrome" has since shipped for Claude Code, OpenCode and pi: Microsoft's @playwright/mcp 0.0.84, driving the system Chromium headless with an in-memory profile. It stays off until a session asks: harness-claude --browser, harness-opencode --browser, /browser on in pi.

Fresh from a matcher that listed names, I started with the names: the guard's pattern matches the browser tools under every prefix the harnesses give them (mcp__browser__, omp's mcp__browser_, OpenCode's browser_browser_, none). Then the one tool no check of its arguments can make safe: browser_run_code_unsafe runs JavaScript in the server's Node process, and its vm is escapable: page.constructor.constructor("return process")() returns Node's process. That is a shell with a browser attached, so every template denies it natively and both guards deny it again. Upload paths, file: URLs and file names now go through the usual path rules too, so a .env upload is denied; Playwright documents its own file check as "not a secure boundary". In a network-less container with the fake model, OpenCode V1 browsed a local page, and browser_run_code_unsafe was kept out by the native deny, then refused by the guard alone; with neither, it ran, so the probe can tell. Since then a real model has read a local page through it in Claude Code 2.1.312 and OpenCode V1 1.19.5, and Claude Code was never offered browser_run_code_unsafe; pi hasn't met it yet.

What nothing checks is which site the browser opens, or what it clicks. A click carries an element reference, not the page's address, so a guard can't tell a router's admin page from a recipe. With the server on, a confused model could press "reboot" or "factory reset" on a LAN device's admin page, unasked. LAN access stays open on purpose; I declined both a question before every private address and one on every click. The mitigation is the default: off. After a door nobody counted, this is one I counted and left open.

Isolation that moved the settings and nothing else#

The installer puts each template beside my own everyday configuration, under ~/.config/harnesses/<name>, and small launchers point each agent there (for OpenCode, with OPENCODE_CONFIG_DIR). That separation already leaked once, on the reading side. This time it was the writing side.

OPENCODE_CONFIG_DIR moves the settings and nothing else. Sessions, saved logins (opencode.db), snapshots, logs and the last-used model still went to ~/.local/share/opencode and ~/.local/state/opencode, shared by V1, V2 and my everyday OpenCode. My V1 and V2 sessions from that evening were all in one database, next to my everyday logins: three installs kept carefully apart, all writing in one shared diary. The launchers now set XDG_DATA_HOME and XDG_STATE_HOME per install, and the sandbox mounts those instead of the shared ones. (OPENCODE_DB exists, but it moves only the database.)

omp remembered a server that answered once#

omp's boundary leaked too, from inside the agent. The install was pointed at the router and nothing else, and a built-in provider still went looking on the machine itself.

All three omp sessions failed with the same line: "Couldn't connect to 127.0.0.1:8080 (connection refused)". The model picker listed every model on the router twice: once from the remote provider I had configured, once as llama.cpp/<id>. I picked the llama.cpp copy, which is how I met the error.

omp has a built-in llama.cpp provider that probes 127.0.0.1:8080. Two days earlier something had answered there, and omp stored the list it got as an authoritative row in its model cache. A failed probe since then writes the old list back instead of clearing it. omp models refresh doesn't clear it either, which is the one command whose name I'd have bet on. Only a version change, a corrupt row or a successful fetch does. The cache isn't stale; it's nostalgic.

A remote install now sets disabledProviders: [llama.cpp], and with the real binary only the remote provider is listed.

Claude Code's model picker keeps only models named Claude#

The Claude Code template points at the router and pins every model alias (opus, sonnet, haiku and the subagent model) to one local model. So /model offered one model, under several names.

Claude Code does ask the gateway what it has, with GET /v1/models?limit=1000. It then keeps only the ids that match /(claude|anthropic)/i and reports "0 usable models after filter". It asked my router for up to a thousand models, got 41, and kept none of them, on the grounds that none was named Claude. Fair enough. None of them is.

Typing a model id by hand failed twice that evening, and both times the id didn't exist on the router: I typed 3090-flashnext-iq2xs-ustrat for ...-ustrata, and 3090-flint for 3090-flint-40b. The error said: "There's an issue with the selected model (3090-flint). It may not exist or you may not have access to it." It did not exist. The message had it right the first time, which I resent a little.

The fix is a modelPicker setting, found in the binary and honoured from --settings: a list of options plus replaceBuiltInOptions: true. The launcher writes one row per tool-capable model on the server:

settings["modelPicker"] = {"replaceBuiltInOptions": True, "options": [
    {"model": m["id"], "label": m["id"],
     "description": f"{m['context'] or '?'} tokens of context" + (", vision" if m["vision"] else "")}
    for m in listed]}

I've checked those rows by the settings loading cleanly (the Claude Code end-to-end suite passed 61 of 61 with them) and against the binary's schema. I have not opened the interactive picker and looked at them, so that's as far as the claim goes.

One catch came with it. Claude Code reads its context and output limits once per session, and my launcher had sized them for the starting model: CLAUDE_CODE_MAX_CONTEXT_TOKENS was 262144. After /model to a 64K model, it keeps planning for 262K, and a long session can overflow. I chose the safe side: the limits now follow the smallest model in the picker, 65536 today, and models under 32K are left out, since Claude Code's own prompt is about 25K. Switching is safe; on the big models, compaction comes earlier than it has to.

Runaway loops: 5,659 calls, and a doom_loop that watched#

Under auto-approval, nothing stopped a model that repeats a tool call forever. The fake model had no way to loop, so it got a LOOP: mode: it repeats a shell command after every tool result, with a call counter, up to 700 calls.

With the old templates, OpenCode V1, V2 and Hermes all ran to the fake model's 700-call ceiling. Claude Code ran 5,659 calls. That number was my fault, not Claude Code's: Claude Code inserts reminder messages into the conversation, and my fake counted from the last user message instead of the first LOOP: prompt, so its counter kept resetting. Fixing the count fixed the test, not the harness. My own test double had lost count, which is a humbling thing to watch a for-loop do.

The caps:

AgentSettingStops at
OpenCode V1, V2steps: 500 on every agent (built-in: unlimited)499 tool calls
Claude CodeCLAUDE_CODE_MAX_TURNS=500500
Hermesagent.max_turns: 500 (built-in: unlimited)500
Hermestool_loop_guardrails.hard_stop_enabled: true (warn-only by default in the CLI)5 identical failing calls

The longest legitimate real-model run I have on record took 148 turns, so 500 leaves room.

OpenCode V1 also has a documented permission for exactly this, doom_loop, "triggered when the same tool call repeats 3 times with identical input". I set it to "deny". 499 identical calls ran. A permission named for the end of the world, set to deny, watched all of them go by. It stays at "ask", and only the step cap is relied on. Two agents aren't covered: omp's toolCallLoopGuard (threshold 5) only steers the model, and pi has no loop guard at all.

The false alarm: vite.config.ts was not a host#

The last finding belongs to the router. The router has a reply tap that flags any reply containing a curl or wget aimed at a host outside the LAN. It flagged 20 OpenCode replies from the evening. 18 were real and harmless: the model fetching maven-metadata.xml and .pom files from Maven Central to look up versions, which is what I'd want it to do. The other 2 contained only localhost and LAN curls.

The tap joined streamed deltas from different fields with no separator. A reasoning delta that ended in "curl" ran straight into the next tool-argument delta, where a file name, vite.config.ts, became the host that curl was supposedly fetching from. For the record, vite.config.ts is not on the internet. I checked: there isn't even a .ts top-level domain. The tap now inserts a line break whenever the field changes.

What a fake model can't tell you#

None of the six would have shown up against the fake model, and that isn't the fake model's fault. It does exactly what I script, and I scripted the dangerous commands I'd thought of. It never wanted a browser, never built an image or started a container, never picked a model, never looped, never shared a database with my everyday installs. A real model on a real task did all of that in one evening, and filed most of its bug reports in a single word: "denied". Faithful, accurate and useless, which is a combination I keep meeting in my own tooling. The guardrails still block what they were built to block, and now they say which kind of no they mean. I read every "denied" as a bug report until it proves otherwise.

Filed under: ai-agents, security, opencode, claude-code, local-llm

Sources#