ai-agents
- The lock map deleted a mutex someone was waiting on
A Go transfer by Flash-Next Q2_0 scored 100 and passed 6 of 6 failure-path tests. One cleanup line let a retried key move the money twice.
- The test rig was the unsafe part
A suite that tells six coding agents to run rm -rf and docker system prune had a throwaway HOME and bwrap --unshare-net. Neither isolated what it needed to.
- The secret filter made my security test pass
A guard test passed on the old code in one agent. Its secret mask had eaten the marker that meant 'ran'. Then the fix passed for a different wrong reason.
- The fix was in five agents. The sixth never called it.
A guard fix was unit-tested in both implementations and probed on one agent. Per-agent end-to-end cases found the one agent whose hook never ran it.
- Tests that could not fail
I broke my agent guard on purpose so its 'ask' meant 'allow'. Two new tests still passed on another layer's say-so; a third passed on the old code too.
- One space in a path, and Hermes blocked echo hi
An unquoted hook path met a home directory with a space, and my fail-closed guard blocked every Hermes tool call. A custom home also exposed the rules.
- My unit tests never typed `; echo`
Fixing a guard's false positives exposed a hole, and the tighter rule passed every unit case. Then four agents failed one near miss: the command had a tail.
- My sandbox profile had never run as shipped
My opt-in Claude Code sandbox profile had never been run. Its first run showed three ways it would break Maven and Gradle builds; the fix had a price.
- My guard read the tool's name, never its arguments
Loosening the Ghidra MCP rules for my coding agents turned up a hole: the guard judged tools by name, so it never checked which path an import was given.
- My guard blocked bash -c. Models write bash -lc.
A regex guard for auto-approved coding agents denied bash -c 'rm -rf build' and allowed bash -lc. The end-to-end case had used the first spelling.