The leak scan found me in my own guard's tests

· 8 min read
Contents

Publishing a private repository as a single orphan commit sounds like the safe kind of open source. No history goes out, so nothing buried in the history can leak: you choose the files, you commit them once, you push. It's the plan that looks best on a whiteboard, and that is exactly why I didn't let it go out on its own.

The repository holds my hardened configuration templates for six terminal coding agents that run on local models (OpenCode V1 and V2, pi, omp, Hermes Agent, Claude Code), plus the guard code that stops them running destructive commands. It's public now, under MIT, as one commit. On the first builds, the checks I'd put in front of that commit, plus one review of the plan, found six things. Each pointed at a real defect, two of them in the scanner itself. One of them was me, by name, in the test cases of that guard.

One orphan commit, built from a KEEP and DROP list#

The publisher is a script that lives in the private repository and builds one orphan commit from tracked files only. Every tracked file has to match a KEEP pattern or a DROP pattern. A file that matches neither aborts the run, so a file I add next month gets an explicit decision instead of a default.

What goes out: the templates, the shared guard code, the scripts, the tests, the docs, the research notes and the experiment scripts the research is built on (the "labs"). What stays in: the agent transcripts and journals (16 files, 22 MB), the work plan, handoff notes, the instruction files and skills I give my own agents, the CI file and a link log. Code comments stay in too, unlike my Go mirrors, which strip them.

The commit time is randomized into a night window.

Four gates before the commit, one around the push#

The checks run in this order, and any one of them stops the build:

  1. A forbidden-content scan: my LAN's address prefix, user and host names, my names and my e-mail domain, private group names. Each is matched as a substring and as a whole word.
  2. A Markdown link check: every relative link has to resolve inside the mirror, not inside the private repository it came from.
  3. Mirror-only text edits, each of which has to match exactly once. If I reword the source line an edit targets, the build fails instead of quietly shipping the old text.
  4. The repository's own fast test suite, run inside the built tree rather than the source.

Then the push: it happens only with --push, which refuses a tree with uncommitted changes, and it opens force push on the protected branch only for the length of the push itself.

That's the machinery. Here's what it caught.

Catch 1: the scan patterns flagged my security tests#

The first scan list was written from the shape of a leak rather than its content: 192.168., 10.0.0., /home/ and PRIVATE KEY. It matched 19 files, and every match was legitimate test content: guard rules that block ~/.ssh keys, and test cases full of private address ranges, because private address ranges are what those tests are about.

So the leak scanner's first report was a list of my security tests, accused of describing the things they exist to block. They do describe them. That's the job. A scanner that can't tell a rule against PRIVATE KEY from a private key can only be turned off or ignored, and a scanner nobody listens to is no scanner at all.

The fix was to scan for the real identifiers, the actual LAN prefix and the actual names, and stop scanning for generic shapes. That turned the scan from noisy into specific, and it found something on its first run.

Catch 2: my user name, inside five guard test cases#

The narrow scan found five guard test cases carrying my real user name and a real LAN address. Their shapes, with my details swapped for placeholders:

loginctl terminate-user <user>
rm -rf /home/<user>/x
/var/spool/cron/<user>
curl -T ... http://<LAN IP>/upload

Each is a case the guard has to judge: ending a user's session, deleting inside a home directory, a user's cron spool, uploading a file to a machine on the network. So the case with rm -rf inside a home directory spelled out the real path to mine, in a repository about to go public.

They're fixed in the source repository, not just papered over in the mirror: a placeholder user and a 192.168.1.x address, and every case still expects the same outcome it did before. The tests never needed to know who I am. They only needed a user and an address that looked like one.

Catch 3: a forbidden name that was also a file extension#

The whole-word match on one of the forbidden names also hit a common file extension in the research notes. I won't print the extension, for the obvious reason: it's spelled like the name. For a moment the scan had decided a file format was me. The word rule now skips a match that has a dot in front of it, which is how extensions are written and how names, in my experience, aren't.

The link check found six relative links pointing into dropped files: the work plan, agent notes and an audit's findings file. In the mirror, each of those would have led nowhere.

Two of them were more interesting. They sat in sentences in the README and in the safety doc that said the audit's minor findings were "still open". All of those findings had been fixed a week earlier. The sentences had outlived their truth by a week, and they were caught by a tool that only checks whether a link points somewhere. It was never asked whether the sentence around the link was still true; it just happened to be standing next to two that weren't.

I fixed those in the source, not only in the mirror. A mirror-only correction would have left my private copy saying something false about my own security work, and the private copy is the one I read.

Catch 5: a test for a folder I'd dropped#

The test suite failed inside the built tree. One test exercised a bootstrap hook for cloud sessions, and that hook lives in the agent folder the mirror drops. In the private repository that folder is there. In the mirror the test was checking something that no longer existed.

The fix was one of the exact-match edits from gate 3: the mirror drops that test. Running the suite in the built tree is why this showed up before a stranger cloned the repository and ran it.

Catch 6: evidence cited in plain text, invisible to the link check#

The last catch came from a review of the plan, not from a gate, and it found the one thing my link check was built not to see. My docs cite experiment scripts as evidence in plain text, not as links: in dozens of places, a sentence cites something like "checked by research/labs/sandbox/egress-probe/". A link checker looks at links. A path in backticks is just a word to it.

The first plan dropped the labs folder. That would have shipped dozens of claims, each pointing at proof that wasn't in the box, and none of my gates looks for a path written in plain text. Including the labs cost 145 files and 0.33 MB. I ran a wider scan over them, for any IPv4 address, any /home/<name> and anything shaped like an API key, and it found only 127.0.0.1, cloud-container paths, a test that uses the cloud metadata address, and 1.1.1.1.

One commit, public, and a source repo with two fewer stale sentences#

The mirror is public: one commit, MIT. An anonymous request to the API for the transcripts path returns 404, which is the one answer I wanted to hear as a stranger.

What I didn't plan for is where the fixes landed. I set out to protect the mirror from the private repository. Of six catches, two were wrong in the source itself and got fixed there: my name in the guard tests, and the two sentences that called fixed findings open. Two were my scanner being wrong, one was the mirror needing its own edit, and one was a gap no gate of mine could see. I built a filter for what leaves the house, and on its first builds it tidied the house. I'd call that a good start, if I could stop wondering how long a test case about rm -rf in my home directory had been carrying the path to it.

Filed under: security, git, ai-agents

Sources#