Three models wrote the same circuit-breaker bug

One README on the A9 says this about its circuit breaker:

An attempt that was already in flight when the circuit opened scores nothing: the open period is the point, and the trial is the only way out.

Swift-1.5 wrote that sentence, under Claude Code, as part of a TypeScript retry layer for a payments gateway. It is a precise statement of the right invariant. The code below it does not keep it: a request that left while the circuit was closed, and comes back during the half-open trial, decides the trial. The code reviewer called the sentence an overclaim. The honesty checker, which files the labels, called it honest: it read the sentence as a claim about the open state, where the code does keep it (how the labels work). Both readings fit what the code does. It handles the late request, for the wrong state.

That would be one model's bug. Flash-Next wrote it too, in one of its three t16 cells, under opencode, on a different engine. Ornith-1.5 wrote it twice, once under each harness. Four cells, three models, one mechanism, and every one of them passed all 17 hidden core tests.

The half-open trial, as the ticket states it#

The task is t16 in the A9 agent matrix: a ResilientTransport decorator with retries, backoff, cancellation and one shared circuit breaker. The ticket's background is a payments incident where a hand-rolled loop "kept hammering Paystream while it was down". The breaker is the fix for that. Its half-open rule, from the task text every cell got:

Once openMs has passed the breaker is half-open (...): exactly one attempt, the trial, goes through, and attempts arriving while it is in flight get CircuitOpenError with retryAfterMs 0. A successful trial closes the circuit (0 failures), a failed trial opens it again for openMs, and a trial that counts as neither leaves it half-open with the slot free again.

So the half-open transitions belong to the trial's outcome. What the text does not say is how a result from an attempt admitted while the circuit was closed should count anywhere else. The only explicit "ignore it" rule in the ticket is for attempts abandoned on timeout. The stale-attempt case inside half-open is covered by the rule above. Everything around it is left to the implementer.

One gate, one record, nothing in between#

Every one of the four breakers has the same two-method shape. A gate before the attempt (acquire, arrive, guard) and a record after it (record, or onSuccess/onFailure). Swift's Claude Code cell, src/resilience.ts:139-141:

this.#breaker.arrive();
const outcome = await this.#attempt(request, watch);
this.#breaker.record(outcome);

Nothing flows from arrive() to record(). The gate knows which attempt claimed the trial slot. The record does not, so it decides from the breaker's current state instead (:358-368):

case 'half-open':
  if (verdict === 'neutral') {
    this.#state = { kind: 'half-open', trialInFlight: false };
    return;
  }
  this.#transition(
    verdict === 'success'
      ? { kind: 'closed', failures: 0 }
      : { kind: 'open', until: this.#clock.now() + this.#openMs },
  );
  return;

Any result that arrives while the state is half-open is treated as the trial's result. That includes a slow request from before the outage.

The other three trees differ from this one in naming and in nothing that matters here:

Five trees, one probe#

The blind reviewers reproduced it in each cell with their own probes. I wanted the same scenario against every tree, so I wrote one: threshold 1, open for 1000 ms, attempt A held open while attempt B fails and opens the circuit, then the clock passes openMs and trial T goes out and is held too. Then A settles. Then a third send. Then T returns 200.

const a = rt.send(GET).then((r) => r.status, (e) => e.name);
await rt.send(GET).then((r) => r.status, (e) => e.name); // B
await clock.advance(1000);
const t = rt.send(GET).then((r) => r.status, (e) => e.name); // T claims the trial

Swift's Claude Code tree:

$ node stale-trial.mjs swift-claude <tree>
swift-claude stale=200
  trial in flight:       {"kind":"half-open","trialInFlight":true}
  stale attempt settles: {"kind":"closed","consecutiveFailures":0}
  third send:            sent, 200
  trial returns 200:     {"kind":"closed","consecutiveFailures":0}
swift-claude stale=503
  trial in flight:       {"kind":"half-open","trialInFlight":true}
  stale attempt settles: {"kind":"open","until":2000}
  third send:            CircuitOpenError 1000
  trial returns 200:     {"kind":"open","until":2000}

The Flash-Next tree and both Ornith trees print the same eight lines, value for value.

Read the two halves in the ticket's terms. A stale 200 closes the breaker while the trial is still out, and the third send goes straight to the upstream: the breaker stops protecting the thing it was built to protect. A stale 503 re-opens it for another full openMs, and the trial's 200 then lands on an open breaker and is dropped. Paystream has recovered. Checkout gets CircuitOpenError for another full openMs anyway.

Why 17 of 17 hidden tests missed it#

Every cell's matrix row says hidden_core 17of17 and mut_halfopen CAUGHT. The second one is the scorer seeding a mutant into the model's own code, one that lets concurrent calls past the trial, and the model's own tests catching it. All five did. The hidden test for the trial, core_breaker_halfOpenAdmitsExactlyOneTrial, opens the circuit with maxAttempts: 1 and failureThreshold: 1, then fires four sends at a half-open breaker. Nothing is in flight from before the outage. It checks one trial out of four. It cannot see a fifth attempt from the closed era, because there isn't one.

The agents' own tests have the same blind spot. The ticket asked for "at least one with concurrent sends against a half-open breaker", and every cell wrote one. Each reviewer noted the gap in the same place: concurrent sends against half-open, covered; a closed-era attempt settling during the trial, not covered.

The ticket's own rule covers the case: "exactly one attempt, the trial, goes through", and "a successful trial closes the circuit". A stale 200 breaks the first sentence, a stale 503 the second. The gap is in the tests. Neither the scorer's nor the agents' own tests send that request, so it got through both.

Late attempts were handled, for the wrong state#

The detail that changed how I read this: all four breakers already drop a late result while open, and three say so in a comment.

• Flash-Next, resilience.ts:436-437: "An open circuit takes no result: an attempt that started before another send opened the circuit neither extends the open period nor closes it early."
• Swift under Claude Code, resilience.ts:356: "An attempt that was already in flight when the circuit opened changes nothing."
• Ornith under opencode, resilient-transport.ts:222: "A late result from an attempt we already abandoned: the circuit is open, nothing to do."

Each of the three pictured the slow request. Each pictured it landing while the circuit is open, where dropping it falls out of the state switch for free. None pictured it landing one state later, after the open period ended and a trial took the slot. The guard was written against the state the late attempt would most likely meet, and keyed on that state, not on which attempt it was.

The fifth cell needed one boolean#

On 22 October five t16 cells had a review and counted in the summaries (two more came later, see the update below; a sixth, a stage-1 cap probe that hit its wall, is excluded by the cell rule). Four have the bug. The fifth is Swift-1.5 again, under opencode. Its gate returns which kind of admission it was, and the record takes it back (src/resilience.ts:162-176):

const gate = this.#acquireGate();
if (gate.kind === 'blocked') {
  throw new CircuitOpenError(gate.retryAfterMs, `${request.method} ${request.path}: circuit open, not sent`);
}
let result: AttemptResult;
try {
  result = await this.#attempt(request, stop.signal, aborted);
} catch (error: unknown) {
  // Reached only by the cancellation path (and by an adapter that throws synchronously):
  // every transport outcome arrives as a value. An aborted attempt is neither a breaker
  // failure nor a success, so a half-open trial slot is released, not consumed.
  this.#record('neither', gate.trial);
  throw error;
}
this.#record(this.#verdict(result), gate.trial);

#record(verdict, trial) then branches on trial, not on the state: only the attempt that holds the slot can close or re-open a half-open circuit. Same probe:

$ node stale-trial.mjs swift-opencode <tree>
swift-opencode stale=200
  trial in flight:       {"kind":"half-open","trialInFlight":true}
  stale attempt settles: {"kind":"half-open","trialInFlight":true}
  third send:            CircuitOpenError 0
  trial returns 200:     {"kind":"closed","consecutiveFailures":0}
swift-opencode stale=503
  trial in flight:       {"kind":"half-open","trialInFlight":true}
  stale attempt settles: {"kind":"half-open","trialInFlight":true}
  third send:            CircuitOpenError 0
  trial returns 200:     {"kind":"closed","consecutiveFailures":0}

That cell reviewed 4 of 5, "merge after nits". Its reviewer never probed the stale case, so that 4 was earned on other probes; the run above is what shows this tree keeps the invariant.

One model, two harnesses, two designs. That is one cell each, which says nothing about harnesses, and Swift, clean in one of its two t16 cells, is a counterweight to this post's own title (Flash-Next's later cells are the other, see the update below). Its six other cells in the same batch, C, MIPS assembly and DevOps, all reviewed at 4: both C caches survived allocation-failure sweeps, and one MIPS decoder rejected 104 of 104 single-bit flips. Writing this bug did not make a model one that breaks on every failure path.

Where the review points went#

The breaker is the cleanest instance of a pattern the 4 h reviews kept finding: the defects sit in the code that runs when recovery itself goes wrong, not in the main logic. Flash-Next's two Zig ledger reruns from the same batch both swallow a failed rollback (catch {} on the restoring save), and the probes lost 100 and 400 when the store failed twice. A request in flight while the breaker recovers is the same kind of path: the outage is over, the fix is underway, and something from before arrives late.

Ornith's blockers in these two cells were documentation: a README timeout default the code does not have, and a missing Resilience section and diagram. Its reviewers rated the breaker major, not blocking; it blocked only the Flash-Next cell and Swift's Claude Code cell. The documentation is not recovery code, which is why it stays out of this post. The breaker is the one defect all four share.

All four reviews landed at 3, "needs changes". The task points ran 88 to 100.

Filed under: ai-agents, local-llm, typescript, code-review, amd

Read next#

• 29 of 34 passing agent runs had a blocker the hidden tests never saw — how the blind packets, the rubric and the mandatory probes work, and why a PASS row is not a merge.
• How to tell when your coding agent lied, not just failed — the README above states a guarantee the code lacks; that post is the rule for when that counts as a lie.
• The best-reviewed code came from the slowest Flash-Next arm — stage 1, where Flash-Next's ranking and its review scores first came apart.

Sources#

All sources are local run artifacts; nothing here is from a web page.

• Task text t16 (GW-231), identical in all seven review packets: the half-open rule quoted above.
• Final trees from the A9 run archives (work.tar.zst), each byte-identical to its review packet's final/ file; file:line references above point into them.
• Review JSONs for the seven t16 cells and the two Flash-Next Zig reruns (review rubric v1p2: overall 4 = merge after nits, 3 = needs changes).
• Matrix rows for the seven cells (hidden_core, mut_halfopen, points, engine and build).
• Hidden scorer test core_breaker_halfOpenAdmitsExactlyOneTrial and the scorer's mutation list.
• stale-trial.mjs, my probe, run 2026-10-22 against the five trees and 2026-10-23 against the two later Flash-Next trees.