proxy
- Bun's fetch had no way to skip the proxy
With HTTP_PROXY set, my agents found no local models: fetch sent the LAN request to the proxy. Bun 1.3.10 had no per-request way out, and end() lost the reply.
- The shutdown grace that never ran
A Go proxy's 5-second shutdown grace was dead code. A real drain and socket activation fixed restarts; KillMode, a WaitGroup and a lock needed fixes too.
- The proxy said unloaded. The model was loading.
A context probe gave up on four models loading behind my LLM proxy: its status was a snapshot up to 30 s old. Then a 336 ms stream timed at 120 s.
- The first engine on the A9 was started to be stopped
v0.3.1 would refuse any engine start on the A9 until its agent was replaced. Then my proxy started its first engine there, for a benchmark to take away.
- The design review found a Konsole tab
My own review of my LLM proxy found a measured MEDIUM. The design review's HIGHs included ssh riding a terminal tab and a help command that started the proxy.
- The benchmark said it was waiting. Once.
My LLM proxy must never pause the A9's benchmark queue. It couldn't see the queue between jobs, while it waited behind the proxy, or mid-handover.
- The fixes had the bugs
Seven re-review passes over a Go LLM proxy found 11, 8, 10, 4, 2, 1 and 0 bugs, nearly all in the fixes. Two could have crashed the process.
- Cutting a stream doesn't cancel the turn
A LAN proxy for two GPU boxes learned to switch llama.cpp engines. Cutting streams to switch faster failed: agent clients re-send the turn.