llama.cpp
- To a coding agent, 503 means try again
What Claude Code, opencode and Node clients do with a self-hosted LLM endpoint's 503s and silences, and the status codes that stop the retry loops.
- Cutting a stream doesn't cancel the turn
A LAN proxy for two GPU boxes learned to switch llama.cpp engines. Cutting streams to switch faster failed: agent clients re-send the turn.
- "Terminating immediately" took 90 seconds
systemd and a wrapper's EXIT trap sent llama-server two SIGTERMs. Its Ctrl+C-twice exit() ran inside the signal handler and deadlocked in glibc.
- MTP acceptance is a side effect, not a target
A friend said MTP acceptance near 0.8 meant an idle GPU and I should draft more. I swept draft length on a 27B. Acceptance fell; speed peaked early.
- It fits in 16 GB. That was the easy question
Qwen3.8-27B with its KV cache in system RAM fits a 16 GB card. Generation drops from 42 to 15 tokens/s at once, and to 1.5 at 200k context.
- My benchmark served the model at half speed
The launcher turned on MTP drafting for five models. The benchmark's own server script didn't, and its result rows had no field that could show it.