benchmarks
- On a laptop iGPU, memory bandwidth picked the model
Coding agents on a Radeon 680M: a dense 27B generated 5.7 tokens a second whatever I tried; MoEs with 2-3B active on the GPU, 27 to 48.
- Two rounds, seventeen arms: the engine set the speed, not the quant
17 local-model setups on one 128 GB box. Q2_0 generated 2.2x faster than IQ4_XS; a same-file engine A/B puts most of that on the engine.
- Five languages, two reps: the round-two ranking
Five local models in six setups, 144 coding-agent runs on one 128 GB box. Flash-Next leads in both quants, and the bugs line up by ticket.
- Nine accusations, one lie: re-reading round two's judges
137 reviewed local-model agent runs, 14 "false" verdicts from two judges. On the raw sessions 9 fell, 4 were about docs, and 1 was a lie.
- The third run mattered where it cost the most
739 three-run groups from my agent bench: 19% split on basic tasks, 56% on Zig. So basic drops to two runs, and Zig runs only after the hard tier earns it.
- llama.cpp's MoE cache wants gigabytes and long prompts
On a 24 GB RTX 3090, llama.cpp's MoE expert cache read long prompts up to 85% faster, did nothing for a Q8 model, and left a 2-4x gap to Strata.
- Measuring OpenTelemetry Overhead Without Fooling Yourself
What OpenTelemetry costs a Spring Boot app in MiB and CPU per order, and the proofs each test cell needed before its number counted.
- The 3090 unlock is real. Where it lands is the catch
gpu_unlock lifted my RTX 3090's BF16 GEMM from 74.9 to 128 TFLOPS. My daily 27B in llama.cpp got 0.7% faster at short context. Here's where the rest went.
- The first engine on the A9 was started to be stopped
v0.3.1 would refuse any engine start on the A9 until its agent was replaced. Then my proxy started its first engine there, for a benchmark to take away.
- The benchmark said it was waiting. Once.
My LLM proxy must never pause the A9's benchmark queue. It couldn't see the queue between jobs, while it waited behind the proxy, or mid-handover.