The 3090 unlock is real. Where it lands is the catch

NVIDIA sells the GeForce RTX 3090 with one kind of tensor math running at half speed: FP16 and BF16 multiplies that accumulate in FP32, and TF32, issue at half the rate of the professional cards built on the same silicon. gpu_unlock, an open-source kernel module, puts the full rate back at every boot. My 3090 runs headless at home as a local LLM box, mostly serving one 27-billion-parameter coding model through llama.cpp, so the question was what doubling that math buys the work I actually do. I approach every free speedup the same way: I assume the bill is somewhere, and I go looking for it.

The module's README sets the expectation: matrix-heavy work speeds up, with LLM prefill (reading the prompt) gains of 14–36% in the reports it links, while decode (writing the answer) stays about the same. On my card the matrix part came true at once. A cuBLAS (NVIDIA's math library) BF16 multiply went from 74.9 to 128 TFLOPS, same card, same system image, same afternoon. Then my daily coding model read a short prompt at 1,292 tokens/s before and 1,301 after. A card with 71% more BF16 math had produced a 0.7% faster model, and my first instinct was to work out which of the two was lying. Neither was. Both numbers are measured, both are correct, and the 0.7% is the one I use every day. The two are related through a list of conditions about as long as a firmware changelog, and this post is the audit of where the other 70 points went.

TL;DR — On an RTX 3090 at a 280 W power limit, the open-source gpu_unlock module lifts FP16/BF16 math with FP32 accumulation by 66–71% and TF32 by 74%. What a local LLM gets out of that depends on which math path its engine takes:

• llama.cpp with quantized models: prompt processing about 0% at short context, growing with context: +8.5–13% for my daily 27B from 32k to 256k, +5–9% at 64k for four MoE models. Generation 0%.
• llama.cpp with BF16 weights: +42–43% on prompts (+45% at 320 W). F16 weights gain nothing.
• vLLM: +37–52% on prompts, +15–31% on batched output, and more at 320 W (up to +59% and +37%).
• Strata (an engine that runs a 125B mixture-of-experts model at 2-bit on one consumer GPU, with most of the weights in system RAM): +8.5–13% on prefill, decode unchanged.
• One user on a 27B: MTP drafting (the model's own multi-token-prediction layer proposing tokens for the full model to check) adds +45–50% to generation on prose and close to double in agent coding runs (+83–91%), measured on the locked card, and +58–63% in my temp-0 run on the unlocked one, at a cost of 4–13% in prompt speed. A newer llama.cpp build read prompts 11–21% faster up to 64k and about 6% faster at 128k. Whether that build keeps the unlock's gain is untested.

The unlock: half-rate tensor math, fixed before the driver loads#

GeForce Ampere cards (GA102, GA103, GA104, GA106) run FP16 and BF16 tensor math with FP32 accumulation, and TF32, at half the issue rate of the professional cards built on the same silicon. gpu_unlock is a GPL-2.0 kernel module that undoes this from the initramfs, before nvidia binds the card. It runs NVIDIA's own boot firmware with a patched (V67) signature, writes two speed-selector registers (SS0 0x88888888, SS1 0x8), and resets the card with an FLR. Then the stock driver loads as usual.

It doesn't patch the driver, so driver updates don't break it. It does nothing unless the kernel command line carries gpu_unlock.targets=all. The selector values don't survive a power cycle, so the module repeats the unlock on every boot and after every resume from suspend to RAM.

The A/B was strict. Same image, same scripts, the only difference was whether the boot carried the option. Each results file records the kernel option, the module's gpu_unlock: unlocked log line (or its absence), the power limit and the temperature. Before every run, load average was under 1 and nothing else held the GPU. The Qwen3-8B rows are means of three repetitions and the Swift rows up to 64k means of two; the 128k and 256k rows, all MoE rows and every vLLM throughput cell are single runs, which I mark where they carry a conclusion. The raw-math check is an 8192³ cublasGemmEx in a CUDA 13.0 container:

Two rows deserve a second look. The last one is the control: FP16 with FP16 accumulation runs at about 129 TFLOPS locked or not, because NVIDIA never halved that path. Unlocked BF16 now runs at the speed the card could always do with FP16 accumulation. And unlocked BF16 measured 127.87 TFLOPS at 280 W and 127.88 at 320 W. Locked, the extra 40 W buys 3.2 TFLOPS. Unlocked, it buys nothing for this GEMM, because 280 W is already enough to saturate the card on it. A result that tells me to spend less electricity is rare enough that I trusted it at once, which was a mistake: vLLM, further down, disagrees.

One warning before anyone pulls the latest upstream. A refactor, fc83321, broke the unlock on RTX 3090s: #20 reports two 3090s failing on a GSP HALTED wait, and #23 a timeout on cards that weren't cold, which left them unusable by nvidia until a power off. Upstream reverted main to the working state (4189b17). My module is built from af28cbb, which has the working kernel code. Upstream's main has moved on since, and #20 was still open when I last read it. Read the tracker before rebuilding. Both issues were already on the tracker when the revert landed (#20 about three hours earlier, #23 about twenty minutes), and the revert's commit message doesn't mention the 3090 at all. A fix that won't say what it fixed sends me back to the tracker before every rebuild.

Bazzite: no DKMS, a prebuilt initramfs, and a modprobe rule#

The box runs Bazzite 44, an immutable Fedora Atomic (rpm-ostree) image, with NVIDIA's open driver 615.78.08. Upstream's make install builds DKMS and distro packages, and both assumptions fail here. /usr is read-only, so DKMS has nowhere to put the module. The initramfs ships prebuilt with the image, and the image forces nvidia into it, so nvidia binds the card before anything on the root file system gets a chance to run.

The route that works keeps everything in /etc. The module, its firmware and a small loader script go to /etc/gpu-unlock/. rpm-ostree initramfs-etc adds those files to the initramfs as an overlay, with no local initramfs rebuild. A modprobe.d rule, install nvidia /bin/sh /etc/gpu-unlock/load.sh; /sbin/modprobe --ignore-install nvidia $CMDLINE_OPTS, runs the loader before every load of nvidia, including the forced one inside the initramfs. The loader reads the gpu_unlock.* options from /proc/cmdline itself, because insmod doesn't, and it always exits 0, so nvidia loads whatever happens. It also sticks to shell builtins and absolute paths: that rule runs through /bin/sh -c inside the initramfs, and a missing binary there would stop nvidia from loading at all. My one design rule for a speed hack is that its worst day looks like no speed, never like no GPU.

The failure mode I care about is a kernel update. The module is built for one kernel, so an image update that brings a new kernel should make insmod refuse it: the card stays locked and nvidia loads as usual. No new kernel has arrived since the install, so that path is designed, not yet seen. That is the most optimistic phrase in systems work, and I am using it about my own shell script. A login check warns when the option is set but the boot didn't unlock. The full procedure, with recovery and revert, is in the manual: Load gpu_unlock before nvidia on Bazzite.

Where the doubled rate applies: FP32 accumulation only#

The unlock doubles one thing: tensor math that accumulates in FP32. Whether a model speeds up depends on how much of its work lands there, and that's a property of the engine's kernels, not of the model or the card.

Qwen3-8B makes the cleanest test, because one model can be run three ways in llama.cpp. I ran it as BF16, as F16, and as Q4_K_M requantized locally from the BF16 GGUF. Prompt processing, pp512, 280 W:

The source explains all three rows. In llama.cpp's CUDA backend (build f95b0d9), ggml_cuda_mul_mat_cublas keeps BF16 weights as BF16 and runs cuBLAS with CUBLAS_COMPUTE_32F, the halved path. F16 weights run with CUBLAS_COMPUTE_16F: FP16 accumulation, the path that was never halved. Quantized weights take llama.cpp's integer kernels for prompts and don't touch the FP16 tensor rate at all.

Which leaves a small irony. Locked, the F16 copy of the model was 37% faster than the BF16 copy (4,981 vs 3,628). The unlock lets BF16 catch up, to 5,184, but anyone already running F16 weights in llama.cpp had that speed all along. In llama.cpp, the unlock's headline win is letting one file format catch up with another. At 320 W the BF16 gain was +44.7%.

Long context: the attention half that accumulates in FP32#

My daily model is Swift 1.5, a Qwen3.8-27B tune, as IQ4_XS: quantized weights, so the integer kernels do the weight math. What's left is attention. llama.cpp's FlashAttention kernel for this card (fattn-mma-f16.cuh) declares its tiles plainly: the Q·Kᵀ product accumulates into float, the product with V into half2. The first half gains from the unlock, the second doesn't.

The KV cache type doesn't get in the way. With a q8_0 or q4_0 cache, fattn-common.cuh converts K and V to F16 before the kernel runs, so the prompt path hits the same FP16 tensor math as an F16 cache would. Attention's share of the work grows with context, while the weight math per token stays put, so the gain grows with context too:

The 128k and 256k rows are one run per side, so read their exact percentages loosely; the rise with depth holds across all five. The ceiling is low because this model is a hybrid. Its GGUF header lists 64 layers plus one MTP layer, with full attention in every fourth layer: 16 layers keep a KV cache, the rest carry a recurrent state. I covered that layout when I moved its KV cache into system RAM. Three quarters of the layers have no attention for the unlock to speed up.

Four MoE models, all on the GPU with a q8_0 cache (one run per side at each depth), tell the same story. At 0 context they land between −0.4% and +2.9%. At 64k: Ornith 1.5 35B-A3B +8.1%, Qwen3.6 35B-A3B +8.8%, gemma-4 26B-A4B +9.0%, LFM2.5 24B-A2B +5.2%. LFM2.5 gains least, and I can't tell you why from what I measured. I would rather leave that hole in the post than fill it with a reason that merely sounds right. Its header puts attention in 10 of 40 blocks, the same quarter as Swift's, so "fewer attention layers" isn't the answer.

Generation: memory-bound, so nothing#

Every llama.cpp generation row is flat. The largest move in any of them is +1.9%, and most are within ±0.5%. Generating one token for one user reads every active weight once, and the 3090's 936 GB/s of memory bandwidth sets the pace long before tensor throughput does. Doubling a rate the decode loop never reaches changes nothing.

Strata's decode wobbled more, −3.9% to +3.7% between locked and unlocked at 280 W. It decodes with speculative drafting, so its tokens per second moves about ±4% from run to run anyway. That's noise, and I'm filing it as noise.

vLLM and Strata: engines that live in FP32 accumulation#

vLLM v0.31.0 was the biggest winner. Its AWQ models run through Marlin kernels (the logs say Using MarlinLinearKernel for both AWQ checkpoints), and its BF16 model through BF16 GEMMs. The prompt-heavy test is 64 prompts of 2,048 tokens with 1 output token each; the batch-output test is 128 requests of 512 tokens in, 256 out. Each cell is one run:

Two things differ from llama.cpp. Batched output gains, because 128 sequences decoding together turn the weight reads into real matrix multiplies, and those are compute-bound again. And vLLM's gain grows with power: unlocked, vLLM runs into the 280 W limit, while llama.cpp's gains at 320 W stayed within 3.3 points of its 280 W ones, up or down. The GEMM table said 280 W saturates the card. For one kernel shape it does. For vLLM's mix of work it doesn't. I believed my own table for five sections, which is about my average.

Strata runs Qwen3.8-Flash-Next, which its README describes as a 125-billion-parameter model, here at Q2_0. Its prefill dequantizes weights to BF16 and calls cuBLAS with CUBLAS_COMPUTE_32F (src/prefill/gemm.cu), and its prompt-attention and selection kernels use FP16 and TF32 tensor-core MMA with FP32 accumulation. Prefill at 280 W, warm medians: +8.5% on a 4K prompt, +12.9% at 32K, +12.7% at 100K.

One user, one 27B: vLLM wins the prompt, llama.cpp wins the session#

The obvious next question was whether the unlock makes vLLM the better engine for me: one user, the Qwen3.8-27B family, long prompts. vLLM ran the community cyankiwi/Qwen3.8-27B-AWQ-INT4 with one sequence; llama.cpp ran Qwen3.8-27B as UD-IQ4_XS with a q8_0 cache. 280 W:

Locked, the two engines were tied. Unlocked, vLLM reads a 32k prompt about 34% faster than before and finishes the whole request in 29.8 s against 37.5. On this table alone, vLLM wins.

Then MTP. Qwen3.8 ships a multi-token-prediction layer, and llama.cpp uses it to draft tokens that the full model verifies. I measured it through a real llama-server with my daily serving flags (128k context, q8_0 cache, 3 draft tokens), temperature 0, 512-token answers, medians of two warm runs:

Here MTP cost about 5% of prompt speed and added 58–63% to generation. That's one point in a wide range: the gain depends on the sampler and on what is being written. Everything else I have on MTP comes from before the unlock, on the locked card, with 3 draft tokens:

The first four rows are from my earlier MTP study, where MTP also cost 13% of prompt speed on the short prompt and 4% on the long one. The last row compares two separate benchmark runs of my coding agents, token-weighted over every request, so it isn't a paired test. Roughly: +45–50% for prose, close to double for agent coding. The study used Swift; the agent runs used Swift, the base Qwen3.8-27B and a third Qwen3.8-27B tune; my run above used the base model. The unlock left generation within 2% in every llama.cpp row, so I expect MTP's gain to carry over to the unlocked card. That's an inference across two models and two methods, not a paired test. I also once found out the hard way what a benchmark without MTP looks like.

vLLM can't follow it there on this card. With MTP on, it died at startup with a CUDA out-of-memory error: its log said "Tried to allocate 2.37 GiB" for the MTP layer, on a GPU with 1.66 GiB free. Losing by less than a gigabyte is still losing, just with better bookkeeping. Without MTP, vLLM as a server refused a 33k context window, and its log put the most it could fit at 17,248 tokens. In llama.cpp, the same 27B family serves 128k every day. As a server with a 10k window and no MTP, vLLM read the 8k prompt at 1,628 t/s and generated at 41.5 t/s, against llama.cpp's 61.4 with MTP.

For one user on a 27B in 24 GB, llama.cpp with MTP wins the session: 61 vs 41.5 tokens/s generation and more than seven times the context. vLLM's case on this card is batch serving and smaller models, where the unlock pays it best.

The newer llama.cpp build: faster prompts, measured on the unlocked card only#

The last test wasn't about the unlock. With everything else fixed, I compared the llama.cpp build I'd been running (f95b0d9, 22 September) against a fresh master build (caa8191, 26 October), both on the unlocked card at 280 W:

The 128k and 256k rows are single runs; the −3.8% at 256k is one repetition and I wouldn't build on it. The MoE models gained 5–29% at 0 context (one run per side). Through the real server with my daily flags, the new build read prompts 13–15% faster and generated 1–4% faster with MTP. Each request also carried a retrieval check, a code buried in the prompt that the answer has to quote, and it passed on both builds.

Up to 64k, five weeks of llama.cpp commits sped up my prompts more than the unlock did, which is a sobering thing to find in a post about running boot firmware with a patched signature on my GPU; at 128k the build still gained 5.9%, the unlock 11.2%. Only the 256k q4_0 row went backwards, on one repetition. What I can't tell you is whether the two gains add up: I ran the new build only on the unlocked card, so how much of the unlock's gain survives on it is untested. I switched my daily llama.cpp to caa8191 and kept the old build as a fallback.

What stayed on#

The unlock stays. Every boot entry carries the option now: I verified the unlocked boot, then dropped the old locked deployment with rpm-ostree cleanup -r. Image updates copy the option and the overlay from the deployment they build on. The login check stays too. On a box that runs headless most of the time, the check that matters after an update is a journalctl -k | grep gpu_unlock.

The card gained 53 TFLOPS of BF16, and my daily model got 0.7% faster from it at short context. Somewhere there's a datasheet that considers this a success, and on its own terms it's right. On long sessions, where the waiting hurts, it got 8–13% faster, and on generation not at all. MTP speeds up the other half, generation, by about half to nearly double depending on the work, and costs a little prompt speed. The newer build read my prompts faster up to 64k than the unlock did. The rest of the 53 TFLOPS is there for whichever engine accumulates in FP32, and on this machine that's vLLM, for the day I serve more than one user. The household has one user, so I have booked the remaining TFLOPS as an asset and stopped expecting dividends.

Filed under: nvidia, local-llm, llama.cpp, benchmarks, hardware

Read next#

• It fits in 16 GB. That was the easy question — the same 27B's hybrid layout, and why its cache has to stay in VRAM.
• MTP acceptance is a side effect, not a target — a draft-length sweep on the same 27B, and why more drafts stopped paying.
• My benchmark served the model at half speed — what a missing MTP flag did to a whole benchmark.
• Load gpu_unlock before nvidia on Bazzite — the install, recovery and revert steps for rpm-ostree.

Sources#

• Zagitta/gpu-unlock — the module; README at af28cbb for what it does, the tested-card table and upstream's own GEMM and prefill numbers.
• gpu-unlock #20 and #23 — the fc83321 regression on RTX 3090s.
• llama.cpp at f95b0d9: ggml-cuda.cu (cuBLAS compute types), fattn-mma-f16.cuh (FlashAttention tile accumulators), fattn-common.cuh (KV cache converted to F16).
• Strata at 1678de3: src/prefill/gemm.cu and the README.
• vLLM, image vllm/vllm-openai:v0.31.0.
• Models: Qwen/Qwen3-8B, Qwen/Qwen3-8B-AWQ, unsloth/Qwen3-8B-GGUF, cyankiwi/Qwen3.8-27B-AWQ-INT4.