IOMMU off buys a 128 GB iGPU 5 %, and costs the NPU

The mental model went like this: the stock Fedora kernel boots the IOMMU in translated mode, so every GPU read of the 120 GiB of GTT memory on this box pays for an address translation, and iommu=pt takes that cost away. Round one of the A/B even seemed to agree. gpt-oss-120b's Vulkan prefill at 64k context came out 14.7 % faster with passthrough, then 4.0 % in a second pair.

The model was wrong at the first step. On the A9 Mega's stock boot the GPU is already identity-mapped. So is the NPU. iommu=pt moved 32 other IOMMU groups (NVMe, network, USB) and left the GPU's DMA path exactly where it was. Only amd_iommu=off changes it. Turning the IOMMU off buys about 5 % of prefill on Strata, about 4 % on ROCm, and costs the NPU, which refuses to start. The box stays on iommu=pt.

Which devices the IOMMU translates: read the group types#

The kernel publishes the answer in sysfs, readable without root. Every IOMMU group has a type file, and counting them gives the whole picture in one line. On the passthrough boot this box runs now:

$ cat /sys/kernel/iommu_groups/*/type | sort | uniq -c
      1 DMA
     34 identity

The stock translated boot, from the bench run's meta.txt, has 33 groups that translate (32 DMA-FQ, the flush-queue variant of a DMA domain, plus one strict DMA group) and two that don't:

iommu group types:       1 DMA      32 DMA-FQ       2 identity 

Round one recorded exactly that line, and the old explanation never asked which two groups were identity. On that boot they were group 22, holding the GPU at c6:00.0, and the NPU's group, holding c7:00.1. The two devices this whole experiment was about were the two the kernel had already left untranslated.

Finding which device sits in a group of a given type takes a loop over the group's devices/ directory and lspci -s on each entry. On the passthrough boot the odd one out is the single translated group:

$ for g in /sys/kernel/iommu_groups/*; do [ "$(cat $g/type)" = DMA ] && for d in $g/devices/*; do echo "group ${g##*/} $(cat $g/type): $(lspci -s ${d##*/})"; done; done
group 18 DMA: c2:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS525A PCI Express Card Reader (rev 01)

Under iommu=pt the only device still behind translation is the SD card reader. Swap DMA for identity on a translated boot and the same loop names the GPU and the NPU. That check is cheaper than any benchmark, and it would have killed round one's explanation before the first reboot.

Round one: a real number with a dead explanation#

Round one ran on 13 October: four Vulkan boots alternating translated and passthrough, plus a ROCm pair, an idle gate before each side, llama.cpp's Vulkan build (b11538) with five reps per row. gpt-oss-120b prefill at 64k:

• Pair 1: 112.2 → 128.7 tok/s, +14.7 %.
• Pair 2: 119.2 → 124.0 tok/s, +4.0 %.
• Drift: the two translated boots, same command line, twenty minutes apart, differed by 6.2 % on that row.

Decode, depth 0, Flash-Next and ROCm were all flat. The data still stands. The story built on it, translation misses on long KV-cache scans, assumed a translation that was not happening. It even made a prediction: Flash-Next, with only a few full-attention layers, should start to gain at 64k and 128k. Round two ran that test, and it came out +0.9 % and +0.4 %, inside the spread.

Round two: four boots, three IOMMU modes#

Round two ran on 14 October: passthrough at 19:12, translated at 19:53, amd_iommu=off at 20:33, passthrough again at 21:12. Same kernel (7.2.9-300.fc45), one bench pass per boot, load average under 1 and the GPU idle before every step. The toggle sets exactly one mode on every kernel entry, then reboots. From fix-19-iommu-mode-toggle.sh:

grubby --update-kernel=ALL --remove-args="iommu=pt amd_iommu=off"
case "$MODE" in
  pt)         grubby --update-kernel=ALL --args=iommu=pt ;;
  translated) ;;
  off)        grubby --update-kernel=ALL --args=amd_iommu=off ;;
  *)          echo "usage: $0 pt|translated|off" >&2; exit 2 ;;
esac

The rows cover the places round one and the engine notes pointed at: gpt-oss-120b on Vulkan at 32k and 64k, the same model on ROCm (kyuz0's rocm-10.0 toolbox image) at 64k, Qwen3.8-Flash-Next on Vulkan at 64k and 128k, and Flash-Next on Strata's hcq8 path with a 64,000-token prompt. llama-bench rows are pp512/tg128 with five reps. Strata gets five runs, and "warm" is the median of runs 2-5.

Prefill, tok/s:

The yardstick is the pt column against pt2: same setting, two boots two hours apart. On gpt-oss Vulkan at 32k they differ by 2.8 %. The Vulkan decode rows moved by 1.4-1.8 % between them (ROCm and Strata decode by under 0.5 %). Anything that size is drift until proven otherwise.

Passthrough vs translated: nothing for the GPU#

Every passthrough-against-translated row on Flash-Next, ROCm and Strata sits within ±1 % of the first pt boot, and inside the pt-to-pt2 drift against the second. Strata's warm prefill, 685.0 against 683.9, is the cleanest example: the engine whose repo carries the IOMMU-off report could not tell these two boots apart. That is what an unchanged GPU path should look like.

The one row that won't sit still#

gpt-oss-120b on Vulkan at 64k is the exception, and it has been for three pairs now. Passthrough came out ahead every time: +14.7 % and +4.0 % in round one, about +7 % in round two (117.2 and 115.5 against 108.4).

Three same-sign results look like an effect. But GPU translation cannot cause it, because there was no GPU translation to remove. And the boot with the IOMMU fully off, the one that does change the GPU's path, was the slowest of the four on this row at 105.1. It is also the noisiest row in the set, with a standard deviation around 5 % on every boot.

The cause is unknown. A passthrough boot moves the NVMe, the NICs and USB onto identity domains, and nothing in this data ties any of those to a Vulkan prefill row at depth. That is where the measurement stops.

amd_iommu=off: +5 % on Strata, +4 % on ROCm#

The kernel documentation's whole description of amd_iommu=off is "do not initialize any AMD IOMMU found in the system". On that boot sysfs lists zero IOMMU groups, and the uniq -c line comes back empty. The kernel log still prints iommu: Default domain type: Translated, so grepping for that line is not a check of whether the IOMMU is on.

With no IOMMU at all, the GPU's DMA finally takes a different path. Two rows moved past drift, and a third sits at its edge:

• Strata, Flash-Next at 64k: warm prefill +5.2 % (717.3 against 682.1), cold prefill +5.3 % (749.4 against 711.7), decode +2.3 % (45.35 against 44.3).
• gpt-oss-120b on ROCm: prefill at 64k +4.3 % (338.3 against 324.4).
• Flash-Next on Vulkan: prefill at 64k +2.5 %, at 128k +0.9 %: both under the 2.8 % drift yardstick.

That last 128k figure is inside drift, and the decode gains sit right at the edge of it: Flash-Next's tg at 64k went from 23.8 to 24.2, the size of the pt-to-pt2 wobble. The prefill gains on Strata and ROCm clear the 2.8 % drift, but not by a margin anyone should quote to a decimal.

Strata's own docs say its maintainers' box boots with amd_iommu=off, and call it a host setting the engine "does not need to run": no speedup claimed. The only number comes from a community laptop report in Strata's repo: 32k prefill went from 959-986 to 1,041-1,067 tok/s, about +6-11 %, across a reboot that also raised the GTT from 104 to 112 GiB, so not the IOMMU alone. This box measured about +5 % prefill and +2 % decode with only the IOMMU changed: same sign, smaller. Unlike round one's story, that report was at least about the right setting.

The cost: amdxdna refuses to start#

The same boot that bought the 5 % has no NPU. From the kernel log of the amd_iommu=off boot:

$ journalctl -k -b -1 --no-pager | grep -i amdxdna
Oct 14 20:33:35 a9-mega kernel: amdxdna 0000:c7:00.1: [drm] *ERROR* aie2_init: Running without IOMMU not supported
Oct 14 20:33:35 a9-mega kernel: amdxdna 0000:c7:00.1: [drm] *ERROR* amdxdna_probe: Hardware init failed, ret -22
Oct 14 20:33:35 a9-mega kernel: amdxdna 0000:c7:00.1: probe with driver amdxdna failed with error -22

No /dev/accel, and FastFlowLM's flm validate printed [ERROR] No NPU device found. and exited 1. The driver is not being fussy about a mode. Upstream aie2_pci.c checks whether the device has an IOMMU group at all and returns -EINVAL when it doesn't, and with amd_iommu=off there are zero groups on the whole system. The next passthrough boot brought /dev/accel/accel0 back.

Losing the NPU comes bundled with losing interrupt remapping and DMA isolation for every other device, the card reader included.

Verdict: iommu=pt stays, off is a documented switch#

The A9 stays on iommu=pt. For the GPU it is the same as the stock translated boot. It keeps the NPU, interrupt remapping and the one translated group, and nothing measured here says it costs anything.

amd_iommu=off is the only setting that changes the GPU's path, and it is worth about 5 % of long-context prefill on Strata and ROCm, and little or nothing measurable on Vulkan. That is real, and smaller than what it costs: an accelerator and the isolation for every device on the bus. It stays a known, scripted option: one argument to fix-19-iommu-mode-toggle.sh and a reboot, if a future workload ever needs the last 5 % more than the accelerator next to it.

The cheapest result in this whole post is the uniq -c line. Before benchmarking what an IOMMU mode does to a device, read the type of the group the device is in.

Filed under: amd, local-llm, linux, benchmarks

Read next#

• Vulkan or ROCm on Strix Halo? Ask at which depth — part 6: the backend gap that dwarfs any IOMMU setting at 64k.
• The benchmark was talking to the previous server — part 8: Strata, the engine whose repo holds the IOMMU-off report checked here.
• 128k context is the floor, and prefill is the wall — part 5: why a few percent of deep prefill was worth chasing at all.
• 120 GiB for the GPU, set in the kernel, not the BIOS — part 2: the GTT setup the GPU's DMA goes through.

Sources#

• Linux kernel parameters documentation — the iommu=pt and amd_iommu=off entries, fetched 14 October 2026.
• drivers/accel/amdxdna/aie2_pci.c — the Running without IOMMU not supported check, returning -EINVAL when the device has no IOMMU group; fetched 14 October 2026.
• Strata docs/STRIX_HALO.md — the maintainers' host settings (112 GiB GTT, amd_iommu=off), rev 61b3fb5d.
• Strata community report, Strix Halo laptop at 70 W — the 104 GiB + IOMMU on → 112 GiB + amd_iommu=off before/after row, rev 61b3fb5d.
• Benchmark data: llama-bench JSONL, Strata measure.py logs and per-boot meta.txt (kernel command line, IOMMU group types, start load) for every boot of both rounds, 13 and 14 October 2026, from the A9 Mega test repository.