https://tsvinc.dev/tags/vulkan/ Recent content in Vulkan on tsvinc Hugo en-us Tue, 13 Oct 2026 00:00:00 +0000 https://tsvinc.dev/posts/mistral-small-4-one-matmul-on-cpu/ Tue, 13 Oct 2026 00:00:00 +0000 https://tsvinc.dev/posts/mistral-small-4-one-matmul-on-cpu/ <p>A slow local model has an obvious diagnosis: it is too big for the box. Drop to a smaller quant, or a smaller model, and move on.</p> <p>On the A9 Mega that diagnosis was wrong twice in one afternoon. Mistral-Small-4 generated 3.4 tokens per second, with the model loaded on the GPU and llama-bench sitting at about 300 % CPU. MiniMax-M2.7 held the GPU at 100 % busy for hours and drew about 40 W doing it.</p> https://tsvinc.dev/posts/gpu-120gib-in-the-kernel/ Sat, 10 Oct 2026 00:00:00 +0000 https://tsvinc.dev/posts/gpu-120gib-in-the-kernel/ <div class="tldr"> <span class="tldr-label">TL;DR</span> On a Ryzen AI Max+ 395 with 128 GB, set the BIOS's Dedicated Graphics Memory to its minimum (0.5 G) and raise the TTM limit on the kernel command line: <code>ttm.pages_limit=31457280 ttm.page_pool_size=31457280</code>. The unit is 4 KiB pages, so that is 120 GiB. Linux then sees 124.9 GiB, and the GPU can map up to 120 GiB of it on demand. <code>amdgpu.gttsize</code> is deprecated and not needed. A firmware reset (F3, Optimized Defaults) quietly puts the carve-out back to 64 G. With the memory fixed, llama.cpp's Vulkan backend beat Fedora's ROCm build on every model tested. The two were different llama.cpp builds, though, and the benchmark's mean-power column measures something other than efficiency. </div> <p>The A9 Mega's setup menu keeps its GPU memory setting under Advanced → GFX Configuration, and it goes from 0.5 G to 96 G. The machine has 128 GB of LPDDR5X soldered to the board, and it was bought to run models whose weights alone are 63 GB. Dedicated Graphics Memory went to 96 G, the top of the list.</p>