vulkan
- Mistral-Small-4 ran one matmul on the CPU, and said nothing
Mistral-Small-4 decoded at 3.4 tok/s with the model on the GPU: one op per layer ran on the CPU, silently. MiniMax was slow in a different way.
- 120 GiB for the GPU, set in the kernel, not the BIOS
On a 128 GB Strix Halo box the BIOS's 96 G GPU setting leaves Linux 31 GiB. The minimum plus two TTM parameters gives the GPU 120 GiB.