The GPU wasn't to blame

· 13 min read
Contents

TL;DR — On Fedora, Mesa (the open-source graphics stack) ships an AMD driver for VA-API (the interface Linux programs use to hand video decoding to the GPU) that offers no H.264 and no HEVC decode, although the GPU does both. Fedora's Mesa keeps upstream's video-codecs=all_free default, and all_free builds only the codecs without patent encumbrance. RPM Fusion, a third-party repository for Fedora, ships mesa-va-drivers-freeworld, which adds H.264 and HEVC back, and libva, the VA-API loader, loads it before Fedora's driver; 64-bit programs using Mesa's OpenGL get RPM Fusion's build too. When you test, ignore ffmpeg's exit code (it is 0 when ffmpeg falls back to decoding on the CPU; only the log says so), and don't test the mpv player's --hwdec=vaapi with --vo=null, which gives the GPU decoder no device, so mpv decodes in software (vaapi-copy works headless).

This is a correction, and the person being corrected is me. In July I wrote that the GPU in my Fedora 44 desktop, an AMD Radeon 780M built into the processor, can't decode 10-bit HEVC video in hardware. It can. What stopped it was Fedora's build of Mesa, which leaves H.264 and HEVC out of its VA-API driver on purpose. And ffmpeg, the tool I checked with, reports a clean run while the CPU quietly does the decoding. The mistake sat in a footnote for more than three months, which is a long time to be wrong in writing.

It started with mpv, the command-line player. In July an mpv trace reported that hardware decoding of an HEVC stream was unsupported, on mpv's Vulkan decode path (hevc-vulkan). My July notes turned it into a sentence about VA-API and HEVC Main 10, the 10-bit profile. Then the post about Haruna, KDE's video player built on libmpv, turned the notes into a sentence about the hardware: "This GPU genuinely can't VA-API-decode HEVC Main10." Its footnote said it again, this time with ffmpeg's error attached as proof: hwaccel initialisation returned error. A game of telephone with one player, and I still managed to garble the message.

Each copy was more certain than the one before. None of the three had asked the kernel what the video block can do. And I wrote "genuinely" twice, which in hindsight is the word I reach for at the exact point where the checking stops.

In October I went back to fix the footnote, and this time I asked. The kernel had been willing to answer the whole time.

Three questions inside "can this GPU decode HEVC?"#

On Linux the question splits into three, and each one has its own witness. What the silicon does: ask the kernel. What the installed VA-API driver offers: ask libva. Whether the test in front of you used the hardware at all: read the log, not the exit code. The July claim was an answer to the second question, filed under the first.

vainfo wasn't installed, so the first two answers came from two short Python scripts that call the C libraries through ctypes, amdgpu_video_caps.py (27 lines) and va_profiles.py (32 lines). The box: Fedora 44, a Radeon 780M ("phoenix"), kernel 7.2.10, Mesa 26.2.5, libva 2.23.0, ffmpeg 8.1.3, mpv 0.41.0.

The silicon says yes#

The amdgpu kernel driver answers a query called AMDGPU_INFO_VIDEO_CAPS for anyone who can open the render node; libdrm wraps it as amdgpu_query_video_caps_info().

$ python3 amdgpu_video_caps.py
decode:
  H264  4096x4096 level 52
  HEVC  8192x4352 level 186
  JPEG  16384x16384 level 0
  VP9   8192x4352 level 0
  AV1   8192x4352 level 0
encode:
  H264  4096x4096 level 0
  HEVC  8192x4352 level 0
  AV1   8192x4352 level 0

HEVC up to 8192×4352, H.264 up to 4096×4096, and encode for both. The kernel's struct has no profile field, so this layer can't tell Main from Main 10. It can tell "decodes HEVC" from "doesn't", and the answer is that it decodes HEVC.

The driver says no#

libva doesn't load "the" driver. It walks a list of directories and opens the first radeonsi_drv_video.so it finds. The list on Fedora 44 is Fedora's own packaging choice: its libva.spec builds with -Ddriverdir="%{_libdir}/dri-nonfree:%{_libdir}/dri-freeworld:%{_libdir}/dri", and the last of the three is where Fedora's Mesa puts its driver (in mesa-dri-drivers; Fedora 44 has no separate mesa-va-drivers package any more).

RPM Fusion's driver is installed on this box now, so to show the state before the fix I pointed libva at Fedora's directory alone. LIBVA_DRIVERS_PATH replaces the built-in list, and the profiles it reports are the ones the box listed before the fix:

$ LIBVA_DRIVERS_PATH=/usr/lib64/dri python3 va_profiles.py
libva info: VA-API version 1.23.0
libva info: Trying to open /usr/lib64/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_23
libva info: va_openDriver() returns 0
Mesa Gallium driver 26.2.5 for AMD Radeon 780M Graphics (radeonsi, phoenix, ACO, DRM 3.64, 7.2.10-200.fc44.x86_64)
  JPEGBaseline               VLD
  VP9Profile0                VLD
  VP9Profile2                VLD
  AV1Profile0                VLD EncSlice
  None                       VideoProc

JPEG, VP9, AV1. No H.264. No HEVC, not even 8-bit Main. The hardware has both, and the driver Fedora ships offers neither. In July I had the right symptom and pinned it on the hardware, the one layer nobody had asked.

Fedora left them out on purpose#

Mesa's build has an option for exactly this. From meson.options in Mesa 26.2.5:

option(
  'video-codecs',
  type : 'array',
  value : ['all_free'],
  choices: [
    'all', 'all_free', 'vc1dec', 'h264dec', 'h264enc', 'h265dec', 'h265enc',
    'av1dec', 'av1enc', 'vp9dec', 'mpeg12dec', 'jpegdec'
  ],
  description : 'List of codecs to build support for. ' +
                'Distros might want to consult their legal department before ' +
                'enabling these. This is used for all video APIs (vaapi, ' +
                'vulkan). Non-patent encumbered codecs will be ' +
                'enabled by default with the all_free default value.'
)

Fedora's mesa.spec for f44 doesn't pass -Dvideo-codecs, so the default applies. That is a decision, not an oversight. The spec used to list the codecs, and a commit on 20 September 2022 removed the line. Its subject is "Drop codecs." and its whole explanation is two sentences: "We don't have legal approval for this. Previously it was accidentally shipped."

   -Dvulkan-drivers=%{?vulkan_drivers} \
   -Dvulkan-layers=device-select \
-  -Dvideo-codecs=h264dec,h264enc,h265dec,h265enc,vc1dec \
   -Dshared-glapi=enabled \

LWN's account of the change quotes the reasoning: behind the VA-API interface, Mesa does part of the decoding work in software, so the driver as shipped could infringe the codec patents. So on this desktop, HEVC decoding on the CPU traces back to a commit with a two-word subject line. I respect the economy.

ffmpeg reports a clean run either way#

The third question is the one the July post tripped over. To answer it without downloading anything, I generated 2-second 1080p clips with ffmpeg's testsrc2: HEVC Main 10, HEVC Main, and AV1 as a control, since the driver claims AV1.

ffmpeg -f lavfi -i testsrc2=size=1920x1080:rate=30 -t 2 -c:v libx265 -pix_fmt yuv420p10le main10.mkv
ffmpeg -f lavfi -i testsrc2=size=1920x1080:rate=30 -t 2 -c:v libx265 -pix_fmt yuv420p main8.mkv
ffmpeg -f lavfi -i testsrc2=size=1920x1080:rate=30 -t 2 -c:v libsvtav1 -pix_fmt yuv420p av1.mkv

Then a decode through VA-API, with libva pinned to Fedora's driver again:

$ LIBVA_DRIVERS_PATH=/usr/lib64/dri ffmpeg -hide_banner -v verbose -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i main10.mkv -f null -
[…]
[hevc @ 0x5604257bec40] No support for codec hevc profile 2.
[hevc @ 0x5604257bec40] Failed setup for format vaapi: hwaccel initialisation returned error.
[graph -1 input from stream 0:0 @ 0x7fb718004e80] w:1920 h:1080 pixfmt:yuv420p10le tb:1/1000 fr:30/1 sar:1/1 csp:unknown range:tv alpha:unspecified
[…]
[in#0/matroska,webm @ 0x5604256cc580]   Input stream #0:0 (video): 60 packets read (1211848 bytes); 60 frames decoded; 0 decode errors;

Read the last line first: 60 frames decoded, 0 decode errors. The exit status was 0. Thank you, ffmpeg. Very reassuring.

The two lines in the middle are the actual result. The driver has no profile for HEVC Main 10 (profile 2 is libavcodec's AV_PROFILE_HEVC_MAIN_10), so hardware setup failed. libavcodec then dropped the VA-API format from its list of candidate pixel formats and called get_format again, and the last format on that list is the software one. The graph -1 input line says where the frames landed: pixfmt:yuv420p10le, plain system memory. 8-bit Main fails the same way with profile 1. The AV1 clip goes through VA-API on the same driver, so the plumbing itself works; the driver just has nothing to connect HEVC to.

ffmpeg's documentation describes this accurately, if you read it as a contract: -hwaccel "has no effect if the selected hwaccel is not available or not supported by the chosen decoder." No effect. Not a failure, not a line in the summary, not a non-zero exit. ffmpeg asked for help, didn't get any, did the work itself and reported a clean run, which is what it was. I went looking for something to be angry at in that sentence and came back empty-handed, which is somehow worse.

RPM Fusion's driver, and what it costs#

RPM Fusion ships the same Mesa built with the codecs: mesa-va-drivers-freeworld, which installs into /usr/lib64/dri-freeworld/, the middle directory in libva's list. A dnf install --assumeno dry run showed one package, 53 MiB, removing nothing. Then:

sudo dnf install mesa-va-drivers-freeworld

It came from rpmfusion-free-updates as 26.2.5-1.fc44, the same Mesa version as Fedora's. Now libva stops one directory earlier:

$ python3 va_profiles.py
libva info: VA-API version 1.23.0
libva info: Trying to open /usr/lib64/dri-nonfree/radeonsi_drv_video.so
libva info: Trying to open /usr/lib64/dri-freeworld/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_23
libva info: va_openDriver() returns 0
Mesa Gallium driver 26.2.5 for AMD Radeon 780M Graphics (radeonsi, phoenix, ACO, DRM 3.64, 7.2.10-200.fc44.x86_64)
  H264ConstrainedBaseline    VLD EncSlice
  H264Main                   VLD EncSlice
  H264High                   VLD EncSlice
  HEVCMain                   VLD EncSlice
  HEVCMain10                 VLD EncSlice
  VP9Profile0                VLD
  VP9Profile2                VLD
  AV1Profile0                VLD EncSlice
  None                       VideoProc

H.264 and HEVC, Main and Main 10, decode (VLD) and encode (EncSlice). And one line is gone: JPEGBaseline. RPM Fusion's spec passes an explicit list, -Dvideo-codecs=h264dec,h264enc,h265dec,h265enc,vc1dec,av1dec,av1enc,vp9dec, with no jpegdec in it, so VA-API JPEG decode disappears when this driver wins the search. Nothing on this desktop that I know of uses it. It is still a trade, made in a spec file.

The package also reaches past VA-API. Its driver file is a link, and it installs a file for the dynamic linker:

$ cd /usr/lib64
$ ls -l dri/radeonsi_drv_video.so dri-freeworld/radeonsi_drv_video.so
lrwxrwxrwx. 1 root root 20 Oct 15 03:00 dri-freeworld/radeonsi_drv_video.so -> libgallium-26.2.5.so
lrwxrwxrwx. 1 root root 23 Oct 14 03:00 dri/radeonsi_drv_video.so -> ../libgallium-26.2.5.so
$ rpm -qf /etc/ld.so.conf.d/mesa-freeworld-lib64.conf
mesa-va-drivers-freeworld-26.2.5-1.fc44.x86_64
$ cat /etc/ld.so.conf.d/mesa-freeworld-lib64.conf
/usr/lib64/dri-freeworld/

$ ldd libEGL_mesa.so.0 | grep gallium
	libgallium-26.2.5.so => /usr/lib64/dri-freeworld/libgallium-26.2.5.so (0x00007fd5d6400000)
$ ldd libGLX_mesa.so.0 | grep gallium
	libgallium-26.2.5.so => /usr/lib64/dri-freeworld/libgallium-26.2.5.so (0x00007fc630800000)
$ ldd /usr/lib/libEGL_mesa.so.0 | grep gallium
	libgallium-26.2.5.so => /lib/libgallium-26.2.5.so (0xf456d000)

In both packages the VA driver file is a symlink to libgallium-26.2.5.so, and Mesa's EGL and GLX libraries link against that library by name. With the freeworld directory in the 64-bit linker's search path, every 64-bit program that loads Mesa's EGL or GLX library now gets RPM Fusion's build of the same Mesa release. 32-bit programs keep Fedora's: the x86_64 package that dnf installed adds only the 64-bit directory, and the last line above is the 32-bit EGL library resolving to /lib. RPM Fusion's repository also carries an i686 build of the package, with its own /usr/lib/dri-freeworld/; I didn't install it. (Pinning libva to Fedora's directory, as in the runs above, opens Fedora's symlink, so those runs map Fedora's own libgallium and really are Fedora's build.)

The same command as before, without the override:

$ ffmpeg -hide_banner -v verbose -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i main10.mkv -f null -
[…]
[graph -1 input from stream 0:0 @ 0x7f5fa4004ec0] w:1920 h:1080 pixfmt:vaapi tb:1/1000 fr:30/1 sar:1/1 csp:unknown range:tv alpha:unspecified
[…]
[in#0/matroska,webm @ 0x55bc56358580]   Input stream #0:0 (video): 60 packets read (1211848 bytes); 60 frames decoded; 0 decode errors;

The summary line matches the failed run apart from a memory address. What changed is above it: the No support and Failed setup lines are gone, and the graph -1 input line now says pixfmt:vaapi, frames that stayed on the GPU.

There is a catch in the dependencies:

$ rpm -q --requires mesa-va-drivers-freeworld
[…]
mesa-filesystem(x86-64) = 26.2.5
[…]

= 26.2.5 is an exact match on Fedora's Mesa version. RPM Fusion's changelog says the pin went in in 2023 "to avoid miss-match with fedora". The other side of it: when Fedora ships a newer Mesa before RPM Fusion rebuilds, the two packages have to move together. That is read from the dependency; I haven't watched an upgrade run into it, and I'm not volunteering.

mpv has the same trap, with a quieter log#

The first mpv check after the fix said software. The same command, re-run for this post:

$ mpv --hwdec=vaapi --vo=null -v main10.mkv
[…]
[vd] Looking at hwdec hevc-vaapi...
[vd] Could not create device.
[vd] Using software decoding.

Right driver, wrong test. The fix worked, and my first check of it said it hadn't: the July mistake again, this time on the far side of the fix. With --hwdec=vaapi, decoded frames stay on the GPU and go straight to the video output, and the mpv manual lists what that needs: --vo=gpu, --vo=gpu-next, --vo=vaapi or --vo=dmabuf-wayland. In the source, a non-copy hwdec gets its device from the video output's hwdec_devs; vo_gpu creates that list and vo_null never does. No device, Could not create device., software. A copy mode creates its own device, so it works without a window:

$ mpv --hwdec=vaapi-copy --vo=null -v main10.mkv
[…]
[vd] Looking at hwdec hevc-vaapi-copy...
[…]
[vd] Using hardware decoding (vaapi-copy).

With a real window (--vo=gpu) the log said Using hardware decoding (vaapi). There is one more asymmetry in mpv's source: Using hardware decoding is logged at info level, while Could not create device. and Using software decoding. are verbose. So without -v, this --vo=null fallback prints nothing at all. The other kind, a driver without the profile, is louder: mpv passes libavcodec's errors through at error level, so the No support for codec hevc profile 2. and Failed setup lines from the ffmpeg run show up in mpv's default output too.

What the Haruna post says now#

Part 6's footnote now blames the build, names the package, and says what the GPU does. Its body no longer says "genuinely". The same pass caught the footnote below it, which claimed xdg-mime was broken on Plasma 6. Retested with a scratch config directory, xdg-mime default org.kde.haruna.desktop video/mp4 prints qtpaths: command not found (it asks for Qt 5's qtpaths, and Fedora 44 ships only qtpaths6), exits 0, and writes the handler into mimeapps.list anyway; xdg-mime query default video/mp4 reads it back.

Both corrected footnotes are about a tool that exits 0. xdg-mime prints an error and does its job. ffmpeg does a different job from the one I asked for and prints a summary with zero errors in it. In July I read an error message as a verdict, twice.

Part 6 has lost its "genuinely", and in this post the word appears only in quotation marks, like a specimen in a jar. If it turns up loose in a later post, that is the sentence to check first.

Filed under: linux, fedora, amd, vaapi, ffmpeg

Sources#