vainfo vaGetDriverNames error

[drhashes]

`Case 1: vainfo
Trying display: wayland
Trying display: x11

and failing detection:
libva error: vaGetDriverNames() failed with unknown libva error

Case 2: vainfo --display drm
All good when forcing to display drm.

Why is testing not done for drm in Case 1?
Should be:
Trying display: wayland
Trying display: x11
Trying display: drm

---

Case 1: vainfo
Trying display: wayland
Trying display: x11
libva info: VA-API version 1.21.0
libva error: vaGetDriverNames() failed with unknown libva error
libva info: User environment variable requested driver 'nvidia'
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/nvidia_drv_video.so
libva info: Found init function __vaDriverInit_1_0
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.21 (libva 2.21.0.pre1)
vainfo: Driver version: VA-API NVDEC driver [direct backend]
vainfo: Supported profile and entrypoints
VAProfileMPEG2Simple : VAEntrypointVLD
VAProfileMPEG2Main : VAEntrypointVLD
VAProfileVC1Simple : VAEntrypointVLD
VAProfileVC1Main : VAEntrypointVLD
VAProfileVC1Advanced : VAEntrypointVLD
VAProfileH264Main : VAEntrypointVLD
VAProfileH264High : VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointVLD
VAProfileHEVCMain : VAEntrypointVLD
VAProfileVP9Profile0 : VAEntrypointVLD
VAProfileHEVCMain10 : VAEntrypointVLD
VAProfileHEVCMain12 : VAEntrypointVLD

---

Case 2: vainfo --display drm
Trying display: drm
libva info: VA-API version 1.21.0
libva info: User environment variable requested driver 'nvidia'
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/nvidia_drv_video.so
libva info: Found init function __vaDriverInit_1_0
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.21 (libva 2.21.0.pre1)
vainfo: Driver version: VA-API NVDEC driver [direct backend]
vainfo: Supported profile and entrypoints
VAProfileMPEG2Simple : VAEntrypointVLD
VAProfileMPEG2Main : VAEntrypointVLD
VAProfileVC1Simple : VAEntrypointVLD
VAProfileVC1Main : VAEntrypointVLD
VAProfileVC1Advanced : VAEntrypointVLD
VAProfileH264Main : VAEntrypointVLD
VAProfileH264High : VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointVLD
VAProfileHEVCMain : VAEntrypointVLD
VAProfileVP9Profile0 : VAEntrypointVLD
VAProfileHEVCMain10 : VAEntrypointVLD
VAProfileHEVCMain12 : VAEntrypointVLD
`

[XinfengZhang]

suppose it is vainfo issue, should transfer to https://github.com/intel/libva-utils

[s1nceri7y]

I believe I have the similar error.

Cross-posting my old debug session.
At the time of debugging I've used the "linux-zen" and "nvidia" packages from arch repository. However, I didn't used nvidia-dkms package, and I'm also not sure if I had the "linux-zen-headers".
The error itself is still persists afterwards, but I didn't had a chance to debug again to see if it change anything inside.

I spent some time learning how to debug using gdb. I'm getting the -1 here, which produces the error.
Stack-trace

Just in case I'll leave the strace log here. log.txt

elFarto/nvidia-vaapi-driver#261 (comment)

[XinfengZhang]

reply #356 (comment)
the issue is caused by the
https://github.com/intel/libva-utils/blob/master/vainfo/vainfo.c#L526
success , but
https://github.com/intel/libva-utils/blob/master/vainfo/vainfo.c#L532
failed.
the protocol trying is in va_open_display, could not cover failed of vaInitialize

[XinfengZhang]

I spent some time learning how to debug using gdb. I'm getting the -1 here, which produces the error.
Stack-trace

from this description , looks
va_DRI3_GetDriverNames failed,
then it will try va_DRI2_GetDriverNames , it also failed?

[drhashes]

A work around is suggested here but would it not be better to fix this for real?

elFarto/nvidia-vaapi-driver#245 (comment)

[mirh]

Indeed, as I said in intel/libva#617 the rationale was completely misplaced.

[QuaternionsRock]

I have also been running into this issue, and I thought I'd post my debugging results here.

---

First, this behavior occurs irrespective of (and is arguably easier to observe without) environment variables:

$ vainfo
libva info: VA-API version 1.20.0
libva error: vaGetDriverNames() failed with unknown libva error
vaInitialize failed with error code -1 (unknown libva error),exit

$ vainfo --display drm
libva info: VA-API version 1.20.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/nvidia_drv_video.so
libva info: Found init function __vaDriverInit_1_0
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.20 (libva 2.12.0)
vainfo: Driver version: VA-API NVDEC driver [egl backend]
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileVC1Simple              :	VAEntrypointVLD
      VAProfileVC1Main                :	VAEntrypointVLD
      VAProfileVC1Advanced            :	VAEntrypointVLD
      <unknown profile>               :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointVLD
      VAProfileVP8Version0_3          :	VAEntrypointVLD
      VAProfileVP9Profile0            :	VAEntrypointVLD
      VAProfileAV1Profile0            :	VAEntrypointVLD
      VAProfileHEVCMain10             :	VAEntrypointVLD
      VAProfileHEVCMain12             :	VAEntrypointVLD
      VAProfileVP9Profile2            :	VAEntrypointVLD

---

Second, this is definitely caused by the X11 display hook.

The order of the display hooks in this array determines the order in which va_open_display tries them. Therefore, when no display argument is provided, vainfo will try the Wayland, X11, and DRM display hooks on a typical Linux system, in that order.

There are clearly no issues with the DRM display hook, meaning the Wayland and/or X11 display hooks' open_display functions must be returning a non-NULL VADisplay when it should probably return NULL. I tried both of them manually to see which is the culprit:

$ vainfo --display wayland
error: failed to initialize display 'wayland'

This error message is only printed if va_open_display is about to return NULL. I am using X11, this makes sense; the Wayland display hook's open_display function is is correctly returning NULL here.

$ vainfo --display x11
libva info: VA-API version 1.20.0
libva error: vaGetDriverNames() failed with unknown libva error
vaInitialize failed with error code -1 (unknown libva error),exit

So the X11 display hook's open_display function is definitely returning a non-NULL VADisplay, and vainfo gets all the way to the vaInitialize call before failing. This doesn't seem like great behavior to me, but I don't know enough about VA-API to formally question it. Regardless, it's worth noting that there's practically no way va_open_display_x11 will return NULL so long as XOpenDisplay doesn't return NULL (vaGetDisplay only returns NULL if one of the three calloc calls fails).

Anyway, as the error message notes, vaInitialize calls vaGetDriverNames here. For an X11 display, this points to va_DisplayContextGetDriverNames, which tries va_DRI3_GetDriverNames first, followed by va_DRI2_GetDriverNames. Unless HAVE_NVCTRL and/or HAVE_FGLRX are defined, that's it.

While it is tempting to recompile libva with HAVE_NVCTRL and be done with it, it appears to be considered "legacy. Additionally, recompiling libva doesn't help with snap packages, so doing so wouldn't enable Firefox hardware acceleration on my system.

I want to compare a few things with my laptop before I proceed. Will post an update soon.

[mirh]

I already linked the relative PR, and I already commented that the libva rationale was wrong.
I guess you can also argue that regardless the error could be far more graceful (surely GetDisplay working but no VA driver available, isn't some crazy unknown edge case?) but still.

As for firefox, of course recompiling system libraries doesn't affect the snap environment that ships their own.

[mirh]

I couldn't check this very thoroughly, but at least on the newer nvidia cards with ≥565 drivers they appear to satisfy the DRI3 querying now.