dGPU with nvidia-drivers and Gnome Wayland

[Faraclas]

Hey guys,

I got an eGPU enclosure and an nvidia 3060 Ti and am trying to get it running in Gnome Wayland. The desired end result it to have my laptop screen and another display driven by the Intel Iris XE GPU (via a thunderbolt hub) and then have another monitor connected to the DisplayPort output of the nvidia card. This is working great in windows (dual-boot), but Windows makes me sad and I would like to get this going in my gentoo system. Here is what I have done so far:

* Added nvidia to make.conf

[logrusx]

Hello Faraclas,

Generally, NVIDIA doesn't care too much about Linux and especially Wayland. That's why the drivers are not up to par with their Windows counterpart and GDM won't start Gnome in Wayland mode if NVIDIA or hybrid graphics are detected.

You didn't mention if this is a laptop with hybrid graphics or a desktop.

According to Arch Wiki you can disable udev rules for GDM by

[Faraclas]

This is a laptop with only the built in Intel Iris XE onboqard. The nvidia card is in an eGPU enclosure connected via thunderbolt.

In my system there are not rules in /etc/udev/rules.d . However, I found them in /usr/lib/udev/rules.d and there is a 61-gdm.rules that exists with a bunch of tests that can wind up with wayland disabled. And it looks like this is happening.

With eGPU powered off on boot:

[Faraclas]

So issue number 1, is that I am definitely not in wayland. However in the Gnome X environment I wound up in, the nvidia card does show up in nvidia-setting (GUI) and in the CLI. In the nvidia-settings (GUI) while the card shows up, there are no instances of X Screens. This is probably because I normally am in Wayland and have never configured them.

$ nvidia-smi
Sat Jan 7 09:46:59 2023
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.60.13 Driver Version: 525.60.13 CUDA Version: 12.0 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... Off | 00000000:4F:00.0 Off | N/A |
| 0% 44C P8 12W / 200W | 1MiB / 8192MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+

elias@gentoo ~ $ sudo lshw -c display
*-display
description: VGA compatible controller
product: TigerLake-LP GT2 [Iris Xe Graphics]
vendor: Intel Corporation
physical id: 2
bus info: pci@0000:00:02.0
logical name: /dev/fb0
version: 01
width: 64 bits
clock: 33MHz
capabilities: pciexpress msi pm vga_controller bus_master cap_list rom fb
configuration: depth=32 driver=i915 latency=0 mode=3840x2160 resolution=3840,2400 visual=truecolor xres=3840 yres=2160
resources: iomemory:600-5ff iomemory:400-3ff irq:184 memory:6054000000-6054ffffff memory:4000000000-400fffffff ioport:4000(size=64) memory:c0000-dffff memory:4010000000-4016ffffff memory:4020000000-40ffffffff
*-display
description: VGA compatible controller
product: GA104 [GeForce RTX 3060 Ti Lite Hash Rate]
vendor: NVIDIA Corporation
physical id: 0
bus info: pci@0000:4f:00.0
version: a1
width: 64 bits
clock: 33MHz
capabilities: pm msi pciexpress vga_controller bus_master cap_list rom
configuration: driver=nvidia latency=0
resources: iomemory:600-5ff iomemory:600-5ff irq:185 memory:74000000-74ffffff memory:6000000000-600fffffff memory:6010000000-6011ffffff ioport:7000(size=128) memory:75080000-750fffff

Any suggestions?

[logrusx]

[Faraclas]

With respect to the defaults, the current nvidia-drivers do work fine in wayland. However, they are missing the functionality to sleep, hibernate, and resume. For this reason, nvidia asked the Gnome team to set the defaults for GDM to start X instead of Wayland. So, part one of getting this to work (in Wayland) would be to get into Wayland (reliably).

I edited the 61-GDM.rules file, to test, and the system I deed went I to Wayland. The monitor connected to the eGPU found the I put on scal and then stayed on with a greenish blank screen. The laptop monitor, and a 3rd monitor (connected to the laptop intel gpu) were both stuck on some terminal looking screen.

Step 2 will be to have the drivers get loaded properly such that this is avoided. I really want to use Wayland as I work on this machine daily and things like independent hdpi scaling work much better. If I need to dual-boot to use the nvidia card then I might as well use the windows partition (where the nvidia card works great, but everything else is miserable).

With respect to udev, there is literally nothing in /etc/udev/rules.d/ . I tried to put some rules in there and they were ignored. I'm really not sure which directory is the "install" and which is for "user edits". But typically isn't /etc used for install and /usr for users?

[logrusx]

Faraclas,

The way to edit udev rules is to create overriding files in /etc/udev/rules.d. Placing symlink to /dev/null is equivalent to negating them. Do not touch files that came from portage. And this is a general rule if you don't want to break your system.

To make nvidia sleep and hibernate, enable respective nvidia systemd services and change the following line in /lib/modprobe.d/nvidia.conf from 0 to 1, as described in the commented line above it:

[logrusx]

I just tried to use Wayland with the dGPU and it worked. I only tested sleep and it worked, however there are glitches with some applications and it's worse than it was an year ago.

Best Regards,
Georgi

[Fitap]