Suspend / resume on NVIDIA w. OpenRC

sunox · Tux's lil' helper Joined: 26 Jan 2022 Posts: 147

I'm just wondering if suspend/resume with nvidia-drivers using OpenRC has been figured out yet.

I have been able to suspend and resume successfully for the past few weeks, but my windows would all be empty on resume, and to get the content to come back I would have to interact with the window e.g. scroll a bit.

This was my semi-working configuration:

I had the NVreg_PreserveVideoMemoryAllocations=0 commented out in /etc/modprobe.d/nvidia.conf, and I have NVreg_PreserveVideoMemoryAllocations=1 set as a kernel parameter. It seems this kernel parameter does not apply, because in /proc/driver/nvidia/params there is a line PreserveVideoMemoryAllocations: 0. (Very weirdly, when testing later I noticed a change what happens during suspend/resume if I had the "=0" line in nvidia.conf uncommented.)

I had stupidly copied over the nvidia.sleep.sh script from /usr/bin into /lib64/elogind/system-sleep and did no modifications to it. The script would just fail after running the first few lines as the argument "pre" doesn't match anything.

So in short, I think I did nothing besides commenting a line out in nvidia.conf and adding a kernel parameter line that I think (?) had no effect.

This morning I started to experiment with the nvidia-sleep.sh script which gets called when I do loginctl suspend. I got it to write "suspend" to /proc/driver/nvidia/suspend when suspending, and "resume" when resuming. Unfortunately I was never able to get it to work. The best I've been able to do was to resume to a frozen desktop showing my wallpaper. That was from following the script given in the last post here (https://forums.gentoo.org/viewtopic-t-1133268-highlight-.html), which was inspired by something in this bug report (https://bugs.gentoo.org/693384). One time somehow the graphical session was on a different tty than the one it was launched from, and I could go to the original tty and see the wayland compositor was complaining of broken pipes.

Another very weird thing is I can't seem to get it back to the semi-broken-but-still-ok-I-guess state it was in a few hours ago, even though I am fairly certain I have undone all changes. At this point I would be happy-ish just getting back to the old state.

Any advice on this complicated issue really appreciated.

sunox · Tux's lil' helper Joined: 26 Jan 2022 Posts: 147

If I just do "echo mem | sudo tee /sys/power/state" I get exactly the same suspend/resume behaviour I had before I started today: Empty windows that have to be interacted with before they repopulate. I used to get this behaviour with loginctl suspend. I guess this might be some sort of clue.

Edit: I got back to my old state by complete fluke. I had an elogind script in system-sleep that unbind-ed the drivers on a usb hub on my motherboard while suspending and then binds them again on resume. This was because a while ago my system would resume quickly after suspending and I traced it back to devices on this usb hub. Anyway somehow if I do not have this script in system-sleep loginctl suspend will just seemingly turn the screen off but not suspend.

I guess I can go back to experimenting with custom scripts etc. now that I think this was getting in the way earlier.

Again advice welcome during this troubling time of partially broken suspend/resume.

Hu · Administrator Joined: 06 Mar 2007 Posts: 22857

echo mem > /sys/power/state instructs the kernel to enter suspend-to-RAM. The kernel will notify all relevant drivers on the way down, and on the way back up. Your problem looks to me like the nVidia driver does not handle suspend well, and needed the application to explicitly redraw the window because the original contents had been lost. Does this work properly if you blacklist the nVidia driver and use Nouveau? What version of the nVidia drivers are you using, and on what kernel?

sunox · Tux's lil' helper Joined: 26 Jan 2022 Posts: 147

I'm on the stable 535 driver and I'm on the newest kernel 6.7.1.

I might try to call the custom nvidia suspend scripts again, and play around with differenet preservememoryallocation values.

eeckwrk99 · Posted: Sun Jan 21, 2024 5:46 pm Post subject:

sunox · Tux's lil' helper Joined: 26 Jan 2022 Posts: 147

I was actually able to get it to work. The last thing I did was switch to the proprietary kernel modules and set the 'preserve memory' option as a kernel parameter. I am not sure which of these did the trick, but I suspect the former. Maybe it was both.

So in case anyone else stumbles on this here is my configuration. Of course some of these might be unnecessary (esp. #5).

1. Comment out the NVreg_PreserveVideoMemoryAllocations=0 " in /etc/modprobe.d/nvidia.conf
2. Put nvidia.NVreg_PreserveVideoMemoryAllocations=1 in kernel commandline.
3. Put this script in /lib64/elogind/system-sleep:

sunox · Tux's lil' helper Joined: 26 Jan 2022 Posts: 147

PreserveVideoMemoryAllocation=1 is requested by my Wayland compositor https://wiki.hyprland.org/Nvidia/#fixing-suspendwakeup-issues. Maybe this is a wayland thing, or something specific to that compositor.

gekk2 · n00b Joined: 18 Sep 2024 Posts: 2

Hi, I encountered the same issue with kernel versions 6.6 and later.

After some investigation, I found that the problem is linked to the following kernel option: Mitigations for CPU vulnerabilities > Mitigate RSB underflow with call depth tracking

This option was introduced in version 6.2 and is specifically intended for Intel CPUs based on the Skylake architecture (like mine).

Once I disabled it, the resume from suspend started working correctly with NVIDIA again.