| View previous topic :: View next topic |
| Author |
Message |
F. Russell
n00b
Joined: 12 Oct 2017
Posts: 14
|
 Posted: Wed Jul 31, 2024 1:55 pm Post subject: NVidia Open Source Requires KVM (Kernel Virtual Machine) |
 |
|
I have an NVidia card based on the Turing architecture and I want to install the open source drivers. However, when I attempt to emerge nvidia-drivers with USE='kernel-open" I receive the message that MMU_NOTIFIER must be enabled in the kernel which in turn requires enabling KVM, or the kernel-based virtual machine.
Since I have no need for a virtual machine I have never enabled it in the kernel. Also, I always have virtualization disabled in my BIOS settings.
Why does NVidia suddenly require KVM for its open-source driver when KVM is not and has never been required for its proprieary driver?
I will have to enable KVM/MMU_NOTIFIER in my kernels to install NVidia but do I also have to enable virtualization in the BIOS? I have no plans to ever implement a virtual machine.
Any insight or advice will be much appreciated.
_________________
F. Russell |
|
| Back to top |
|
 |
wjb
l33t
Joined: 10 Jul 2005
Posts: 629
Location: Fife, Scotland
|
| Posted: Wed Jul 31, 2024 3:18 pm Post subject: |
|
|
The ebuild message is
| Quote: |
local ERROR_MMU_NOTIFIER="CONFIG_MMU_NOTIFIER: is not set but needed to build with USE=kernel-open.
Cannot be directly selected in the kernel's menuconfig, and may need
selection of another option that requires it such as CONFIG_KVM."
|
So MMU_NOTIFIER is required, and a way to achieve this is to set KVM.
Check the config dependencies for alternatives to setting KVM? |
|
| Back to top |
|
|
F. Russell
n00b
Joined: 12 Oct 2017
Posts: 14
|
| Posted: Wed Jul 31, 2024 3:48 pm Post subject: |
|
|
| Quote: | So MMU_NOTIFIER is required, and a way to achieve this is to set KVM.
Check the config dependencies for alternatives to setting KVM? |
AFAIK, there is no other way other than to enable KVM.
This web page describes the dependencies of MMU_NOTIFIER:
https://www.kernelconfig.io/config_mmu_notifier
I don't undestand the details, but MMU_NOTIFIER seems directly related to virtual machine environments and nothing else.
It appears that I would need to enable KVM and then CONFIG_KVM_GENERIC_MMU_NOTIFIER in order to get MMU_NOTIFIER and I may still be missing some details.
Since I've never implemented VMs in the past I need to reseach this area, but it still seems odd that NVidia would suddenly require a virtual machine environment.
_________________
F. Russell |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22619
|
| Posted: Wed Jul 31, 2024 4:23 pm Post subject: |
|
|
| As wjb explained above, nVidia does not require a virtual machine environment. It requires MMU notifier support to be enabled in the kernel. MMU notifier support is an implementation detail, so it is not directly exposed as a user-selectable option, and is instead enabled by kernel features which need it. |
|
| Back to top |
|
|
grknight
Retired Dev
Joined: 20 Feb 2015
Posts: 1910
|
| Posted: Wed Jul 31, 2024 4:56 pm Post subject: |
|
|
| F. Russell wrote: | | It appears that I would need to enable KVM and then CONFIG_KVM_GENERIC_MMU_NOTIFIER in order to get MMU_NOTIFIER and I may still be missing some details. |
That same page also shows many other ways including CONFIG_AMD_IOMMU and CONFIG_INTEL_IOMMU_SVM which either is of high probability of use to a user who is implementing an nVidia graphics card. |
|
| Back to top |
|
|
F. Russell
n00b
Joined: 12 Oct 2017
Posts: 14
|
| Posted: Wed Jul 31, 2024 5:02 pm Post subject: |
|
|
| Hu wrote: | | As wjb explained above, nVidia does not require a virtual machine environment. It requires MMU notifier support to be enabled in the kernel. MMU notifier support is an implementation detail, so it is not directly exposed as a user-selectable option, and is instead enabled by kernel features which need it. |
NVidia may not *directly* require KVM but on my machine the MMU_NOTIFIER option is not present during kernel configuration and the only way to get that option is to enable KVM. This web page also seems to indicate the same:
https://www.reddit.com/r/Gentoo/comments/unpl3z/for_those_testing_the_new_nvidiadriverskernelopen/?rdt=43578
Furthermore, a problem with MMU_NOTIFIER has been reported previously, although this report pre-dates the release of the open-source drivers:
https://forums.developer.nvidia.com/t/build-failure-450-57-and-450-66-with-kernel-5-8-implicit-declaration-mmu-notifier-register/147977
In this report, NVidia fails to check for the absence of MMU_NOTIFIER. Could this be related to the current issue?
Due to my lack of knowledge I cannot say but I will consider bringing this up on the NVidia forum.
_________________
F. Russell |
|
| Back to top |
|
|
F. Russell
n00b
Joined: 12 Oct 2017
Posts: 14
|
| Posted: Wed Jul 31, 2024 5:09 pm Post subject: |
|
|
| grknight wrote: | | F. Russell wrote: | | It appears that I would need to enable KVM and then CONFIG_KVM_GENERIC_MMU_NOTIFIER in order to get MMU_NOTIFIER and I may still be missing some details. |
That same page also shows many other ways including CONFIG_AMD_IOMMU and CONFIG_INTEL_IOMMU_SVM which either is of high probability of use to a user who is implementing an nVidia graphics card. |
Yes, but CONFIG_INTEL_IOMMU_SVM is the option for an Intel virtual machine, which is present only on some Intel CPUs. On my Xeon CPU it has been replaced by Intel VT-d.
It is still, however, a virtual machine.
I need to research this more deeply if I can but, as I indicated in a previous post, the problem *may* be that NVidia does not check for the absence of MMU_NOTIFIER.
_________________
F. Russell |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22619
|
| Posted: Wed Jul 31, 2024 5:43 pm Post subject: |
|
|
I see 18 different symbols that can enable MMU notifiers, though a few of them are architecture specific.
That is not how I interpret the help text of CONFIG_INTEL_IOMMU_SVM. It makes no mention of virtual machines. |
|
| Back to top |
|
|
grknight
Retired Dev
Joined: 20 Feb 2015
Posts: 1910
|
| Posted: Wed Jul 31, 2024 6:24 pm Post subject: |
|
|
| F. Russell wrote: | | Yes, but CONFIG_INTEL_IOMMU_SVM is the option for an Intel virtual machine, which is present only on some Intel CPUs. On my Xeon CPU it has been replaced by Intel VT-d. |
No, it is for virtual memory, not a virtual machine. These are totally different concepts. | Code: | | Shared Virtual Memory (SVM) provides a facility for devices to access DMA resources through process address space by means of a Process Address Space ID (PASID). |
|
|
| Back to top |
|
|
F. Russell
n00b
Joined: 12 Oct 2017
Posts: 14
|
| Posted: Wed Jul 31, 2024 6:25 pm Post subject: |
|
|
| Hu wrote: | I see 18 different symbols that can enable MMU notifiers, though a few of them are architecture specific.
That is not how I interpret the help text of CONFIG_INTEL_IOMMU_SVM. It makes no mention of virtual machines. |
CONFIG_INTEL_IOMMU_SVM is intended to expose Shared Virtual Addressing to virtual machines:
https://patchwork.kernel.org/project/qemu-devel/cover/1571920483-3382-1-git-send-email-yi.l.liu@intel.com/
That's not surprising since it is enabled only under the CONFIG_KVM.
But I think that I've found the source of the problem. I have examined the source code of the open-source driver
and find this in /kernel-open/nvidia-uvm/uvm_linux.h:
| Code: | // The callback was added in commit 0f0a327fa12cd55de5e7f8c05a70ac3d047f405e,
// v3.19 (2014-11-13) and renamed in commit 1af5a8109904.
#if defined(NV_MMU_NOTIFIER_OPS_HAS_INVALIDATE_RANGE) || \
defined(NV_MMU_NOTIFIER_OPS_HAS_ARCH_INVALIDATE_SECONDARY_TLBS)
#define UVM_CAN_USE_MMU_NOTIFIERS() 1
#else
#define UVM_CAN_USE_MMU_NOTIFIERS() 0
#endif |
This seems to be the exact same problem first reported here:
https://forums.developer.nvidia.com/t/build-failure-450-57-and-450-66-with-kernel-5-8-implicit-declaration-mmu-notifier-register/147977
I could be wrong, but NVidia does not account for the possibility that a VM does not exist. UVM_CAN_USE_MMU_NOTIFIERS should be set to 0 if a VM, and hence MMU_NOTIFY, is not present.
Maybe a bug report to Gentoo bugzilla is appropriate? I'll first contact the NVidia forum.// The callback was added in commit 0f0a327fa12cd55de5e7f8c05a70ac3d047f405e,
// v3.19 (2014-11-13) and renamed in commit 1af5a8109904.
#if defined(NV_MMU_NOTIFIER_OPS_HAS_INVALIDATE_RANGE) || \
defined(NV_MMU_NOTIFIER_OPS_HAS_ARCH_INVALIDATE_SECONDARY_TLBS)
#define UVM_CAN_USE_MMU_NOTIFIERS() 1
#else
#define UVM_CAN_USE_MMU_NOTIFIERS() 0
#endif
_________________
F. Russell |
|
| Back to top |
|
|
grknight
Retired Dev
Joined: 20 Feb 2015
Posts: 1910
|
| Posted: Wed Jul 31, 2024 6:34 pm Post subject: |
|
|
| F. Russell wrote: | | That's not surprising since it is enabled only under the CONFIG_KVM. |
I don't know where you get this from. The option CONFIG_INTEL_IOMMU_SVM does not require any such KVM. While the main intention could have been originally created for VMs it is not limited to it. |
|
| Back to top |
|
|
pietinger
Moderator
Joined: 17 Oct 2006
Posts: 5093
Location: Bavaria
|
| Posted: Wed Jul 31, 2024 6:39 pm Post subject: |
|
|
CONFIG_INTEL_IOMMU_SVM is not necessary
Enable this (it does not hurt) in the driver-> graphics:
| Code: | [*] Intel 8xx/9xx/G3x/G4x/HD Graphics
[*] Always enable userptr support |
and you will get MMU_NOTIFIER
_________________
https://wiki.gentoo.org/wiki/User:Pietinger |
|
| Back to top |
|
|
F. Russell
n00b
Joined: 12 Oct 2017
Posts: 14
|
| Posted: Wed Jul 31, 2024 6:49 pm Post subject: |
|
|
Unless I am mistaken, this clearly seems to be a bug in NVidia open-source.
In addition to this in /kernel-open/nvidia-uvm/uvm_linux.h:
| Code: | // The callback was added in commit 0f0a327fa12cd55de5e7f8c05a70ac3d047f405e,
// v3.19 (2014-11-13) and renamed in commit 1af5a8109904.
#if defined(NV_MMU_NOTIFIER_OPS_HAS_INVALIDATE_RANGE) || \
defined(NV_MMU_NOTIFIER_OPS_HAS_ARCH_INVALIDATE_SECONDARY_TLBS)
#define UVM_CAN_USE_MMU_NOTIFIERS() 1
#else
#define UVM_CAN_USE_MMU_NOTIFIERS() 0
#endif |
I find this in /kernel-open/nvidia-uvm/nvidia-uvm.Kbuild:
| Code: | NV_CONFTEST_TYPE_COMPILE_TESTS += mmu_notifier_ops_invalidate_range
NV_CONFTEST_TYPE_COMPILE_TESTS += mmu_notifier_ops_arch_invalidate_secondary_tlbs
|
This matches exactly with the ommision reported here:
https://forums.developer.nvidia.com/t/build-failure-450-57-and-450-66-with-kernel-5-8-implicit-declaration-mmu-notifier-register/147977/2
NV_CONFTEST_FUNCTION_COMPILE_TESTS += __mmu_notifier_register
However, all of this is quite beyond my expertise which is why I am reporting it here.
I need to patch the source code according to the above link, modify the ebuild to avoid the check, and then re-emerge the driver.
_________________
F. Russell |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22619
|
| Posted: Wed Jul 31, 2024 7:00 pm Post subject: |
|
|
| What exactly do you think is the nVidia bug here? They're using a kernel feature that the user may or may not have enabled. They try to check for it. As a courtesy, the ebuild also checks for it, with the hope of getting you a better error message if your current configuration is wrong. Are you saying the nVidia check is incorrect? What does it do versus what should it do? |
|
| Back to top |
|
|
F. Russell
n00b
Joined: 12 Oct 2017
Posts: 14
|
| Posted: Wed Jul 31, 2024 7:47 pm Post subject: |
|
|
| Hu wrote: | | Are you saying the nVidia check is incorrect? What does it do versus what should it do? |
I'm not saying anything. I only indicate the following problem report:
https://forums.developer.nvidia.com/t/build-failure-450-57-and-450-66-with-kernel-5-8-implicit-declaration-mmu-notifier-register/147977/2
In that report, the NV_MMU_NOTIFIER_REGISTER is not being checked which results in a failure to build.
Checking the current kernel-open source code it seems that the that same thing is happening.
Again, this is beyond my expertise but the NVidia code should check for MMU_NOTIFIER and if absent
it should take the proper course.
This is how the proprietary code works.
I will report this to the NVidia forum.
_________________
F. Russell |
|
| Back to top |
|
|
wjb
l33t
Joined: 10 Jul 2005
Posts: 629
Location: Fife, Scotland
|
| Posted: Wed Jul 31, 2024 7:50 pm Post subject: |
|
|
| F. Russell wrote: | | This web page describes the dependencies of MMU_NOTIFIER: (actually not really) |
From gentoo-sources-6.6.38, make nconfig, then symsearch for mmu_notifier
| Quote: |
Symbol: MMU_NOTIFIER [=y]
Type : bool
Defined at mm/Kconfig:719
Selects: INTERVAL_TREE [=y]
Selected by [m]:
- KVM [=m] && VIRTUALIZATION [=y] && HAVE_KVM [=y] && HIGH_RES_TIMERS [=y] && X86_LOCAL_APIC [=y]
Selected by [n]:
- X86_SGX [=n] && X86_64 [=y] && CPU_SUP_INTEL [=y] && X86_X2APIC [=n] && CRYPTO [=y]=y [=y] && CRYPTO_SHA256 [=y]=y [=y]
- SGI_GRU [=n] && X86_UV [=n] && SMP [=y]
- DRM_RADEON_USERPTR [=n] && HAS_IOMEM [=y] && DRM_RADEON [=n]
- DRM_AMDGPU_USERPTR [=n] && HAS_IOMEM [=y] && DRM_AMDGPU [=n] && MMU [=y]
- HSA_AMD [=n] && HAS_IOMEM [=y] && DRM_AMDGPU [=n] && (X86_64 [=y] || ARM64 || PPC64)
- HSA_AMD_SVM [=n] && HAS_IOMEM [=y] && HSA_AMD [=n] && DEVICE_PRIVATE [=n]
- DRM_NOUVEAU_SVM [=n] && HAS_IOMEM [=y] && DEVICE_PRIVATE [=n] && DRM_NOUVEAU [=m] && MMU [=y] && STAGING [=n]
- DRM_I915_USERPTR [=n] && HAS_IOMEM [=y] && DRM_I915 [=n] │
- INFINIBAND_ON_DEMAND_PAGING [=n] && INFINIBAND [=n] && INFINIBAND_USER_MEM [=n]
- INFINIBAND_HFI1 [=n] && INFINIBAND [=n] && (INFINIBAND_USER_ACCESS [=n] || !INFINIBAND_USER_ACCESS [=n]) && X86_64 [=y] && INFINIBAND_RDMAVT [=n] && I2C [=y] && !UML
- XEN_GNTDEV [=n] && XEN [=n]
- AMD_IOMMU_V2 [=n] && IOMMU_SUPPORT [=y] && AMD_IOMMU [=y]
- ARM_SMMU_V3_SVA [=n] && IOMMU_SUPPORT [=y] && ARM_SMMU_V3 [=n]
- INTEL_IOMMU_SVM [=n] && IOMMU_SUPPORT [=y] && INTEL_IOMMU [=y] && X86_64 [=y]
- TEST_HMM [=n] && RUNTIME_TESTING_MENU [=y] && TRANSPARENT_HUGEPAGE [=y] && DEVICE_PRIVATE [=n]
|
I already have KVM set (because I have a use for it) hence the first match. Several of the other selections agree with what people have suggested above, note that none of them involve KVM.
And if nvidia needed you to set KVM, why wouldn't they just tell you to set that (as well) rather than doing it indirectly.
Bill |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22619
|
| Posted: Wed Jul 31, 2024 9:13 pm Post subject: |
|
|
That report is almost 4 years old. You have shown no output indicating that your current issue relates to that report. You have shown nothing that supports the original claim that nVidia requires KVM support. I think there is nothing we can do to help you with what you have provided.
If you think the nVidia forum is a better venue for your problem report, please seek their aid, and link us to their response. |
|
| Back to top |
|
|
F. Russell
n00b
Joined: 12 Oct 2017
Posts: 14
|
| Posted: Wed Jul 31, 2024 9:24 pm Post subject: |
|
|
| Hu wrote: | | That report is almost 4 years old. You have shown no output indicating that your current issue relates to that report. |
I certainly have shown.
The current NVidia open-source code (555.58.02) contains the same omissions as the report from 2020. See my posts above.
I am certainly not denigrating any Gentoo forum contributions but it still remains odd that NVidia proprietary drivers
accept an absence of MMU_NOTIFIER whereas the open-source drivers suddenly do not.
_________________
F. Russell |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22619
|
| Posted: Wed Jul 31, 2024 9:42 pm Post subject: |
|
|
| I saw that you quoted a snippet of code, which you said in the same post is beyond your expertise. I don't see where you posted any error messages, from Portage, from gcc, or any nVidia scripts. If you don't have an error message, why do you think that code is wrong? Circumstances may (or may not) have changed in the intervening 4 years, causing code which once was right to become wrong, or vice versa. |
|
| Back to top |
|
|
wjb
l33t
Joined: 10 Jul 2005
Posts: 629
Location: Fife, Scotland
|
| Posted: Wed Jul 31, 2024 10:15 pm Post subject: |
|
|
Why not just try doing some experiments and see how it goes?
- KVM & bios disable kvm
- KVM & bios enable kvm
- not KVM & bios disable kvm & what pietinger suggested
I don't think there's going to be any noticeable difference. |
|
| Back to top |
|
|
|
Kernel & Hardware
