rphii
n00b
Joined: 08 Jan 2022
Posts: 18
|
 Posted: Fri Dec 22, 2023 7:46 pm Post subject: qemu GPU passthrough missing vfio |
 |
|
Hello wonderful people
Today I decided to finally try to tackle this thought of wanting to have a qemu GPU passthrough virtual machine... And now I am clueless how to proceed
When I first had the thought I would've wanted to try it with windows, but not anymore. Heard some things about Nobara, so I am trying that one instead.
Anyways, first things first: Kernel config ($ cat /usr/src/linux/.config) and some things about my system:
WM: Hyprland (Wayland)
CPU: AMD Ryzen 7 2700X (16) @ 3.7 GHz
GPU: NVIDIA RTX 2070
Mostly following the official wiki, and also some snippets from this gist.
I installed packages and added the user to the following groups, which worked just fine, didn't take long at all XD: kvm,input,libvirt.
Then I checked for NVIDIA stuff.
| Code: | $ lspci -nn | grep NVIDIA
1c:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU106 [GeForce RTX 2070 Rev. A] [10de:1f07] (rev a1)
1c:00.1 Audio device [0403]: NVIDIA Corporation TU106 High Definition Audio Controller [10de:10f9] (rev a1)
1c:00.2 USB controller [0c03]: NVIDIA Corporation TU106 USB 3.1 Host Controller [10de:1ada] (rev a1)
1c:00.3 Serial bus controller [0c80]: NVIDIA Corporation TU106 USB Type-C UCSI Controller [10de:1adb] (rev a1) |
I tried this GRUB config before, as I've seen it in other sources...
| Code: | $ cat /etc/default/grub | grep GRUB_CMDLINE_LINUX
GRUB_CMDLINE_LINUX="iommu=pt amd_iommu=on pcie_acs_override=downstream,multifunction vfio-pci.ids=10de:1f07,10de:10f9"
|
However, the current GRUB config is this:
| Code: | $ cat /etc/default/grub | grep GRUB_CMDLINE_LINUX
GRUB_CMDLINE_LINUX="iommu=pt amd_iommu=on pcie_acs_override=downstream,multifunction
|
Verifying IOMMU has been enabled:
| Code: | $ dmesg | grep 'IOMMU'
[ 0.183027] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[ 0.183997] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 0.186630] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
[ 0.465610] AMD-Vi: AMD IOMMUv2 loaded and initialized |
Search for VGA card IDs:
| Code: | $ for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*}; printf 'IOMMU Group %s ' "$n"; lspci -nns "${d##*/}"; done | grep NVIDIA
IOMMU Group 16 1c:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU106 [GeForce RTX 2070 Rev. A] [10de:1f07] (rev a1)
IOMMU Group 16 1c:00.1 Audio device [0403]: NVIDIA Corporation TU106 High Definition Audio Controller [10de:10f9] (rev a1)
IOMMU Group 16 1c:00.2 USB controller [0c03]: NVIDIA Corporation TU106 USB 3.1 Host Controller [10de:1ada] (rev a1)
IOMMU Group 16 1c:00.3 Serial bus controller [0c80]: NVIDIA Corporation TU106 USB Type-C UCSI Controller [10de:1adb] (rev a1) |
I added the VGA PCI IDs to VFIO:
| Code: | $ cat /etc/modprobe.d/vfio.conf
options vfio-pci ids=10de:1f07,10de:10f9
|
I also did those steps (almost identically as mentioned in the official wiki):
| Code: | root # mkdir /home/qemu
root # cp /home/<user>/.config/pulse/default.pa /home/qemu -r
root # chown qemu:qemu -R /home/qemu
root # usermod -d /home/qemu qemu
|
And also added all my input devices to /etc/libvirt/qemu.conf, as stated in the official wiki.
I downloaded, created an image and have a start script:
| Code: | ~/VM-nobara $ qemu-img create -f qcow2 image.img 72G
Formatting 'image.img', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=77309411328 lazy_refcounts=off refcount_bits=16
~/VM-nobara $ ls
Nobara-38-KDE-Nvidia-2023-11-28.iso image.img start-nobara.sh
~/VM-nobara $ cat start-nobara.sh
#!/bin/sh
set -e
qemu-system-x86_64 \
-nodefaults -enable-kvm -boot menu=on \
./Nobara-38-KDE-Nvidia-2023-11-28.iso \
-drive file=image.img \
-cpu host,kvm=off -smp 8 -m 16G \
-name "Nobara" \
-device vfio-pci,host=1c:00.0,x-vga=on,multifunction=on \
-device vfio-pci,host=1c:00.1 \
-monitor stdio \
-nographic \
-vga none \
"$@"
virsh nodetach-reattach pci_0000_1c_00_0
virsh nodetach-reattach pci_0000_1c_00_1
|
However, and this is the fun stuff, that start script tells me this:
| Code: | ~/VM-nobara $ ./start-nobara.sh
WARNING: Image format was not specified for './Nobara-38-KDE-Nvidia-2023-11-28.iso' and probing guessed raw.
Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
Specify the 'raw' format explicitly to remove the restrictions.
QEMU 8.0.4 monitor - type 'help' for more information
(qemu) qemu-system-x86_64: -device vfio-pci,host=1c:00.0,x-vga=on,multifunction=on: vfio 0000:1c:00.0: failed to open /dev/vfio/16: No such file or directory
~/VM-nobara $ ls /dev/vfio
vfio
|
I then went ahead and tried doing this, which resulted in a freeze when I rebooted, so I undid that again:
| Code: | $ cat /etc/modprobe.d/vfio.conf
options vfio-pci ids=10de:1f07,10de:10f9
softdep nvidia pre: vfio-pci
|
And that's it. If anyone can provide me with insight I will be very grateful 
Have a nice day! |
|
Other Things Gentoo
