Some questions about passthrough of CPU GPU on qemu

FAb7D

I can't figure out for sure if I have "CPU passthrough" enabled or not.

On the EFI-BIOS I set "virtualization enabled", from terminal I get:

~ $  dmesg | grep -i VT-x
~ $  dmesg | grep -i VT-d
 ~ $ grep --color vmx /proc/cpuinfo 
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp vnmi md_clear flush_l1d arch_capabilities
vmx flags	: vnmi preemption_timer invvpid ept_x_only ept_ad ept_1gb flexpriority tsc_offset vtpr mtf vapic ept vpid unrestricted_guest ple pml ept_mode_based_exec

.....

On QEMU/KVM on CPU menu there is the option vCPU host-passthrough.
I don't understand if CPU passthrough in really enabled.

That said I have a system with intell i7 CPU (Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz) https://www.intel.com/content/www/us/en/products/sku/191045/intel-core-i79750h-processor-12m-cache-up-to-4-50-ghz/specifications.html?wapkw=i7-9750H with also a GPU componentCoffeeLake-H GT2 [UHD Graphics 630] disabled because I use the GPU NVIDIA GeForce GTX 1650 Mobile/Max-Q,
is there a way to use the CoffeeLake GPU in passthrough mode only on quemu/KVM?
One GPU for Solus, the other one for qemu.

silke

CPU passthrough and GPU passthrough are two very different concepts:

CPU passthrough via the host-passthrough option is QEMU passing along the CPU model details to the VM. The alternative is that it passes along a CPU with a subset of instructions. This setting is useful when moving a VM between hypervisors with different CPU models, as you can select a model that is supported by both. If you select host-passthrough and your VM starts, this works. You can do cat /proc/cpuinfo in the VM and you should see the same info as outside of the VM.
GPU passthough is actually passing a PCI device from your bus to your VM. This happens via IOMMU.

is there a way to use the CoffeeLake GPU in passthrough mode only on quemu/KVM?

Theoretically yes (IOMMU is enabled in Solus kernels), but this really depends on how the GPU is connected to the PCI bus and what other devices are part of the IOMMU group. The Arch Wiki has more details on how to find out and how to use it.

Or you could just add the device via virt-manager and see what happens 🙂. Worst-case is a kernel panic. Make sure the VM doesn't autostart on boot.

FAb7D

silke Many tutorials specify
"Edit the file /etc/default/grub as root (use sudo), adding the following to the line beginning GRUB_CMDLINE_LINUX_DEFAULT:
For Intel, intel_iommu=on iommu=pt"
But Grub there's no more.
I don't know how to do (in the easiest way) the same thing with clr-boot-manager + systemd-boot ... (those are 2 things I've never used before)

Actualy there is a "intel_iommu=o" 

> cat /proc/cmdline
initrd=\EFI\com.solus-project\initrd-com.solus-project.current.6.4.15-254 root=UUID=8b2c9e33-66c2-4e67-bc57-cf7da5be2965 rd.luks.uuid=37a5141e-2b28-4141-a969-47d73866ae78 quiet loglevel=3 splash systemd.show_status=false rw radeon.si_support=0 radeon.cik_support=0 amdgpu.si_support=1 amdgpu.cik_support=1 nvidia-drm.modeset=1 nvidia.NVreg_PreserveVideoMemoryAllocations=1 nvidia.NVreg_TemporaryFilePath=/var/tmp resume=UUID=37565307-e63d-44bd-9dc0-8630d0eaf596 intel_iommu=o pci=nommconf
[16:38:29]

In the Arch tutorial they talk about an alternative to GRUB: "the files in /proc/sys/ (see sysctl) and /sys/."

Harvey

If you need that one missing option:
echo "iommu=pt" | sudo tee /etc/kernel/cmdline.d/50_iommu.conf
sudo clr-boot-manager update

Reboot.

FAb7D

Harvey thanks! It seams to work:

> cat /proc/cmdline
initrd=\EFI\com.solus-project\initrd-com.solus-project.current.6.4.15-254 root=UUID=8b2c9e33-66c2-4e67-bc57-cf7da5be2965 rd.luks.uuid=37a5141e-2b28-4141-a969-47d73866ae78 quiet loglevel=3 splash systemd.show_status=false rw radeon.si_support=0 radeon.cik_support=0 amdgpu.si_support=1 amdgpu.cik_support=1 nvidia-drm.modeset=1 nvidia.NVreg_PreserveVideoMemoryAllocations=1 nvidia.NVreg_TemporaryFilePath=/var/tmp resume=UUID=37565307-e63d-44bd-9dc0-8630d0eaf596 iommu=pt intel_iommu=o pci=nommconf

I am left with a doubt , is it normal that "intel_iommu = o" (in the end of the output up here )? I expected " intel_iommu = on" with the ending n

Intrigued I found this (if it can serve):

> cat /etc/kernel/cmdline.d/sd-card-reader.conf
intel_iommu=o 
pci=nommconf

Harvey

Only thing I could find about sd-card-reader.conf was a post on our old phab where Joey said:

So for example, create a file /etc/kernel/cmdline.d/sd-card-reader.conf with:
intel_iommu=on

To which fabio replied.

I have tried

intel_iommu=on
pci=nommconf
intel_iommu=o pci=nommconf

So I assume this file was created by you and with o instead of on.

FAb7D

Harvey Oh my god I'm just stupid, in fact the sd-card-reader has never worked properly,sorry I wasted your time, but now (I've changed from o to on) iommu work!

dmesg | grep -i -e DMAR -e IOMMU
[    0.000000] Command line: initrd=\EFI\com.solus-project\initrd-com.solus-project.current.6.4.15-254 root=UUID=8b2c9e33-66c2-4e67-bc57-cf7da5be2965 rd.luks.uuid=37a5141e-2b28-4141-a969-47d73866ae78 quiet loglevel=3 splash systemd.show_status=false rw radeon.si_support=0 radeon.cik_support=0 amdgpu.si_support=1 amdgpu.cik_support=1 nvidia-drm.modeset=1 nvidia.NVreg_PreserveVideoMemoryAllocations=1 nvidia.NVreg_TemporaryFilePath=/var/tmp resume=UUID=37565307-e63d-44bd-9dc0-8630d0eaf596 iommu=pt intel_iommu=on pci=nommconf
[    0.005941] ACPI: DMAR 0x0000000079B9A000 0000A8 (v01 HPQOEM 85FC     00000002 HP   00040000)
[    0.005973] ACPI: Reserving DMAR table memory at [mem 0x79b9a000-0x79b9a0a7]
[    0.030560] Kernel command line: initrd=\EFI\com.solus-project\initrd-com.solus-project.current.6.4.15-254 root=UUID=8b2c9e33-66c2-4e67-bc57-cf7da5be2965 rd.luks.uuid=37a5141e-2b28-4141-a969-47d73866ae78 quiet loglevel=3 splash systemd.show_status=false rw radeon.si_support=0 radeon.cik_support=0 amdgpu.si_support=1 amdgpu.cik_support=1 nvidia-drm.modeset=1 nvidia.NVreg_PreserveVideoMemoryAllocations=1 nvidia.NVreg_TemporaryFilePath=/var/tmp resume=UUID=37565307-e63d-44bd-9dc0-8630d0eaf596 iommu=pt intel_iommu=on pci=nommconf
[    0.030704] DMAR: IOMMU enabled
[    0.071924] DMAR: Host address width 39
[    0.071925] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[    0.071930] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 1c0000c40660462 ecap 19e2ff0505e
[    0.071931] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[    0.071934] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap d2008c40660462 ecap f050da
[    0.071935] DMAR: RMRR base: 0x00000078e8d000 end: 0x00000078eacfff
[    0.071937] DMAR: RMRR base: 0x0000007b000000 end: 0x0000007f7fffff
[    0.071938] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.071939] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.071940] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.075043] DMAR-IR: Enabled IRQ remapping in x2apic mode
[    0.299069] iommu: Default domain type: Passthrough (set via kernel command line)
[    0.318314] DMAR: No ATSR found
[    0.318315] DMAR: No SATC found
[    0.318316] DMAR: IOMMU feature fl1gp_support inconsistent
[    0.318316] DMAR: IOMMU feature pgsel_inv inconsistent
[    0.318317] DMAR: IOMMU feature nwfs inconsistent
[    0.318318] DMAR: IOMMU feature pasid inconsistent
[    0.318319] DMAR: IOMMU feature eafs inconsistent
[    0.318319] DMAR: IOMMU feature prs inconsistent
[    0.318320] DMAR: IOMMU feature nest inconsistent
[    0.318320] DMAR: IOMMU feature mts inconsistent
[    0.318321] DMAR: IOMMU feature sc_support inconsistent
[    0.318321] DMAR: IOMMU feature dev_iotlb_support inconsistent
[    0.318322] DMAR: dmar0: Using Queued invalidation
[    0.318324] DMAR: dmar1: Using Queued invalidation
[    0.318481] pci 0000:00:02.0: Adding to iommu group 0
[    0.318511] pci 0000:00:00.0: Adding to iommu group 1
[    0.318524] pci 0000:00:01.0: Adding to iommu group 2
[    0.318533] pci 0000:00:04.0: Adding to iommu group 3
[    0.318542] pci 0000:00:08.0: Adding to iommu group 4
[    0.318554] pci 0000:00:12.0: Adding to iommu group 5
[    0.318571] pci 0000:00:14.0: Adding to iommu group 6
[    0.318579] pci 0000:00:14.2: Adding to iommu group 6
[    0.318591] pci 0000:00:15.0: Adding to iommu group 7
[    0.318604] pci 0000:00:16.0: Adding to iommu group 8
[    0.318612] pci 0000:00:17.0: Adding to iommu group 9
[    0.318637] pci 0000:00:1d.0: Adding to iommu group 10
[    0.318646] pci 0000:00:1d.5: Adding to iommu group 10
[    0.318655] pci 0000:00:1d.6: Adding to iommu group 10
[    0.318664] pci 0000:00:1d.7: Adding to iommu group 10
[    0.318688] pci 0000:00:1f.0: Adding to iommu group 11
[    0.318698] pci 0000:00:1f.3: Adding to iommu group 11
[    0.318708] pci 0000:00:1f.4: Adding to iommu group 11
[    0.318717] pci 0000:00:1f.5: Adding to iommu group 11
[    0.318720] pci 0000:01:00.0: Adding to iommu group 2
[    0.318723] pci 0000:01:00.1: Adding to iommu group 2
[    0.318727] pci 0000:02:00.0: Adding to iommu group 10
[    0.318730] pci 0000:03:00.0: Adding to iommu group 10
[    0.318734] pci 0000:04:00.0: Adding to iommu group 10
[    0.318737] pci 0000:05:00.0: Adding to iommu group 10
[    0.318763] DMAR: Intel(R) Virtualization Technology for Directed I/O

Thank you!! And tanks to Silke too! All yours replies was useful.