Udon's xLog

1. Background Setup#

The AMD Ryzen 7 5700G integrates the AMD Radeon RX Vega 8, based on the Vega architecture (5th generation GCN), featuring 8 CUs and a total of 512 stream processors. If this integrated graphics performance can be made roughly equivalent to that of a 750ti and passed through to Windows 11, it would save the cost of purchasing an additional graphics card, as well as the extra electricity expenses.

This article roughly follows the setup from PVE7 AMD 5700G iGPU Passthrough.

2. Software and Hardware Environment#

• CPU: AMD Ryzen 7 5700G

• Motherboard: MSI MAG B550M MORTAR WIFI

• BIOS Version: 7C94v1C(E7C94AMS.1C0)

• PVE Version: 7.4-13

• PVE Manager Version: pve-manager/7.4-13/46c37d9c

• Linux Kernel Version: Linux 5.15.107-2-pve #1 SMP PVE 5.15.107-2 (2023-05-10T09:10Z)

3. Change BIOS Settings#

1. Reset BIOS to default settings
   Settings\Save & Exit\Restore Defaults

2. Disable S/3 Modern Standby Support (optional)
   Settings\Advanced\ACPI Settings\S/3 Modern Standby Support [Disabled]

3. Set Initiate Graphic Adaptor to external graphics card
   Settings\Advanced\Initiate Graphic Configuration\Initiate Graphic Adaptor [PEG]

4. Set Integrated Graphics to Force Mode
   Settings\Advanced\Initiate Graphic Configuration\Integrated Graphics [Force]

5. Set UMA Frame Buffer Size (video memory size) to 2G/4G
   Settings\Advanced\Initiate Graphic Configuration\UMA Frame Buffer Size [4G]

6. Enable motherboard IOMMU
   Overclocking\Advanced CPU Configuration\AMD CBS\IOMMU [Enabled]

7. Enable CPU virtualization
   Overclocking\Advanced CPU Configuration\SVM Mode [Enabled]

8. Save and exit

4. Export Integrated Graphics VBIOS for Backup#

1. Open the MSI B550M MORTAR WIFI support page, download the corresponding BIOS version file and extract it to obtain the BIOS source file.

2. Download UEFI BIOS Updater, extract it, place the BIOS file from step one into the UBU directory, and run UBU.bat
   

3. After entering the main menu, input 2 and press Enter
   

4. Input S and press Enter
   

5. Input 0 and press Enter to exit to the main menu, then exit the tool
   

6. Find vbios_1638.dat in UBU_v1.79.17\Extracted\VBIOS\017.010.000.030.000000, which is the VBIOS for the 5700G integrated graphics, and keep it for backup.

7. If the CPU is not a 5700G, you can find the corresponding VBIOS file name in UBU_v1.79.17\Files\AMD\VBIOS\_Listvbios.txt according to the architecture.

5. Install PVE#

Use Rufus to create a bootable USB drive, select the USB drive as the boot option, and follow the prompts to proceed. This article will not cover this in detail.

6. Modify PVE Kernel#

1. Connect to the PVE host via SSH.

2. Modify GRUB

root@pve:~# nano /etc/default/grub

3. Change GRUB_CMDLINE_LINUX_DEFAULT="quiet" to GRUB_CMDLINE_LINUX_DEFAULT="quiet iommu=pt initcall_blacklist=sysfb_init pcie_acs_override=downstream,multifunction", save and exit.

4. Update GRUB

root@pve:~# update-grub

5. Add kernel modules

root@pve:~# nano /etc/modules

Change the file to:

# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.

vfio
vfio_pci
vfio_virqfd
vfio_iommu_type1

Save and exit.

6. Disable device drivers

root@pve:~# nano /etc/modprobe.d/pve-blacklist.conf

Write in the file:

blacklist amdgpu
blacklist snd_hda_intel

Save and exit.

7. Update the modified settings

root@pve:~# update-initramfs -u -k all

8. Restart the machine.

9. Check IOMMU group status

root@pve:~# nano ~/iommu_group.sh

Write in the file:

#!/bin/bash
shopt -s nullglob
for g in $(find /sys/kernel/iommu_groups/* -maxdepth 0 -type d | sort -V); do
        echo "IOMMU Group ${g##*/}:"
        for d in $g/devices/*; do
                echo -e "\t$(lspci -D -nns ${d##*/})"
        done;
done;

Save and exit.

10. Grant execution permission to the file and run it to check if the IOMMU groups are mostly one device per group

root@pve:~# chmod +x ~/iommu_group.sh && ./iommu_group.sh 

7. Bind PCI Device#

1. List all PCI devices on the machine

root@pve:~# lspci -D -nnk

Find the line with VGA compatible controller:

0000:30:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Cezanne [1002:1638] (rev c8)
        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Device [1002:1636]
        Kernel modules: amdgpu

Note down the integrated graphics PCI ID 0000:30:00.0 and the device ID [1002:1638], as these IDs may vary depending on the device.

2. Bind the PCI device to vfio

root@pve:~# nano //etc/modprobe.d/pt.conf

Write in the file:

options vfio-pci ids=xxxx:xxxx    #"xxxx:xxxx" is your integrated graphics device ID
options vfio-pci disable_idle_d3=1

Save and exit.

3. Update the kernel again

root@pve:~# update-initramfs -u -k all

4. Restart the machine.

8. Install Windows 11 Virtual Machine#

Since there are many tutorials on installing Windows virtual machines on PVE, this section will only show some necessary changes for integrated graphics passthrough. First, enter the PVE web console, create a new virtual machine, and configure it according to the image below.

The machine type must be q35, the BIOS must be SeaBIOS, the SCSI controller should be set to VirtIO SCSI Single for better performance, and TPM storage must be added.

Similarly, set the disk bus to VirtIO Block for better performance.

The CPU type must be set to host.

During the initial installation, disconnect the network to avoid automatic installation of updates/drivers, and select VirtIO for better performance.

Since Windows itself does not include support for VirtIO, you need to mount the VirtIO drivers in ISO format to a CD drive. Click on the virtual machine to enter the details page, select the hardware tab, and add a CD drive.

Start the virtual machine and install Windows. When selecting the drive for installation, if the drive list is empty, click the load driver option, select the VirtIO driver, navigate to the directory CD Drive:\amd64\w11, select the w11 folder, and confirm. Wait for the driver to load, and the mounted hard drive will appear in the list. Continue with the installation.

9. Modify Windows Settings#

1. Go to settings and pause updates.

2. Open advanced system settings, click on device installation settings in the hardware tab, and disable automatic downloads.
   

3. Install VirtIO drivers
   Open the mounted CD drive and run virtio-win-gt-x64.msi, and proceed with the default installation.

4. (Optional) Enable Remote Desktop.

5. Shut down.

10. Set Up Graphics Card Passthrough#

1. Upload the extracted VBIOS to PVE

scp .\vbios_1638.dat root@192.168.3.2:/usr/share/kvm

2. Add a PCI device to the virtual machine
   

3. Select the previously noted integrated graphics PCI ID 0000:30:00.0, check the main GPU and PCI-Express options, then add it.
   

4. Modify the virtual machine configuration file

root@pve:~# nano /etc/pve/qemu-server/104.conf    #104 is your virtual machine ID

Change the cpu and hostpci0 items to:

cpu: host,hidden=1
hostpci0: 0000:30:00.0,pcie=1,x-vga=1,romfile=vbios_1638.dat

Save and exit.

5. Change the display of the virtual machine hardware to none
   

6. Reconnect the network card, power on, use RDP to connect to the Windows 11 virtual machine, and install the AMD graphics driver.

7. All done!

11. Resolve Some Bugs#

In Proxmox VE 7.x, when passing through AMD Vega integrated graphics to a Windows environment, there may be an issue where the graphics card cannot be automatically passed through after a normal shutdown and reboot, meaning Windows does not recognize the integrated graphics and there is no output. To resolve this issue, there are two methods:

1. Manually eject the graphics card in Device Manager and then shut down, but this is not very elegant.

2. Install Devcon and write a bat file to automatically eject the integrated graphics and shut down.
   2-1. Obtain the Devcon Installer from GitHub.
   2-2. Run the Devcon Installer as an administrator and select Update Sources to update the source if there is a proxy.
   2-3. Check Add DevCon to environment PATH, select Windows 11 version 22H2, choose x64 architecture, and then install.
   2-4. Create a bat file with the following content:

@echo off 
chcp 65001
echo Ejecting graphics card...
set devmgr_show_nonpresent_devices=1
start devmgmt.msc
echo Please wait...
ping 127.0.0.1 -n 6 > nul
echo Uninstalling graphics card...
devcon.exe remove "PCI\VEN_1002&DEV_1638"    #Enter your integrated graphics device ID
echo Shutting down in five seconds...
ping 127.0.0.1 -n 6 > nul
shutdown /s /t 0

2-5. Save and run the bat file as an administrator to automatically shut down!