How I set up Nvidia GPU passthrough on Ubuntu 22.10 and 23.04

Tutorials
1

How I set up Nvidia GPU passthrough on Ubuntu 22.10 and 23.04

This is not a definitive guide. It’s simply how i’ve achieved success on my machine running Ubuntu 22.10 and 23.04.

This will disable all non-primary GPUs in order to use them for passthrough. There’s ways to only bind specific cards but it’s more complicated, see information here.

Just a warning, most guides don’t work, contain a lot of cruft or are incomplete. I found it easy to find guides but difficult to find one that worked and was simple.

Step 0 - Enable virtualization options in BIOS

The method varies by the vendor, model and BIOS version of your motherboard but you need to enable anything listed as:

  • Intel VT-x
  • AMD-V
  • IOMMU

Step 1 - Enable iommu and bind cards to vfio

To make life easy we’ll use passthrough_helper_ubuntu_20. It’s a set of two mini shell scripts so if you’d prefer to confirm each step as you go and do them manually just view gpu_passthrough.sh.

This script will:

  • Update grub with the correct environment variable for your processor that turns on iommu
  • Install vfio-pci-override-vga.sh into initramfs scripts which binds all non-primary GPUs to vfio 
git clone https://github.com/pavolelsig/passthrough_helper_ubuntu_20
sudo bash ./passthrough_helper_ubuntu_20/gpu_passthrough.sh
# Follow instructions

Step 2 - Disable nvidia drivers

This was mandatory for me to prevent an avalanche of driver errors.

# Confirm nvidia hasn't already been blacklisted
cat /etc/modprobe.d/blacklist.conf

# Blacklist nvidia drivers (paste all lines into terminal at once)
cat << 'EOF' | sudo tee -a /etc/modprobe.d/blacklist.conf
blacklist nvidia
blacklist nvidia_uvm
blacklist nvidia_drm
blacklist nvidia_modeset
blacklist nouveau
EOF

Step 3 - Add cards to your VM

In this example i’m using virt-manager

  1. Open the VM
  2. Click the bulb icon or view > details
  3. Click “Add Hardware”
  4. Choose “PCI Host Device”
  5. Choose a GPU device for passthrough and after #5 repeat from #2 and choose it’s audio controller.
  6. Click “Finish”

image
image1037×865 175 KB

Start the VM and install the appropriate nvidia drivers.