KVM/QEMU via virt-manager. I would imagine that your use case would work if you pass the USB device or the entire usb host controller through to the VM, but I’m not sure. Please check the video linked in my other comment for more information on the single GPU setup