I cant do PCI passthru of a NIC to a OPNsense. bare metal performance wasnt great either, so I switched to openWRT and bought a different NIC capable of SR-IOV. my motherboard groups things sloppily so PCI passthrough wont work, but SR-IOV will supposedly get me most of the way there. I am just not wrapping my head around making this work.
I just need toilet paper pass the ports on the i350 NIC into the VM. I dont want the host to use them, it has its own onboard NIC for rescue usage. I just need this stupid thing to pass create the VFs at boot, pass them to the VM, and let the VM use them.
I’ve resorted to using chatGPT to sort through this and obviously that is not going well.
Does anyone have a guide somewhere about this? everything I’m finding is for GPUs, not NICs.
SR-IOV works by presenting one device as many, which you can passthrough one of those to your VM. Meaning SR-IOV only works through PCIe passthrough, so you’d have to figure that out first. The GPU guides should get you most of the way there.
Some distros include an ACS patch into their kernel (e.g. Proxmox, and I think CachyOS), which lets you passthrough devices without hardware support (but lacking some security features).
I believe it might be possible to ‘passthrough’ the VF from the host without PCIe passthrough (I’ve only done this with containers though), but performance is often worse than just using a bridge.
My problem with using a bridge as it means packets just keep getting copied multiple times and on gigabyte that’s really choking my system. On the other hand, the wind port on that neck is going to be public facing so I kind of want all those security features in place, even if it’s unlikely they’re gonna be necessary
Yeah thats fair enough. The ACS override patch should still have better isolation and speed than anything else you can do without native ACS, the security implications are just it’s theoretically possible to intercept another PCIe device’s traffic through the NIC; you can read more here.
You can start with this, but does your motherboard support sr-iov? If you can’t use normal PCI passthrough because of lack of IOMMU granularity, the odds of it supporting SR-IOV are slim.
It does support SR-IOV. It supports IOMMU and ACS too, but only for the GPU slot and one of the M2 slots too, so I kind of have to dance around this. Everything else is dumped into a single IOMMU group.
I considered the proxmox approach but decided against it because my GPU is supposedly really difficult to make cooperate with these virtualized environments, so I made it go bare metal and the only VM that would need run would then be the router/firewall stuff. As you can see, thats not going super well.
I’m likely going to end up getting another motherboard just for the freaking passthrough. The whole point of this was to get myself off the cloud and give myself the netboot setup I actually wanted rather than what firewalla has decided I should have but its just not going well :(.
What is your mobo model?
