On Tuesday 04 February 2014 12:10:55 Jason Gunthorpe wrote:
For instance to support peer-to-peer IO you need to have a consisent, non-overlapping set of bus/device/function/tag to uniquely route TLPs within the chip. Cross domain TLP routing in HW is non-trivial.
Yes, that is a good reason.
IOMMUs (and SR-IOv) rely on the BDF to identify the originating device for each TLP. Multiple domains means a much more complex IOMMU environment.
I fear we already have to support complex IOMMU setups on ARM, whether there are multiple PCI domains or not. But it would be nice in theory not to require it.
Failure to integrate on-chip devices into the PCI world also means thing like SR-IOv won't work sanely with on-chip devices.
I'd consider this a feature ;)
But you are probably right: people will do SR-IOV whether we like it or not, and they will try to do it on non-PCI devices too, and great suffering will be involved.
Arnd