On 1/12/26 16:35, Jason Gunthorpe wrote:
On Mon, Jan 12, 2026 at 03:56:32PM +0100, Christian König wrote:
The problem revoke is designed to solve is that many importers have hardware that can either be DMA'ing or failing. There is no fault mechanims that can be used to implement the full "move around for no reason" semantics that are implied by move_notify.
In this case just call dma_buf_pin(). We already support that approach for RDMA devices which can't do ODP.
That alone isn't good enough - the patch adding the non-ODP support also contained this:
static void ib_umem_dmabuf_unsupported_move_notify(struct dma_buf_attachment *attach) { struct ib_umem_dmabuf *umem_dmabuf = attach->importer_priv;
ibdev_warn_ratelimited(umem_dmabuf->umem.ibdev, "Invalidate callback should not be called when memory is pinned\n"); }
Yeah, I know. That's what I meant we have to better document this.
static struct dma_buf_attach_ops ib_umem_dmabuf_attach_pinned_ops = { .allow_peer2peer = true, .move_notify = ib_umem_dmabuf_unsupported_move_notify, };
So we can't just allow it to attach to exporters that are going to start calling move_notify while pinned.
The point is exporters are already doing this.
Looking around I don't see anyone else doing something like this, and reading your remarks I think EFA guys got it wrong. So I'm wondering if this should not have been allowed. Unfortunately 5 years later I'm pretty sure it is being used in places where we don't have HW support to invalidate at all, and it is now well established uAPI that we can't just break.
Which is why we are coming to negotiation because at least the above isn't going to work if move_notify is called for revoke reasons, and we'd like to block attaching exporters that need revoke for the above.
Ah, yes that makes sense. This is clearly a new requirement.
So basically for PCIe hotplug was a rare event were we said we have some problems with non-ODP but we can live with that, but for this use case here it's more like a perfectly normal condition that userspace can trigger.
So the exporter wants to reject importers which can't handle a mapping invalidation while the BO is pinned, correct?
So, would you be happier with this if we documented that move_notify can be called for pinned importers for revoke purposes and figure out something to mark the above as special so exporters can fail pin if they are going to call move_notify?
That would work for me. I mean it is already current practice, we just never fully documented it.
Then this series would transform into documentation, making VFIO accept pin and continue to call move_notify as it does right now, and some logic to reject the RDMA non-ODP importer.
I think we just need to expose this with flags or similar from the importer side. As far as I know RDMA without ODP is currently the only one really needing this (except for cross device scanout, but that is special anyway).
Christian.
Jason