On 10/16/26 22:26, Karl Mehltretter wrote:
On Tue, Feb 17, 2026 at 08:11:34PM -0400, Jason Gunthorpe wrote:
- match_args.attach = attach;
- if (dmabuf->ops->match_mapping) {
ret = dmabuf->ops->match_mapping(&match_args);- }
- attach->dmabuf = dmabuf; attach->importer_ops = importer_ops; attach->importer_priv = importer_priv;
I may misunderstand the intended contract. match_mapping() runs while the attachment is still zeroed and without dmabuf->resv held.
Patch 11's amdgpu callback needs importer_ops and importer_priv, so its XGMI match cannot work here. Patch 12's VFIO callback asserts dmabuf->resv and reads priv->vdev, which cleanup changes under that lock.
Should the core initialize the attachment and hold dmabuf->resv for match_mapping(), or should callbacks avoid attachment and lock-protected state?
Right. This completely misplaced.
The matching can't be done at attachment time, but rather needs to be done before creating a mapping while holding the resv lock so that the matching decision can't change.
Regards, Christian.
Thanks, Karl
On Mon, 2026-10-19 at 10:35 +0200, Christian König wrote:
On 10/16/26 22:26, Karl Mehltretter wrote:
On Tue, Feb 17, 2026 at 08:11:34PM -0400, Jason Gunthorpe wrote:
- match_args.attach = attach;
- if (dmabuf->ops->match_mapping) {
ret = dmabuf->ops->match_mapping(&match_args);- }
attach->dmabuf = dmabuf; attach->importer_ops = importer_ops; attach->importer_priv = importer_priv;
I may misunderstand the intended contract. match_mapping() runs while the attachment is still zeroed and without dmabuf->resv held.
Patch 11's amdgpu callback needs importer_ops and importer_priv, so its XGMI match cannot work here. Patch 12's VFIO callback asserts dmabuf-
resv and
reads priv->vdev, which cleanup changes under that lock.
Should the core initialize the attachment and hold dmabuf->resv for match_mapping(), or should callbacks avoid attachment and lock- protected state?
Right. This completely misplaced.
The matching can't be done at attachment time, but rather needs to be done before creating a mapping while holding the resv lock so that the matching decision can't change.
So question is what exactly should the contract be? Looking at the pcie_p2p negotiation today it looks somewhat like.
1) Importer and exporter agree at attach time what mappings are allowed pending placement conditions and other attachers. 2) At map-time the mapping is negotiated taking all constraints into account. 3) The negotiated mapping is valid until an invalidate_mappings() callback is received.
Following this would allow a re-implementation of pcie_p2p as a mapping type and would allow a controlled link-severing to send an invalidate_mappings() notifier and the mappings to be re-negotiated.
Importers requiring an early hint as to what the actual mapping would look like would need to be prepared to re-consider at map-time, with a reasonable assumption that the reconsideration and re-mapping would happen seldom enough.
Thanks, Thomas
Regards, Christian.
Thanks, Karl
On Mon, Oct 19, 2026 at 02:43:29PM +0200, Thomas Hellström wrote:
On Mon, 2026-10-19 at 10:35 +0200, Christian König wrote:
On 10/16/26 22:26, Karl Mehltretter wrote:
On Tue, Feb 17, 2026 at 08:11:34PM -0400, Jason Gunthorpe wrote:
- match_args.attach = attach;
- if (dmabuf->ops->match_mapping) {
ret = dmabuf->ops->match_mapping(&match_args);- }
attach->dmabuf = dmabuf; attach->importer_ops = importer_ops; attach->importer_priv = importer_priv;
I may misunderstand the intended contract. match_mapping() runs while the attachment is still zeroed and without dmabuf->resv held.
Patch 11's amdgpu callback needs importer_ops and importer_priv, so its XGMI match cannot work here. Patch 12's VFIO callback asserts dmabuf-
resv and
reads priv->vdev, which cleanup changes under that lock.
Should the core initialize the attachment and hold dmabuf->resv for match_mapping(), or should callbacks avoid attachment and lock- protected state?
Right. This completely misplaced.
The matching can't be done at attachment time, but rather needs to be done before creating a mapping while holding the resv lock so that the matching decision can't change.
So question is what exactly should the contract be? Looking at the pcie_p2p negotiation today it looks somewhat like.
- Importer and exporter agree at attach time what mappings are allowed
pending placement conditions and other attachers. 2) At map-time the mapping is negotiated taking all constraints into account. 3) The negotiated mapping is valid until an invalidate_mappings() callback is received.
Following this would allow a re-implementation of pcie_p2p as a mapping type and would allow a controlled link-severing to send an invalidate_mappings() notifier and the mappings to be re-negotiated.
Importers requiring an early hint as to what the actual mapping would look like would need to be prepared to re-consider at map-time, with a reasonable assumption that the reconsideration and re-mapping would happen seldom enough.
The idea was the supported mapping types are fixed properties of the importer and exporter.
Like iommufd is not suddenly going to start understanding ualink after an invalidation.
Further, I had the (perhaps naive) expectation that the interconnect option was not something that would change during mapping. I did not think you would loose your PCIe option if ualink becomes available, or vice versa. Though on deeper reflection I suppose someone could build HW so you might loose your ualink option if the dmabuf is moved to system memory while PCIe would still work.
Making the decision right before map seems like a reasonable enhancement, but I can no longer remember off hand if that would cause complications :)
Jason
On Mon, 2026-10-19 at 10:30 -0300, Jason Gunthorpe wrote:
On Mon, Oct 19, 2026 at 02:43:29PM +0200, Thomas Hellström wrote:
On Mon, 2026-10-19 at 10:35 +0200, Christian König wrote:
On 10/16/26 22:26, Karl Mehltretter wrote:
On Tue, Feb 17, 2026 at 08:11:34PM -0400, Jason Gunthorpe wrote:
- match_args.attach = attach;
- if (dmabuf->ops->match_mapping) {
ret = dmabuf->ops-match_mapping(&match_args);
- }
attach->dmabuf = dmabuf; attach->importer_ops = importer_ops; attach->importer_priv = importer_priv;
I may misunderstand the intended contract. match_mapping() runs while the attachment is still zeroed and without dmabuf->resv held.
Patch 11's amdgpu callback needs importer_ops and importer_priv, so its XGMI match cannot work here. Patch 12's VFIO callback asserts dmabuf-
resv and
reads priv->vdev, which cleanup changes under that lock.
Should the core initialize the attachment and hold dmabuf->resv for match_mapping(), or should callbacks avoid attachment and lock- protected state?
Right. This completely misplaced.
The matching can't be done at attachment time, but rather needs to be done before creating a mapping while holding the resv lock so that the matching decision can't change.
So question is what exactly should the contract be? Looking at the pcie_p2p negotiation today it looks somewhat like.
- Importer and exporter agree at attach time what mappings are
allowed pending placement conditions and other attachers. 2) At map-time the mapping is negotiated taking all constraints into account. 3) The negotiated mapping is valid until an invalidate_mappings() callback is received.
Following this would allow a re-implementation of pcie_p2p as a mapping type and would allow a controlled link-severing to send an invalidate_mappings() notifier and the mappings to be re- negotiated.
Importers requiring an early hint as to what the actual mapping would look like would need to be prepared to re-consider at map-time, with a reasonable assumption that the reconsideration and re-mapping would happen seldom enough.
The idea was the supported mapping types are fixed properties of the importer and exporter.
Like iommufd is not suddenly going to start understanding ualink after an invalidation.
Further, I had the (perhaps naive) expectation that the interconnect option was not something that would change during mapping. I did not think you would loose your PCIe option if ualink becomes available, or vice versa. Though on deeper reflection I suppose someone could build HW so you might loose your ualink option if the dmabuf is moved to system memory while PCIe would still work.
True, but there are other complications. Let's say at map time there is another importer that don't support anything but system memory dma- bufs. Then both ualink and PCIe p2p could not be used, and we'd fall back to system. Or a map() move to VRAM from system for a fast interconnect failed for some reason. (Somebody might have pinned it in system).
But agreed it sounds to me like the interconnect capabilities should be fixed at attach time.
Making the decision right before map seems like a reasonable enhancement, but I can no longer remember off hand if that would cause complications :)
It was that mellanox ATS thing, right? That's the only case I heard of that needed the mapping type decision (or at least a strong hint) early.
/Thomas
Jason
linaro-mm-sig@lists.linaro.org
-
Christian König -
Jason Gunthorpe -
Thomas Hellström