Linaro-mm-sig March 2024

linaro-mm-sig@lists.linaro.org

19 participants
32 discussions

[PATCH v8 0/6] iio: new DMABUF based API

by Paul Cercueil

Hi Jonathan, Here's the final(tm) version of the IIO DMABUF patchset. This v8 fixes the remaining few issues that Christian reported. I also updated the documentation patch as there has been changes to index.rst. This was based on next-20240308. Changelog: - [3/6]: - Fix swapped fence direction - Simplify fence wait mechanism - Remove "Buffer closed with active transfers" print, as it was dead code - Un-export iio_buffer_dmabuf_{get,put}. They are not used anywhere else so they can even be static. - Prevent attaching already-attached DMABUFs - [6/6]: Renamed dmabuf_api.rst -> iio_dmabuf_api.rst, and updated index.rst whose format changed in iio/togreg. Cheers, -Paul Paul Cercueil (6): dmaengine: Add API function dmaengine_prep_peripheral_dma_vec() dmaengine: dma-axi-dmac: Implement device_prep_peripheral_dma_vec iio: core: Add new DMABUF interface infrastructure iio: buffer-dma: Enable support for DMABUFs iio: buffer-dmaengine: Support new DMABUF based userspace API Documentation: iio: Document high-speed DMABUF based API Documentation/iio/iio_dmabuf_api.rst | 54 ++ Documentation/iio/index.rst | 1 + drivers/dma/dma-axi-dmac.c | 40 ++ drivers/iio/buffer/industrialio-buffer-dma.c | 181 ++++++- .../buffer/industrialio-buffer-dmaengine.c | 59 ++- drivers/iio/industrialio-buffer.c | 462 ++++++++++++++++++ include/linux/dmaengine.h | 27 + include/linux/iio/buffer-dma.h | 31 ++ include/linux/iio/buffer_impl.h | 30 ++ include/uapi/linux/iio/buffer.h | 22 + 10 files changed, 890 insertions(+), 17 deletions(-) create mode 100644 Documentation/iio/iio_dmabuf_api.rst -- 2.43.0

1 month, 4 weeks

Re: [PATCH RESEND drm-misc 4/4] dma-buf: heaps: make dma_heap_class constant

by T.J. Mercier

On Tue, Mar 5, 2024 at 3:34 AM Ricardo B. Marliere <ricardo(a)marliere.net> wrote: > > Since commit 43a7206b0963 ("driver core: class: make class_register() take > a const *"), the driver core allows for struct class to be in read-only > memory, so move the dma_heap_class structure to be declared at build time > placing it into read-only memory, instead of having to be dynamically > allocated at boot time. > > Cc: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> > Suggested-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> > Signed-off-by: Ricardo B. Marliere <ricardo(a)marliere.net> > --- > drivers/dma-buf/dma-heap.c | 26 ++++++++++++++------------ > 1 file changed, 14 insertions(+), 12 deletions(-) > > diff --git a/drivers/dma-buf/dma-heap.c b/drivers/dma-buf/dma-heap.c > index 84ae708fafe7..bcca6a2bbce8 100644 > --- a/drivers/dma-buf/dma-heap.c > +++ b/drivers/dma-buf/dma-heap.c > @@ -43,10 +43,18 @@ struct dma_heap { > struct cdev heap_cdev; > }; > > +static char *dma_heap_devnode(const struct device *dev, umode_t *mode) > +{ > + return kasprintf(GFP_KERNEL, "dma_heap/%s", dev_name(dev)); > +} > + > static LIST_HEAD(heap_list); > static DEFINE_MUTEX(heap_list_lock); > static dev_t dma_heap_devt; > -static struct class *dma_heap_class; > +static struct class dma_heap_class = { > + .name = DEVNAME, > + .devnode = dma_heap_devnode, > +}; > static DEFINE_XARRAY_ALLOC(dma_heap_minors); > > static int dma_heap_buffer_alloc(struct dma_heap *heap, size_t len, > @@ -261,7 +269,7 @@ struct dma_heap *dma_heap_add(const struct dma_heap_export_info *exp_info) > goto err1; > } > > - dev_ret = device_create(dma_heap_class, > + dev_ret = device_create(&dma_heap_class, > NULL, > heap->heap_devt, > NULL, > @@ -291,7 +299,7 @@ struct dma_heap *dma_heap_add(const struct dma_heap_export_info *exp_info) > return heap; > > err3: > - device_destroy(dma_heap_class, heap->heap_devt); > + device_destroy(&dma_heap_class, heap->heap_devt); > err2: > cdev_del(&heap->heap_cdev); > err1: > @@ -301,11 +309,6 @@ struct dma_heap *dma_heap_add(const struct dma_heap_export_info *exp_info) > return err_ret; > } > > -static char *dma_heap_devnode(const struct device *dev, umode_t *mode) > -{ > - return kasprintf(GFP_KERNEL, "dma_heap/%s", dev_name(dev)); > -} > - > static int dma_heap_init(void) > { > int ret; > @@ -314,12 +317,11 @@ static int dma_heap_init(void) > if (ret) > return ret; > > - dma_heap_class = class_create(DEVNAME); > - if (IS_ERR(dma_heap_class)) { > + ret = class_register(&dma_heap_class); > + if (ret) { > unregister_chrdev_region(dma_heap_devt, NUM_HEAP_MINORS); > - return PTR_ERR(dma_heap_class); > + return ret; > } > - dma_heap_class->devnode = dma_heap_devnode; > > return 0; > } > > -- > 2.43.0 Reviewed-by: T.J. Mercier <tjmercier(a)google.com> Is this really a resend? I don't see anything on lore and I can't recall seeing this patch in my inbox before.

2 months

Re: [PATCH RESEND drm-misc 4/4] dma-buf: heaps: make dma_heap_class constant

by T.J. Mercier

On Tue, Mar 5, 2024 at 10:02 AM Ricardo B. Marliere <ricardo(a)marliere.net> wrote: > > On 5 Mar 09:07, T.J. Mercier wrote: > > > > Reviewed-by: T.J. Mercier <tjmercier(a)google.com> > > > > Is this really a resend? I don't see anything on lore and I can't > > recall seeing this patch in my inbox before. > > Hi T.J. thanks for reviewing! > > I'm sorry about that, I sent the series only to Greg before but I > thought it had Cc'ed the lists as well. Then I realized it was sent > publicly only once. Double mistake :( > > Best regards, > - Ricardo. Cheers, glad I don't have to try to rework my email filters. :)

2 months

Re: [PATCH] dma-buf: Add syntax highlighting to code listings in the document

by T.J. Mercier

On Thu, Jan 18, 2024 at 7:33 PM Tommy Chiang <ototot(a)chromium.org> wrote: > > This patch tries to improve the display of the code listing > on The Linux Kernel documentation website for dma-buf [1] . > > Originally, it appears that it was attempting to escape > the '*' character, but looks like it's not necessary (now), > so we are seeing something like '\*' on the webite. > > This patch removes these unnecessary backslashes and adds syntax > highlighting to improve the readability of the code listing. > > [1] https://docs.kernel.org/driver-api/dma-buf.html > > Signed-off-by: Tommy Chiang <ototot(a)chromium.org> > --- > drivers/dma-buf/dma-buf.c | 15 +++++++++------ > 1 file changed, 9 insertions(+), 6 deletions(-) > > diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c > index 8fe5aa67b167..e083a0ab06d7 100644 > --- a/drivers/dma-buf/dma-buf.c > +++ b/drivers/dma-buf/dma-buf.c > @@ -1282,10 +1282,12 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_move_notify, DMA_BUF); > * vmap interface is introduced. Note that on very old 32-bit architectures > * vmalloc space might be limited and result in vmap calls failing. > * > - * Interfaces:: > + * Interfaces: > * > - * void \*dma_buf_vmap(struct dma_buf \*dmabuf, struct iosys_map \*map) > - * void dma_buf_vunmap(struct dma_buf \*dmabuf, struct iosys_map \*map) > + * .. code-block:: c > + * > + * void *dma_buf_vmap(struct dma_buf *dmabuf, struct iosys_map *map) > + * void dma_buf_vunmap(struct dma_buf *dmabuf, struct iosys_map *map) > * > * The vmap call can fail if there is no vmap support in the exporter, or if > * it runs out of vmalloc space. Note that the dma-buf layer keeps a reference > @@ -1342,10 +1344,11 @@ EXPORT_SYMBOL_NS_GPL(dma_buf_move_notify, DMA_BUF); > * enough, since adding interfaces to intercept pagefaults and allow pte > * shootdowns would increase the complexity quite a bit. > * > - * Interface:: > + * Interface: > + * > + * .. code-block:: c > * > - * int dma_buf_mmap(struct dma_buf \*, struct vm_area_struct \*, > - * unsigned long); > + * int dma_buf_mmap(struct dma_buf *, struct vm_area_struct *, unsigned long); > * > * If the importing subsystem simply provides a special-purpose mmap call to > * set up a mapping in userspace, calling do_mmap with &dma_buf.file will > -- > 2.43.0.381.gb435a96ce8-goog Reviewed-by: T.J. Mercier <tjmercier(a)google.com> The code block highlighting is nice.

2 months

Re: [PATCH v2 2/2] net: ethernet: ti: am65-cpsw: Add minimal XDP support

by Andrew Lunn

> 3) From 2), am65_cpsw_alloc_skb() function removed and replaced by > netdev_alloc_skb_ip_align(), as used by the driver before -> res = 506 > Conclusion: Here is where the loss comes from. > IOW, My am65_cpsw_alloc_skb() function is not good. > > Initially, I mainly created this 'custom' am65_cpsw_alloc_skb() function > because I thought that none of XDP memory models could be used along > with netdev_alloc_skb_ip_align() function. Was I wrong ? > By creating this custom am65_cpsw_alloc_skb(), I also wanted to handle > the way headroom is reserved differently. What is special about your device? Why would netdev_alloc_skb_ip_align() not work? Andrew

2 months

Re: [PATCH v2 2/2] net: ethernet: ti: am65-cpsw: Add minimal XDP support

by Andrew Lunn

On Tue, Mar 05, 2024 at 11:46:00AM +0100, Julien Panis wrote: > On 3/1/24 17:38, Andrew Lunn wrote: > > On Fri, Mar 01, 2024 at 04:02:53PM +0100, Julien Panis wrote: > > > This patch adds XDP (eXpress Data Path) support to TI AM65 CPSW > > > Ethernet driver. The following features are implemented: > > > - NETDEV_XDP_ACT_BASIC (XDP_PASS, XDP_TX, XDP_DROP, XDP_ABORTED) > > > - NETDEV_XDP_ACT_REDIRECT (XDP_REDIRECT) > > > - NETDEV_XDP_ACT_NDO_XMIT (ndo_xdp_xmit callback) > > > > > > The page pool memory model is used to get better performance. > > Do you have any benchmark numbers? It should help with none XDP > > traffic as well. So maybe iperf numbers before and after? > > > > Andrew > > Argh...Houston, we have a problem. I checked my v3, which is ready for > submission, with iperf3: > 1) Before = without page pool -> 500 MBits/sec > 2) After = with page pool -> 442 MBits/sec > -> ~ 10% worse with page pool here. > > Unless the difference is not due to page pool. Maybe there's something else > which is not good in my patch. I'm going to send the v3 which uses page pool, > hopefully someone will find out something suspicious. Meanwhile, I'll carry on > investigating: I'll check the results with my patch, by removing only the using of > page pool. You can also go the other way. First add page pool support. For the FEC, that improved its performance. Then add XDP, which i think decreased the performance a little. It is extra processing in the hot path, so a little loss is not unsurprising. What tends to be expensive with ARM is cache invalidation and flush. So make sure you have the lengths correct. You don't want to operate on more memory than necessary. No point flushing the full MTU for a 64 byte TCP ACK, etc. Andrew

2 months

Re: [PATCH v7 0/6] iio: new DMABUF based API

by Paul Cercueil

Hi Jonathan, Le mardi 05 mars 2024 à 10:07 +0000, Jonathan Cameron a écrit : > On Mon, 04 Mar 2024 08:59:47 +0100 > Nuno Sá <noname.nuno(a)gmail.com> wrote: > > > On Sun, 2024-03-03 at 17:42 +0000, Jonathan Cameron wrote: > > > On Fri, 23 Feb 2024 13:13:58 +0100 > > > Nuno Sa <nuno.sa(a)analog.com> wrote: > > > > > > > Hi Jonathan, likely you're wondering why I'm sending v7. Well, > > > > to be > > > > honest, we're hoping to get this merged this for the 6.9 merge > > > > window. > > > > Main reason is because the USB part is already in (so it would > > > > be nice > > > > to get the whole thing in). Moreover, the changes asked in v6 > > > > were simple > > > > (even though I'm not quite sure in one of them) and Paul has no > > > > access to > > > > it's laptop so he can't send v7 himself. So he kind of > > > > said/asked for me to > > > > do it. > > > > > > So, we are cutting this very fine. If Linus hints strongly at an > > > rc8 maybe we > > > can sneak this in. However, I need an Ack from Vinod for the dma > > > engine > > > changes first. > > > > > > Also I'd love a final 'looks ok' comment from DMABUF folk (Ack > > > even better!) > > > > > > Seems that the other side got resolved in the USB gadget, but > > > last we heard > > > form > > > Daniel and Christian looks to have been back on v5. I'd like them > > > to confirm > > > they are fine with the changes made as a result. > > > > > > > I can ask Christian or Daniel for some acks but my feeling (I still > > need, at > > some point, to get really familiar with all of this) is that this > > should be > > pretty similar to the USB series (from a DMABUF point of view) as > > they are both > > importers. > > > > > I've been happy with the IIO parts for a few versions now but my > > > ability to > > > review > > > the DMABUF and DMA engine bits is limited. > > > > > > A realistic path to get this in is rc8 is happening, is all Acks > > > in place by > > > Wednesday, > > > I get apply it and hits Linux-next Thursday, Pull request to Greg > > > on Saturday > > > and Greg > > > is feeling particularly generous to take one on the day he > > > normally closes his > > > trees. > > > > > > > Well, it looks like we still have a shot. I'll try to see if Vinod > > is fine with > > the DMAENGINE stuff. > > > > Sadly, looks like rc7 was at the end of a quiet week, so almost > certain to not > be an rc8 in the end. Let's aim to get this in at the start of the > next cycle > so we can build on it from there. And it looks like I'll need a V8 for the few things noted by Christian. Having it in 6.9 would have been great but having it eventually merged is all that matters - so I'm fine to have it queued for 6.10 instead. Cheers, -Paul

2 months

Re: ECC memory semantics for heaps

by John Stultz

On Mon, Mar 4, 2024 at 5:46 AM Maxime Ripard <mripard(a)redhat.com> wrote: > On Wed, Feb 28, 2024 at 08:17:55PM -0800, John Stultz wrote: > > On Wed, Feb 28, 2024 at 7:24 AM Maxime Ripard <mripard(a)redhat.com> wrote: > > > > > > I'm currently working on a platform that seems to have togglable RAM ECC > > > support. Enabling ECC reduces the memory capacity and memory bandwidth, > > > so while it's a good idea to protect most of the system, it's not worth > > > it for things like framebuffers that won't really be affected by a > > > bitflip. > > > > > > It's currently setup by enabling ECC on the entire memory, and then > > > having a region of memory where ECC is disabled and where we're supposed > > > to allocate from for allocations that don't need it. > > > > > > My first thought to support this was to create a reserved memory region > > > for the !ECC memory, and to create a heap to allocate buffers from that > > > region. That would leave the system protected by ECC, while enabling > > > userspace to be nicer to the system by allocating buffers from the !ECC > > > region if it doesn't need it. > > > > > > However, this creates basically a new combination compared to the one we > > > already have (ie, physically contiguous vs virtually contiguous), and we > > > probably would want to throw in cacheable vs non-cacheable too. > > > > > > If we had to provide new heaps for each variation, we would have 8 heaps > > > (and 6 new ones), which could be fine I guess but would still increase > > > quite a lot the number of heaps we have so far. > > > > > > Is it something that would be a problem? If it is, do you see another > > > way to support those kind of allocations (like providing hints through > > > the ioctl maybe?)? > > > > So, the dma-buf heaps interface uses chardevs so that we can have a > > lot of flexibility in the types of heaps (and don't have the risk of > > bitmask exhaustion like ION had). So I don't see adding many > > differently named heaps as particularly problematic. > > Ok > > > That said, if there are truly generic properties (cacheable vs > > non-cachable is maybe one of those) which apply to most heaps, I'm > > open to making use of the flags. But I want to avoid having per-heap > > flags, it really needs to be a generic attribute. > > > > And I personally don't mind the idea of having things added as heaps > > initially, and potentially upgrading them to flags if needed (allowing > > heap drivers to optionally enumerate the old chardevs behind a config > > option for backwards compatibility). > > > > How common is the hardware that is going to provide this configurable > > ECC option > > In terms of number of SoCs with the feature, it's probably a handful. In > terms of number of units shipped, we're in the fairly common range :) > Sure, I guess I was trying to get a sense of is this a feature we'll likely be seeing commonly across hardware (such that internal kernel allocators would be considering it as a flag), or is it more tied to a single vendor such that enabling/isolating it in a driver is the right place in the abstraction to put it. > > and will you really want the option on all of the heap types? > > Aside from the cacheable/uncacheable discussion, yes. We could probably > get away with only physically contiguous allocations at the moment > though, I'll double check. Ok, that will be useful to know. > > Will there be any hardware constraint limitations caused by the > > ECC/!ECC flags? (ie: Devices that can't use !ECC allocated buffers?) > > My understanding is that there's no device restriction. It will be a > carved out memory so we will need to maintain a separate pool and it > will be limited in size, but that's pretty much the only one afaik. Ok. > > If not, I wonder if it would make sense to have something more along > > the lines using a fcntl() like how F_SEAL_* is used with memfds? > > With some of the discussion around "restricted"/"secure" heaps that > > can change state, I've liked this idea of just allocating dmabufs from > > normal heaps and then using fcntl or something similar to modify > > properties of the buffer that are separate from the type of memory > > that is needed to be allocated to satisfy device constraints. > > Sorry, I'm not super familiar with F_SEAL so I don't really follow what > you have in mind here. Do you have any additional resources I could read > to understand better what you're thinking about? See the File Sealing section: https://man7.org/linux/man-pages/man2/fcntl.2.html > Also, if we were to modify the ECC attributes after the dma-buf has been > allocated by dma-buf, and if the !ECC memory is carved out only, then > wouldn't that mean we would need to reallocate the backing buffer for > that dma-buf? So yea, having to work on a larger pool likely makes this not useful here, so apologies for the tangent. To explain myself, part of what I'm thinking of is, the dmabuf heaps (and really ION before it) try to solve how to allocate a buffer type that can be used across a number of devices that may have different constraints. So the focus was on "types of memory" to satisfy the constraint (contiguous, non-contiguous, secure/restricted, etc), which come down to what pages actually get used. However, outside of the "constraint type" the buffer may have, there are other "properties" that may affect performance (like cacheability, and some variants of "restricted buffers" which can change over their lifetime). With ION vendors would mix these two together in their vendor heaps, and with out-of-tree dmabuf heaps it is also common to tangle types and properties together. So I'm sort of stewing on how to best distinguish between heaps for "types of memory/pages" (ie: what's *required* to share the buffer between devices) vs these buffer properties (which affect performance) that may apply to multiple memory types. (And I'm also not 100% convinced that distinguishing between this is necessary, but casually mixing them feels messy to me) For buffers where those properties might change over time (like some variants of restricted buffers), I think the fnctl/F_SEAL_* idea makes sense to allow the buffer to become restricted. For cacheability, it seems likely an allocation flag would be nicest, but we don't have upstream users and not a lot of heap types yet, thus the out-of-tree "system-uncached" heap which sort of mixes types and properties. With ECC I was trying to get a sense of where it would sit between this "type of memory" vs a "buffer property". If internal allocators are likely to consider it in a way similar to CMA (and with the pool granular control, it sounds like it), then yeah, it probably should be a type of memory, so a new heap name is likely the way to go - but there is still the question of how to best support the various combinations of (contiguous, cacheable) along with ECC. But if it were something that was dynamically controllable at a finer grained level in the future, maybe it would be something like a buffer property. thanks -john

2 months

Re: [PATCH v7 3/6] iio: core: Add new DMABUF interface infrastructure

by Christian König

Am 23.02.24 um 13:14 schrieb Nuno Sa: > From: Paul Cercueil <paul(a)crapouillou.net> > > Add the necessary infrastructure to the IIO core to support a new > optional DMABUF based interface. > > With this new interface, DMABUF objects (externally created) can be > attached to a IIO buffer, and subsequently used for data transfer. > > A userspace application can then use this interface to share DMABUF > objects between several interfaces, allowing it to transfer data in a > zero-copy fashion, for instance between IIO and the USB stack. > > The userspace application can also memory-map the DMABUF objects, and > access the sample data directly. The advantage of doing this vs. the > read() interface is that it avoids an extra copy of the data between the > kernel and userspace. This is particularly userful for high-speed > devices which produce several megabytes or even gigabytes of data per > second. > > As part of the interface, 3 new IOCTLs have been added: > > IIO_BUFFER_DMABUF_ATTACH_IOCTL(int fd): > Attach the DMABUF object identified by the given file descriptor to the > buffer. > > IIO_BUFFER_DMABUF_DETACH_IOCTL(int fd): > Detach the DMABUF object identified by the given file descriptor from > the buffer. Note that closing the IIO buffer's file descriptor will > automatically detach all previously attached DMABUF objects. > > IIO_BUFFER_DMABUF_ENQUEUE_IOCTL(struct iio_dmabuf *): > Request a data transfer to/from the given DMABUF object. Its file > descriptor, as well as the transfer size and flags are provided in the > "iio_dmabuf" structure. > > These three IOCTLs have to be performed on the IIO buffer's file > descriptor, obtained using the IIO_BUFFER_GET_FD_IOCTL() ioctl. A few nit picks and one bug below, apart from that looks good to me. > > Signed-off-by: Paul Cercueil <paul(a)crapouillou.net> > Signed-off-by: Nuno Sa <nuno.sa(a)analog.com> > --- > drivers/iio/industrialio-buffer.c | 462 ++++++++++++++++++++++++++++++++++++++ > include/linux/iio/buffer_impl.h | 33 +++ > include/uapi/linux/iio/buffer.h | 22 ++ > 3 files changed, 517 insertions(+) > > diff --git a/drivers/iio/industrialio-buffer.c b/drivers/iio/industrialio-buffer.c > index b581a7e80566..0e63a09fa90a 100644 > --- a/drivers/iio/industrialio-buffer.c > +++ b/drivers/iio/industrialio-buffer.c > @@ -9,14 +9,19 @@ > * - Better memory allocation techniques? > * - Alternative access techniques? > */ > +#include <linux/atomic.h> > #include <linux/anon_inodes.h> > #include <linux/kernel.h> > #include <linux/export.h> > #include <linux/device.h> > +#include <linux/dma-buf.h> > +#include <linux/dma-fence.h> > +#include <linux/dma-resv.h> > #include <linux/file.h> > #include <linux/fs.h> > #include <linux/cdev.h> > #include <linux/slab.h> > +#include <linux/mm.h> > #include <linux/poll.h> > #include <linux/sched/signal.h> > > @@ -28,6 +33,32 @@ > #include <linux/iio/buffer.h> > #include <linux/iio/buffer_impl.h> > > +#define DMABUF_ENQUEUE_TIMEOUT_MS 5000 > + > +struct iio_dma_fence; > + > +struct iio_dmabuf_priv { > + struct list_head entry; > + struct kref ref; > + > + struct iio_buffer *buffer; > + struct iio_dma_buffer_block *block; > + > + u64 context; > + spinlock_t lock; > + > + struct dma_buf_attachment *attach; > + struct sg_table *sgt; > + enum dma_data_direction dir; > + atomic_t seqno; > +}; > + > +struct iio_dma_fence { > + struct dma_fence base; > + struct iio_dmabuf_priv *priv; > + struct work_struct work; > +}; > + > static const char * const iio_endian_prefix[] = { > [IIO_BE] = "be", > [IIO_LE] = "le", > @@ -332,6 +363,8 @@ void iio_buffer_init(struct iio_buffer *buffer) > { > INIT_LIST_HEAD(&buffer->demux_list); > INIT_LIST_HEAD(&buffer->buffer_list); > + INIT_LIST_HEAD(&buffer->dmabufs); > + mutex_init(&buffer->dmabufs_mutex); > init_waitqueue_head(&buffer->pollq); > kref_init(&buffer->ref); > if (!buffer->watermark) > @@ -1519,14 +1552,62 @@ static void iio_buffer_unregister_legacy_sysfs_groups(struct iio_dev *indio_dev) > kfree(iio_dev_opaque->legacy_scan_el_group.attrs); > } > > +static void iio_buffer_dmabuf_release(struct kref *ref) > +{ > + struct iio_dmabuf_priv *priv = container_of(ref, struct iio_dmabuf_priv, ref); > + struct dma_buf_attachment *attach = priv->attach; > + struct iio_buffer *buffer = priv->buffer; > + struct dma_buf *dmabuf = attach->dmabuf; > + > + dma_resv_lock(dmabuf->resv, NULL); > + dma_buf_unmap_attachment(attach, priv->sgt, priv->dir); > + dma_resv_unlock(dmabuf->resv); > + > + buffer->access->detach_dmabuf(buffer, priv->block); > + > + dma_buf_detach(attach->dmabuf, attach); > + dma_buf_put(dmabuf); > + kfree(priv); > +} > + > +void iio_buffer_dmabuf_get(struct dma_buf_attachment *attach) > +{ > + struct iio_dmabuf_priv *priv = attach->importer_priv; > + > + kref_get(&priv->ref); > +} > +EXPORT_SYMBOL_GPL(iio_buffer_dmabuf_get); > + > +void iio_buffer_dmabuf_put(struct dma_buf_attachment *attach) > +{ > + struct iio_dmabuf_priv *priv = attach->importer_priv; > + > + kref_put(&priv->ref, iio_buffer_dmabuf_release); > +} > +EXPORT_SYMBOL_GPL(iio_buffer_dmabuf_put); > + > static int iio_buffer_chrdev_release(struct inode *inode, struct file *filep) > { > struct iio_dev_buffer_pair *ib = filep->private_data; > struct iio_dev *indio_dev = ib->indio_dev; > struct iio_buffer *buffer = ib->buffer; > + struct iio_dmabuf_priv *priv, *tmp; > > wake_up(&buffer->pollq); > > + mutex_lock(&buffer->dmabufs_mutex); > + > + /* Close all attached DMABUFs */ > + list_for_each_entry_safe(priv, tmp, &buffer->dmabufs, entry) { > + list_del_init(&priv->entry); > + iio_buffer_dmabuf_put(priv->attach); > + } > + > + if (!list_empty(&buffer->dmabufs)) > + dev_warn(&indio_dev->dev, "Buffer FD closed with active transfers\n"); That check here smells fishy. Either the list_for_each_entry_safe() above has removed all entries and then it can never happen or the buffer list is not always properly protected by the mutex. Do you really need that? If yes then please justify. BTW: When the transfers are async it's perfectly possible that there are active transfers on close (when the application is just killed for example). > + > + mutex_unlock(&buffer->dmabufs_mutex); > + > kfree(ib); > clear_bit(IIO_BUSY_BIT_POS, &buffer->flags); > iio_device_put(indio_dev); > @@ -1534,11 +1615,391 @@ static int iio_buffer_chrdev_release(struct inode *inode, struct file *filep) > return 0; > } > > +static int iio_dma_resv_lock(struct dma_buf *dmabuf, bool nonblock) > +{ > + if (!nonblock) > + return dma_resv_lock_interruptible(dmabuf->resv, NULL); > + > + if (!dma_resv_trylock(dmabuf->resv)) > + return -EBUSY; > + > + return 0; > +} > + > +static struct dma_buf_attachment * > +iio_buffer_find_attachment(struct iio_dev_buffer_pair *ib, > + struct dma_buf *dmabuf, bool nonblock) > +{ > + struct device *dev = ib->indio_dev->dev.parent; > + struct iio_buffer *buffer = ib->buffer; > + struct dma_buf_attachment *attach = NULL; > + struct iio_dmabuf_priv *priv; > + > + mutex_lock(&buffer->dmabufs_mutex); > + > + list_for_each_entry(priv, &buffer->dmabufs, entry) { > + if (priv->attach->dev == dev > + && priv->attach->dmabuf == dmabuf) { > + attach = priv->attach; > + break; > + } > + } > + > + if (attach) > + iio_buffer_dmabuf_get(attach); > + > + mutex_unlock(&buffer->dmabufs_mutex); > + > + return attach ?: ERR_PTR(-EPERM); > +} > + > +static int iio_buffer_attach_dmabuf(struct iio_dev_buffer_pair *ib, > + int __user *user_fd, bool nonblock) > +{ > + struct iio_dev *indio_dev = ib->indio_dev; > + struct iio_buffer *buffer = ib->buffer; > + struct dma_buf_attachment *attach; > + struct iio_dmabuf_priv *priv; > + struct dma_buf *dmabuf; > + int err, fd; > + > + if (!buffer->access->attach_dmabuf > + || !buffer->access->detach_dmabuf > + || !buffer->access->enqueue_dmabuf) > + return -EPERM; > + > + if (copy_from_user(&fd, user_fd, sizeof(fd))) > + return -EFAULT; > + > + priv = kzalloc(sizeof(*priv), GFP_KERNEL); > + if (!priv) > + return -ENOMEM; > + > + spin_lock_init(&priv->lock); > + priv->context = dma_fence_context_alloc(1); > + > + dmabuf = dma_buf_get(fd); > + if (IS_ERR(dmabuf)) { > + err = PTR_ERR(dmabuf); > + goto err_free_priv; > + } > + > + attach = dma_buf_attach(dmabuf, indio_dev->dev.parent); > + if (IS_ERR(attach)) { > + err = PTR_ERR(attach); > + goto err_dmabuf_put; > + } > + > + err = iio_dma_resv_lock(dmabuf, nonblock); > + if (err) > + goto err_dmabuf_detach; > + > + priv->dir = buffer->direction == IIO_BUFFER_DIRECTION_IN > + ? DMA_FROM_DEVICE : DMA_TO_DEVICE; > + > + priv->sgt = dma_buf_map_attachment(attach, priv->dir); > + if (IS_ERR(priv->sgt)) { > + err = PTR_ERR(priv->sgt); > + dev_err(&indio_dev->dev, "Unable to map attachment: %d\n", err); > + goto err_resv_unlock; > + } > + > + kref_init(&priv->ref); > + priv->buffer = buffer; > + priv->attach = attach; > + attach->importer_priv = priv; > + > + priv->block = buffer->access->attach_dmabuf(buffer, attach); > + if (IS_ERR(priv->block)) { > + err = PTR_ERR(priv->block); > + goto err_dmabuf_unmap_attachment; > + } > + > + dma_resv_unlock(dmabuf->resv); > + > + mutex_lock(&buffer->dmabufs_mutex); > + list_add(&priv->entry, &buffer->dmabufs); > + mutex_unlock(&buffer->dmabufs_mutex); Is it valid to attach the same dma_buf multiple times? If not that code here should probably check that. > + > + return 0; > + > +err_dmabuf_unmap_attachment: > + dma_buf_unmap_attachment(attach, priv->sgt, priv->dir); > +err_resv_unlock: > + dma_resv_unlock(dmabuf->resv); > +err_dmabuf_detach: > + dma_buf_detach(dmabuf, attach); > +err_dmabuf_put: > + dma_buf_put(dmabuf); > +err_free_priv: > + kfree(priv); > + > + return err; > +} > + > +static int iio_buffer_detach_dmabuf(struct iio_dev_buffer_pair *ib, > + int *user_req, bool nonblock) > +{ > + struct iio_buffer *buffer = ib->buffer; > + struct iio_dev *indio_dev = ib->indio_dev; > + struct iio_dmabuf_priv *priv; > + struct dma_buf *dmabuf; > + int dmabuf_fd, ret = -EPERM; ENOENT is more common to use when something can't be found I think, but perfectly up to you what you prefer. > + > + if (copy_from_user(&dmabuf_fd, user_req, sizeof(dmabuf_fd))) > + return -EFAULT; > + > + dmabuf = dma_buf_get(dmabuf_fd); > + if (IS_ERR(dmabuf)) > + return PTR_ERR(dmabuf); > + > + mutex_lock(&buffer->dmabufs_mutex); > + > + list_for_each_entry(priv, &buffer->dmabufs, entry) { > + if (priv->attach->dev == indio_dev->dev.parent > + && priv->attach->dmabuf == dmabuf) { > + list_del(&priv->entry); > + > + /* Unref the reference from iio_buffer_attach_dmabuf() */ > + iio_buffer_dmabuf_put(priv->attach); > + ret = 0; > + break; > + } > + } > + > + mutex_unlock(&buffer->dmabufs_mutex); > + dma_buf_put(dmabuf); > + > + return ret; > +} > + > +static const char * > +iio_buffer_dma_fence_get_driver_name(struct dma_fence *fence) > +{ > + return "iio"; > +} > + > +static void iio_buffer_dma_fence_release(struct dma_fence *fence) > +{ > + struct iio_dma_fence *iio_fence = > + container_of(fence, struct iio_dma_fence, base); > + > + kfree(iio_fence); > +} > + > +static const struct dma_fence_ops iio_buffer_dma_fence_ops = { > + .get_driver_name = iio_buffer_dma_fence_get_driver_name, > + .get_timeline_name = iio_buffer_dma_fence_get_driver_name, > + .release = iio_buffer_dma_fence_release, > +}; > + > +static int iio_buffer_enqueue_dmabuf(struct iio_dev_buffer_pair *ib, > + struct iio_dmabuf __user *iio_dmabuf_req, > + bool nonblock) > +{ > + struct iio_buffer *buffer = ib->buffer; > + struct iio_dmabuf iio_dmabuf; > + struct dma_buf_attachment *attach; > + struct iio_dmabuf_priv *priv; > + struct iio_dma_fence *fence; > + struct dma_buf *dmabuf; > + unsigned long timeout; > + bool cookie, cyclic, dma_to_ram; > + long retl; > + u32 seqno; > + int ret; > + > + if (copy_from_user(&iio_dmabuf, iio_dmabuf_req, sizeof(iio_dmabuf))) > + return -EFAULT; > + > + if (iio_dmabuf.flags & ~IIO_BUFFER_DMABUF_SUPPORTED_FLAGS) > + return -EINVAL; > + > + cyclic = iio_dmabuf.flags & IIO_BUFFER_DMABUF_CYCLIC; > + > + /* Cyclic flag is only supported on output buffers */ > + if (cyclic && buffer->direction != IIO_BUFFER_DIRECTION_OUT) > + return -EINVAL; > + > + dmabuf = dma_buf_get(iio_dmabuf.fd); > + if (IS_ERR(dmabuf)) > + return PTR_ERR(dmabuf); > + > + if (!iio_dmabuf.bytes_used || iio_dmabuf.bytes_used > dmabuf->size) { > + ret = -EINVAL; > + goto err_dmabuf_put; > + } > + > + attach = iio_buffer_find_attachment(ib, dmabuf, nonblock); > + if (IS_ERR(attach)) { > + ret = PTR_ERR(attach); > + goto err_dmabuf_put; > + } > + > + priv = attach->importer_priv; > + > + fence = kmalloc(sizeof(*fence), GFP_KERNEL); > + if (!fence) { > + ret = -ENOMEM; > + goto err_attachment_put; > + } > + > + fence->priv = priv; > + > + seqno = atomic_add_return(1, &priv->seqno); > + > + /* > + * The transfers are guaranteed to be processed in the order they are > + * enqueued, so we can use a simple incrementing sequence number for > + * the dma_fence. > + */ > + dma_fence_init(&fence->base, &iio_buffer_dma_fence_ops, > + &priv->lock, priv->context, seqno); > + > + ret = iio_dma_resv_lock(dmabuf, nonblock); > + if (ret) > + goto err_fence_put; > + > + timeout = nonblock ? 0 : msecs_to_jiffies(DMABUF_ENQUEUE_TIMEOUT_MS); > + > + /* Make sure we don't have writers */ > + retl = dma_resv_wait_timeout(dmabuf->resv, DMA_RESV_USAGE_WRITE, > + true, timeout); > + if (retl == 0) > + retl = -EBUSY; > + if (retl < 0) { > + ret = (int)retl; > + goto err_resv_unlock; > + } > + > + dma_to_ram = buffer->direction == IIO_BUFFER_DIRECTION_IN; > + > + if (dma_to_ram) { > + /* > + * If we're writing to the DMABUF, make sure we don't have > + * readers > + */ > + retl = dma_resv_wait_timeout(dmabuf->resv, > + DMA_RESV_USAGE_READ, true, > + timeout); > + if (retl == 0) > + retl = -EBUSY; > + if (retl < 0) { > + ret = (int)retl; > + goto err_resv_unlock; > + } > + } > + > + if (buffer->access->lock_queue) > + buffer->access->lock_queue(buffer); > + > + ret = dma_resv_reserve_fences(dmabuf->resv, 1); > + if (ret) > + goto err_queue_unlock; > + > + dma_resv_add_fence(dmabuf->resv, &fence->base, > + dma_resv_usage_rw(dma_to_ram)); That is incorrect use of the function dma_resv_usage_rw(). That function is for retrieving fences and not adding them. See the function implementation and comments, when you use it like this you get exactly what you don't want. Regards, Christian. > + dma_resv_unlock(dmabuf->resv); > + > + cookie = dma_fence_begin_signalling(); > + > + ret = buffer->access->enqueue_dmabuf(buffer, priv->block, &fence->base, > + priv->sgt, iio_dmabuf.bytes_used, > + cyclic); > + if (ret) { > + /* > + * DMABUF enqueue failed, but we already added the fence. > + * Signal the error through the fence completion mechanism. > + */ > + iio_buffer_signal_dmabuf_done(&fence->base, ret); > + } > + > + if (buffer->access->unlock_queue) > + buffer->access->unlock_queue(buffer); > + > + dma_fence_end_signalling(cookie); > + dma_buf_put(dmabuf); > + > + return ret; > + > +err_queue_unlock: > + if (buffer->access->unlock_queue) > + buffer->access->unlock_queue(buffer); > +err_resv_unlock: > + dma_resv_unlock(dmabuf->resv); > +err_fence_put: > + dma_fence_put(&fence->base); > +err_attachment_put: > + iio_buffer_dmabuf_put(attach); > +err_dmabuf_put: > + dma_buf_put(dmabuf); > + > + return ret; > +} > + > +static void iio_buffer_cleanup(struct work_struct *work) > +{ > + struct iio_dma_fence *fence = > + container_of(work, struct iio_dma_fence, work); > + struct iio_dmabuf_priv *priv = fence->priv; > + struct dma_buf_attachment *attach = priv->attach; > + > + dma_fence_put(&fence->base); > + iio_buffer_dmabuf_put(attach); > +} > + > +void iio_buffer_signal_dmabuf_done(struct dma_fence *fence, int ret) > +{ > + struct iio_dma_fence *iio_fence = > + container_of(fence, struct iio_dma_fence, base); > + bool cookie = dma_fence_begin_signalling(); > + > + /* > + * Get a reference to the fence, so that it's not freed as soon as > + * it's signaled. > + */ > + dma_fence_get(fence); > + > + fence->error = ret; > + dma_fence_signal(fence); > + dma_fence_end_signalling(cookie); > + > + /* > + * The fence will be unref'd in iio_buffer_cleanup. > + * It can't be done here, as the unref functions might try to lock the > + * resv object, which can deadlock. > + */ > + INIT_WORK(&iio_fence->work, iio_buffer_cleanup); > + schedule_work(&iio_fence->work); > +} > +EXPORT_SYMBOL_GPL(iio_buffer_signal_dmabuf_done); > + > +static long iio_buffer_chrdev_ioctl(struct file *filp, > + unsigned int cmd, unsigned long arg) > +{ > + struct iio_dev_buffer_pair *ib = filp->private_data; > + void __user *_arg = (void __user *)arg; > + bool nonblock = filp->f_flags & O_NONBLOCK; > + > + switch (cmd) { > + case IIO_BUFFER_DMABUF_ATTACH_IOCTL: > + return iio_buffer_attach_dmabuf(ib, _arg, nonblock); > + case IIO_BUFFER_DMABUF_DETACH_IOCTL: > + return iio_buffer_detach_dmabuf(ib, _arg, nonblock); > + case IIO_BUFFER_DMABUF_ENQUEUE_IOCTL: > + return iio_buffer_enqueue_dmabuf(ib, _arg, nonblock); > + default: > + return -EINVAL; > + } > +} > + > static const struct file_operations iio_buffer_chrdev_fileops = { > .owner = THIS_MODULE, > .llseek = noop_llseek, > .read = iio_buffer_read, > .write = iio_buffer_write, > + .unlocked_ioctl = iio_buffer_chrdev_ioctl, > + .compat_ioctl = compat_ptr_ioctl, > .poll = iio_buffer_poll, > .release = iio_buffer_chrdev_release, > }; > @@ -1953,6 +2414,7 @@ static void iio_buffer_release(struct kref *ref) > { > struct iio_buffer *buffer = container_of(ref, struct iio_buffer, ref); > > + mutex_destroy(&buffer->dmabufs_mutex); > buffer->access->release(buffer); > } > > diff --git a/include/linux/iio/buffer_impl.h b/include/linux/iio/buffer_impl.h > index 89c3fd7c29ca..f4b1147291e5 100644 > --- a/include/linux/iio/buffer_impl.h > +++ b/include/linux/iio/buffer_impl.h > @@ -9,8 +9,12 @@ > #include <uapi/linux/iio/buffer.h> > #include <linux/iio/buffer.h> > > +struct dma_buf_attachment; > +struct dma_fence; > struct iio_dev; > +struct iio_dma_buffer_block; > struct iio_buffer; > +struct sg_table; > > /** > * INDIO_BUFFER_FLAG_FIXED_WATERMARK - Watermark level of the buffer can not be > @@ -39,6 +43,13 @@ struct iio_buffer; > * device stops sampling. Calles are balanced with @enable. > * @release: called when the last reference to the buffer is dropped, > * should free all resources allocated by the buffer. > + * @attach_dmabuf: called from userspace via ioctl to attach one external > + * DMABUF. > + * @detach_dmabuf: called from userspace via ioctl to detach one previously > + * attached DMABUF. > + * @enqueue_dmabuf: called from userspace via ioctl to queue this DMABUF > + * object to this buffer. Requires a valid DMABUF fd, that > + * was previouly attached to this buffer. > * @modes: Supported operating modes by this buffer type > * @flags: A bitmask combination of INDIO_BUFFER_FLAG_* > * > @@ -68,6 +79,17 @@ struct iio_buffer_access_funcs { > > void (*release)(struct iio_buffer *buffer); > > + struct iio_dma_buffer_block * (*attach_dmabuf)(struct iio_buffer *buffer, > + struct dma_buf_attachment *attach); > + void (*detach_dmabuf)(struct iio_buffer *buffer, > + struct iio_dma_buffer_block *block); > + int (*enqueue_dmabuf)(struct iio_buffer *buffer, > + struct iio_dma_buffer_block *block, > + struct dma_fence *fence, struct sg_table *sgt, > + size_t size, bool cyclic); > + void (*lock_queue)(struct iio_buffer *buffer); > + void (*unlock_queue)(struct iio_buffer *buffer); > + > unsigned int modes; > unsigned int flags; > }; > @@ -136,6 +158,12 @@ struct iio_buffer { > > /* @ref: Reference count of the buffer. */ > struct kref ref; > + > + /* @dmabufs: List of DMABUF attachments */ > + struct list_head dmabufs; /* P: dmabufs_mutex */ > + > + /* @dmabufs_mutex: Protects dmabufs */ > + struct mutex dmabufs_mutex; > }; > > /** > @@ -156,9 +184,14 @@ int iio_update_buffers(struct iio_dev *indio_dev, > **/ > void iio_buffer_init(struct iio_buffer *buffer); > > +void iio_buffer_dmabuf_get(struct dma_buf_attachment *attach); > +void iio_buffer_dmabuf_put(struct dma_buf_attachment *attach); > + > struct iio_buffer *iio_buffer_get(struct iio_buffer *buffer); > void iio_buffer_put(struct iio_buffer *buffer); > > +void iio_buffer_signal_dmabuf_done(struct dma_fence *fence, int ret); > + > #else /* CONFIG_IIO_BUFFER */ > > static inline void iio_buffer_get(struct iio_buffer *buffer) {} > diff --git a/include/uapi/linux/iio/buffer.h b/include/uapi/linux/iio/buffer.h > index 13939032b3f6..c666aa95e532 100644 > --- a/include/uapi/linux/iio/buffer.h > +++ b/include/uapi/linux/iio/buffer.h > @@ -5,6 +5,28 @@ > #ifndef _UAPI_IIO_BUFFER_H_ > #define _UAPI_IIO_BUFFER_H_ > > +#include <linux/types.h> > + > +/* Flags for iio_dmabuf.flags */ > +#define IIO_BUFFER_DMABUF_CYCLIC (1 << 0) > +#define IIO_BUFFER_DMABUF_SUPPORTED_FLAGS 0x00000001 > + > +/** > + * struct iio_dmabuf - Descriptor for a single IIO DMABUF object > + * @fd: file descriptor of the DMABUF object > + * @flags: one or more IIO_BUFFER_DMABUF_* flags > + * @bytes_used: number of bytes used in this DMABUF for the data transfer. > + * Should generally be set to the DMABUF's size. > + */ > +struct iio_dmabuf { > + __u32 fd; > + __u32 flags; > + __u64 bytes_used; > +}; > + > #define IIO_BUFFER_GET_FD_IOCTL _IOWR('i', 0x91, int) > +#define IIO_BUFFER_DMABUF_ATTACH_IOCTL _IOW('i', 0x92, int) > +#define IIO_BUFFER_DMABUF_DETACH_IOCTL _IOW('i', 0x93, int) > +#define IIO_BUFFER_DMABUF_ENQUEUE_IOCTL _IOW('i', 0x94, struct iio_dmabuf) > > #endif /* _UAPI_IIO_BUFFER_H_ */ >

2 months

Re: [PATCH v7 1/6] dmaengine: Add API function dmaengine_prep_peripheral_dma_vec()

by Nuno Sá

On Fri, 2024-02-23 at 13:13 +0100, Nuno Sa wrote: > From: Paul Cercueil <paul(a)crapouillou.net> > > This function can be used to initiate a scatter-gather DMA transfer, > where the address and size of each segment is located in one entry of > the dma_vec array. > > The major difference with dmaengine_prep_slave_sg() is that it supports > specifying the lengths of each DMA transfer; as trying to override the > length of the transfer with dmaengine_prep_slave_sg() is a very tedious > process. The introduction of a new API function is also justified by the > fact that scatterlists are on their way out. > > Note that dmaengine_prep_interleaved_dma() is not helpful either in that > case, as it assumes that the address of each segment will be higher than > the one of the previous segment, which we just cannot guarantee in case > of a scatter-gather transfer. > > Signed-off-by: Paul Cercueil <paul(a)crapouillou.net> > Signed-off-by: Nuno Sa <nuno.sa(a)analog.com> > --- Hi Vinod, Is this already good for you? I do not want to be pushy but we're trying to see if we can have this in the 6.9 cycle and Jonathan definitely wants an ack from you before merging this in his tree. I've more or less till Wednesday so that's why I'm asking already today so I still have time to re-spin if you want some changes. - Nuno Sá

2 months

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

Linaro-mm-sig March 2024