Linaro-mm-sig May 2012

linaro-mm-sig@lists.linaro.org

28 participants
37 discussions

by Rob Clark

From: Rob Clark <rob(a)ti.com> Works in a similar way to get_file(), and is needed in cases such as when the exporter needs to also keep a reference to the dmabuf (that is later released with a dma_buf_put()), and possibly other similar cases. Signed-off-by: Rob Clark <rob(a)ti.com> --- include/linux/dma-buf.h | 14 ++++++++++++++ 1 files changed, 14 insertions(+), 0 deletions(-) diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index cbdff81..25eb287 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -132,6 +132,20 @@ struct dma_buf_attachment { void *priv; }; +/** + * get_dma_buf - convenience wrapper for get_file. + * @dmabuf: [in] pointer to dma_buf + * + * Increments the reference count on the dma-buf, needed in case of drivers + * that either need to create additional references to the dmabuf on the + * kernel side. For example, an exporter that needs to keep a dmabuf ptr + * so that subsequent exports don't create a new dmabuf. + */ +static inline void get_dma_buf(struct dma_buf *dmabuf) +{ + get_file(dmabuf->file); +} + #ifdef CONFIG_DMA_SHARED_BUFFER struct dma_buf_attachment *dma_buf_attach(struct dma_buf *dmabuf, struct device *dev); -- 1.7.5.4

11 years, 11 months

[PATCH] dma-buf: add initial vmap documentation

by Dave Airlie

From: Dave Airlie <airlied(a)redhat.com> Signed-off-by: Dave Airlie <airlied(a)redhat.com> --- Documentation/dma-buf-sharing.txt | 11 +++++++++++ 1 files changed, 11 insertions(+), 0 deletions(-) diff --git a/Documentation/dma-buf-sharing.txt b/Documentation/dma-buf-sharing.txt index 3bbd5c5..98e9fa0 100644 --- a/Documentation/dma-buf-sharing.txt +++ b/Documentation/dma-buf-sharing.txt @@ -300,6 +300,17 @@ Access to a dma_buf from the kernel context involves three steps: Note that these calls need to always succeed. The exporter needs to complete any preparations that might fail in begin_cpu_access. + For some circumstances the overhead of kmap can be too high, a vmap interface + is introduced. This interface shouldn't be used very carefully, as vmalloc + space is a limited resources on many architectures. + + Interfaces: + void *dma_buf_vmap(struct dma_buf *dmabuf) + void dma_buf_vunmap(struct dma_buf *dmabuf, void *vaddr) + + This call can fail if there is no vmap support in the exporter, or if it + runs out of vmalloc space. Fallback to kmap should be implemented. + 3. Finish access When the importer is done accessing the range specified in begin_cpu_access, -- 1.7.6

11 years, 11 months

[RFC 00/13] Support for dmabuf exporting for videobuf2

by Tomasz Stanislawski

Hello everyone, The patches adds support for DMABUF [1] exporting to V4L2 stack. It was updated from [7] after Laurent Pinchart's review. The previous patchset was split into two parts. The support for DMABUF importing was posted in [2]. The exporter part is dependant on DMA mapping redesign [3] which is not merged into the mainline. Therefore it is posted as a separate patchset. This patchset is rebased on 3.4-rc1 plus the following patchsets: - support for DMABUF importing in V4L2 [2] - DMA mapping redesign [3] - support for dma_get_pages extension [4] - support for vmap extension to dmabuf framework by Dave Airlie [5][6] [1] https://lkml.org/lkml/2011/12/26/29 [2] http://thread.gmane.org/gmane.linux.drivers.video-input-infrastructure/46586 [3] http://thread.gmane.org/gmane.linux.kernel.cross-arch/12819 [4] http://thread.gmane.org/gmane.linux.drivers.video-input-infrastructure/4379… [5] http://thread.gmane.org/gmane.linux.drivers.video-input-infrastructure/46713 [6] http://cgit.freedesktop.org/~airlied/linux/commit/?h=drm-dmabuf2&id=c481a54… [7] http://thread.gmane.org/gmane.comp.video.dri.devel/66213 Marek Szyprowski (1): v4l: vb2-dma-contig: let mmap method to use dma_mmap_coherent call Tomasz Stanislawski (12): v4l: add buffer exporting via dmabuf v4l: vb2: add buffer exporting via dmabuf v4l: vb2-dma-contig: add setup of sglist for MMAP buffers v4l: vb2-dma-contig: add support for DMABUF exporting v4l: vb2-dma-contig: add vmap/kmap for dmabuf exporting v4l: vb2-dma-contig: change map/unmap behaviour for importers v4l: vb2-dma-contig: change map/unmap behaviour for exporters v4l: s5p-tv: mixer: support for dmabuf importing v4l: s5p-tv: mixer: support for dmabuf exporting v4l: fimc: support for dmabuf importing v4l: fimc: support for dmabuf exporting v4l: vivi: support for dmabuf exporting drivers/media/video/Kconfig | 1 + drivers/media/video/s5p-fimc/fimc-capture.c | 11 ++- drivers/media/video/s5p-tv/Kconfig | 1 + drivers/media/video/s5p-tv/mixer_video.c | 12 ++- drivers/media/video/v4l2-compat-ioctl32.c | 1 + drivers/media/video/v4l2-ioctl.c | 7 + drivers/media/video/videobuf2-core.c | 66 ++++++++ drivers/media/video/videobuf2-dma-contig.c | 224 +++++++++++++++++++++++++- drivers/media/video/vivi.c | 9 + include/linux/videodev2.h | 23 +++ include/media/v4l2-ioctl.h | 2 + include/media/videobuf2-core.h | 2 + 12 files changed, 348 insertions(+), 11 deletions(-) -- 1.7.5.4

11 years, 11 months

New "xf86-video-armsoc" DDX driver

by Tom Cooksey

Hi All, For the last few months we (ARM MPD... "The Mali guys") have been working on getting X.Org up and running with Mali T6xx (ARM's next-generation GPU IP). The approach is very similar (well identical I think) to how things work on OMAP: We use a DRM driver to manage the display controller via KMS. The KMS driver also allocates both scan-out and pixmap/back buffers via the DRM_IOCTL_MODE_CREATE_DUMB ioctl which is internally implemented with GEM. When returning buffers to DRI clients, the x-server uses flink to get a global handle to a buffer which it passes back to the DRI client (in our case the Mali-T600 X11 EGL winsys). The client then uses the new PRIME ioctls to export the GEM buffer it received from the x-server to a dma_buf fd. This fd is then passed into the T6xx kernel driver via our own job dispatch user/kernel API (we're not using DRM for driving the GPU, only the display controller). Note: ARM doesn't generally provide the display controller IP block, so this is really for our customers/Linaro to develop, though we do have something hacked up for ARM's own PL111 display controller on our Versatile Express development platform which we'll be open sourcing/up-streaming asap via Linaro. We believe most ARM SoCs are likely to work the same way, at least those with 3rd-party GPU IP blocks/drivers (so everyone except Qualcomm & nVidia). As mentioned, this is certainly how the OMAP integration works. As such, we've taken the OMAP DDX driver Rob Clark wrote and hacked on it to make it work for Mali. The patch is actually relatively small, which is not really too surprising as all the driver is doing is allocating buffers and managing a display controller via a device-agnostic interface (KMS). All the device-specific code is kept in the DRM driver and the client GLES/EGL library. Given that the DDX driver doesn't contain any device-specific code, we're going to take the OMAP DDX as a baseline and try and make it more generic. Our immediate goals are to make it work on our own Versatile Express development platform and on Samsung's Exynos 5250 SoC, however our hope is to have a single DDX driver which can cover OMAP, Exynos, ST-E's Nova/Thor platforms and probably others too. It's even been suggested it could work with Mesa's sw backend(?). Anyway, the DDX is very much a work-in-progress and is still heavily branded OMAP, even though it's working (almost) perfectly on VExpress & Exynos too (re-branding isn't too high-up our priority list at the moment). We are actively developing this driver and will be doing so in a public git repository hosted by Linaro. We will not be maintaining any private repository behind ARM's firewall or anything like that - you'll see what we see. The first patches have now been pushed, so if anyone's interested in seeing what we have so far or wants to track development, the tree is here: http://git.linaro.org/gitweb?p=arm/xorg/driver/xf86-video-armsoc.git;a=summa ry Note: When we originally spoke to Rob Clark about this, he suggested we take the already-generic xf86-video-modesetting and just add the dri2 code to it. This is indeed how we started out, however as we progressed it became clear that the majority of the code we wanted was in the omap driver and were having to work fairly hard to keep some of the original modesetting code. This is why we've now changed tactic and just forked the OMAP driver, something Rob is more than happy for us to do. One thing the DDX driver isn't doing yet is making use of 2D hw blocks. In the short-term, we will simply create a branch off of the "generic" master for each SoC and add 2D hardware support there. We do however want a more permanent solution which doesn't need a separate branch per SoC. Some of the suggested solutions are: * Add a new generic DRM ioctl API for larger 2D operations (I would imagine small blits/blends would be done in SW). * Use SW rendering for everything other than solid blits and use v4l2's blitting API for those (importing/exporting buffers to be blitted using dma_buf). The theory here is that most UIs are rendered with GLES and so you only need 2D hardware for blits. I think we'll prototype this approach on Exynos. * Define a new x-server sub-module interface to allow a seperate .so 2D driver to be loaded (this is the approach the current OMAP DDX uses). We are hoping someone might have some advice & suggestions on how to proceed with regards to 2D. We're also very interested in any feedback, both on the DDX driver specifically and on the approach we're taking in general. Cheers, Tom

11 years, 11 months

[PATCH] dma-buf: add vmap interface (v3)

by Dave Airlie

From: Dave Airlie <airlied(a)redhat.com> The main requirement I have for this interface is for scanning out using the USB gpu devices. Since these devices have to read the framebuffer on updates and linearly compress it, using kmaps is a major overhead for every update. v2: fix warn issues pointed out by Sylwester Nawrocki. v3: fix compile !CONFIG_DMA_SHARED_BUFFER and add _GPL for now Signed-off-by: Dave Airlie <airlied(a)redhat.com> --- drivers/base/dma-buf.c | 34 ++++++++++++++++++++++++++++++++++ include/linux/dma-buf.h | 14 ++++++++++++++ 2 files changed, 48 insertions(+), 0 deletions(-) diff --git a/drivers/base/dma-buf.c b/drivers/base/dma-buf.c index 07cbbc6..0d8197e 100644 --- a/drivers/base/dma-buf.c +++ b/drivers/base/dma-buf.c @@ -406,3 +406,37 @@ void dma_buf_kunmap(struct dma_buf *dmabuf, unsigned long page_num, dmabuf->ops->kunmap(dmabuf, page_num, vaddr); } EXPORT_SYMBOL_GPL(dma_buf_kunmap); + +/** + * dma_buf_vmap - Create virtual mapping for the buffer object into kernel address space. The same restrictions as for vmap and friends apply. + * @dma_buf: [in] buffer to vmap + * + * This call may fail due to lack of virtual mapping address space. + * These calls are optional in drivers. The intended use for them + * is for mapping objects linear in kernel space for high use objects. + * Please attempt to use kmap/kunmap before thinking about these interfaces. + */ +void *dma_buf_vmap(struct dma_buf *dmabuf) +{ + if (WARN_ON(!dmabuf)) + return NULL; + + if (dmabuf->ops->vmap) + return dmabuf->ops->vmap(dmabuf); + return NULL; +} +EXPORT_SYMBOL_GPL(dma_buf_vmap); + +/** + * dma_buf_vunmap - Unmap a vmap obtained by dma_buf_vmap. + * @dma_buf: [in] buffer to vmap + */ +void dma_buf_vunmap(struct dma_buf *dmabuf, void *vaddr) +{ + if (WARN_ON(!dmabuf)) + return; + + if (dmabuf->ops->vunmap) + dmabuf->ops->vunmap(dmabuf, vaddr); +} +EXPORT_SYMBOL_GPL(dma_buf_vunmap); diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index 3efbfc2..d8c2865 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -92,6 +92,9 @@ struct dma_buf_ops { void (*kunmap_atomic)(struct dma_buf *, unsigned long, void *); void *(*kmap)(struct dma_buf *, unsigned long); void (*kunmap)(struct dma_buf *, unsigned long, void *); + + void *(*vmap)(struct dma_buf *); + void (*vunmap)(struct dma_buf *, void *vaddr); }; /** @@ -167,6 +170,9 @@ void *dma_buf_kmap_atomic(struct dma_buf *, unsigned long); void dma_buf_kunmap_atomic(struct dma_buf *, unsigned long, void *); void *dma_buf_kmap(struct dma_buf *, unsigned long); void dma_buf_kunmap(struct dma_buf *, unsigned long, void *); + +void *dma_buf_vmap(struct dma_buf *); +void dma_buf_vunmap(struct dma_buf *, void *vaddr); #else static inline struct dma_buf_attachment *dma_buf_attach(struct dma_buf *dmabuf, @@ -248,6 +254,14 @@ static inline void dma_buf_kunmap(struct dma_buf *dmabuf, unsigned long pnum, void *vaddr) { } + +static inline void *dma_buf_vmap(struct dma_buf *dmabuf) +{ +} + +static inline void dma_buf_vunmap(struct dma_buf *dmabuf, void *vaddr) +{ +} #endif /* CONFIG_DMA_SHARED_BUFFER */ #endif /* __DMA_BUF_H__ */ -- 1.7.6

11 years, 11 months

[RFC 0/2] dma-mapping: Introduce new IOVA API with address specified

by Hiroshi DOYU

Hello, The following patchset is our enhancement for the upstream DMA mapping API(v9), where new IOVA API is introduced with the version of IOVA address specified. The current upstream DMA mapping API cannot specify any specific IOVA address at allocation. We need to specify IOVA address. This is necessary because some HWAs requre some specific address, for example, AVP vector and also some data buffer alignement can improve better performance from H/W constraints POV. Hiroshi DOYU (2): dma-mapping: Export arm_iommu_{alloc,free}_iova() functions dma-mapping: Enable IOVA mapping with specific address arch/arm/include/asm/dma-iommu.h | 31 ++++++ arch/arm/include/asm/dma-mapping.h | 1 + arch/arm/mm/dma-mapping.c | 181 +++++++++++++++++++++++++++++------- 3 files changed, 180 insertions(+), 33 deletions(-) -- 1.7.5.4

11 years, 11 months

[PATCH] dma-buf: mmap support

by Daniel Vetter

Compared to Rob Clark's RFC I've ditched the prepare/finish hooks and corresponding ioctls on the dma_buf file. The major reason for that is that many people seem to be under the impression that this is also for synchronization with outstanding asynchronous processsing. I'm pretty massively opposed to this because: - It boils down reinventing a new rather general-purpose userspace synchronization interface. If we look at things like futexes, this is hard to get right. - Furthermore a lot of kernel code has to interact with this synchronization primitive. This smells a look like the dri1 hw_lock, a horror show I prefer not to reinvent. - Even more fun is that multiple different subsystems would interact here, so we have plenty of opportunities to create funny deadlock scenarios. I think synchronization is a wholesale different problem from data sharing and should be tackled as an orthogonal problem. Now we could demand that prepare/finish may only ensure cache coherency (as Rob intended), but that runs up into the next problem: We not only need mmap support to facilitate sw-only processing nodes in a pipeline (without jumping through hoops by importing the dma_buf into some sw-access only importer), which allows for a nicer ION->dma-buf upgrade path for existing Android userspace. We also need mmap support for existing importing subsystems to support existing userspace libraries. And a loot of these subsystems are expected to export coherent userspace mappings. So prepare/finish can only ever be optional and the exporter /needs/ to support coherent mappings. Given that mmap access is always somewhat fallback-y in nature I've decided to drop this optimization, instead of just making it optional. If we demonstrate a clear need for this, supported by benchmark results, we can always add it in again later as an optional extension. Other differences compared to Rob's RFC is the above mentioned support for mapping a dma-buf through facilities provided by the importer. Which results in mmap support no longer being optional. Note that this dma-buf mmap patch does _not_ support every possible insanity an existing subsystem could pull of with mmap: Because it does not allow to intercept pagefaults and shoot down ptes importing subsystems can't add some magic of their own at these points (e.g. to automatically synchronize with outstanding rendering or set up some special resources). I've done a cursory read through a few mmap implementions of various subsytems and I'm hopeful that we can avoid this (and the complexity it'd bring with it). Additonally I've extended the documentation a bit to explain the hows and whys of this mmap extension. In case we ever want to add support for explicitly cache maneged userspace mmap with a prepare/finish ioctl pair, we could specify that userspace needs to mmap a different part of the dma_buf, e.g. the range starting at dma_buf->size up to dma_buf->size*2. This works because the size of a dma_buf is invariant over it's lifetime. The exporter would obviously need to fall back to coherent mappings for both ranges if a legacy clients maps the coherent range and the architecture cannot suppor conflicting caching policies. Also, this would obviously be optional and userspace needs to be able to fall back to coherent mappings. v2: - Spelling fixes from Rob Clark. - Compile fix for !DMA_BUF from Rob Clark. - Extend commit message to explain how explicitly cache managed mmap support could be added later. - Extend the documentation with implementations notes for exporters that need to manually fake coherency. v3: - dma_buf pointer initialization goof-up noticed by Rebecca Schultz Zavin. Cc: Rob Clark <rob.clark(a)linaro.org> Cc: Rebecca Schultz Zavin <rebecca(a)android.com> Signed-Off-by: Daniel Vetter <daniel.vetter(a)ffwll.ch> --- Documentation/dma-buf-sharing.txt | 98 ++++++++++++++++++++++++++++++++++--- drivers/base/dma-buf.c | 64 +++++++++++++++++++++++- include/linux/dma-buf.h | 16 ++++++ 3 files changed, 170 insertions(+), 8 deletions(-) diff --git a/Documentation/dma-buf-sharing.txt b/Documentation/dma-buf-sharing.txt index 3bbd5c5..5ff4d2b 100644 --- a/Documentation/dma-buf-sharing.txt +++ b/Documentation/dma-buf-sharing.txt @@ -29,13 +29,6 @@ The buffer-user in memory, mapped into its own address space, so it can access the same area of memory. -*IMPORTANT*: [see https://lkml.org/lkml/2011/12/20/211 for more details] -For this first version, A buffer shared using the dma_buf sharing API: -- *may* be exported to user space using "mmap" *ONLY* by exporter, outside of - this framework. -- with this new iteration of the dma-buf api cpu access from the kernel has been - enable, see below for the details. - dma-buf operations for device dma only -------------------------------------- @@ -313,6 +306,83 @@ Access to a dma_buf from the kernel context involves three steps: enum dma_data_direction dir); +Direct Userspace Access/mmap Support +------------------------------------ + +Being able to mmap an export dma-buf buffer object has 2 main use-cases: +- CPU fallback processing in a pipeline and +- supporting existing mmap interfaces in importers. + +1. CPU fallback processing in a pipeline + + In many processing pipelines it is sometimes required that the cpu can access + the data in a dma-buf (e.g. for thumbnail creation, snapshots, ...). To avoid + the need to handle this specially in userspace frameworks for buffer sharing + it's ideal if the dma_buf fd itself can be used to access the backing storage + from userspace using mmap. + + Furthermore Android's ION framework already supports this (and is otherwise + rather similar to dma-buf from a userspace consumer side with using fds as + handles, too). So it's beneficial to support this in a similar fashion on + dma-buf to have a good transition path for existing Android userspace. + + No special interfaces, userspace simply calls mmap on the dma-buf fd. + +2. Supporting existing mmap interfaces in exporters + + Similar to the motivation for kernel cpu access it is again important that + the userspace code of a given importing subsystem can use the same interfaces + with a imported dma-buf buffer object as with a native buffer object. This is + especially important for drm where the userspace part of contemporary OpenGL, + X, and other drivers is huge, and reworking them to use a different way to + mmap a buffer rather invasive. + + The assumption in the current dma-buf interfaces is that redirecting the + initial mmap is all that's needed. A survey of some of the existing + subsystems shows that no driver seems to do any nefarious thing like syncing + up with outstanding asynchronous processing on the device or allocating + special resources at fault time. So hopefully this is good enough, since + adding interfaces to intercept pagefaults and allow pte shootdowns would + increase the complexity quite a bit. + + Interface: + int dma_buf_mmap(struct dma_buf *, struct vm_area_struct *, + unsigned long); + + If the importing subsystem simply provides a special-purpose mmap call to set + up a mapping in userspace, calling do_mmap with dma_buf->file will equally + achieve that for a dma-buf object. + +3. Implementation notes for exporters + + Because dma-buf buffers have invariant size over their lifetime, the dma-buf + core checks whether a vma is too large and rejects such mappings. The + exporter hence does not need to duplicate this check. + + Because existing importing subsystems might presume coherent mappings for + userspace, the exporter needs to set up a coherent mapping. If that's not + possible, it needs to fake coherency by manually shooting down ptes when + leaving the cpu domain and flushing caches at fault time. Note that all the + dma_buf files share the same anon inode, hence the exporter needs to replace + the dma_buf file stored in vma->vm_file with it's own if pte shootdown is + requred. This is because the kernel uses the underlying inode's address_space + for vma tracking (and hence pte tracking at shootdown time with + unmap_mapping_range). + + If the above shootdown dance turns out to be too expensive in certain + scenarios, we can extend dma-buf with a more explicit cache tracking scheme + for userspace mappings. But the current assumption is that using mmap is + always a slower path, so some inefficiencies should be acceptable. + + Exporters that shoot down mappings (for any reasons) shall not do any + synchronization at fault time with outstanding device operations. + Synchronization is an orthogonal issue to sharing the backing storage of a + buffer and hence should not be handled by dma-buf itself. This is explictly + mentioned here because many people seem to want something like this, but if + different exporters handle this differently, buffer sharing can fail in + interesting ways depending upong the exporter (if userspace starts depending + upon this implicit synchronization). + Miscellaneous notes ------------------- @@ -336,6 +406,20 @@ Miscellaneous notes the exporting driver to create a dmabuf fd must provide a way to let userspace control setting of O_CLOEXEC flag passed in to dma_buf_fd(). +- If an exporter needs to manually flush caches and hence needs to fake + coherency for mmap support, it needs to be able to zap all the ptes pointing + at the backing storage. Now linux mm needs a struct address_space associated + with the struct file stored in vma->vm_file to do that with the function + unmap_mapping_range. But the dma_buf framework only backs every dma_buf fd + with the anon_file struct file, i.e. all dma_bufs share the same file. + + Hence exporters need to setup their own file (and address_space) association + by setting vma->vm_file and adjusting vma->vm_pgoff in the dma_buf mmap + callback. In the specific case of a gem driver the exporter could use the + shmem file already provided by gem (and set vm_pgoff = 0). Exporters can then + zap ptes by unmapping the corresponding range of the struct address_space + associated with their own file. + References: [1] struct dma_buf_ops in include/linux/dma-buf.h [2] All interfaces mentioned above defined in include/linux/dma-buf.h diff --git a/drivers/base/dma-buf.c b/drivers/base/dma-buf.c index 07cbbc6..7cfb405 100644 --- a/drivers/base/dma-buf.c +++ b/drivers/base/dma-buf.c @@ -44,8 +44,26 @@ static int dma_buf_release(struct inode *inode, struct file *file) return 0; } +static int dma_buf_mmap_internal(struct file *file, struct vm_area_struct *vma) +{ + struct dma_buf *dmabuf; + + if (!is_dma_buf_file(file)) + return -EINVAL; + + dmabuf = file->private_data; + + /* check for overflowing the buffer's size */ + if (vma->vm_pgoff + ((vma->vm_end - vma->vm_start) >> PAGE_SHIFT) > + dmabuf->size >> PAGE_SHIFT) + return -EINVAL; + + return dmabuf->ops->mmap(dmabuf, vma); +} + static const struct file_operations dma_buf_fops = { .release = dma_buf_release, + .mmap = dma_buf_mmap_internal, }; /* @@ -82,7 +100,8 @@ struct dma_buf *dma_buf_export(void *priv, const struct dma_buf_ops *ops, || !ops->unmap_dma_buf || !ops->release || !ops->kmap_atomic - || !ops->kmap)) { + || !ops->kmap + || !ops->mmap)) { return ERR_PTR(-EINVAL); } @@ -406,3 +425,46 @@ void dma_buf_kunmap(struct dma_buf *dmabuf, unsigned long page_num, dmabuf->ops->kunmap(dmabuf, page_num, vaddr); } EXPORT_SYMBOL_GPL(dma_buf_kunmap); + + +/** + * dma_buf_mmap - Setup up a userspace mmap with the given vma + * @dma_buf: [in] buffer that should back the vma + * @vma: [in] vma for the mmap + * @pgoff: [in] offset in pages where this mmap should start within the + * dma-buf buffer. + * + * This function adjusts the passed in vma so that it points at the file of the + * dma_buf operation. It alsog adjusts the starting pgoff and does bounds + * checking on the size of the vma. Then it calls the exporters mmap function to + * set up the mapping. + * + * Can return negative error values, returns 0 on success. + */ +int dma_buf_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma, + unsigned long pgoff) +{ + if (WARN_ON(!dmabuf || !vma)) + return -EINVAL; + + /* check for offset overflow */ + if (pgoff + ((vma->vm_end - vma->vm_start) >> PAGE_SHIFT) < pgoff) + return -EOVERFLOW; + + /* check for overflowing the buffer's size */ + if (pgoff + ((vma->vm_end - vma->vm_start) >> PAGE_SHIFT) > + dmabuf->size >> PAGE_SHIFT) + return -EINVAL; + + /* readjust the vma */ + if (vma->vm_file) + fput(vma->vm_file); + + vma->vm_file = dmabuf->file; + get_file(vma->vm_file); + + vma->vm_pgoff = pgoff; + + return dmabuf->ops->mmap(dmabuf, vma); +} +EXPORT_SYMBOL_GPL(dma_buf_mmap); diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index 3efbfc2..1f78d15 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -61,6 +61,10 @@ struct dma_buf_attachment; * This Callback must not sleep. * @kmap: maps a page from the buffer into kernel address space. * @kunmap: [optional] unmaps a page from the buffer. + * @mmap: used to expose the backing storage to userspace. Note that the + * mapping needs to be coherent - if the exporter doesn't directly + * support this, it needs to fake coherency by shooting down any ptes + * when transitioning away from the cpu domain. */ struct dma_buf_ops { int (*attach)(struct dma_buf *, struct device *, @@ -92,6 +96,8 @@ struct dma_buf_ops { void (*kunmap_atomic)(struct dma_buf *, unsigned long, void *); void *(*kmap)(struct dma_buf *, unsigned long); void (*kunmap)(struct dma_buf *, unsigned long, void *); + + int (*mmap)(struct dma_buf *, struct vm_area_struct *vma); }; /** @@ -167,6 +173,9 @@ void *dma_buf_kmap_atomic(struct dma_buf *, unsigned long); void dma_buf_kunmap_atomic(struct dma_buf *, unsigned long, void *); void *dma_buf_kmap(struct dma_buf *, unsigned long); void dma_buf_kunmap(struct dma_buf *, unsigned long, void *); + +int dma_buf_mmap(struct dma_buf *, struct vm_area_struct *, + unsigned long); #else static inline struct dma_buf_attachment *dma_buf_attach(struct dma_buf *dmabuf, @@ -248,6 +257,13 @@ static inline void dma_buf_kunmap(struct dma_buf *dmabuf, unsigned long pnum, void *vaddr) { } + +static inline int dma_buf_mmap(struct dma_buf *dmabuf, + struct vm_area_struct *vma, + unsigned long pgoff) +{ + return -ENODEV; +} #endif /* CONFIG_DMA_SHARED_BUFFER */ #endif /* __DMA_BUF_H__ */ -- 1.7.10

11 years, 11 months

[PATCH] dma-buf: add vmap interface (v2)

by Dave Airlie

From: Dave Airlie <airlied(a)redhat.com> The main requirement I have for this interface is for scanning out using the USB gpu devices. Since these devices have to read the framebuffer on updates and linearly compress it, using kmaps is a major overhead for every update. v2: fix warn issues pointed out by Sylwester Nawrocki. Signed-off-by: Dave Airlie <airlied(a)redhat.com> --- drivers/base/dma-buf.c | 34 ++++++++++++++++++++++++++++++++++ include/linux/dma-buf.h | 14 ++++++++++++++ 2 files changed, 48 insertions(+), 0 deletions(-) diff --git a/drivers/base/dma-buf.c b/drivers/base/dma-buf.c index 07cbbc6..750f92c 100644 --- a/drivers/base/dma-buf.c +++ b/drivers/base/dma-buf.c @@ -406,3 +406,37 @@ void dma_buf_kunmap(struct dma_buf *dmabuf, unsigned long page_num, dmabuf->ops->kunmap(dmabuf, page_num, vaddr); } EXPORT_SYMBOL_GPL(dma_buf_kunmap); + +/** + * dma_buf_vmap - Create virtual mapping for the buffer object into kernel address space. The same restrictions as for vmap and friends apply. + * @dma_buf: [in] buffer to vmap + * + * This call may fail due to lack of virtual mapping address space. + * These calls are optional in drivers. The intended use for them + * is for mapping objects linear in kernel space for high use objects. + * Please attempt to use kmap/kunmap before thinking about these interfaces. + */ +void *dma_buf_vmap(struct dma_buf *dmabuf) +{ + if (WARN_ON(!dmabuf)) + return NULL; + + if (dmabuf->ops->vmap) + return dmabuf->ops->vmap(dmabuf); + return NULL; +} +EXPORT_SYMBOL(dma_buf_vmap); + +/** + * dma_buf_vunmap - Unmap a vmap obtained by dma_buf_vmap. + * @dma_buf: [in] buffer to vmap + */ +void dma_buf_vunmap(struct dma_buf *dmabuf, void *vaddr) +{ + if (WARN_ON(!dmabuf)) + return; + + if (dmabuf->ops->vunmap) + dmabuf->ops->vunmap(dmabuf, vaddr); +} +EXPORT_SYMBOL(dma_buf_vunmap); diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index 3efbfc2..b92b6de 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -92,6 +92,9 @@ struct dma_buf_ops { void (*kunmap_atomic)(struct dma_buf *, unsigned long, void *); void *(*kmap)(struct dma_buf *, unsigned long); void (*kunmap)(struct dma_buf *, unsigned long, void *); + + void *(*vmap)(struct dma_buf *); + void (*vunmap)(struct dma_buf *, void *vaddr); }; /** @@ -167,6 +170,9 @@ void *dma_buf_kmap_atomic(struct dma_buf *, unsigned long); void dma_buf_kunmap_atomic(struct dma_buf *, unsigned long, void *); void *dma_buf_kmap(struct dma_buf *, unsigned long); void dma_buf_kunmap(struct dma_buf *, unsigned long, void *); + +void *dma_buf_vmap(struct dma_buf *); +void dma_buf_vunmap(struct dma_buf *, void *vaddr); #else static inline struct dma_buf_attachment *dma_buf_attach(struct dma_buf *dmabuf, @@ -248,6 +254,14 @@ static inline void dma_buf_kunmap(struct dma_buf *dmabuf, unsigned long pnum, void *vaddr) { } + +static inline void *dma_buf_vmap(struct dma_buf *) +{ +} + +static inline void dma_buf_vunmap(struct dma_buf *, void *vaddr); +{ +} #endif /* CONFIG_DMA_SHARED_BUFFER */ #endif /* __DMA_BUF_H__ */ -- 1.7.6

11 years, 11 months

[PATCH/RFC 0/3] ARM: DMA-mapping: new extensions for buffer sharing

by Marek Szyprowski

Hello, This patch series introduces a new features to DMA mapping subsystem to let drivers share the allocated buffers (preferably using recently introduced dma_buf framework) easy and efficient. The first extension is DMA_ATTR_NO_KERNEL_MAPPING attribute. It is intended for use with dma_{alloc, mmap, free}_attrs functions. It can be used to notify dma-mapping core that the driver will not use kernel mapping for the allocated buffer at all, so the core can skip creating it. This saves precious kernel virtual address space. Such buffer can be accessed from userspace, after calling dma_mmap_attrs() for it (a typical use case for multimedia buffers). The value returned by dma_alloc_attrs() with this attribute should be considered as a DMA cookie, which needs to be passed to dma_mmap_attrs() and dma_free_attrs() funtions. The second extension is required to let drivers to share the buffers allocated by DMA-mapping subsystem. Right now the driver gets a dma address of the allocated buffer and the kernel virtual mapping for it. If it wants to share it with other device (= map into its dma address space) it usually hacks around kernel virtual addresses to get pointers to pages or assumes that both devices share the DMA address space. Both solutions are just hacks for the special cases, which should be avoided in the final version of buffer sharing. To solve this issue in a generic way, a new call to DMA mapping has been introduced - dma_get_sgtable(). It allocates a scatter-list which describes the allocated buffer and lets the driver(s) to use it with other device(s) by calling dma_map_sg() on it. The proposed patches have been generated on top of the ARM DMA-mapping redesign patch series on Linux v3.4-rc7. They are also available on the following GIT branch: git://git.linaro.org/people/mszyprowski/linux-dma-mapping.git 3.4-rc7-arm-dma-v10-ext with all require patches on top of vanilla v3.4-rc7 kernel. Best regards Marek Szyprowski Samsung Poland R&D Center Patch summary: Marek Szyprowski (3): common: DMA-mapping: add DMA_ATTR_NO_KERNEL_MAPPING attribute ARM: dma-mapping: add support for DMA_ATTR_NO_KERNEL_MAPPING attribute ARM: dma-mapping: add support for dma_get_sgtable() Documentation/DMA-attributes.txt | 18 +++++++++++++ arch/arm/include/asm/dma-mapping.h | 12 +++++++++ arch/arm/mm/dma-mapping.c | 51 ++++++++++++++++++++++++++++++++---- include/linux/dma-attrs.h | 1 + include/linux/dma-mapping.h | 3 +++ 5 files changed, 80 insertions(+), 5 deletions(-) -- 1.7.10.1

11 years, 11 months

[PATCHv2 updated] ARM: dma-mapping: remove custom consistent dma region

by Marek Szyprowski

This patch changes dma-mapping subsystem to use generic vmalloc areas for all consistent dma allocations. This increases the total size limit of the consistent allocations and removes platform hacks and a lot of duplicated code. Atomic allocations are served from special pool preallocated on boot, becasue vmalloc areas cannot be reliably created in atomic context. Signed-off-by: Marek Szyprowski <m.szyprowski(a)samsung.com> --- Hello, This is an updated version of the patch posted in the following thread: http://www.spinics.net/lists/kernel/msg1342885.html This one has been rebased onto the ARM DMA-mapping redesign patches and includes a part for IOMMU-aware ARM DMA-mapping implementation. The ARM DMA-mapping redesign patches are available in the following thread: http://www.spinics.net/lists/arm-kernel/msg175729.html Best regards Marek Szyprowski Samsung Poland R&D Center --- Documentation/kernel-parameters.txt | 4 + arch/arm/include/asm/dma-mapping.h | 2 +- arch/arm/mm/dma-mapping.c | 497 ++++++++++++++++------------------- 3 files changed, 228 insertions(+), 275 deletions(-) diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt index c1601e5..ba58f50 100644 --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -515,6 +515,10 @@ bytes respectively. Such letter suffixes can also be entirely omitted. a hypervisor. Default: yes + coherent_pool=nn[KMG] [ARM,KNL] + Sets the size of memory pool for coherent, atomic dma + allocations. + code_bytes [X86] How many bytes of object code to print in an oops report. Range: 0 - 8192 diff --git a/arch/arm/include/asm/dma-mapping.h b/arch/arm/include/asm/dma-mapping.h index bbef15d..80777d87 100644 --- a/arch/arm/include/asm/dma-mapping.h +++ b/arch/arm/include/asm/dma-mapping.h @@ -226,7 +226,7 @@ static inline int dma_mmap_writecombine(struct device *dev, struct vm_area_struc * DMA region above it's default value of 2MB. It must be called before the * memory allocator is initialised, i.e. before any core_initcall. */ -extern void __init init_consistent_dma_size(unsigned long size); +static inline void init_consistent_dma_size(unsigned long size) { } /* * For SA-1111, IXP425, and ADI systems the dma-mapping functions are "magic" diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c index 3ac4760..2e98403 100644 --- a/arch/arm/mm/dma-mapping.c +++ b/arch/arm/mm/dma-mapping.c @@ -20,6 +20,7 @@ #include <linux/highmem.h> #include <linux/slab.h> #include <linux/iommu.h> +#include <linux/io.h> #include <linux/vmalloc.h> #include <asm/memory.h> @@ -228,204 +229,170 @@ static void __dma_free_buffer(struct page *page, size_t size) } #ifdef CONFIG_MMU - -#define CONSISTENT_OFFSET(x) (((unsigned long)(x) - consistent_base) >> PAGE_SHIFT) -#define CONSISTENT_PTE_INDEX(x) (((unsigned long)(x) - consistent_base) >> PMD_SHIFT) - -/* - * These are the page tables (2MB each) covering uncached, DMA consistent allocations - */ -static pte_t **consistent_pte; - -#define DEFAULT_CONSISTENT_DMA_SIZE SZ_2M - -unsigned long consistent_base = CONSISTENT_END - DEFAULT_CONSISTENT_DMA_SIZE; - -void __init init_consistent_dma_size(unsigned long size) -{ - unsigned long base = CONSISTENT_END - ALIGN(size, SZ_2M); - - BUG_ON(consistent_pte); /* Check we're called before DMA region init */ - BUG_ON(base < VMALLOC_END); - - /* Grow region to accommodate specified size */ - if (base < consistent_base) - consistent_base = base; -} - -#include "vmregion.h" - -static struct arm_vmregion_head consistent_head = { - .vm_lock = __SPIN_LOCK_UNLOCKED(&consistent_head.vm_lock), - .vm_list = LIST_HEAD_INIT(consistent_head.vm_list), - .vm_end = CONSISTENT_END, -}; - #ifdef CONFIG_HUGETLB_PAGE #error ARM Coherent DMA allocator does not (yet) support huge TLB #endif -/* - * Initialise the consistent memory allocation. - */ -static int __init consistent_init(void) -{ - int ret = 0; - pgd_t *pgd; - pud_t *pud; - pmd_t *pmd; - pte_t *pte; - int i = 0; - unsigned long base = consistent_base; - unsigned long num_ptes = (CONSISTENT_END - base) >> PMD_SHIFT; - - consistent_pte = kmalloc(num_ptes * sizeof(pte_t), GFP_KERNEL); - if (!consistent_pte) { - pr_err("%s: no memory\n", __func__); - return -ENOMEM; - } - - pr_debug("DMA memory: 0x%08lx - 0x%08lx:\n", base, CONSISTENT_END); - consistent_head.vm_start = base; - - do { - pgd = pgd_offset(&init_mm, base); - - pud = pud_alloc(&init_mm, pgd, base); - if (!pud) { - pr_err("%s: no pud tables\n", __func__); - ret = -ENOMEM; - break; - } - - pmd = pmd_alloc(&init_mm, pud, base); - if (!pmd) { - pr_err("%s: no pmd tables\n", __func__); - ret = -ENOMEM; - break; - } - WARN_ON(!pmd_none(*pmd)); - - pte = pte_alloc_kernel(pmd, base); - if (!pte) { - pr_err("%s: no pte tables\n", __func__); - ret = -ENOMEM; - break; - } - - consistent_pte[i++] = pte; - base += PMD_SIZE; - } while (base < CONSISTENT_END); - - return ret; -} - -core_initcall(consistent_init); - static void * __dma_alloc_remap(struct page *page, size_t size, gfp_t gfp, pgprot_t prot, const void *caller) { - struct arm_vmregion *c; - size_t align; - int bit; + struct vm_struct *area; + unsigned long addr; - if (!consistent_pte) { - pr_err("%s: not initialised\n", __func__); + area = get_vm_area_caller(size, VM_DMA | VM_USERMAP, caller); + if (!area) + return NULL; + addr = (unsigned long)area->addr; + area->phys_addr = __pfn_to_phys(page_to_pfn(page)); + + if (ioremap_page_range(addr, addr + size, area->phys_addr, prot)) { + vunmap((void *)addr); + return NULL; + } + return (void *)addr; +} + +static void __dma_free_remap(void *cpu_addr, size_t size) +{ + struct vm_struct *area; + + read_lock(&vmlist_lock); + area = find_vm_area(cpu_addr); + if (!area) { + pr_err("%s: trying to free invalid coherent area: %p\n", + __func__, cpu_addr); + dump_stack(); + read_unlock(&vmlist_lock); + return; + } + unmap_kernel_range((unsigned long)cpu_addr, size); + read_unlock(&vmlist_lock); + vunmap(cpu_addr); +} + +struct dma_pool { + size_t size; + spinlock_t lock; + unsigned long *bitmap; + unsigned long count; + void *vaddr; + struct page *page; +}; + +static struct dma_pool atomic_pool = { + .size = SZ_256K, +}; + +static int __init early_coherent_pool(char *p) +{ + atomic_pool.size = memparse(p, &p); + return 0; +} +early_param("coherent_pool", early_coherent_pool); + +/* + * Initialise the coherent pool for atomic allocations. + */ +static int __init atomic_pool_init(void) +{ + struct dma_pool *pool = &atomic_pool; + pgprot_t prot = pgprot_dmacoherent(pgprot_kernel); + unsigned long count = pool->size >> PAGE_SHIFT; + gfp_t gfp = GFP_KERNEL | GFP_DMA; + unsigned long *bitmap; + struct page *page; + void *ptr; + int bitmap_size = BITS_TO_LONGS(count) * sizeof(long); + + bitmap = kzalloc(bitmap_size, GFP_KERNEL); + if (!bitmap) + goto no_bitmap; + + page = __dma_alloc_buffer(NULL, pool->size, gfp); + if (!page) + goto no_page; + + ptr = __dma_alloc_remap(page, pool->size, gfp, prot, NULL); + if (ptr) { + spin_lock_init(&pool->lock); + pool->vaddr = ptr; + pool->page = page; + pool->bitmap = bitmap; + pool->count = count; + pr_info("DMA: preallocated %u KiB pool for atomic coherent allocations\n", + (unsigned)pool->size / 1024); + return 0; + } + + __dma_free_buffer(page, pool->size); +no_page: + kfree(bitmap); +no_bitmap: + pr_err("DMA: failed to allocate %u KiB pool for atomic coherent allocation\n", + (unsigned)pool->size / 1024); + return -ENOMEM; +} +core_initcall(atomic_pool_init); + +static void *__alloc_from_pool(size_t size, struct page **ret_page) +{ + struct dma_pool *pool = &atomic_pool; + unsigned int count = size >> PAGE_SHIFT; + unsigned int pageno; + unsigned long flags; + void *ptr = NULL; + size_t align; + + if (!pool->vaddr) { + pr_err("%s: coherent pool not initialised!\n", __func__); dump_stack(); return NULL; } /* - * Align the virtual region allocation - maximum alignment is - * a section size, minimum is a page size. This helps reduce - * fragmentation of the DMA space, and also prevents allocations - * smaller than a section from crossing a section boundary. + * Align the region allocation - allocations from pool are rather + * small, so align them to their order in pages, minimum is a page + * size. This helps reduce fragmentation of the DMA space. */ - bit = fls(size - 1); - if (bit > SECTION_SHIFT) - bit = SECTION_SHIFT; - align = 1 << bit; + align = PAGE_SIZE << get_order(size); - /* - * Allocate a virtual address in the consistent mapping region. - */ - c = arm_vmregion_alloc(&consistent_head, align, size, - gfp & ~(__GFP_DMA | __GFP_HIGHMEM), caller); - if (c) { - pte_t *pte; - int idx = CONSISTENT_PTE_INDEX(c->vm_start); - u32 off = CONSISTENT_OFFSET(c->vm_start) & (PTRS_PER_PTE-1); - - pte = consistent_pte[idx] + off; - c->priv = page; - - do { - BUG_ON(!pte_none(*pte)); - - set_pte_ext(pte, mk_pte(page, prot), 0); - page++; - pte++; - off++; - if (off >= PTRS_PER_PTE) { - off = 0; - pte = consistent_pte[++idx]; - } - } while (size -= PAGE_SIZE); - - dsb(); - - return (void *)c->vm_start; + spin_lock_irqsave(&pool->lock, flags); + pageno = bitmap_find_next_zero_area(pool->bitmap, pool->count, + 0, count, (1 << align) - 1); + if (pageno < pool->count) { + bitmap_set(pool->bitmap, pageno, count); + ptr = pool->vaddr + PAGE_SIZE * pageno; + *ret_page = pool->page + pageno; } - return NULL; + spin_unlock_irqrestore(&pool->lock, flags); + + return ptr; } -static void __dma_free_remap(void *cpu_addr, size_t size) +static int __free_from_pool(void *start, size_t size) { - struct arm_vmregion *c; - unsigned long addr; - pte_t *ptep; - int idx; - u32 off; + struct dma_pool *pool = &atomic_pool; + unsigned long pageno, count; + unsigned long flags; - c = arm_vmregion_find_remove(&consistent_head, (unsigned long)cpu_addr); - if (!c) { - pr_err("%s: trying to free invalid coherent area: %p\n", - __func__, cpu_addr); + if (start < pool->vaddr || start > pool->vaddr + pool->size) + return 0; + + if (start + size > pool->vaddr + pool->size) { + pr_err("%s: freeing wrong coherent size from pool\n", __func__); dump_stack(); - return; + return 0; } - if ((c->vm_end - c->vm_start) != size) { - pr_err("%s: freeing wrong coherent size (%ld != %d)\n", - __func__, c->vm_end - c->vm_start, size); - dump_stack(); - size = c->vm_end - c->vm_start; - } + pageno = (start - pool->vaddr) >> PAGE_SHIFT; + count = size >> PAGE_SHIFT; - idx = CONSISTENT_PTE_INDEX(c->vm_start); - off = CONSISTENT_OFFSET(c->vm_start) & (PTRS_PER_PTE-1); - ptep = consistent_pte[idx] + off; - addr = c->vm_start; - do { - pte_t pte = ptep_get_and_clear(&init_mm, addr, ptep); + spin_lock_irqsave(&pool->lock, flags); + bitmap_clear(pool->bitmap, pageno, count); + spin_unlock_irqrestore(&pool->lock, flags); - ptep++; - addr += PAGE_SIZE; - off++; - if (off >= PTRS_PER_PTE) { - off = 0; - ptep = consistent_pte[++idx]; - } - - if (pte_none(pte) || !pte_present(pte)) - pr_crit("%s: bad page in kernel page table\n", - __func__); - } while (size -= PAGE_SIZE); - - flush_tlb_kernel_range(c->vm_start, c->vm_end); - - arm_vmregion_free(&consistent_head, c); + return 1; } static inline pgprot_t __get_dma_pgprot(struct dma_attrs *attrs, pgprot_t prot) @@ -441,6 +408,8 @@ static inline pgprot_t __get_dma_pgprot(struct dma_attrs *attrs, pgprot_t prot) #define __dma_alloc_remap(page, size, gfp, prot, c) page_address(page) #define __dma_free_remap(addr, size) do { } while (0) #define __get_dma_pgprot(attrs, prot) __pgprot(0) +#define __alloc_from_pool(size, ret_page) NULL +#define __free_from_pool(addr, size) 0 #endif /* CONFIG_MMU */ @@ -463,6 +432,16 @@ __dma_alloc(struct device *dev, size_t size, dma_addr_t *handle, gfp_t gfp, *handle = DMA_ERROR_CODE; size = PAGE_ALIGN(size); + /* + * Atomic allocations need special handling + */ + if (gfp & GFP_ATOMIC && !arch_is_coherent()) { + addr = __alloc_from_pool(size, &page); + if (addr) + *handle = pfn_to_dma(dev, page_to_pfn(page)); + return addr; + } + page = __dma_alloc_buffer(dev, size, gfp); if (!page) return NULL; @@ -506,30 +485,21 @@ int arm_dma_mmap(struct device *dev, struct vm_area_struct *vma, { int ret = -ENXIO; #ifdef CONFIG_MMU - unsigned long user_size, kern_size; - struct arm_vmregion *c; + unsigned long user_count = (vma->vm_end - vma->vm_start) >> PAGE_SHIFT; + unsigned long count = PAGE_ALIGN(size) >> PAGE_SHIFT; + unsigned long pfn = dma_to_pfn(dev, dma_addr); + unsigned long off = vma->vm_pgoff; vma->vm_page_prot = __get_dma_pgprot(attrs, vma->vm_page_prot); if (dma_mmap_from_coherent(dev, vma, cpu_addr, size, &ret)) return ret; - user_size = (vma->vm_end - vma->vm_start) >> PAGE_SHIFT; - - c = arm_vmregion_find(&consistent_head, (unsigned long)cpu_addr); - if (c) { - unsigned long off = vma->vm_pgoff; - struct page *pages = c->priv; - - kern_size = (c->vm_end - c->vm_start) >> PAGE_SHIFT; - - if (off < kern_size && - user_size <= (kern_size - off)) { - ret = remap_pfn_range(vma, vma->vm_start, - page_to_pfn(pages) + off, - user_size << PAGE_SHIFT, - vma->vm_page_prot); - } + if (off < count && user_count <= (count - off)) { + ret = remap_pfn_range(vma, vma->vm_start, + pfn + off, + user_count << PAGE_SHIFT, + vma->vm_page_prot); } #endif /* CONFIG_MMU */ @@ -543,13 +513,16 @@ int arm_dma_mmap(struct device *dev, struct vm_area_struct *vma, void arm_dma_free(struct device *dev, size_t size, void *cpu_addr, dma_addr_t handle, struct dma_attrs *attrs) { - WARN_ON(irqs_disabled()); - if (dma_release_from_coherent(dev, get_order(size), cpu_addr)) return; size = PAGE_ALIGN(size); + if (__free_from_pool(cpu_addr, size)) + return; + + WARN_ON(irqs_disabled()); + if (!arch_is_coherent()) __dma_free_remap(cpu_addr, size); @@ -769,9 +742,6 @@ static int arm_dma_set_mask(struct device *dev, u64 dma_mask) static int __init dma_debug_do_init(void) { -#ifdef CONFIG_MMU - arm_vmregion_create_proc("dma-mappings", &consistent_head); -#endif dma_debug_init(PREALLOC_DMA_DEBUG_ENTRIES); return 0; } @@ -888,61 +858,30 @@ static int __iommu_free_buffer(struct device *dev, struct page **pages, size_t s * Create a CPU mapping for a specified pages */ static void * -__iommu_alloc_remap(struct page **pages, size_t size, gfp_t gfp, pgprot_t prot) +__iommu_alloc_remap(struct page **pages, size_t size, gfp_t gfp, pgprot_t prot, + const void *caller) { - struct arm_vmregion *c; - size_t align; - size_t count = size >> PAGE_SHIFT; - int bit; + unsigned int i, count = size >> PAGE_SHIFT; + struct vm_struct *area; + unsigned long p; - if (!consistent_pte[0]) { - pr_err("%s: not initialised\n", __func__); - dump_stack(); + area = get_vm_area_caller(size, VM_DMA | VM_USERMAP, caller); + if (!area) return NULL; + + area->pages = pages; + p = (unsigned long)area->addr; + + for (i = 0; i < count; i++) { + phys_addr_t phys = __pfn_to_phys(page_to_pfn(pages[i])); + if (ioremap_page_range(p, p + PAGE_SIZE, phys, prot)) + goto err; + p += PAGE_SIZE; } - - /* - * Align the virtual region allocation - maximum alignment is - * a section size, minimum is a page size. This helps reduce - * fragmentation of the DMA space, and also prevents allocations - * smaller than a section from crossing a section boundary. - */ - bit = fls(size - 1); - if (bit > SECTION_SHIFT) - bit = SECTION_SHIFT; - align = 1 << bit; - - /* - * Allocate a virtual address in the consistent mapping region. - */ - c = arm_vmregion_alloc(&consistent_head, align, size, - gfp & ~(__GFP_DMA | __GFP_HIGHMEM), NULL); - if (c) { - pte_t *pte; - int idx = CONSISTENT_PTE_INDEX(c->vm_start); - int i = 0; - u32 off = CONSISTENT_OFFSET(c->vm_start) & (PTRS_PER_PTE-1); - - pte = consistent_pte[idx] + off; - c->priv = pages; - - do { - BUG_ON(!pte_none(*pte)); - - set_pte_ext(pte, mk_pte(pages[i], prot), 0); - pte++; - off++; - i++; - if (off >= PTRS_PER_PTE) { - off = 0; - pte = consistent_pte[++idx]; - } - } while (i < count); - - dsb(); - - return (void *)c->vm_start; - } + return area->addr; +err: + unmap_kernel_range((unsigned long)area->addr, size); + vunmap(area->addr); return NULL; } @@ -1001,6 +940,17 @@ static int __iommu_remove_mapping(struct device *dev, dma_addr_t iova, size_t si return 0; } +static struct page **__iommu_get_pages(void *cpu_addr) +{ + struct vm_struct *area; + read_lock(&vmlist_lock); + area = find_vm_area(cpu_addr); + read_unlock(&vmlist_lock); + if (area) + return area->pages; + return NULL; +} + static void *arm_iommu_alloc_attrs(struct device *dev, size_t size, dma_addr_t *handle, gfp_t gfp, struct dma_attrs *attrs) { @@ -1019,7 +969,8 @@ static void *arm_iommu_alloc_attrs(struct device *dev, size_t size, if (*handle == DMA_ERROR_CODE) goto err_buffer; - addr = __iommu_alloc_remap(pages, size, gfp, prot); + addr = __iommu_alloc_remap(pages, size, gfp, prot, + __builtin_return_address(0)); if (!addr) goto err_mapping; @@ -1036,31 +987,25 @@ static int arm_iommu_mmap_attrs(struct device *dev, struct vm_area_struct *vma, void *cpu_addr, dma_addr_t dma_addr, size_t size, struct dma_attrs *attrs) { - struct arm_vmregion *c; + unsigned long uaddr = vma->vm_start; + unsigned long usize = vma->vm_end - vma->vm_start; + struct page **pages = __iommu_get_pages(cpu_addr); vma->vm_page_prot = __get_dma_pgprot(attrs, vma->vm_page_prot); - c = arm_vmregion_find(&consistent_head, (unsigned long)cpu_addr); - if (c) { - struct page **pages = c->priv; + if (!pages) + return -ENXIO; - unsigned long uaddr = vma->vm_start; - unsigned long usize = vma->vm_end - vma->vm_start; - int i = 0; + do { + int ret = vm_insert_page(vma, uaddr, *pages++); + if (ret) { + pr_err("Remapping memory failed: %d\n", ret); + return ret; + } + uaddr += PAGE_SIZE; + usize -= PAGE_SIZE; + } while (usize > 0); - do { - int ret; - - ret = vm_insert_page(vma, uaddr, pages[i++]); - if (ret) { - pr_err("Remapping memory, error: %d\n", ret); - return ret; - } - - uaddr += PAGE_SIZE; - usize -= PAGE_SIZE; - } while (usize > 0); - } return 0; } @@ -1071,16 +1016,20 @@ static int arm_iommu_mmap_attrs(struct device *dev, struct vm_area_struct *vma, void arm_iommu_free_attrs(struct device *dev, size_t size, void *cpu_addr, dma_addr_t handle, struct dma_attrs *attrs) { - struct arm_vmregion *c; + struct page **pages = __iommu_get_pages(cpu_addr); size = PAGE_ALIGN(size); - c = arm_vmregion_find(&consistent_head, (unsigned long)cpu_addr); - if (c) { - struct page **pages = c->priv; - __dma_free_remap(cpu_addr, size); - __iommu_remove_mapping(dev, handle, size); - __iommu_free_buffer(dev, pages, size); + if (!pages) { + pr_err("%s: trying to free invalid coherent area: %p\n", + __func__, cpu_addr); + dump_stack(); + return; } + + unmap_kernel_range((unsigned long)cpu_addr, size); + vunmap(cpu_addr); + __iommu_remove_mapping(dev, handle, size); + __iommu_free_buffer(dev, pages, size); } /* -- 1.7.10.1

11 years, 11 months

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

Linaro-mm-sig May 2012