Hello,
This patchset adds reference counting for an exporter module to DMABUF
framework. Moreover, it adds setup of an owner field for exporters in DRM
subsystem.
v1: Original
v2:
- split patch into DMABUF and DRM part
- allow owner to be NULL
Regards,
Tomasz Stanislawski
Tomasz Stanislawski (2):
dma-buf: add reference counting for exporter module
drm: set owner field to for all DMABUF exporters
Documentation/dma-buf-sharing.txt | 3 ++-
drivers/base/dma-buf.c | 9 ++++++++-
drivers/gpu/drm/exynos/exynos_drm_dmabuf.c | 1 +
drivers/gpu/drm/i915/i915_gem_dmabuf.c | 1 +
drivers/gpu/drm/nouveau/nouveau_prime.c | 1 +
drivers/gpu/drm/radeon/radeon_prime.c | 1 +
drivers/staging/omapdrm/omap_gem_dmabuf.c | 1 +
include/linux/dma-buf.h | 2 ++
8 files changed, 17 insertions(+), 2 deletions(-)
--
1.7.9.5
A dma-fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A dma-fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence.
+ dma_fence_signal()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
TODO maybe need some helper fxn for simple devices, like a display-
only drm/kms device which simply wants to wait for exclusive fence to
be signaled, and then attach a non-exclusive fence while scanout is in
progress.
The one pending on the fence can add an async callback:
+ dma_fence_add_callback()
The callback can optionally be cancelled with remove_wait_queue()
Or wait synchronously (optionally with timeout or interruptible):
+ dma_fence_wait()
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = dma_buf_get_fence(dmabuf);
if (fence->ops == &bikeshed_fence_ops) {
dma_buf *fence_buf;
dma_bikeshed_fence_get_buf(fence, &fence_buf, &offset);
... tell the hw the memory location to wait on ...
} else {
/* fall-back to sw sync * /
dma_fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
To facilitate other non-sw implementations, the enable_signaling callback
can be used to keep track if a device not supporting hw sync is waiting
on the fence, and in this case should arrange to call dma_fence_signal()
at some point after the condition has changed, to notify other devices
waiting on the fence. If there are no sw waiters, this can be skipped to
avoid waking the CPU unnecessarily. The handler of the enable_signaling
op should take a refcount until the fence is signaled, then release its ref.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
v6: [ Maarten Lankhorst ] I removed dma_fence_cancel_callback and some comments
about checking if fence fired or not. This is broken by design.
waitqueue_active during destruction is now fatal, since the signaller
should be holding a reference in enable_signalling until it signalled
the fence. Pass the original dma_fence_cb along, and call __remove_wait
in the dma_fence_callback handler, so that no cleanup needs to be
performed.
v7: [ Maarten Lankhorst ] Set cb->func and only enable sw signaling if
fence wasn't signaled yet, for example for hardware fences that may
choose to signal blindly.
Signed-off-by: Maarten Lankhorst <maarten.lankhorst(a)canonical.com>
---
drivers/base/Makefile | 2
drivers/base/dma-fence.c | 287 +++++++++++++++++++++++++++++++++++++++++++++
include/linux/dma-fence.h | 96 +++++++++++++++
3 files changed, 384 insertions(+), 1 deletion(-)
create mode 100644 drivers/base/dma-fence.c
create mode 100644 include/linux/dma-fence.h
diff --git a/drivers/base/Makefile b/drivers/base/Makefile
index 5aa2d70..6e9f217 100644
--- a/drivers/base/Makefile
+++ b/drivers/base/Makefile
@@ -10,7 +10,7 @@ obj-$(CONFIG_CMA) += dma-contiguous.o
obj-y += power/
obj-$(CONFIG_HAS_DMA) += dma-mapping.o
obj-$(CONFIG_HAVE_GENERIC_DMA_COHERENT) += dma-coherent.o
-obj-$(CONFIG_DMA_SHARED_BUFFER) += dma-buf.o
+obj-$(CONFIG_DMA_SHARED_BUFFER) += dma-buf.o dma-fence.o
obj-$(CONFIG_ISA) += isa.o
obj-$(CONFIG_FW_LOADER) += firmware_class.o
obj-$(CONFIG_NUMA) += node.o
diff --git a/drivers/base/dma-fence.c b/drivers/base/dma-fence.c
new file mode 100644
index 0000000..c280ee7
--- /dev/null
+++ b/drivers/base/dma-fence.c
@@ -0,0 +1,287 @@
+/*
+ * Fence mechanism for dma-buf to allow for asynchronous dma access
+ *
+ * Copyright (C) 2012 Texas Instruments
+ * Author: Rob Clark <rob.clark(a)linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/slab.h>
+#include <linux/sched.h>
+#include <linux/export.h>
+#include <linux/dma-fence.h>
+
+/**
+ * dma_fence_signal - Signal a fence.
+ *
+ * @fence: The fence to signal
+ *
+ * All registered callbacks will be called directly (synchronously) and
+ * all blocked waters will be awoken.
+ *
+ * TODO: any value in adding a dma_fence_cancel(), for example to recov
+ * from hung gpu? It would behave like dma_fence_signal() but return
+ * an error to waiters and cb's to let them know that the condition they
+ * are waiting for will never happen.
+ */
+int dma_fence_signal(struct dma_fence *fence)
+{
+ unsigned long flags;
+ int ret = -EINVAL;
+
+ if (WARN_ON(!fence))
+ return -EINVAL;
+
+ spin_lock_irqsave(&fence->event_queue.lock, flags);
+ if (!fence->signaled) {
+ fence->signaled = true;
+ __wake_up_locked_key(&fence->event_queue, TASK_NORMAL,
+ &fence->event_queue);
+ ret = 0;
+ } else WARN(1, "Already signaled");
+ spin_unlock_irqrestore(&fence->event_queue.lock, flags);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(dma_fence_signal);
+
+static void release_fence(struct kref *kref)
+{
+ struct dma_fence *fence =
+ container_of(kref, struct dma_fence, refcount);
+
+ BUG_ON(waitqueue_active(&fence->event_queue));
+
+ if (fence->ops->release)
+ fence->ops->release(fence);
+
+ kfree(fence);
+}
+
+/**
+ * dma_fence_put - Release a reference to the fence.
+ */
+void dma_fence_put(struct dma_fence *fence)
+{
+ WARN_ON(!fence);
+ kref_put(&fence->refcount, release_fence);
+}
+EXPORT_SYMBOL_GPL(dma_fence_put);
+
+/**
+ * dma_fence_get - Take a reference to the fence.
+ *
+ * In most cases this is used only internally by dma-fence.
+ */
+void dma_fence_get(struct dma_fence *fence)
+{
+ WARN_ON(!fence);
+ kref_get(&fence->refcount);
+}
+EXPORT_SYMBOL_GPL(dma_fence_get);
+
+static int check_signaling(struct dma_fence *fence)
+{
+ bool enable_signaling = false, signaled;
+ unsigned long flags;
+
+ spin_lock_irqsave(&fence->event_queue.lock, flags);
+ signaled = fence->signaled;
+ if (!signaled && !fence->needs_sw_signal)
+ enable_signaling = fence->needs_sw_signal = true;
+ spin_unlock_irqrestore(&fence->event_queue.lock, flags);
+
+ if (enable_signaling) {
+ int ret;
+
+ /* At this point, if enable_signaling returns any error
+ * a wakeup has to be performanced regardless.
+ * -ENOENT signals fence was already signaled. Any other error
+ * inidicates a catastrophic hardware error.
+ *
+ * If any hardware error occurs, nothing can be done against
+ * it, so it's treated like the fence was already signaled.
+ * No synchronization can be performed, so we have to assume
+ * the fence was already signaled.
+ */
+ ret = fence->ops->enable_signaling(fence);
+ if (ret) {
+ signaled = true;
+ dma_fence_signal(fence);
+ }
+ }
+
+ if (!signaled)
+ return 0;
+ else
+ return -ENOENT;
+}
+
+int __dma_fence_wake_func(wait_queue_t *wait, unsigned mode,
+ int flags, void *key)
+{
+ struct dma_fence_cb *cb =
+ container_of(wait, struct dma_fence_cb, base);
+
+ __remove_wait_queue(key, wait);
+ return cb->func(cb, wait->private);
+}
+
+/**
+ * dma_fence_add_callback - Add a callback to be called when the fence
+ * is signaled.
+ *
+ * @fence: The fence to wait on
+ * @cb: The callback to register
+ *
+ * Any number of callbacks can be registered to a fence, but a callback
+ * can only be registered to once fence at a time.
+ *
+ * Note that the callback can be called from an atomic context. If
+ * fence is already signaled, this function will return -ENOENT (and
+ * *not* call the callback)
+ */
+int dma_fence_add_callback(struct dma_fence *fence, struct dma_fence_cb *cb,
+ dma_fence_func_t func, void *priv)
+{
+ unsigned long flags;
+ int ret;
+
+ if (WARN_ON(!fence || !func))
+ return -EINVAL;
+
+ ret = check_signaling(fence);
+
+ spin_lock_irqsave(&fence->event_queue.lock, flags);
+ if (!ret && fence->signaled)
+ ret = -ENOENT;
+
+ if (!ret) {
+ cb->base.flags = 0;
+ cb->base.func = __dma_fence_wake_func;
+ cb->base.private = priv;
+ cb->fence = fence;
+ cb->func = func;
+ __add_wait_queue(&fence->event_queue, &cb->base);
+ }
+ spin_unlock_irqrestore(&fence->event_queue.lock, flags);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(dma_fence_add_callback);
+
+/**
+ * dma_fence_wait - Wait for a fence to be signaled.
+ *
+ * @fence: The fence to wait on
+ * @interruptible: if true, do an interruptible wait
+ * @timeout: absolute time for timeout, in jiffies.
+ *
+ * Returns 0 on success, -EBUSY if a timeout occured,
+ * -ERESTARTSYS if the wait was interrupted by a signal.
+ */
+int dma_fence_wait(struct dma_fence *fence, bool interruptible, unsigned long timeout)
+{
+ unsigned long cur;
+ int ret;
+
+ if (WARN_ON(!fence))
+ return -EINVAL;
+
+ cur = jiffies;
+ if (time_after_eq(cur, timeout))
+ return -EBUSY;
+
+ timeout -= cur;
+
+ ret = check_signaling(fence);
+ if (ret == -ENOENT)
+ return 0;
+ else if (ret)
+ return ret;
+
+ if (interruptible)
+ ret = wait_event_interruptible_timeout(fence->event_queue,
+ fence->signaled,
+ timeout);
+ else
+ ret = wait_event_timeout(fence->event_queue,
+ fence->signaled, timeout);
+
+ if (ret > 0)
+ return 0;
+ else if (!ret)
+ return -EBUSY;
+ else
+ return ret;
+}
+EXPORT_SYMBOL_GPL(dma_fence_wait);
+
+/*
+ * Helpers intended to be used by the ops of the dma_fence implementation:
+ *
+ * NOTE: helpers and fxns intended to be used by other dma-fence
+ * implementations are not exported.. I'm not really sure if it makes
+ * sense to have a dma-fence implementation that is itself a module.
+ */
+
+void __dma_fence_init(struct dma_fence *fence, struct dma_fence_ops *ops, void *priv)
+{
+ WARN_ON(!ops || !ops->enable_signaling);
+
+ kref_init(&fence->refcount);
+ fence->ops = ops;
+ fence->priv = priv;
+ init_waitqueue_head(&fence->event_queue);
+}
+EXPORT_SYMBOL_GPL(__dma_fence_init);
+
+/*
+ * Pure sw implementation for dma-fence. The CPU always gets involved.
+ */
+
+static int sw_enable_signaling(struct dma_fence *fence)
+{
+ /*
+ * pure sw, no irq's to enable, because the fence creator will
+ * always call dma_fence_signal()
+ */
+ return 0;
+}
+
+static struct dma_fence_ops sw_fence_ops = {
+ .enable_signaling = sw_enable_signaling,
+};
+
+/**
+ * dma_fence_create - Create a simple sw-only fence.
+ *
+ * This fence only supports signaling from/to CPU. Other implementations
+ * of dma-fence can be used to support hardware to hardware signaling, if
+ * supported by the hardware, and use the dma_fence_helper_* functions for
+ * compatibility with other devices that only support sw signaling.
+ */
+struct dma_fence *dma_fence_create(void)
+{
+ struct dma_fence *fence;
+
+ fence = kzalloc(sizeof(struct dma_fence), GFP_KERNEL);
+ if (!fence)
+ return ERR_PTR(-ENOMEM);
+
+ __dma_fence_init(fence, &sw_fence_ops, 0);
+
+ return fence;
+}
+EXPORT_SYMBOL_GPL(dma_fence_create);
diff --git a/include/linux/dma-fence.h b/include/linux/dma-fence.h
new file mode 100644
index 0000000..70d12c0
--- /dev/null
+++ b/include/linux/dma-fence.h
@@ -0,0 +1,96 @@
+/*
+ * Fence mechanism for dma-buf to allow for asynchronous dma access
+ *
+ * Copyright (C) 2012 Texas Instruments
+ * Author: Rob Clark <rob.clark(a)linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __DMA_FENCE_H__
+#define __DMA_FENCE_H__
+
+#include <linux/err.h>
+#include <linux/list.h>
+#include <linux/wait.h>
+#include <linux/list.h>
+#include <linux/dma-buf.h>
+
+struct dma_fence;
+struct dma_fence_ops;
+struct dma_fence_cb;
+
+struct dma_fence {
+ struct kref refcount;
+ struct dma_fence_ops *ops;
+ wait_queue_head_t event_queue;
+ void *priv;
+
+ /* has this fence been signaled yet? */
+ bool signaled : 1;
+
+ /* do we have one or more waiters or callbacks? */
+ bool needs_sw_signal : 1;
+};
+
+typedef int (*dma_fence_func_t)(struct dma_fence_cb *cb, void *priv);
+
+struct dma_fence_cb {
+ wait_queue_t base;
+ dma_fence_func_t func;
+ struct dma_fence *fence;
+};
+
+struct dma_fence_ops {
+ /**
+ * For fence implementations that have the capability for hw->hw
+ * signaling, they can implement this op to enable the necessary
+ * irqs, or insert commands into cmdstream, etc. This is called
+ * in the first wait() or add_callback() path to let the fence
+ * implementation know that there is another driver waiting on
+ * the signal (ie. hw->sw case).
+ *
+ * A return value of -ENOENT will indicate that the fence has
+ * already passed. Any other errors will be treated as -ENOENT,
+ * and can happen because of hardware failure.
+ */
+ int (*enable_signaling)(struct dma_fence *fence);
+ void (*release)(struct dma_fence *fence);
+};
+
+/*
+ * TODO does it make sense to be able to enable dma-fence without dma-buf,
+ * or visa versa?
+ */
+#ifdef CONFIG_DMA_SHARED_BUFFER
+
+/* create a basic (pure sw) fence: */
+struct dma_fence *dma_fence_create(void);
+
+/* intended to be used by other dma_fence implementations: */
+void __dma_fence_init(struct dma_fence *fence,
+ struct dma_fence_ops *ops, void *priv);
+
+void dma_fence_get(struct dma_fence *fence);
+void dma_fence_put(struct dma_fence *fence);
+
+int dma_fence_signal(struct dma_fence *fence);
+int dma_fence_wait(struct dma_fence *fence, bool interruptible, unsigned long timeout);
+int dma_fence_add_callback(struct dma_fence *fence, struct dma_fence_cb *cb,
+ dma_fence_func_t func, void *priv);
+
+#else
+// TODO
+#endif /* CONFIG_DMA_SHARED_BUFFER */
+
+#endif /* __DMA_FENCE_H__ */
Hi Linus,
I would like to ask for pulling a set of fixup patches for ARM
dma-mapping extensions merged in v3.6-rc1.
The following changes since commit 0d7614f09c1ebdbaa1599a5aba7593f147bf96ee:
Linux 3.6-rc1 (2012-08-02 16:38:10 -0700)
with the top-most commit d9e0d149b5dcc2ef4688afc572b9906bcda941ef
ARM: dma-mapping: fix incorrect freeing of atomic allocations
are available in the git repository at:
git://git.linaro.org/people/mszyprowski/linux-dma-mapping.git fixes-for-linus-for-3.6-rc2
Thanks!
Best regards
Marek Szyprowski
Samsung Poland R&D Center
Patch summary:
Aaro Koskinen (2):
ARM: dma-mapping: fix atomic allocation alignment
ARM: dma-mapping: fix incorrect freeing of atomic allocations
Chris Brand (1):
ARM: mm: fix MMU mapping of CMA regions
arch/arm/mm/dma-mapping.c | 12 ++++++------
1 files changed, 6 insertions(+), 6 deletions(-)
Hello everyone,
The patches adds support for DMABUF exporting to V4L2 stack. The latest
support for DMABUF importing was posted in [1]. The exporter part is dependant
on DMA mapping redesign [2] which is expected to be merged into the mainline.
Therefore it is posted as a separate patchset. Moreover some patches depends on
vmap extension for DMABUF by Dave Airlie [3] and sg_alloc_table_from_pages
function [4]. The last patch 'v4l: vb2-dma-contig: use dma_get_sgtable' depends
on dma_get_sgtable extension to DMA api [5].
The tree with all the patches and extensions is available at:
repo: git://git.infradead.org/users/kmpark/linux-2.6-samsung
branch: media-for3.5-vb2-dmabuf-v7
Changelog:
v2:
- add documentation for DMABUF exporting
- squashed 'let mmap method to use dma_mmap_coherent call' with
'remove vb2_mmap_pfn_range function'
- move setup of scatterlist for MMAP buffers from alloc to DMABUF export code
- use locking to serialize map/unmap of DMABUF attachments
- squash vmap/kmap, setup of sg lists, allocation in attachments into
dma-contig exporter patch
- fix occasional failure of follow_pfn trick by using init_mm in artificial
VMA
- add support for exporting in s5p-mfc driver
- drop all code that duplicates sg_alloc_table_from_pages
- introduce usage of dma_get_sgtable as generic solution
to follow_pfn trick
v1:
- updated setup of VIDIOC_EXPBUF ioctl
- doc updates
- introduced workaround to avoid using dma_get_pages,
- removed caching of exported dmabuf to avoid existence of circular reference
between dmabuf and vb2_dc_buf or resource leakage
- removed all 'change behaviour' patches
- inital support for exporting in s5p-mfs driver
- removal of vb2_mmap_pfn_range that is no longer used
- use sg_alloc_table_from_pages instead of creating sglist in vb2_dc code
- move attachment allocation to exporter's attach callback
v0: RFC
- initial version
[1] http://thread.gmane.org/gmane.linux.drivers.video-input-infrastructure/49438
[2] http://thread.gmane.org/gmane.linux.kernel.cross-arch/14098
[3] http://permalink.gmane.org/gmane.comp.video.dri.devel/69302
[4] This patchset is rebased on 3.4-rc1 plus the following patchsets:
[5] http://www.spinics.net/lists/linux-arch/msg18282.html
Marek Szyprowski (1):
v4l: vb2-dma-contig: let mmap method to use dma_mmap_coherent call
Tomasz Stanislawski (8):
Documentation: media: description of DMABUF exporting in V4L2
v4l: add buffer exporting via dmabuf
v4l: vb2: add buffer exporting via dmabuf
v4l: vb2-dma-contig: add support for DMABUF exporting
v4l: s5p-fimc: support for dmabuf exporting
v4l: s5p-tv: mixer: support for dmabuf exporting
v4l: s5p-mfc: support for dmabuf exporting
v4l: vb2-dma-contig: use dma_get_sgtable
Documentation/DocBook/media/v4l/compat.xml | 3 +
Documentation/DocBook/media/v4l/io.xml | 3 +
Documentation/DocBook/media/v4l/v4l2.xml | 1 +
Documentation/DocBook/media/v4l/vidioc-expbuf.xml | 223 ++++++++++++++++++++
drivers/media/video/s5p-fimc/fimc-capture.c | 9 +
drivers/media/video/s5p-mfc/s5p_mfc_dec.c | 18 ++
drivers/media/video/s5p-mfc/s5p_mfc_enc.c | 18 ++
drivers/media/video/s5p-tv/mixer_video.c | 10 +
drivers/media/video/v4l2-compat-ioctl32.c | 1 +
drivers/media/video/v4l2-dev.c | 1 +
drivers/media/video/v4l2-ioctl.c | 6 +
drivers/media/video/videobuf2-core.c | 67 ++++++
drivers/media/video/videobuf2-dma-contig.c | 224 ++++++++++++++++++++-
drivers/media/video/videobuf2-memops.c | 40 ----
include/linux/videodev2.h | 26 +++
include/media/v4l2-ioctl.h | 2 +
include/media/videobuf2-core.h | 2 +
include/media/videobuf2-memops.h | 5 -
18 files changed, 612 insertions(+), 47 deletions(-)
create mode 100644 Documentation/DocBook/media/v4l/vidioc-expbuf.xml
--
1.7.9.5
From: Rob Clark <rob(a)ti.com>
Re-sending first patch, with a wider audience. Apparently I didn't
spam enough inboxes the first time.
And, at Daniel Vetter's suggestion, adding some helper functions in
dma-buf to get the most restrictive parameters of all the attached
devices.
Rob Clark (2):
device: add dma_params->max_segment_count
dma-buf: add helpers for attacher dma-parms
drivers/base/dma-buf.c | 63 +++++++++++++++++++++++++++++++++++++++++++
include/linux/device.h | 1 +
include/linux/dma-buf.h | 19 +++++++++++++
include/linux/dma-mapping.h | 16 +++++++++++
4 files changed, 99 insertions(+)
--
1.7.9.5
A dma-fence can be attached to a buffer which is being filled or consumed
by hw, to allow userspace to pass the buffer without waiting to another
device. For example, userspace can call page_flip ioctl to display the
next frame of graphics after kicking the GPU but while the GPU is still
rendering. The display device sharing the buffer with the GPU would
attach a callback to get notified when the GPU's rendering-complete IRQ
fires, to update the scan-out address of the display, without having to
wake up userspace.
A dma-fence is transient, one-shot deal. It is allocated and attached
to one or more dma-buf's. When the one that attached it is done, with
the pending operation, it can signal the fence.
+ dma_fence_signal()
The dma-buf-mgr handles tracking, and waiting on, the fences associated
with a dma-buf.
TODO maybe need some helper fxn for simple devices, like a display-
only drm/kms device which simply wants to wait for exclusive fence to
be signaled, and then attach a non-exclusive fence while scanout is in
progress.
The one pending on the fence can add an async callback (and optionally
cancel it.. for example, to recover from GPU hangs):
+ dma_fence_add_callback()
+ dma_fence_cancel_callback()
Or wait synchronously (optionally with timeout or from atomic context):
+ dma_fence_wait()
A default software-only implementation is provided, which can be used
by drivers attaching a fence to a buffer when they have no other means
for hw sync. But a memory backed fence is also envisioned, because it
is common that GPU's can write to, or poll on some memory location for
synchronization. For example:
fence = dma_buf_get_fence(dmabuf);
if (fence->ops == &bikeshed_fence_ops) {
dma_buf *fence_buf;
dma_bikeshed_fence_get_buf(fence, &fence_buf, &offset);
... tell the hw the memory location to wait on ...
} else {
/* fall-back to sw sync * /
dma_fence_add_callback(fence, my_cb);
}
On SoC platforms, if some other hw mechanism is provided for synchronizing
between IP blocks, it could be supported as an alternate implementation
with it's own fence ops in a similar way.
To facilitate other non-sw implementations, the enable_signaling callback
can be used to keep track if a device not supporting hw sync is waiting
on the fence, and in this case should arrange to call dma_fence_signal()
at some point after the condition has changed, to notify other devices
waiting on the fence. If there are no sw waiters, this can be skipped to
avoid waking the CPU unnecessarily.
The intention is to provide a userspace interface (presumably via eventfd)
later, to be used in conjunction with dma-buf's mmap support for sw access
to buffers (or for userspace apps that would prefer to do their own
synchronization).
v1: Original
v2: After discussion w/ danvet and mlankhorst on #dri-devel, we decided
that dma-fence didn't need to care about the sw->hw signaling path
(it can be handled same as sw->sw case), and therefore the fence->ops
can be simplified and more handled in the core. So remove the signal,
add_callback, cancel_callback, and wait ops, and replace with a simple
enable_signaling() op which can be used to inform a fence supporting
hw->hw signaling that one or more devices which do not support hw
signaling are waiting (and therefore it should enable an irq or do
whatever is necessary in order that the CPU is notified when the
fence is passed).
v3: Fix locking fail in attach_fence() and get_fence()
v4: Remove tie-in w/ dma-buf.. after discussion w/ danvet and mlankorst
we decided that we need to be able to attach one fence to N dma-buf's,
so using the list_head in dma-fence struct would be problematic.
v5: [ Maarten Lankhorst ] Updated for dma-bikeshed-fence and dma-buf-manager.
---
drivers/base/Makefile | 2
drivers/base/dma-fence.c | 317 +++++++++++++++++++++++++++++++++++++++++++++
include/linux/dma-fence.h | 123 +++++++++++++++++
3 files changed, 441 insertions(+), 1 deletion(-)
create mode 100644 drivers/base/dma-fence.c
create mode 100644 include/linux/dma-fence.h
diff --git a/drivers/base/Makefile b/drivers/base/Makefile
index 5aa2d70..6e9f217 100644
--- a/drivers/base/Makefile
+++ b/drivers/base/Makefile
@@ -10,7 +10,7 @@ obj-$(CONFIG_CMA) += dma-contiguous.o
obj-y += power/
obj-$(CONFIG_HAS_DMA) += dma-mapping.o
obj-$(CONFIG_HAVE_GENERIC_DMA_COHERENT) += dma-coherent.o
-obj-$(CONFIG_DMA_SHARED_BUFFER) += dma-buf.o
+obj-$(CONFIG_DMA_SHARED_BUFFER) += dma-buf.o dma-fence.o
obj-$(CONFIG_ISA) += isa.o
obj-$(CONFIG_FW_LOADER) += firmware_class.o
obj-$(CONFIG_NUMA) += node.o
diff --git a/drivers/base/dma-fence.c b/drivers/base/dma-fence.c
new file mode 100644
index 0000000..6798dc4
--- /dev/null
+++ b/drivers/base/dma-fence.c
@@ -0,0 +1,317 @@
+/*
+ * Fence mechanism for dma-buf to allow for asynchronous dma access
+ *
+ * Copyright (C) 2012 Texas Instruments
+ * Author: Rob Clark <rob.clark(a)linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/slab.h>
+#include <linux/sched.h>
+#include <linux/export.h>
+#include <linux/dma-fence.h>
+
+/**
+ * dma_fence_signal - Signal a fence.
+ *
+ * @fence: The fence to signal
+ *
+ * All registered callbacks will be called directly (synchronously) and
+ * all blocked waters will be awoken.
+ *
+ * TODO: any value in adding a dma_fence_cancel(), for example to recov
+ * from hung gpu? It would behave like dma_fence_signal() but return
+ * an error to waiters and cb's to let them know that the condition they
+ * are waiting for will never happen.
+ */
+int dma_fence_signal(struct dma_fence *fence)
+{
+ unsigned long flags;
+ int ret = -EINVAL;
+
+ if (WARN_ON(!fence))
+ return -EINVAL;
+
+ spin_lock_irqsave(&fence->event_queue.lock, flags);
+ if (!fence->signaled) {
+ fence->signaled = true;
+ wake_up_all_locked(&fence->event_queue);
+ ret = 0;
+ } else WARN(1, "Already signaled");
+ spin_unlock_irqrestore(&fence->event_queue.lock, flags);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(dma_fence_signal);
+
+static void release_fence(struct kref *kref)
+{
+ struct dma_fence *fence =
+ container_of(kref, struct dma_fence, refcount);
+
+ WARN_ON(waitqueue_active(&fence->event_queue));
+ if (fence->ops->release)
+ fence->ops->release(fence);
+
+ kfree(fence);
+}
+
+/**
+ * dma_fence_put - Release a reference to the fence.
+ */
+void dma_fence_put(struct dma_fence *fence)
+{
+ WARN_ON(!fence);
+ kref_put(&fence->refcount, release_fence);
+}
+EXPORT_SYMBOL_GPL(dma_fence_put);
+
+/**
+ * dma_fence_get - Take a reference to the fence.
+ *
+ * In most cases this is used only internally by dma-fence.
+ */
+void dma_fence_get(struct dma_fence *fence)
+{
+ WARN_ON(!fence);
+ kref_get(&fence->refcount);
+}
+EXPORT_SYMBOL_GPL(dma_fence_get);
+
+static int check_signaling(struct dma_fence *fence)
+{
+ bool enable_signaling = false, signaled;
+ unsigned long flags;
+
+ spin_lock_irqsave(&fence->event_queue.lock, flags);
+ if (!fence->needs_sw_signal)
+ enable_signaling = fence->needs_sw_signal = true;
+ signaled = fence->signaled;
+ spin_unlock_irqrestore(&fence->event_queue.lock, flags);
+
+ if (enable_signaling) {
+ int ret;
+
+ /* At this point, if enable_signaling returns any error
+ * a wakeup has to be performanced regardless.
+ * -ENOENT signals fence was already signaled. Any other error
+ * inidicates a catastrophic hardware error.
+ *
+ * If any hardware error occurs, nothing can be done against
+ * it, so it's treated like the fence was already signaled.
+ * No synchronization can be performed, so we have to assume
+ * the fence was already signaled.
+ */
+ ret = fence->ops->enable_signaling(fence);
+ if (ret) {
+ signaled = true;
+ dma_fence_signal(fence);
+ }
+ }
+
+ if (!signaled)
+ return 0;
+ else
+ return -ENOENT;
+}
+
+/**
+ * dma_fence_add_callback - Add a callback to be called when the fence
+ * is signaled.
+ *
+ * @fence: The fence to wait on
+ * @cb: The callback to register
+ *
+ * Any number of callbacks can be registered to a fence, but a callback
+ * can only be registered to once fence at a time.
+ *
+ * Note that the callback can be called from an atomic context. If
+ * fence is already signaled, this function will return -ENOENT (and
+ * *not* call the callback)
+ */
+int dma_fence_add_callback(struct dma_fence *fence,
+ struct dma_fence_cb *cb)
+{
+ unsigned long flags;
+ int ret;
+
+ if (WARN_ON(!fence || !cb))
+ return -EINVAL;
+
+ ret = check_signaling(fence);
+
+ spin_lock_irqsave(&fence->event_queue.lock, flags);
+ if (!ret && !fence->signaled) {
+ cb->fence = fence;
+ __add_wait_queue(&fence->event_queue, &cb->base);
+ ret = 0;
+ } else if (!ret)
+ ret = -ENOENT;
+ spin_unlock_irqrestore(&fence->event_queue.lock, flags);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(dma_fence_add_callback);
+
+/**
+ * dma_fence_cancel_callback - Remove a previously registered callback.
+ *
+ * @cb: The callback to unregister
+ *
+ * The callback will not be called after this function returns, but could
+ * be called before this function returns.
+ */
+int dma_fence_cancel_callback(struct dma_fence_cb *cb)
+{
+ struct dma_fence *fence;
+ unsigned long flags;
+ int ret = -EINVAL;
+
+ if (WARN_ON(!cb))
+ return -EINVAL;
+
+ fence = cb->fence;
+
+ spin_lock_irqsave(&fence->event_queue.lock, flags);
+ if (fence) {
+ __remove_wait_queue(&fence->event_queue, &cb->base);
+ cb->fence = NULL;
+ ret = 0;
+ }
+ spin_unlock_irqrestore(&fence->event_queue.lock, flags);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(dma_fence_cancel_callback);
+
+/**
+ * dma_fence_wait - Wait for a fence to be signaled.
+ *
+ * @fence: The fence to wait on
+ * @interruptible: if true, do an interruptible wait
+ * @timeout: absolute time for timeout, in jiffies.
+ *
+ * Returns 0 on success, -EBUSY if a timeout occured,
+ * -ERESTARTSYS if the wait was interrupted by a signal.
+ */
+int dma_fence_wait(struct dma_fence *fence, bool interruptible, unsigned long timeout)
+{
+ unsigned long cur;
+ int ret;
+
+ if (WARN_ON(!fence))
+ return -EINVAL;
+
+ cur = jiffies;
+ if (time_after_eq(cur, timeout))
+ return -EBUSY;
+
+ timeout -= cur;
+
+ ret = check_signaling(fence);
+ if (ret == -ENOENT)
+ return 0;
+ else if (ret)
+ return ret;
+
+ if (interruptible)
+ ret = wait_event_interruptible_timeout(fence->event_queue,
+ fence->signaled,
+ timeout);
+ else
+ ret = wait_event_timeout(fence->event_queue,
+ fence->signaled, timeout);
+
+ WARN(1, "wait_event_timeout(%u) returns %i", interruptible, ret);
+ if (ret > 0)
+ return 0;
+ else if (!ret)
+ return -EBUSY;
+ else
+ return ret;
+}
+EXPORT_SYMBOL_GPL(dma_fence_wait);
+
+int __dma_fence_wake_func(wait_queue_t *wait, unsigned mode,
+ int flags, void *key)
+{
+ struct dma_fence_cb *cb =
+ container_of(wait, struct dma_fence_cb, base);
+ struct dma_fence *fence = cb->fence;
+ int ret;
+
+ ret = cb->func(cb, fence);
+ cb->fence = NULL;
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(__dma_fence_wake_func);
+
+/*
+ * Helpers intended to be used by the ops of the dma_fence implementation:
+ *
+ * NOTE: helpers and fxns intended to be used by other dma-fence
+ * implementations are not exported.. I'm not really sure if it makes
+ * sense to have a dma-fence implementation that is itself a module.
+ */
+
+void __dma_fence_init(struct dma_fence *fence, struct dma_fence_ops *ops, void *priv)
+{
+ WARN_ON(!ops || !ops->enable_signaling);
+
+ kref_init(&fence->refcount);
+ fence->ops = ops;
+ fence->priv = priv;
+ init_waitqueue_head(&fence->event_queue);
+}
+EXPORT_SYMBOL_GPL(__dma_fence_init);
+
+/*
+ * Pure sw implementation for dma-fence. The CPU always gets involved.
+ */
+
+static int sw_enable_signaling(struct dma_fence *fence)
+{
+ /*
+ * pure sw, no irq's to enable, because the fence creator will
+ * always call dma_fence_signal()
+ */
+ return 0;
+}
+
+static struct dma_fence_ops sw_fence_ops = {
+ .enable_signaling = sw_enable_signaling,
+};
+
+/**
+ * dma_fence_create - Create a simple sw-only fence.
+ *
+ * This fence only supports signaling from/to CPU. Other implementations
+ * of dma-fence can be used to support hardware to hardware signaling, if
+ * supported by the hardware, and use the dma_fence_helper_* functions for
+ * compatibility with other devices that only support sw signaling.
+ */
+struct dma_fence *dma_fence_create(void)
+{
+ struct dma_fence *fence;
+
+ fence = kzalloc(sizeof(struct dma_fence), GFP_KERNEL);
+ if (!fence)
+ return ERR_PTR(-ENOMEM);
+
+ __dma_fence_init(fence, &sw_fence_ops, 0);
+
+ return fence;
+}
+EXPORT_SYMBOL_GPL(dma_fence_create);
diff --git a/include/linux/dma-fence.h b/include/linux/dma-fence.h
new file mode 100644
index 0000000..648f136
--- /dev/null
+++ b/include/linux/dma-fence.h
@@ -0,0 +1,123 @@
+/*
+ * Fence mechanism for dma-buf to allow for asynchronous dma access
+ *
+ * Copyright (C) 2012 Texas Instruments
+ * Author: Rob Clark <rob.clark(a)linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __DMA_FENCE_H__
+#define __DMA_FENCE_H__
+
+#include <linux/err.h>
+#include <linux/list.h>
+#include <linux/wait.h>
+#include <linux/list.h>
+#include <linux/dma-buf.h>
+
+struct dma_fence;
+struct dma_fence_ops;
+struct dma_fence_cb;
+
+struct dma_fence {
+ struct kref refcount;
+ struct dma_fence_ops *ops;
+ wait_queue_head_t event_queue;
+ void *priv;
+
+ /* has this fence been signaled yet? */
+ bool signaled : 1;
+
+ /* do we have one or more waiters or callbacks? */
+ bool needs_sw_signal : 1;
+};
+
+typedef int (*dma_fence_func_t)(struct dma_fence_cb *cb,
+ struct dma_fence *fence);
+
+struct dma_fence_cb {
+ wait_queue_t base;
+ dma_fence_func_t func;
+
+ /*
+ * This is initialized when the cb is added, and NULL'd when it
+ * is canceled or expired, so can be used to for error checking
+ * if the cb is already pending. A dma_fence_cb can be pending
+ * on at most one fence at a time.
+ */
+ struct dma_fence *fence;
+};
+
+struct dma_fence_ops {
+ /**
+ * For fence implementations that have the capability for hw->hw
+ * signaling, they can implement this op to enable the necessary
+ * irqs, or insert commands into cmdstream, etc. This is called
+ * in the first wait() or add_callback() path to let the fence
+ * implementation know that there is another driver waiting on
+ * the signal (ie. hw->sw case).
+ *
+ * A return value of -ENOENT will indicate that the fence has
+ * already passed.
+ */
+ int (*enable_signaling)(struct dma_fence *fence);
+ void (*release)(struct dma_fence *fence);
+};
+
+int __dma_fence_wake_func(wait_queue_t *wait, unsigned mode,
+ int flags, void *key);
+
+#define DMA_FENCE_CB_INITIALIZER(cb_func) { \
+ .base = { .func = __dma_fence_wake_func }, \
+ .func = (cb_func), \
+ }
+
+#define DECLARE_DMA_FENCE_CB(name, cb_func) \
+ struct dma_fence_cb name = DMA_FENCE_CB_INITIALIZER(cb_func)
+
+
+/*
+ * TODO does it make sense to be able to enable dma-fence without dma-buf,
+ * or visa versa?
+ */
+#ifdef CONFIG_DMA_SHARED_BUFFER
+
+/* create a basic (pure sw) fence: */
+struct dma_fence *dma_fence_create(void);
+
+/* intended to be used by other dma_fence implementations: */
+void __dma_fence_init(struct dma_fence *fence, struct dma_fence_ops *ops, void *priv);
+
+void dma_fence_get(struct dma_fence *fence);
+void dma_fence_put(struct dma_fence *fence);
+int dma_fence_signal(struct dma_fence *fence);
+
+int dma_fence_add_callback(struct dma_fence *fence,
+ struct dma_fence_cb *cb);
+int dma_fence_cancel_callback(struct dma_fence_cb *cb);
+int dma_fence_wait(struct dma_fence *fence, bool interruptible, unsigned long timeout);
+
+/* helpers intended to be used by the ops of the dma_fence implementation: */
+int dma_fence_helper_signal(struct dma_fence *fence);
+int dma_fence_helper_add_callback(struct dma_fence *fence,
+ struct dma_fence_cb *cb);
+int dma_fence_helper_cancel_callback(struct dma_fence_cb *cb);
+int dma_fence_helper_wait(struct dma_fence *fence, bool interruptible,
+ long timeout);
+
+#else
+// TODO
+#endif /* CONFIG_DMA_SHARED_BUFFER */
+
+#endif /* __DMA_FENCE_H__ */
Hi Linus,
I would like to ask for pulling another set of DMA-mapping framework
updates for v3.6.
The following changes since commit 28a33cbc24e4256c143dce96c7d93bf423229f92:
Linux 3.5 (2012-07-21 13:58:29 -0700)
with the top-most commit 97ef952a20853fad72087a53fa556fbec45edd8f
ARM: dma-mapping: add support for DMA_ATTR_SKIP_CPU_SYNC attribute
are available in the git repository at:
git://git.linaro.org/people/mszyprowski/linux-dma-mapping.git for-linus-for-3.6-rc1
Those patches are continuation of my earlier works merged in commits
58bca4a8fa90 and d484864dd96e1. They contains extensions to DMA-mapping
framework to remove limitation of the current ARM implementation (like
limited total size of DMA coherent/write combine buffers), improve
performance of buffer sharing between devices (attributes to skip cpu
cache operations or creation of additional kernel mapping for some
specific use cases) as well as some unification of the common code for
dma_mmap_attrs()/dma_mmap_coherent() functions. All extensions have been
implemented and tested for ARM architecture.
Thanks!
Best regards
Marek Szyprowski
Samsung Poland R&D Center
Patch summary:
Marek Szyprowski (11):
mm: vmalloc: use const void * for caller argument
ARM: dma-mapping: remove custom consistent dma region
ARM: dma-mapping: add more sanity checks in arm_dma_mmap()
ARM: dma-mapping: fix error path for memory allocation failure
common: dma-mapping: add support for generic dma_mmap_* calls
common: DMA-mapping: add DMA_ATTR_NO_KERNEL_MAPPING attribute
ARM: dma-mapping: add support for DMA_ATTR_NO_KERNEL_MAPPING attribute
common: dma-mapping: introduce dma_get_sgtable() function
ARM: dma-mapping: add support for dma_get_sgtable()
common: DMA-mapping: add DMA_ATTR_SKIP_CPU_SYNC attribute
ARM: dma-mapping: add support for DMA_ATTR_SKIP_CPU_SYNC attribute
Tomasz Stanislawski (1):
scatterlist: add sg_alloc_table_from_pages function
Documentation/DMA-attributes.txt | 42 +++
Documentation/kernel-parameters.txt | 2 +-
arch/arm/common/dmabounce.c | 1 +
arch/arm/include/asm/dma-mapping.h | 24 +-
arch/arm/mm/dma-mapping.c | 561 ++++++++++++------------------
arch/arm/mm/mm.h | 3 +
arch/powerpc/include/asm/dma-mapping.h | 8 +-
arch/powerpc/kernel/dma-iommu.c | 1 +
arch/powerpc/kernel/dma-swiotlb.c | 1 +
arch/powerpc/kernel/dma.c | 36 +-
arch/powerpc/kernel/vio.c | 1 +
drivers/base/dma-mapping.c | 49 +++
include/asm-generic/dma-coherent.h | 1 +
include/asm-generic/dma-mapping-common.h | 55 +++
include/linux/dma-attrs.h | 2 +
include/linux/dma-mapping.h | 3 +
include/linux/scatterlist.h | 4 +
include/linux/vmalloc.h | 9 +-
lib/scatterlist.c | 64 ++++
mm/vmalloc.c | 28 +-
20 files changed, 505 insertions(+), 390 deletions(-)
Hello!
This is yet another quick update on the patchset which replaces custom
consistent dma regions usage in dma-mapping framework in favour of
generic vmalloc areas created on demand for each allocation. The main
purpose for this patchset is to remove 2MiB limit of dma
coherent/writecombine allocations.
This version addresses a few minor issues pointed by Minchan Kim.
This patch is based on vanilla v3.5 release.
Best regards
Marek Szyprowski
Samsung Poland R&D Center
Changelog:
v5:
- fixed another minor issues pointed by Minchan Kim: added more comments
here and there, changed pr_err() + stack_dump() to WARN(), added a fix
for no-MMU systems
v4: http://thread.gmane.org/gmane.linux.kernel.mm/80906
- replaced arch-independent VM_DMA flag with ARM-specific
VM_ARM_DMA_CONSISTENT flag
v3: http://thread.gmane.org/gmane.linux.kernel.mm/80028
- rebased onto v3.4-rc2: added support for IOMMU-aware implementation
of dma-mapping calls, unified with CMA coherent dma pool
- implemented changes requested by Minchan Kim: added more checks for
vmarea->flags & VM_DMA, renamed some variables, removed obsole locks,
squashed find_vm_area() exporting patch into the main redesign patch
v2: http://thread.gmane.org/gmane.linux.kernel.mm/78563
- added support for atomic allocations (served from preallocated pool)
- minor cleanup here and there
- rebased onto v3.4-rc7
v1: http://thread.gmane.org/gmane.linux.kernel.mm/76703
- initial version
Patch summary:
Marek Szyprowski (2):
mm: vmalloc: use const void * for caller argument
ARM: dma-mapping: remove custom consistent dma region
Documentation/kernel-parameters.txt | 2 +-
arch/arm/include/asm/dma-mapping.h | 2 +-
arch/arm/mm/dma-mapping.c | 511 +++++++++++++----------------------
arch/arm/mm/mm.h | 3 +
include/linux/vmalloc.h | 9 +-
mm/vmalloc.c | 28 ++-
6 files changed, 210 insertions(+), 345 deletions(-)
--
1.7.1.569.g6f426