On Mon, Mar 06, 2017 at 08:42:59AM +0100, Michal Hocko wrote:
> On Fri 03-03-17 09:37:55, Laura Abbott wrote:
> > On 03/03/2017 05:29 AM, Michal Hocko wrote:
> > > On Thu 02-03-17 13:44:32, Laura Abbott wrote:
> > >> Hi,
> > >>
> > >> There's been some recent discussions[1] about Ion-like frameworks. There's
> > >> apparently interest in just keeping Ion since it works reasonablly well.
> > >> This series does what should be the final clean ups for it to possibly be
> > >> moved out of staging.
> > >>
> > >> This includes the following:
> > >> - Some general clean up and removal of features that never got a lot of use
> > >> as far as I can tell.
> > >> - Fixing up the caching. This is the series I proposed back in December[2]
> > >> but never heard any feedback on. It will certainly break existing
> > >> applications that rely on the implicit caching. I'd rather make an effort
> > >> to move to a model that isn't going directly against the establishement
> > >> though.
> > >> - Fixing up the platform support. The devicetree approach was never well
> > >> recieved by DT maintainers. The proposal here is to think of Ion less as
> > >> specifying requirements and more of a framework for exposing memory to
> > >> userspace.
> > >> - CMA allocations now happen without the need of a dummy device structure.
> > >> This fixes a bunch of the reasons why I attempted to add devicetree
> > >> support before.
> > >>
> > >> I've had problems getting feedback in the past so if I don't hear any major
> > >> objections I'm going to send out with the RFC dropped to be picked up.
> > >> The only reason there isn't a patch to come out of staging is to discuss any
> > >> other changes to the ABI people might want. Once this comes out of staging,
> > >> I really don't want to mess with the ABI.
> > >
> > > Could you recapitulate concerns preventing the code being merged
> > > normally rather than through the staging tree and how they were
> > > addressed?
> > >
> >
> > Sorry, I'm really not understanding your question here, can you
> > clarify?
>
> There must have been a reason why this code ended up in the staging
> tree, right? So my question is what those reasons were and how they were
> handled in order to move the code from the staging subtree.
No one gave a thing about android in upstream, so Greg KH just dumped it
all into staging/android/. We've discussed ION a bunch of times, recorded
anything we'd like to fix in staging/android/TODO, and Laura's patch
series here addresses a big chunk of that.
This is pretty much the same approach we (gpu folks) used to de-stage the
syncpt stuff.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
On 03/03/2017 05:29 AM, Michal Hocko wrote:
> On Thu 02-03-17 13:44:32, Laura Abbott wrote:
>> Hi,
>>
>> There's been some recent discussions[1] about Ion-like frameworks. There's
>> apparently interest in just keeping Ion since it works reasonablly well.
>> This series does what should be the final clean ups for it to possibly be
>> moved out of staging.
>>
>> This includes the following:
>> - Some general clean up and removal of features that never got a lot of use
>> as far as I can tell.
>> - Fixing up the caching. This is the series I proposed back in December[2]
>> but never heard any feedback on. It will certainly break existing
>> applications that rely on the implicit caching. I'd rather make an effort
>> to move to a model that isn't going directly against the establishement
>> though.
>> - Fixing up the platform support. The devicetree approach was never well
>> recieved by DT maintainers. The proposal here is to think of Ion less as
>> specifying requirements and more of a framework for exposing memory to
>> userspace.
>> - CMA allocations now happen without the need of a dummy device structure.
>> This fixes a bunch of the reasons why I attempted to add devicetree
>> support before.
>>
>> I've had problems getting feedback in the past so if I don't hear any major
>> objections I'm going to send out with the RFC dropped to be picked up.
>> The only reason there isn't a patch to come out of staging is to discuss any
>> other changes to the ABI people might want. Once this comes out of staging,
>> I really don't want to mess with the ABI.
>
> Could you recapitulate concerns preventing the code being merged
> normally rather than through the staging tree and how they were
> addressed?
>
Sorry, I'm really not understanding your question here, can you
clarify?
Thanks,
Laura
On 02/22/2017 05:01 PM, Chen Feng wrote:
>
>
> On 2017/2/22 3:29, Laura Abbott wrote:
>> On 02/20/2017 10:05 PM, Chen Feng wrote:
>>> Hi Laura,
>>>
>>> When we enable kernel v4.4 or newer version on our platform, we meet the issue
>>> of flushing cache without reference device. It seems that this patch set is
>>> a solution. I'm curious the progress of the discussion. Do you have any plan
>>> to fix it in v4.4 and newer kernel verison?
>>>
>>
>> No, I've abandoned this approach based on feedback. The APIs had too much
>> potential for incorrect usage. I'm ripping out the implicit caching in Ion
>> and switching it to a model where there should always be a device available.
>>
>> What's your use case where you don't have a device structure?
>>
> Userspace use ioctl to flush cache for device.
>
> ion_sync_for_device
> dma_sync_sg_for_device(NULL, buffer->sg_table->sgl,
> buffer->sg_table->nents, DMA_BIDIRECTIONAL);
>
> And sys-heap when allocate a zero buffer flush zero data to ddr.
> alloc_buffer_page
> ion_pages_sync_for_device(NULL, page, PAGE_SIZE << order,
> DMA_BIDIRECTIONAL);
>
>
Yes, those calls are being removed. This is what I proposed back in
December https://marc.info/?l=linux-kernel&m=148176054902921&w=2
I never heard any feedback on it so I assume everyone was okay with
the general direction. I plan on pushing a revised and expanded
version of that series once the merge window closes.
Thanks,
Laura
>
>> Thanks,
>> Laura
>>
>>> On 2016/9/14 2:41, Laura Abbott wrote:
>>>> On 09/13/2016 08:14 AM, Will Deacon wrote:
>>>>> On Tue, Sep 13, 2016 at 08:02:20AM -0700, Laura Abbott wrote:
>>>>>> On 09/13/2016 02:19 AM, Will Deacon wrote:
>>>>>>> On Mon, Sep 12, 2016 at 02:32:56PM -0700, Laura Abbott wrote:
>>>>>>>>
>>>>>>>> arm64 may need to guarantee the caches are synced. Implement versions of
>>>>>>>> the kernel_force_cache API to allow this.
>>>>>>>>
>>>>>>>> Signed-off-by: Laura Abbott <labbott(a)redhat.com>
>>>>>>>> ---
>>>>>>>> v3: Switch to calling cache operations directly instead of relying on
>>>>>>>> DMA mapping.
>>>>>>>> ---
>>>>>>>> arch/arm64/include/asm/cacheflush.h | 8 ++++++++
>>>>>>>> arch/arm64/mm/cache.S | 24 ++++++++++++++++++++----
>>>>>>>> arch/arm64/mm/flush.c | 11 +++++++++++
>>>>>>>> 3 files changed, 39 insertions(+), 4 deletions(-)
>>>>>>>
>>>>>>> I'm really hesitant to expose these cache routines as an API solely to
>>>>>>> support a driver sitting in staging/. I appreciate that there's a chicken
>>>>>>> and egg problem here, but we *really* don't want people using these routines
>>>>>>> in preference to the DMA API, and I fear that we'll simply grow a bunch
>>>>>>> more users of these things if we promote it as an API like you're proposing.
>>>>>>>
>>>>>>> Can the code not be contained under staging/, as part of ion?
>>>>>>>
>>>>>>
>>>>>> I proposed that in V1 and it was suggested I make it a proper API
>>>>>>
>>>>>> http://www.mail-archive.com/driverdev-devel@linuxdriverproject.org/msg47654…
>>>>>> http://www.mail-archive.com/driverdev-devel@linuxdriverproject.org/msg47672…
>>>>>
>>>>> :/ then I guess we're in disagreement. If ion really needs this stuff
>>>>> (which I don't fully grok), perhaps we should be exposing something at
>>>>> a higher level from the architecture, so it really can't be used for
>>>>> anything other than ion.
>>>>
>>>> I talked/complained about this at a past plumbers. The gist is that Ion
>>>> ends up acting as a fake DMA layer for clients. It doesn't match nicely
>>>> because clients can allocate both coherent and non-coherent memory.
>>>> Trying to use dma_map doesn't work because a) a device for coherency isn't
>>>> known at allocation time b) it kills performance. Part of the motivation
>>>> for taking this approach is to avoid the need to rework the existing
>>>> Android userspace and keep the existing behavior, as terrible as it
>>>> is. Having Ion out of staging and not actually usable isn't helpful.
>>>>
>>>> I'll give this all some more thought and hopefully have one or two more
>>>> proposals before Connect/Plumbers.
>>>>
>>>>>
>>>>> Will
>>>>>
>>>>
>>>> Thanks,
>>>> Laura
>>>> _______________________________________________
>>>> Linaro-mm-sig mailing list
>>>> Linaro-mm-sig(a)lists.linaro.org
>>>> https://lists.linaro.org/mailman/listinfo/linaro-mm-sig
>>>
>>>
>>> _______________________________________________
>>> linux-arm-kernel mailing list
>>> linux-arm-kernel(a)lists.infradead.org
>>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>>>
>>
>>
>> .
>>
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel(a)lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>
On Tue, Feb 21, 2017 at 4:08 PM, Christian König
<deathsimple(a)vodafone.de> wrote:
> Am 21.02.2017 um 15:55 schrieb Marek Szyprowski:
>>
>> Dear All,
>>
>> On 2017-02-21 15:37, Marek Szyprowski wrote:
>>>
>>> Hi Christian,
>>>
>>> On 2017-02-21 14:59, Christian König wrote:
>>>>
>>>> Am 21.02.2017 um 14:21 schrieb Marek Szyprowski:
>>>>>
>>>>> Add compat ioctl support to dma-buf. This lets one to use
>>>>> DMA_BUF_IOCTL_SYNC
>>>>> ioctl from 32bit application on 64bit kernel. Data structures for both
>>>>> 32
>>>>> and 64bit modes are same, so there is no need for additional
>>>>> translation
>>>>> layer.
>>>>
>>>>
>>>> Well I might be wrong, but IIRC compat_ioctl was just optional and if
>>>> not specified unlocked_ioctl was called instead.
>>>>
>>>> If that is true your patch wouldn't have any effect at all.
>>>
>>>
>>> Well, then why I got -ENOTTY in the 32bit test app for this ioctl on
>>> 64bit ARM64 kernel without this patch?
>>>
>>
>> I've checked in fs/compat_ioctl.c, I see no fallback in
>> COMPAT_SYSCALL_DEFINE3,
>> so one has to provide compat_ioctl callback to have ioctl working with
>> 32bit
>> apps.
>
>
> Then my memory cheated on me.
>
> In this case the patch is Reviewed-by: Christian König
> <christian.koenig(a)amd.com>.
Since you have commit rights for drm-misc, care to push this to
drm-misc-next-fixes pls? Also I think this warrants a cc: stable,
clearly an obvious screw-up in creating this api on our side :( So
feel free to smash my ack on the patch.
Thanks, Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
On 02/20/2017 10:05 PM, Chen Feng wrote:
> Hi Laura,
>
> When we enable kernel v4.4 or newer version on our platform, we meet the issue
> of flushing cache without reference device. It seems that this patch set is
> a solution. I'm curious the progress of the discussion. Do you have any plan
> to fix it in v4.4 and newer kernel verison?
>
No, I've abandoned this approach based on feedback. The APIs had too much
potential for incorrect usage. I'm ripping out the implicit caching in Ion
and switching it to a model where there should always be a device available.
What's your use case where you don't have a device structure?
Thanks,
Laura
> On 2016/9/14 2:41, Laura Abbott wrote:
>> On 09/13/2016 08:14 AM, Will Deacon wrote:
>>> On Tue, Sep 13, 2016 at 08:02:20AM -0700, Laura Abbott wrote:
>>>> On 09/13/2016 02:19 AM, Will Deacon wrote:
>>>>> On Mon, Sep 12, 2016 at 02:32:56PM -0700, Laura Abbott wrote:
>>>>>>
>>>>>> arm64 may need to guarantee the caches are synced. Implement versions of
>>>>>> the kernel_force_cache API to allow this.
>>>>>>
>>>>>> Signed-off-by: Laura Abbott <labbott(a)redhat.com>
>>>>>> ---
>>>>>> v3: Switch to calling cache operations directly instead of relying on
>>>>>> DMA mapping.
>>>>>> ---
>>>>>> arch/arm64/include/asm/cacheflush.h | 8 ++++++++
>>>>>> arch/arm64/mm/cache.S | 24 ++++++++++++++++++++----
>>>>>> arch/arm64/mm/flush.c | 11 +++++++++++
>>>>>> 3 files changed, 39 insertions(+), 4 deletions(-)
>>>>>
>>>>> I'm really hesitant to expose these cache routines as an API solely to
>>>>> support a driver sitting in staging/. I appreciate that there's a chicken
>>>>> and egg problem here, but we *really* don't want people using these routines
>>>>> in preference to the DMA API, and I fear that we'll simply grow a bunch
>>>>> more users of these things if we promote it as an API like you're proposing.
>>>>>
>>>>> Can the code not be contained under staging/, as part of ion?
>>>>>
>>>>
>>>> I proposed that in V1 and it was suggested I make it a proper API
>>>>
>>>> http://www.mail-archive.com/driverdev-devel@linuxdriverproject.org/msg47654…
>>>> http://www.mail-archive.com/driverdev-devel@linuxdriverproject.org/msg47672…
>>>
>>> :/ then I guess we're in disagreement. If ion really needs this stuff
>>> (which I don't fully grok), perhaps we should be exposing something at
>>> a higher level from the architecture, so it really can't be used for
>>> anything other than ion.
>>
>> I talked/complained about this at a past plumbers. The gist is that Ion
>> ends up acting as a fake DMA layer for clients. It doesn't match nicely
>> because clients can allocate both coherent and non-coherent memory.
>> Trying to use dma_map doesn't work because a) a device for coherency isn't
>> known at allocation time b) it kills performance. Part of the motivation
>> for taking this approach is to avoid the need to rework the existing
>> Android userspace and keep the existing behavior, as terrible as it
>> is. Having Ion out of staging and not actually usable isn't helpful.
>>
>> I'll give this all some more thought and hopefully have one or two more
>> proposals before Connect/Plumbers.
>>
>>>
>>> Will
>>>
>>
>> Thanks,
>> Laura
>> _______________________________________________
>> Linaro-mm-sig mailing list
>> Linaro-mm-sig(a)lists.linaro.org
>> https://lists.linaro.org/mailman/listinfo/linaro-mm-sig
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel(a)lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>
Hi Christian,
On 2017-02-21 14:59, Christian König wrote:
> Am 21.02.2017 um 14:21 schrieb Marek Szyprowski:
>> Add compat ioctl support to dma-buf. This lets one to use
>> DMA_BUF_IOCTL_SYNC
>> ioctl from 32bit application on 64bit kernel. Data structures for
>> both 32
>> and 64bit modes are same, so there is no need for additional translation
>> layer.
>
> Well I might be wrong, but IIRC compat_ioctl was just optional and if
> not specified unlocked_ioctl was called instead.
>
> If that is true your patch wouldn't have any effect at all.
Well, then why I got -ENOTTY in the 32bit test app for this ioctl on
64bit ARM64 kernel without this patch?
Best regards
--
Marek Szyprowski, PhD
Samsung R&D Institute Poland
Add compat ioctl support to dma-buf. This lets one to use DMA_BUF_IOCTL_SYNC
ioctl from 32bit application on 64bit kernel. Data structures for both 32
and 64bit modes are same, so there is no need for additional translation
layer.
Signed-off-by: Marek Szyprowski <m.szyprowski(a)samsung.com>
---
drivers/dma-buf/dma-buf.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
index 718f832a5c71..0007b792827b 100644
--- a/drivers/dma-buf/dma-buf.c
+++ b/drivers/dma-buf/dma-buf.c
@@ -325,6 +325,9 @@ static long dma_buf_ioctl(struct file *file,
.llseek = dma_buf_llseek,
.poll = dma_buf_poll,
.unlocked_ioctl = dma_buf_ioctl,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = dma_buf_ioctl,
+#endif
};
/*
--
1.9.1
Rendering operations to the dma-buf are tracked implicitly via the
reservation_object (dmabuf->resv). This is used to allow poll() to
wait upon outstanding rendering (or just query the current status of
rendering). The dma-buf sync ioctl allows userspace to prepare the
dma-buf for CPU access, which should include waiting upon rendering.
(Some drivers may need to do more work to ensure that the dma-buf mmap
is coherent as well as complete.)
Signed-off-by: Chris Wilson <chris(a)chris-wilson.co.uk>
Cc: Sumit Semwal <sumit.semwal(a)linaro.org>
Cc: Daniel Vetter <daniel.vetter(a)ffwll.ch>
Cc: linux-media(a)vger.kernel.org
Cc: dri-devel(a)lists.freedesktop.org
Cc: linaro-mm-sig(a)lists.linaro.org
Cc: linux-kernel(a)vger.kernel.org
---
I'm wondering whether it makes sense just to always do the wait first.
It is one of the first operations every driver has to make. A driver
that wants to implement it differently (e.g. they can special case
native waits) will still require a wait on the reservation object to
finish external rendering.
-Chris
---
drivers/dma-buf/dma-buf.c | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
index ddaee60ae52a..123f14b8e882 100644
--- a/drivers/dma-buf/dma-buf.c
+++ b/drivers/dma-buf/dma-buf.c
@@ -586,6 +586,22 @@ void dma_buf_unmap_attachment(struct dma_buf_attachment *attach,
}
EXPORT_SYMBOL_GPL(dma_buf_unmap_attachment);
+static int __dma_buf_begin_cpu_access(struct dma_buf *dmabuf,
+ enum dma_data_direction direction)
+{
+ bool write = (direction == DMA_BIDIRECTIONAL ||
+ direction == DMA_TO_DEVICE);
+ struct reservation_object *resv = dma_buf->resv;
+ long ret;
+
+ /* Wait on any implicit rendering fences */
+ ret = reservation_object_wait_timeout_rcu(resv, write, true,
+ MAX_SCHEDULE_TIMEOUT);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
/**
* dma_buf_begin_cpu_access - Must be called before accessing a dma_buf from the
@@ -607,6 +623,8 @@ int dma_buf_begin_cpu_access(struct dma_buf *dmabuf,
if (dmabuf->ops->begin_cpu_access)
ret = dmabuf->ops->begin_cpu_access(dmabuf, direction);
+ else
+ ret = __dma_buf_begin_cpu_access(dmabuf, direction);
return ret;
}
--
2.8.1
Hi,
I've been (once again) looking at alternate caching models for Ion. Part of
this work is also to make Ion fit better in to the dma_buf model.
Ion is a bit unusual for dma_buf. Most drivers that support dma_buf have two
parts: exporting buffers that a driver allocates and importing buffers
allocated elsewhere for use by the driver. Ion is basically designed to export
only and not import buffers from other drivers (the need for import is also
on my TODO list) Even more unusual, there is no actual 'driver' to map into.
Ion currently does nothing except pass back the same sg_table each time without
calling dma_map.
The description of the .map_dma_buf function in dma_buf_ops
* @map_dma_buf: returns list of scatter pages allocated, increases usecount
* of the buffer. Requires atleast one attach to be called
* before. Returned sg list should already be mapped into
* _device_ address space. This call may sleep. May also return
* -EINTR. Should return -EINVAL if attach hasn't been called yet.
So Ion is definitely not doing this correctly. This ties back into correcting
the caching model. If we call dma_map_sg/dma_unmap_sg with begin_cpu_access,
this should be enough to allow the caches to always be properly synchronized
and means we can drop the various dma_sync calls floating around. This is going
to violate one of the big fat comments in ion_buffer_create
/*
* this will set up dma addresses for the sglist -- it is not
* technically correct as per the dma api -- a specific
* device isn't really taking ownership here. However, in practice on
* our systems the only dma_address space is physical addresses.
* Additionally, we can't afford the overhead of invalidating every
* allocation via dma_map_sg. The implicit contract here is that
* memory coming from the heaps is ready for dma, ie if it has a
* cached mapping that mapping has been invalidated
*/
for_each_sg(buffer->sg_table->sgl, sg, buffer->sg_table->nents, i) {
sg_dma_address(sg) = sg_phys(sg);
sg_dma_len(sg) = sg->length;
}
The overhead of invalidating is a valid concern. I'm hoping that the
architecture has either evolved such that this won't be a problem or we can
figure out some clever use of DMA_ATTR_SKIP_CPU_SYNC.
As part of this, I'm considering dropping the fault synchronization. If we have
explicit use begin_cpu_access and use of the dma_buf sync ioctls, I don't think
it should be necessary.
I have a 'pre-RFC' tree at https://pagure.io/kernel-ion/branch/ion_cache_proof_dec14
Yes, the patches are not bisectable and there is more to be done. These have
been compile tested only and haven't been hooked up to anything to actually run
(another actually big TODO). I'm mostly looking for feedback if this looks like
the right direction and if there are going to be major problems with this
approach. I don't actually anticipate this getting merged into
drivers/staging/android/ion but this is the easiest way to continue discussion.
Thanks,
Laura
Laura Abbott (4):
staging: android: ion: Some cleanup
staging: android: ion: Duplicate sg_table
staging: android: ion: Remove page faulting support
staging: android: ion: Call dma_map_sg for syncing and mapping
drivers/staging/android/ion/ion-ioctl.c | 6 -
drivers/staging/android/ion/ion.c | 251 ++++++++----------------
drivers/staging/android/ion/ion.h | 5 +-
drivers/staging/android/ion/ion_carveout_heap.c | 16 +-
drivers/staging/android/ion/ion_chunk_heap.c | 15 +-
drivers/staging/android/ion/ion_cma_heap.c | 5 +-
drivers/staging/android/ion/ion_page_pool.c | 3 -
drivers/staging/android/ion/ion_priv.h | 4 +-
drivers/staging/android/ion/ion_system_heap.c | 14 +-
9 files changed, 90 insertions(+), 229 deletions(-)
--
2.7.4