Hi,
I am trying to export an ion_buffer allocate from kernel space to
multiple user-space clients. Eg: Allow multiple process to mmap
framebuffer allocated using ion by fb driver.
The following is the pseudo-code for that. Is this fine? there a
cleaner way to do it?
Or is it expected to share buffers across process only by user-space
sharing fds using sockets/binder and not directly in kernel.
fb driver init/probe: (init process context)
-------------------------------------------------
/* Create an ion client and allocate framebuffer */
init_client = ion_client_create(idev,...);
init_hdl = ion_alloc(init_client,...);
/* Create a global dma_buf instance for the buffer */
fd = ion_share_dma_buf(init_client, init_hdl);
// - Inc refcount of ion_buffer
// - Create a dma_buf and anon-file for the ion buffer
// - Get a free fd and install to anon file
g_dma_buf = dma_buf_get(fd);
// - Get the dma_buf pointer and inc refcount of anon_file
dma_buf_put(g_dma_buf);
// - Dec extra refcount of anon_file which happened in prev command
put_unused_fd(fd);
// - Free up the fd as fd is not exported to user-space here.
fb driver exit: (init process context)
------------------------------------------
/* Free the dma_buf reference */
dma_buf_put(g_dma_buf);
// - Dec refcount of anon_file. Free the dma_buf and dec refcount of
ion_buffer if anon_file refcount = 0
/* Free the framebuffer and destroy the ion client created for init process */
ion_free(init_client, init_hdl);
ion_client_destroy(init_client);
fb device open: (user process context)
-----------------------------------------------
/* Create an ion client for the user process */
p_client = ion_client_create(idev,...);
fb device ioctl to import ion handle for the fb: (user process context)
-----------------------------------------------------------------------------------
/* Import a ion_handle from the global dma_buf */
fd = dma_buf_fd(g_dmabuf, O_CLOEXEC);
// - Get ref to anon file
// - Get a free fd and install to anon file
p_hdl = ion_import_dma_buf(p_client, fd);
// - Inc refcount of ion_buffer
// - create a ion_handle for the buffer for this process/client
dma_buf_put(g_dmabuf);
// - Free the anon file reference taken in first step
put_unused_fd(fd);
// - Free up the fd as fd is not exported to user-space here.
fb device release: (user process context)
---------------------------------------------------
/* Destroy the client created */
ion_client_destroy(p_client);
- Nishanth Peethambaran
Hi,
I've been observing a high rate of failures with CMA allocations on my
ARM system. I've set up a test case set up with a 56MB CMA region that
essentially does the following:
total_failures = 0;
loop forever:
loop_failure = 0;
for (i = 0; i < 56; i++)
chunk[i] = dma_allocate(&cma_dev, 1MB)
if (!chunk[i])
loop_failure = 0
if (loop_failure)
total_failures++
loop_failure = 0
for (i = 0; i < 56; i++)
dma_free(&cma_dev, chunk[i], 1MB)
In the background, I also have a process doing some amount of filesystem
activity (adb push/pull since this is an android system). During the
course of my investigations I generally get ~8500 loops total and ~450
total failures (i.e. one or more buffers could not be allocated). This
is unacceptably high for our use cases.
In every case the allocation failure was ultimately due to a migration
failure; the pages contained buffers which could not be dropped because
the buffers were busy (move_to_new_page -> fallback_migrate_page ->
try_to_release_page -> try_to_free_buffers -> drop_buffers ->
buffer_busy). In every case, the b_count on the buffer head was always 1.
The problem arises because of the LRU lists for buffer heads:
__getblk
__getblk_slow
grow_buffers
grow_dev_page
find_or_create_page -- create a possibly movable page
__find_get_block
__find_get_block_slow
find_get_page -- return the movable page
bh_lru_install
get_bh -- buffer head now has a reference
The reference taken in bh_lru_install won't be dropped until the bh is
evicted from the lru. This means the page cannot be migrated as long as
the buffer exists on an LRU list. The real issue is that unless the
buffer gets evicted quickly the page can remain non-migratible for long
periods of time. This makes CMA regions unusable for long periods of
time given that we generally don't want to size CMA regions any larger
than necessary ergo any failure will cause a problem.
My quick and dirty workaround for testing is to remove the GFP_MOVABLE
flag from find_or_create_page but this seems significantly less than
optimal. Ideally, it seems like the buffers should be evicted from the
LRU when trying to drop (expand on invalid_bh_lru?) but I'm not familiar
enough with the code path to know if this is a good approach.
Any suggestions/feedback is appreciated. Thanks.
Laura
--
Sent by an employee of the Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum.
Hi,
I see that the lowmemkiller.c is changed to use oom_score_adj instead
of oom_adj.
Does this mean I cannot use an ICS system image with 3.4 kernel?
Or is there a workaround?
- Nishanth Peethambaran
looping-in the linaro-mm-sig ML.
On Thu, Aug 30, 2012 at 4:47 PM, Aubertin, Guillaume <g-aubertin(a)ti.com>wrote:
> hi guys,
>
> I've been working for a few days on getting a proper rmmod with the
> remoteproc/rpmsg modules, and I stumbled upon an interesting issue.
>
> when doing sucessive memory allocation and release in the CMA
> reservation (by loading/unloading the firmware several times), the
> following message shows up :
>
> [ 119.908477] cma: dma_alloc_from_contiguous(cma ed10ad00, count 256,
> align 8)
> [ 119.908843] cma: dma_alloc_from_contiguous(): memory range at c0dfb000
> is busy, retrying
> [ 119.909698] cma: dma_alloc_from_contiguous(): returned c0dfd000
>
> dma_alloc_from_contiguous() tries to allocate the following range,
> 0xc0dfd000, succesfully this time.
>
> In some cases, the allocation fails after trying several ranges :
>
> [ 119.912231] cma: dma_alloc_from_contiguous(cma ed10ad00, count 768,
> align 8)
> [ 119.912719] cma: dma_alloc_from_contiguous(): memory range at c0dff000
> is busy, retrying
> [ 119.913055] cma: dma_alloc_from_contiguous(): memory range at c0e01000
> is busy, retrying
> [ 119.913055] rproc remoteproc0: dma_alloc_coherent failed: 3145728
>
> Here is my understanding so far :
>
> First, even if we made a CMA reservation, the kernel can still allocate
> pages in this area, but these pages must be movable (user process page by
> example).
>
> When dma_alloc_from_contiguous() is called to allocate X pages, it looks
> for the next X contiguous free pages in it's CMA bitmap (with respect to
> the memory alignment). Then, alloc_contig_range() is called to allocate the
> given range of pages. Alloc_contig_range() analyses the pages we want to
> allocate, and if a page is already used, it is migrated to a new page
> outside the page array we want to reserve. this is done using
> isolate_migratepages_range() to list the pages to migrate, and
> migrate_pages() to try to migrate the pages, and that's where it fails.
> Below is a list of next function calls :
>
> fallback_migrate_page() --> migrate_page() --> try_to_release_page()
> --> try_to_free_buffer() --> drop_buffers() --> buffer_busy()
>
> I understand here that the page contains used buffers that can't be
> dropped, and so the page can't be migrated. Well, I must admit that once
> here, I'm feeling a little lost in this ocean of memory management code ;).
> After a few researches, I found the following thread on the
> linux-arm-kernel ML talking about the same issue :
>
> http://lists.infradead.org/pipermail/linux-arm-kernel/2012-June/102844.html with
> the following patch :
>
> * mm/page_alloc.c | 3 ++-*
> * 1 files changed, 2 insertions(+), 1 deletions(-)*
> *
> *
> *diff --git a/mm/page_alloc.c b/mm/page_alloc.c*
> *index 0e1c6f5..c9a6483 100644*
> *--- a/mm/page_alloc.c*
> *+++ b/mm/page_alloc.c*
> *@@ -1310,7 +1310,8 @@ void free_hot_cold_page(struct page *page, int
> cold)*
> * * excessively into the page allocator*
> * */*
> * if (migratetype >= MIGRATE_PCPTYPES) {*
> *- if (unlikely(migratetype == MIGRATE_ISOLATE)) {*
> *+ if (unlikely(migratetype == MIGRATE_ISOLATE)*
> *+ || is_migrate_cma(migratetype)) {*
> * free_one_page(zone, page, 0, migratetype);*
> * goto out;*
> * }*
>
> I tried the patch, and it seems to work (I didn't have any "memory range
> busy" in 5000+ tests), but I'm affraid that this could have some nasty side
> effects.
>
> Any idea ?
>
> Thanks in advance,
> Guillaume
>
>
> --
> Texas Instruments France SA, 821 Avenue Jack Kilby, 06270 Villeneuve
> Loubet. 036 420 040 R.C.S Antibes. Capital de EUR 753.920
>
--
Texas Instruments France SA, 821 Avenue Jack Kilby, 06270 Villeneuve
Loubet. 036 420 040 R.C.S Antibes. Capital de EUR 753.920
Hi All,
Over the last few months I've been working on & off with a few people from
Linaro on a new EGL extension. The extension allows constructing an EGLImage
from a (set of) dma_buf file descriptors, including support for multi-plane
YUV. I envisage the primary use-case of this extension to be importing video
frames from v4l2 into the EGL/GLES graphics driver to texture from.
Originally the intent was to develop this as a Khronos-ratified extension.
However, this is a little too platform-specific to be an officially
sanctioned Khronos extension. It also goes against the general "EGLStream"
direction the EGL working group is going in. As such, the general feeling
was to make this an EXT "multi-vendor" extension with no official stamp of
approval from Khronos. As this is no-longer intended to be a Khronos
extension, I've re-written it to be a lot more Linux & dma_buf specific. It
also allows me to circulate the extension more widely (I.e. To those outside
Khronos membership).
ARM are implementing this extension for at least our Mali-T6xx driver and
likely earlier drivers too. I am sending this e-mail to solicit feedback,
both from other vendors who might implement this extension (Mesa3D?) and
from potential users of the extension. However, any feedback is welcome.
Please find the extension text as it currently stands below. There several
open issues which I've proposed solutions for, but I'm not really happy with
those proposals and hoped others could chip-in with better ideas. There are
likely other issues I've not thought about which also need to be added and
addressed.
Once there's a general consensus or if no-one's interested, I'll update the
spec, move it out of Draft status and get it added to the Khronos registry,
which includes assigning values for the new symbols.
Cheers,
Tom
---------8<---------
Name
EXT_image_dma_buf_import
Name Strings
EGL_EXT_image_dma_buf_import
Contributors
Jesse Barker
Rob Clark
Tom Cooksey
Contacts
Jesse Barker (jesse 'dot' barker 'at' linaro 'dot' org)
Tom Cooksey (tom 'dot' cooksey 'at' arm 'dot' com)
Status
DRAFT
Version
Version 3, August 16, 2012
Number
EGL Extension ???
Dependencies
EGL 1.2 is required.
EGL_KHR_image_base is required.
The EGL implementation must be running on a Linux kernel supporting the
dma_buf buffer sharing mechanism.
This extension is written against the wording of the EGL 1.2
Specification.
Overview
This extension allows creating an EGLImage from a Linux dma_buf file
descriptor or multiple file descriptors in the case of multi-plane YUV
images.
New Types
None
New Procedures and Functions
None
New Tokens
Accepted by the <target> parameter of eglCreateImageKHR:
EGL_LINUX_DMA_BUF_EXT
Accepted as an attribute in the <attrib_list> parameter of
eglCreateImageKHR:
EGL_LINUX_DRM_FOURCC_EXT
EGL_DMA_BUF_PLANE0_FD_EXT
EGL_DMA_BUF_PLANE0_OFFSET_EXT
EGL_DMA_BUF_PLANE0_PITCH_EXT
EGL_DMA_BUF_PLANE1_FD_EXT
EGL_DMA_BUF_PLANE1_OFFSET_EXT
EGL_DMA_BUF_PLANE1_PITCH_EXT
EGL_DMA_BUF_PLANE2_FD_EXT
EGL_DMA_BUF_PLANE2_OFFSET_EXT
EGL_DMA_BUF_PLANE2_PITCH_EXT
Additions to Chapter 2 of the EGL 1.2 Specification (EGL Operation)
Add to section 2.5.1 "EGLImage Specification" (as defined by the
EGL_KHR_image_base specification), in the description of
eglCreateImageKHR:
"Values accepted for <target> are listed in Table aaa, below.
+-------------------------+--------------------------------------------+
| <target> | Notes
|
+-------------------------+--------------------------------------------+
| EGL_LINUX_DMA_BUF_EXT | Used for EGLImages imported from Linux
|
| | dma_buf file descriptors
|
+-------------------------+--------------------------------------------+
Table aaa. Legal values for eglCreateImageKHR <target> parameter
...
If <target> is EGL_LINUX_DMA_BUF_EXT, <dpy> must be a valid display,
<ctx>
must be EGL_NO_CONTEXT, and <buffer> must be NULL, cast into the type
EGLClientBuffer. The details of the image is specified by the attributes
passed into eglCreateImageKHR. Required attributes and their values are
as
follows:
* EGL_WIDTH & EGL_HEIGHT: The logical dimensions of the buffer in
pixels
* EGL_LINUX_DRM_FOURCC_EXT: The pixel format of the buffer, as
specified
by drm_fourcc.h and used as the pixel_format parameter of the
drm_mode_fb_cmd2 ioctl.
* EGL_DMA_BUF_PLANE0_FD_EXT: The dma_buf file descriptor of plane 0
of
the image.
* EGL_DMA_BUF_PLANE0_OFFSET_EXT: The offset from the start of the
dma_buf of the first sample in plane 0, in bytes.
* EGL_DMA_BUF_PLANE0_PITCH_EXT: The number of bytes between the
start of
subsequent rows of samples in plane 0. May have special meaning
for
non-linear formats.
For images in an RGB color-space or those using a single-plane YUV
format,
only the first plane's file descriptor, offset & pitch should be
specified.
For semi-planar YUV formats, the chroma samples are stored in plane 1
and
for fully planar formats, U-samples are stored in plane 1 and V-samples
are
stored in plane 2. Planes 1 & 2 are specified by the following
attributes,
which have the same meanings as defined above for plane 0:
* EGL_DMA_BUF_PLANE1_FD_EXT
* EGL_DMA_BUF_PLANE1_OFFSET_EXT
* EGL_DMA_BUF_PLANE1_PITCH_EXT
* EGL_DMA_BUF_PLANE2_FD_EXT
* EGL_DMA_BUF_PLANE2_OFFSET_EXT
* EGL_DMA_BUF_PLANE2_PITCH_EXT
If eglCreateImageKHR is successful for a EGL_LINUX_DMA_BUF_EXT target,
the
EGL takes ownership of the file descriptor and is responsible for
closing
it, which it may do at any time while the EGLDisplay is initialized."
Add to the list of error conditions for eglCreateImageKHR:
"* If <target> is EGL_LINUX_DMA_BUF_EXT and <buffer> is not NULL, the
error EGL_BAD_PARAMETER is generated.
* If <target> is EGL_LINUX_DMA_BUF_EXT, and the list of attributes is
incomplete, EGL_BAD_PARAMETER is generated.
* If <target> is EGL_LINUX_DMA_BUF_EXT, and the
EGL_LINUX_DRM_FOURCC_EXT
attribute is set to a format not supported by the EGL,
EGL_BAD_MATCH
is generated.
* If <target> is EGL_LINUX_DMA_BUF_EXT, and the
EGL_LINUX_DRM_FOURCC_EXT
attribute indicates a single-plane format, EGL_BAD_ATTRIBUTE is
generated if any of the EGL_DMA_BUF_PLANE1_* or
EGL_DMA_BUF_PLANE2_*
attributes are specified.
Issues
1. Should this be a KHR or EXT extension?
ANSWER: EXT. Khronos EGL working group not keen on this extension as it
is
seen as contradicting the EGLStream direction the specification is going
in.
The working group recommends creating additional specs to allow an
EGLStream
producer/consumer connected to v4l2/DRM or any other Linux interface.
2. Should this be a generic any platform extension, or a Linux-only
extension which explicitly states the handles are dma_buf fds?
ANSWER: There's currently no intention to port this extension to any OS
not
based on the Linux kernel. Consequently, this spec can be explicitly
written
against Linux and the dma_buf API.
3. Does ownership of the file descriptor pass to the EGL library?
PROPOSAL: If eglCreateImageKHR is successful, EGL assumes ownership of
the
file descriptors and is responsible for closing them.
4. How are the different YUV color spaces handled (BT.709/BT.601)?
Open issue, still TBD. Doesn't seem to be specified by either the v4l2
or
DRM APIs. PROPOSAL: Undefined and implementation/format dependent.
5. What chroma-siting is used for sub-sampled YUV formats?
Open issue, still TBD. Doesn't seem to be specified by either the v4l2
or
DRM APIs. PROPOSAL: Undefined and implementation/format dependent.
5. How can an application query which formats the EGL implementation
supports?
PROPOSAL: Don't provide a query mechanism but instead add an error
condition
that EGL_BAD_MATCH is raised if the EGL implementation doesn't support
that
particular format.
5. Which image formats should be supported and how is format specified?
Open issue, still TBD. Seem to be two options 1) specify a new enum in
this
specification and enumerate all possible formats. 2) Use an existing
enum
already in Linux, either v4l2_mbus_pixelcode and/or those formats listed
in drm_fourcc.h?
PROPOSAL: Go for option 2) and just use values defined in drm_fourcc.h.
Revision History
#3 (Tom Cooksey, August 16, 2012)
- Changed name from EGL_EXT_image_external and re-written language to
explicitly state this for use with Linux & dma_buf.
- Added a list of issues, including some still open ones.
#2 (Jesse Barker, May 30, 2012)
- Revision to split eglCreateImageKHR functionality from export
Functionality.
- Update definition of EGLNativeBufferType to be a struct containing a
list
of handles to support multi-buffer/multi-planar formats.
#1 (Jesse Barker, March 20, 2012)
- Initial draft.
Hi!
Aaro Koskinen and Josh Coombs reported that commit e9da6e9905e639 ("ARM:
dma-mapping: remove custom consistent dma region") introduced a
regresion. It turned out that the default 256KiB for atomic coherent
pool might not be enough. After that patch, some Kirkwood systems run
out of atomic coherent memory and fail without any meanfull message.
This patch series is an attempt to fix those issues by adding function
for setting coherent pool size from platform initialization code and
increasing the size of the pool for Kirkwood systems.
Best regards
Marek Szyprowski
Samsung Poland R&D Center
Patch summary:
Marek Szyprowski (3):
ARM: DMA-Mapping: add function for setting coherent pool size from
platform code
ARM: DMA-Mapping: print warning when atomic coherent allocation fails
ARM: Kirkwood: increase atomic coherent pool size
arch/arm/include/asm/dma-mapping.h | 7 +++++++
arch/arm/mach-kirkwood/common.c | 7 +++++++
arch/arm/mm/dma-mapping.c | 22 +++++++++++++++++++++-
3 files changed, 35 insertions(+), 1 deletions(-)
--
1.7.1.569.g6f426
Hi, All
We met question about dmac_map_area & dmac_flush_range from user addr.
mcr would not return on armv7 processor.
Existing ion carveout heap does not support partial cache flush.
Total cache will be flushed at all.
There is only one dirty bit for carveout heap, as well as sg_table->nents.
drivers/gpu/ion/ion_carveout_heap.c
ion_carveout_heap_map_dma -> sg_alloc_table(table, 1, GFP_KERNEL);
ion_buffer_alloc_dirty -> pages = buffer->sg_table->nents;
We want to support partial cache flush.
Align to cache line, instead of PAGE_SIZE, for efficiency consideration.
We have considered extended dirty bit, but looks like only align to PAGE_SIZE.
For experiment we modify ioctl ION_IOC_SYNC on armv7.
And directly use dmac_map_area & dmac_flush_range with add from user space.
However, we find dmac_map_area can not work with this addr from user space.
In fact, it is mcr can not work with addr from user space, it would hung.
Also, ion_vm_falut would happen twice.
The first time is from __dabt_usr, when we access the mmaped buffer, it is fine.
The second is from __davt_svc, it is caused by mcr, it is strange?
ION malloc carveout heap
addr = user mmap
user access addr, ion_vm_fault (__dabt_usr), build page table, and
vm_insert_page.
dmac_map_area & dmac_flush_range with addr -> ion_vm_fault (__davt_svc)
mcr hung.
Not understand why ion_vm_fault happen twice, where page table has been build.
Why mcr will hung with addr from user space.
Besides, no problem with ION on 3.0, which do not use ion_vm_fault.
Any suggestion?
Thanks