Hi all,
I have one question on memory migration. As we know, malloc() from user app will allocate MIGRATE_MOVABLE pages. But if we want to use this memory as DMA usage, we can't accept MIGRATE_MOVABLE type. Could we change its behavior before DMA working?
Thanks Haojian
On Fri, 13 Apr 2012, Haojian Zhuang wrote:
I have one question on memory migration. As we know, malloc() from user app will allocate MIGRATE_MOVABLE pages. But if we want to use this memory as DMA usage, we can't accept MIGRATE_MOVABLE type. Could we change its behavior before DMA working?
MIGRATE_MOVABLE works fine for DMA. If you keep a reference from a device driver to user pages then you will have to increase the page refcount which will in turn pin the page and make it non movable for as long as you keep the refcount.
On Sat, Apr 14, 2012 at 2:27 AM, Christoph Lameter cl@linux.com wrote:
On Fri, 13 Apr 2012, Haojian Zhuang wrote:
I have one question on memory migration. As we know, malloc() from user app will allocate MIGRATE_MOVABLE pages. But if we want to use this memory as DMA usage, we can't accept MIGRATE_MOVABLE type. Could we change its behavior before DMA working?
MIGRATE_MOVABLE works fine for DMA. If you keep a reference from a device driver to user pages then you will have to increase the page refcount which will in turn pin the page and make it non movable for as long as you keep the refcount.
Hi Christoph,
Thanks for your illustration. But it's a little abstract. Could you give me a simple example or show me the code?
Best Regards Haojian
On Sat, 14 Apr 2012, Haojian Zhuang wrote:
On Sat, Apr 14, 2012 at 2:27 AM, Christoph Lameter cl@linux.com wrote:
On Fri, 13 Apr 2012, Haojian Zhuang wrote:
I have one question on memory migration. As we know, malloc() from user app will allocate MIGRATE_MOVABLE pages. But if we want to use this memory as DMA usage, we can't accept MIGRATE_MOVABLE type. Could we change its behavior before DMA working?
MIGRATE_MOVABLE works fine for DMA. If you keep a reference from a device driver to user pages then you will have to increase the page refcount which will in turn pin the page and make it non movable for as long as you keep the refcount.
Hi Christoph,
Thanks for your illustration. But it's a little abstract. Could you give me a simple example or show me the code?
Run get_user_pages() on the memory you are interest in pinning. See how other drivers do that by looking up other use cases. F.e. ib_umem_get() does a similar thing.
On Mon, Apr 16, 2012 at 9:55 PM, Christoph Lameter cl@linux.com wrote:
On Sat, 14 Apr 2012, Haojian Zhuang wrote:
On Sat, Apr 14, 2012 at 2:27 AM, Christoph Lameter cl@linux.com wrote:
On Fri, 13 Apr 2012, Haojian Zhuang wrote:
I have one question on memory migration. As we know, malloc() from user app will allocate MIGRATE_MOVABLE pages. But if we want to use this memory as DMA usage, we can't accept MIGRATE_MOVABLE type. Could we change its behavior before DMA working?
MIGRATE_MOVABLE works fine for DMA. If you keep a reference from a device driver to user pages then you will have to increase the page refcount which will in turn pin the page and make it non movable for as long as you keep the refcount.
Hi Christoph,
Thanks for your illustration. But it's a little abstract. Could you give me a simple example or show me the code?
Run get_user_pages() on the memory you are interest in pinning. See how other drivers do that by looking up other use cases. F.e. ib_umem_get() does a similar thing.
Got it. And I think there's conflict in CMA.
For example, user process A malloc() memory, page->_count is 1. After using get_user_pages() in device driver for DMA usage, page->_count becomes 2.
If the page is in CMA region, it results migrate_pages() returns -EAGAIN. But error handling in CMA is in below.
ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA); if (ret == 0) { bitmap_set(cma->bitmap, pageno, count); break; } else if (ret != -EBUSY) { goto error; }
Since EAGAIN doesn't equal to EBUSY, dma_alloc_from_contiguous() aborts. Should dma_alloc_from_contiguous() handle EAGAIN?
Best Regards Haojian
On Wed, 18 Apr 2012, Haojian Zhuang wrote:
Run get_user_pages() on the memory you are interest in pinning. See how other drivers do that by looking up other use cases. F.e. ib_umem_get() does a similar thing.
Got it. And I think there's conflict in CMA.
For example, user process A malloc() memory, page->_count is 1. After using get_user_pages() in device driver for DMA usage, page->_count becomes 2.
If the page is in CMA region, it results migrate_pages() returns -EAGAIN. But error handling in CMA is in below.
The increase of the page count should be temporary. That is why migrate_pages() uses -EAGAIN to signify a temporary inability to migrate the page.
Xen uses a page flag for pinned pages. IMHO that could be generalized and used instead of increasing the page count. Or it could be checked in addition and change the return value of migrate_pages().
ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA); if (ret == 0) { bitmap_set(cma->bitmap, pageno, count); break; } else if (ret != -EBUSY) { goto error; }Since EAGAIN doesn't equal to EBUSY, dma_alloc_from_contiguous() aborts. Should dma_alloc_from_contiguous() handle EAGAIN?
You need to talk to the CMA developers for this one. If there is a pinned page in that range then definitely alloc_contig_range needs to fail. In the case of EAGAIN (and correct marking of pinned pages elsewhere in the kernel) we could handle the EGAIN return value by trying again.
Hello,
On Wednesday, April 18, 2012 9:37 AM Haojian Zhuang wrote:
On Mon, Apr 16, 2012 at 9:55 PM, Christoph Lameter cl@linux.com wrote:
On Sat, 14 Apr 2012, Haojian Zhuang wrote:
On Sat, Apr 14, 2012 at 2:27 AM, Christoph Lameter cl@linux.com wrote:
On Fri, 13 Apr 2012, Haojian Zhuang wrote:
I have one question on memory migration. As we know, malloc() from user app will allocate MIGRATE_MOVABLE pages. But if we want to use this memory as DMA usage, we can't accept MIGRATE_MOVABLE type. Could we change its behavior before DMA working?
MIGRATE_MOVABLE works fine for DMA. If you keep a reference from a device driver to user pages then you will have to increase the page refcount which will in turn pin the page and make it non movable for as long as you keep the refcount.
Hi Christoph,
Thanks for your illustration. But it's a little abstract. Could you give me a simple example or show me the code?
Run get_user_pages() on the memory you are interest in pinning. See how other drivers do that by looking up other use cases. F.e. ib_umem_get() does a similar thing.
Got it. And I think there's conflict in CMA.
For example, user process A malloc() memory, page->_count is 1. After using get_user_pages() in device driver for DMA usage, page->_count becomes 2.
If the page is in CMA region, it results migrate_pages() returns -EAGAIN. But error handling in CMA is in below.
ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA); if (ret == 0) { bitmap_set(cma->bitmap, pageno, count); break; } else if (ret != -EBUSY) { goto error; }Since EAGAIN doesn't equal to EBUSY, dma_alloc_from_contiguous() aborts. Should dma_alloc_from_contiguous() handle EAGAIN?
Yes, it definitely should threat EAGAIN the same way as EBUSY. I think I've double checked that alloc_contig_range return only EBUSY in case of migration failure, but it looks that I need to check it once again. Thanks for spotting the possible bug.
Best regards
Hello,
On Wednesday, April 18, 2012 9:37 AM Haojian Zhuang wrote:
On Mon, Apr 16, 2012 at 9:55 PM, Christoph Lameter cl@linux.com wrote:
On Sat, 14 Apr 2012, Haojian Zhuang wrote:
On Sat, Apr 14, 2012 at 2:27 AM, Christoph Lameter cl@linux.com wrote:
On Fri, 13 Apr 2012, Haojian Zhuang wrote:
I have one question on memory migration. As we know, malloc() from user app will allocate MIGRATE_MOVABLE pages. But if we want to use this memory as DMA usage, we can't accept MIGRATE_MOVABLE type. Could we change its behavior before DMA working?
MIGRATE_MOVABLE works fine for DMA. If you keep a reference from a device driver to user pages then you will have to increase the page refcount which will in turn pin the page and make it non movable for as long as you keep the refcount.
Hi Christoph,
Thanks for your illustration. But it's a little abstract. Could you give me a simple example or show me the code?
Run get_user_pages() on the memory you are interest in pinning. See how other drivers do that by looking up other use cases. F.e. ib_umem_get() does a similar thing.
Got it. And I think there's conflict in CMA.
For example, user process A malloc() memory, page->_count is 1. After using get_user_pages() in device driver for DMA usage, page->_count becomes 2.
If the page is in CMA region, it results migrate_pages() returns -EAGAIN. But error handling in CMA is in below.
ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA); if (ret == 0) { bitmap_set(cma->bitmap, pageno, count); break; } else if (ret != -EBUSY) { goto error; }Since EAGAIN doesn't equal to EBUSY, dma_alloc_from_contiguous() aborts. Should dma_alloc_from_contiguous() handle EAGAIN?
I've checked again and it is really not possible for migrate_pages() to return -EAGAIN, so the CMA code is correct. The only special case which needs retry is -EBUSY.
Best regards
linaro-mm-sig@lists.linaro.org
-
Christoph Lameter -
Haojian Zhuang -
Marek Szyprowski