On Fri, Mar 13, 2020 at 09:17:42AM -0300, Jason Gunthorpe wrote:
On Fri, Mar 13, 2020 at 04:21:39AM -0700, Christoph Hellwig wrote:
On Thu, Mar 12, 2020 at 11:19:28AM -0300, Jason Gunthorpe wrote:
The non-page scatterlist is also a big concern for RDMA as we have drivers that want the page list, so even if we did as this series contemplates I'd have still have to split the drivers and create the notion of a dma-only SGL.
The drivers I looked at want a list of IOVA address, aligned to the device "page size". What other data do drivers want? Execept for the software protocol stack drivers, which of couse need pages for the stack futher down.
In principle it is possible to have just an aligned page list - however the page size is variable, following certain rules, and today the drivers still determine the correct page size largely on their own.
Some progress was made recently to consolidate this, but more is needed.
If the common code doesn't know the device page size in advance then today's approach of sending largest possible dma mapped SGLs into the device driver is best. The driver only has to do splitting.
The point was that drivers don't need pages, drivers need IOVAs. In what form they are stuffed into the hardware is the drivers problem.
Am 16.03.20 um 09:56 schrieb Christoph Hellwig:
On Fri, Mar 13, 2020 at 09:17:42AM -0300, Jason Gunthorpe wrote:
On Fri, Mar 13, 2020 at 04:21:39AM -0700, Christoph Hellwig wrote:
On Thu, Mar 12, 2020 at 11:19:28AM -0300, Jason Gunthorpe wrote:
The non-page scatterlist is also a big concern for RDMA as we have drivers that want the page list, so even if we did as this series contemplates I'd have still have to split the drivers and create the notion of a dma-only SGL.
The drivers I looked at want a list of IOVA address, aligned to the device "page size". What other data do drivers want? Execept for the software protocol stack drivers, which of couse need pages for the stack futher down.
In principle it is possible to have just an aligned page list - however the page size is variable, following certain rules, and today the drivers still determine the correct page size largely on their own.
Some progress was made recently to consolidate this, but more is needed.
If the common code doesn't know the device page size in advance then today's approach of sending largest possible dma mapped SGLs into the device driver is best. The driver only has to do splitting.
The point was that drivers don't need pages, drivers need IOVAs. In what form they are stuffed into the hardware is the drivers problem.
Well I would prefer if the drivers can somehow express their requirements and get IOVA structures already in the form they need.
Converting the IOVA data from one form to another is sometimes quite costly. Especially when it is only temporarily needed.
Regards, Christian.
On Mon, Mar 16, 2020 at 10:41:42AM +0100, Christian König wrote:
Well I would prefer if the drivers can somehow express their requirements and get IOVA structures already in the form they need.
Converting the IOVA data from one form to another is sometimes quite costly. Especially when it is only temporarily needed.
We basically have two ways to generate the IOVA:
- a linear translation for the direct mapping case or some dumb IOMMU drivers - in that case case there is a 1:1 mapping between input segments and output segments in DMA mapping - a non-trivial IOMMU where all aligned segments are merged into a single IOVA range
So I don't really see how the dma layer could help much with any limitation beyond existing max size and dma boundary ones.
linaro-mm-sig@lists.linaro.org
-
Christian König -
Christoph Hellwig