On 12/20/19 1:21 AM, Jan Kara wrote: ...
So, ideas and next steps:
- Assuming that you *are* hitting this, I think I may have to fall back to
implementing the "deferred" part of this design, as part of this series, after all. That means:
For the pin/unpin calls at least, stop treating all pages as if they are a cluster of PAGE_SIZE pages; instead, retrieve a huge page as one page. That's not how it works now, and the need to hand back a huge array of subpages is part of the problem. This affects the callers too, so it's not a super quick change to make. (I was really hoping not to have to do this yet.)
Does that mean that you would need to make all GUP users huge page aware? Otherwise I don't see how what you suggest would work... And I don't think making all GUP users huge page aware is realistic (effort-wise) or even wanted (maintenance overhead in all those places).
Well, pretty much yes. It's really just the pin_user_pages*() callers, but the internals, follow_page() and such, are so interconnected right now that it would probably blow up into a huge effort, as you point out.
I believe there might be also a different solution for this: For transparent huge pages, we could find a space in 'struct page' of the second page in the huge page for proper pin counter and just account pins there so we'd have full width of 32-bits for it.
Honza
OK, let me pursue that. Given that I shouldn't need to handle pages splitting, it should be not *too* bad.
I am starting to think that I should just post the first 9 or so prerequisite patches (first 9 patches, plus the v4l2 fix that arguably should have been earlier in the sequence I guess), as 5.6 candidates, while I go back to the drawing board here.
thanks,