On Tue, Dec 22, 2020 at 02:31:52PM -0500, Andrea Arcangeli wrote:
My previous suggestion to use a mutex to serialize userfaultfd_writeprotect with a mutex will still work, but we can run as many wrprotect and un-wrprotect as we want in parallel, as long as they're not simultaneous, we can do much better than a mutex.
Ideally we would need a new two_group_semaphore, where each group can run as many parallel instances as it wants, but no instance of one group can run in parallel with any instance of the other group. AFIK such a kind of lock doesn't exist right now.
Kent and I worked on one for a bit, and we called it a red-black mutex. If team red had the lock, more members of team red could join in. If team black had the lock, more members of team black could join in. I forget what our rule was around fairness (if team red has the lock, and somebody from team black is waiting, can another member of team red take the lock, or must they block?)
It was to solve the direct-IO vs buffered-IO problem (you can have as many direct-IO readers/writers at once or you can have as many buffered-IO readers/writers at once, but exclude a mix of direct and buffered I/O). In the end, we decided it didn't work all that well.