On Wed, 19 Jul 2023 08:10:58 -0700 Mina Almasry almasrymina@google.com wrote:
On Tue, Jul 18, 2023 at 3:45 PM Jakub Kicinski kuba@kernel.org wrote:
On Tue, 18 Jul 2023 16:35:17 -0600 David Ahern wrote:
I do not see how 1 RSS context (or more specifically a h/w Rx queue) can be used properly with memory from different processes (or dma-buf references).
Right, my experience with dma-bufs from GPUs are that they're allocated from the userspace and owned by the process that allocated the backing GPU memory and generated the dma-buf from it. I.e., we're limited to 1 dma-buf per RX queue. If we enable binding multiple dma-bufs to the same RX queue, we have a problem, because AFAIU the NIC can't decide which dma-buf to put the packet into (it hasn't parsed the packet's destination yet).
When the process dies, that memory needs to be flushed from the H/W queues. Queues with interlaced submissions make that more complicated.
When the process dies, do we really want to flush the memory from the hardware queues? The drivers I looked at don't seem to have a function to flush the rx queues alone, they usually do an entire driver reset to achieve that. Not sure if that's just convenience or there is some technical limitation there. Do we really want to trigger a driver reset at the event a userspace process crashes?
Naive idea. Would it be possible for process to use mmap() on the GPU memory and then do zero copy TCP receive some how? Or is this what is being proposed.