From: Bobby Eshleman bobbyeshleman@meta.com
Update devmem.rst documentation to describe the autorelease netlink attribute used during RX dmabuf binding.
The autorelease attribute is specified at bind-time via the netlink API (NETDEV_CMD_BIND_RX) and controls what happens to outstanding tokens when the socket closes.
Document the two token release modes (automatic vs manual), how to configure the binding for autorelease, the perf benefits, new caveats and restrictions, and the way the mode is enforced system-wide.
Signed-off-by: Bobby Eshleman bobbyeshleman@meta.com --- Changes in v7: - Document netlink instead of sockopt - Mention system-wide locked to one mode --- Documentation/networking/devmem.rst | 70 +++++++++++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+)
diff --git a/Documentation/networking/devmem.rst b/Documentation/networking/devmem.rst index a6cd7236bfbd..67c63bc5a7ae 100644 --- a/Documentation/networking/devmem.rst +++ b/Documentation/networking/devmem.rst @@ -235,6 +235,76 @@ can be less than the tokens provided by the user in case of: (a) an internal kernel leak bug. (b) the user passed more than 1024 frags.
+ +Autorelease Control +~~~~~~~~~~~~~~~~~~~ + +The autorelease mode controls what happens to outstanding tokens (tokens not +released via SO_DEVMEM_DONTNEED) when the socket closes. Autorelease is +configured per-binding at binding creation time via the netlink API:: + + struct netdev_bind_rx_req *req; + struct netdev_bind_rx_rsp *rsp; + struct ynl_sock *ys; + struct ynl_error yerr; + + ys = ynl_sock_create(&ynl_netdev_family, &yerr); + + req = netdev_bind_rx_req_alloc(); + netdev_bind_rx_req_set_ifindex(req, ifindex); + netdev_bind_rx_req_set_fd(req, dmabuf_fd); + netdev_bind_rx_req_set_autorelease(req, 0); /* 0 = manual, 1 = auto */ + __netdev_bind_rx_req_set_queues(req, queues, n_queues); + + rsp = netdev_bind_rx(ys, req); + + dmabuf_id = rsp->id; + +When autorelease is disabled (0): + +- Outstanding tokens are NOT released when the socket closes +- Outstanding tokens are only released when the dmabuf is unbound +- Provides better performance by eliminating xarray overhead (~13% CPU reduction) +- Kernel tracks tokens via atomic reference counters in net_iov structures + +When autorelease is enabled (1): + +- Outstanding tokens are automatically released when the socket closes +- Backwards compatible behavior +- Kernel tracks tokens in an xarray per socket + +The default is autorelease disabled. + +Important: In both modes, applications should call SO_DEVMEM_DONTNEED to +return tokens as soon as they are done processing. The autorelease setting only +affects what happens to tokens that are still outstanding when close() is called. + +The mode is enforced system-wide. Once a binding is created with a specific +autorelease mode, all subsequent bindings system-wide must use the same mode. + + +Performance Considerations +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Disabling autorelease provides approximately ~13% CPU utilization improvement +in RX workloads. That said, applications must ensure all tokens are released +via SO_DEVMEM_DONTNEED before closing the socket, otherwise the backing pages +will remain pinned until the dmabuf is unbound. + + +Caveats +~~~~~~~ + +- Once a system-wide autorelease mode is selected (via the first binding), + all subsequent bindings must use the same mode. Attempts to create bindings + with a different mode will be rejected with -EINVAL. + +- Applications using manual release mode (autorelease=0) must ensure all tokens + are returned via SO_DEVMEM_DONTNEED before socket close to avoid resource + leaks during the lifetime of the dmabuf binding. Tokens not released before + close() will only be freed when the dmabuf is unbound. + + TX Interface ============