On Tue, Sep 08, 2026 at 09:15:45PM +0800, Li Wang wrote:

From: Mengmeng Zhao zhaomengmeng@kylinos.cn

Inspired by the paper published in SC'25 [1], we implemented a character device named fgds that provides two ioctl interfaces: `REG_BUFFER/UNREG_BUFFER`. It enables applications to perform direct I/O between GPU memory and NVMe via POSIX and io_uring APIs. This is particularly useful for LLM workloads, such as model loading, KV cache offloading, and checkpointing. The fgds device corresponds one-to-one with the PCIe GPU on the machine. The usage is straightforward: an application simply opens the corresponding fgds device, calls ioctl on the returned fd with REG_BUFFER, taking the target GPU memory buffer address (represented as a dma-buf fd), and the buffer length as inputs, and then invokes mmap on the fgds device fd, using the return value of ioctl as the input. The mmap call returns a CPU virtual address (call it cpu_vaddr). Afterward, cpu_vaddr can be passed directly to pread/pwrite, or io_uring_prep_read/io_uring_prep_write to perform direct I/O between files on NVMe and GPU memory. A minimal working example can be found in [2]. The underlying mechanism is that, with the support of fgds device, cpu_vaddr is made to point directly to the GPU memory buffer corresponding to the dma-buf fd. This solution is loosely coupled with the GPU vendor's driver; the GPU vendor only needs to support exporting the allocated GPU memory buffer through the standard Linux kernel dma-buf framework, which the vast majority of mainstream GPUs already support. This allows both applications and the fgds device to work seamlessly with GPUs from different vendors without any modifications. Furthermore, applications no longer need to call vendor-specific proprietary APIs (such as NVIDIA's cuFile API) or install vendor-specific kernel modules (such as NVIDIA's nvidia-fs.ko) for different GPU vendors. We have tested fgds on GPU cards from NVIDIA, AMD, and several other vendors, and it works well.

Besides the benefits in ease of use and compatibility, another key advantage of this solution is higher performance. [2] presents the performance comparison results between fgds and NVIDIA GDS. Because fgds eliminates the overhead of phony buffers incurred by NVIDIA GDS, it achieves significantly higher performance. For example, for reads, fgds outperforms GDS by 11% to 109%; for writes, fgds outperforms GDS by 10% to 71%.

To further accelerate the read and write operations of large files or massive data volumes—which are very common in LLM scenarios—we have implemented library functions `fgds_read` and `fgds_write`. Under the hood, these interfaces split large data into chunks and submit them asynchronously and in parallel via io_uring, thereby further boosting I/O performance, with read performance improved by up to 115% and write performance by up to 40%. In addition, we also provide the `fgds_register` library interface to encapsulate the `open`, `ioctl' and `mmap` operations. Readers who are interested can refer to [2].

In addition, we have added the LMCache backend, enabling vLLM to offload KV cache via LMCache using fgds, which accelerates inference performance. We also added PyTorch APIs, compatible with the PyTorch GDS API, to improve the performance of reading and writing checkpoints during LLM training.

We look forward to community feedback and are fully committed to iterating on this series to work towards upstreaming.

That's not really needed in a changelog text, it could be in the 0/X patch :)

Anyway, you didn't cc: the io_uring list, why?

Also, as a first cut, please see the sashiko comments on this patch: https://sashiko.dev/#/patchset/20260908131545.105987-1-liwang@kylinos.cn

[1] https://dl.acm.org/doi/10.1145/3712285.3759862 [2] https://github.com/Storage-and-OS-for-AI/fgds

Signed-off-by: Mengmeng Zhao zhaomengmeng@kylinos.cn Signed-off-by: Li Wang liwang@kylinos.cn

---

drivers/misc/Kconfig | 9 + drivers/misc/Makefile | 1 + drivers/misc/fgds.c | 989 ++++++++++++++++++++++++++++++++++++++ include/uapi/linux/fgds.h | 54 +++ 4 files changed, 1053 insertions(+) create mode 100644 drivers/misc/fgds.c create mode 100644 include/uapi/linux/fgds.h

diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig index 7364931dad3a..2f3a5a5fd0bf 100644 --- a/drivers/misc/Kconfig +++ b/drivers/misc/Kconfig @@ -568,6 +568,15 @@ config MCHP_LAN966X_PCI - lan966x-miim (MDIO_MSCC_MIIM) - lan966x-switch (LAN966X_SWITCH) +config FGDS

• tristate "GPU-NVMe direct I/O control driver"
• depends on PCI && DMA_SHARED_BUFFER && ZONE_DEVICE
• help

•  Say Y here if you want to support GPU-NVME direct I/O
  

•  via POSIX/io_uring interfaces.
  

•  If unsure, say N.
  

Module name is not listed here.

Nor why "fgds" is the name, that's going to be hard to remember, does it stand for something?

source "drivers/misc/c2port/Kconfig" source "drivers/misc/eeprom/Kconfig" source "drivers/misc/cb710/Kconfig" diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile index e8d8d5d88c0d..04985abe1678 100644 --- a/drivers/misc/Makefile +++ b/drivers/misc/Makefile @@ -71,3 +71,4 @@ obj-y += keba/ obj-y += amd-sbi/ obj-$(CONFIG_MISC_RP1) += rp1/ obj-$(CONFIG_INTEL_SSEI) += issei/ +obj-$(CONFIG_FGDS) += fgds.o diff --git a/drivers/misc/fgds.c b/drivers/misc/fgds.c new file mode 100644 index 000000000000..3aa4945f701b --- /dev/null +++ b/drivers/misc/fgds.c @@ -0,0 +1,989 @@ +// SPDX-License-Identifier: GPL-2.0 +/*

• • Fast GPU Direct Storage via dma-buf.
• • Copyright (C) 2026 KylinSoft. Co., Ltd. All rights reserved.
• • Maps GPU memory into user space to enable direct NVME-to-GPU DMA
• • pread/pwrite syscalls. BAR pages are remapped into ZONE_DEVICE via
• • devm_memremap_pages() and populated using dma-buf backing pages.
• */

+#define pr_fmt(fmt) "fgds: " fmt

You are a driver, always use dev_*() print functions, not pr_() functions, as you will loose the device information. For example:

+/*

• • BAR-based mapping requires device physical addresses. When using
• • IOMMU, DMA addresses are IOVAs, which cannot be mapped directly.
• */

+static int fgds_check_gpu_iommu(struct pci_dev *pdev) +{

• struct iommu_domain *domain;
• domain = iommu_get_domain_for_dev(&pdev->dev);
• if (domain && domain->type != IOMMU_DOMAIN_IDENTITY) {

• pr_warn("%s: reject attaching a translating IOMMU domain (requires iommu=pt or off\n",
  

• 	dev_name(&pdev->dev));
  

Should be dev_warn(), right?

But what can userspace do with that warning, did something just break?

• pr_info("loaded successfully: %u GPU(s) active\n", fgds_dev_count);

When drivers work, they are quiet, please remove this, and the other pr_info() lines, as they seem to be left over from your debugging.

thanks,

greg k-h