We observed several NVMe failures when running with SWIOTLB. The root cause of the issue is that when data is mapped via SWIOTLB, the address offset is not preserved. Several device drivers including the NVMe driver relies on this offset to function correctly.
Even though we discovered the error when running using AMD SEV, we have reproduced the same error in Rhel 8 without SEV. By adding swiotlb=force option to the boot command line parameter, NVMe funcionality is impacted. For example formatting a disk into xfs format returns an error.
---- Changes in v2: Rebased patches to 5.10.33 Updated patch description to correct format.
Jianxiong Gao (9): driver core: add a min_align_mask field to struct device_dma_parameters swiotlb: add a IO_TLB_SIZE define swiotlb: factor out an io_tlb_offset helper swiotlb: factor out a nr_slots helper swiotlb: clean up swiotlb_tbl_unmap_single swiotlb: refactor swiotlb_tbl_map_single swiotlb: don't modify orig_addr in swiotlb_tbl_sync_single swiotlb: respect min_align_mask nvme-pci: set min_align_mask
drivers/nvme/host/pci.c | 1 + include/linux/device.h | 1 + include/linux/dma-mapping.h | 16 +++ include/linux/swiotlb.h | 1 + kernel/dma/swiotlb.c | 259 ++++++++++++++++++++---------------- 5 files changed, 164 insertions(+), 114 deletions(-)