┌────────────┐ ┌───────────────────────────────────┐ ┌────────────────┐ │ │ │ │ │ │ │ │ │ PCI Endpoint │ │ PCI Host │ │ │ │ │ │ │ │ │◄──┤ 1.platform_msi_domain_alloc_irqs()│ │ │ │ │ │ │ │ │ │ MSI ├──►│ 2.write_msi_msg() ├──►├─BAR<n> │ │ Controller │ │ update doorbell register address│ │ │ │ │ │ for BAR │ │ │ │ │ │ │ │ 3. Write BAR<n>│ │ │◄──┼───────────────────────────────────┼───┤ │ │ │ │ │ │ │ │ ├──►│ 4.Irq Handle │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ └────────────┘ └───────────────────────────────────┘ └────────────────┘
This patches based on old https://lore.kernel.org/imx/20221124055036.1630573-1-Frank.Li@nxp.com/
Original patch only target to vntb driver. But actually it is common method.
This patches add new API to pci-epf-core, so any EP driver can use it.
Previous v2 discussion here. https://lore.kernel.org/imx/20230911220920.1817033-1-Frank.Li@nxp.com/
Changes in v20: - remove set epf of_node's patch and only support one epf now. - move imx6's patch to first - detail change see each patches' change log - Link to v19: https://lore.kernel.org/r/20250609-ep-msi-v19-0-77362eaa48fa@nxp.com
Changes in v19: - irq part already in v6.16-rc1, only missed pcie/dts part - rebase to v6.16-rc1 - update commit message for patch IMMUTABLE check. - Link to v18: https://lore.kernel.org/r/20250414-ep-msi-v18-0-f69b49917464@nxp.com
Changes in v18: - pci-ep.yaml: sort property order, fix maxvalue to 0x7ffff for msi-map-mask and iommu-map-mask - Link to v17: https://lore.kernel.org/r/20250407-ep-msi-v17-0-633ab45a31d0@nxp.com
Changes in v17: - move document part to pci-ep.yaml - Link to v16: https://lore.kernel.org/r/20250404-ep-msi-v16-0-d4919d68c0d0@nxp.com
Changes in v16: - remove arm64: dts: imx95-19x19-evk: Add PCIe1 endpoint function overlay file because there are better patches, which under review. - Add document for pcie-ep msi-map usage - other change to see each patch's change log About IMMUTABLE (No change for this part, tglx provide feedback)
- This IMMUTABLE thing serves no purpose, because you don't randomly plug this end-point block on any MSI controller. They come as part of an SoC.
"Yes and no. The problem is that the EP implementation is meant to be a generic library and while GIC-ITS guarantees immutability of the address/data pair after setup, there are architectures (x86, loongson, riscv) where the base MSI controller does not and immutability is only achieved when interrupt remapping is enabled. The latter can be disabled at boot-time and then the EP implementation becomes a lottery across affinity changes.
That was my concern about this library implementation and that's why I asked for a mechanism to ensure that the underlying irqdomain provides a immutable address/data pair.
So it does not matter for GIC-ITS, but in the larger picture it matters.
Thanks,
tglx "
So it does not matter for GIC-ITS, but in the larger picture it matters.
- Link to v15: https://lore.kernel.org/r/20250211-ep-msi-v15-0-bcacc1f2b1a9@nxp.com
Changes in v15: - rebase to v6.14-rc1 - fix build issue find by kernel test robot - Link to v14: https://lore.kernel.org/r/20250207-ep-msi-v14-0-9671b136f2b8@nxp.com
Changes in v14: Marc Zyngier raised concerns about adding DOMAIN_BUS_DEVICE_PCI_EP_MSI. As a result, the approach has been reverted to the v9 method. However, there are several improvements:
MSI now supports msi-map in addition to msi-parent. - The struct device: id is used as the endpoint function (EPF) device identity to map to the stream ID (sideband information). - The EPC device tree source (DTS) utilizes msi-map to provide such information. - The EPF device's of_node is set to the EPC controller’s node. This approach is commonly used for multi-function device (MFD) platform child devices, allowing them to inherit properties from the MFD device’s DTS, such as reset-cells and gpio-cells. This method is well-suited for the current case, as the EPF is inherently created/binded to the EPC and should inherit the EPC’s DTS node properties.
Additionally:
Since the basic IMX95 LUT support has already been merged into the mainline, a DTS and driver increment patch is added to complete the solution. The patch is rebased onto the latest linux-next tree and aligned with the new pcitest framework.
- Link to v13: https://lore.kernel.org/r/20241218-ep-msi-v13-0-646e2192dc24@nxp.com
Changes in v13: - Change to use DOMAIN_BUS_PCI_DEVICE_EP_MSI - Change request id as func | vfunc << 3 - Remove IRQ_DOMAIN_MSI_IMMUTABLE
Thomas Gleixner:
I hope capture all your points in review comments. If missed, let me know.
- Link to v12: https://lore.kernel.org/r/20241211-ep-msi-v12-0-33d4532fa520@nxp.com
Changes in v12: - Change to use IRQ_DOMAIN_MSI_IMMUTABLE and add help function irq_domain_msi_is_immuatble(). - split PCI: endpoint: pci-ep-msi: Add MSI address/data pair mutable check to 3 patches - Link to v11: https://lore.kernel.org/r/20241209-ep-msi-v11-0-7434fa8397bd@nxp.com
Changes in v11: - Change to use MSI_FLAG_MSG_IMMUTABLE - Link to v10: https://lore.kernel.org/r/20241204-ep-msi-v10-0-87c378dbcd6d@nxp.com
Changes in v10:
Thomas Gleixner: There are big change in pci-ep-msi.c. I am sure if go on the corrent path. The key improvement is remove only 1 function devices's limitation.
I use new patch for imutable check, which relative additional feature compared to base enablement patch.
- Remove patch Add msi_remove_device_irq_domain() in platform_device_msi_free_irqs_all() - Add new patch irqchip/gic-v3-its: Avoid overwriting msi_prepare callback if provided by msi_domain_info - Remove only support 1 endpoint function limiation. - Create one MSI domain for each endpoint function devices. - Use "msi-map" in pci ep controler node, instead of of msi-parent. first argument is (func_no << 8 | vfunc_no)
- Link to v9: https://lore.kernel.org/r/20241203-ep-msi-v9-0-a60dbc3f15dd@nxp.com
Changes in v9 - Add patch platform-msi: Add msi_remove_device_irq_domain() in platform_device_msi_free_irqs_all() - Remove patch PCI: endpoint: Add pci_epc_get_fn() API for customizable filtering - Remove API pci_epf_align_inbound_addr_lo_hi - Move doorbell_alloc in to doorbell_enable function. - Link to v8: https://lore.kernel.org/r/20241116-ep-msi-v8-0-6f1f68ffd1bb@nxp.com
Changes in v8: - update helper function name to pci_epf_align_inbound_addr() - Link to v7: https://lore.kernel.org/r/20241114-ep-msi-v7-0-d4ac7aafbd2c@nxp.com
Changes in v7: - Add helper function pci_epf_align_addr(); - Link to v6: https://lore.kernel.org/r/20241112-ep-msi-v6-0-45f9722e3c2a@nxp.com
Changes in v6: - change doorbell_addr to doorbell_offset - use round_down() - add Niklas's test by tag - rebase to pci/endpoint - Link to v5: https://lore.kernel.org/r/20241108-ep-msi-v5-0-a14951c0d007@nxp.com
Changes in v5: - Move request_irq to epf test function driver for more flexiable user case - Add fixed size bar handler - Some minor improvememtn to see each patches's changelog. - Link to v4: https://lore.kernel.org/r/20241031-ep-msi-v4-0-717da2d99b28@nxp.com
Changes in v4: - Remove patch genirq/msi: Add cleanup guard define for msi_lock_descs()/msi_unlock_descs() - Use new method to avoid compatible problem. Add new command DOORBELL_ENABLE and DOORBELL_DISABLE. pcitest -B send DOORBELL_ENABLE first, EP test function driver try to remap one of BAR_N (except test register bar) to ITS MSI MMIO space. Old driver don't support new command, so failure return, not side effect. After test, DOORBELL_DISABLE command send out to recover original map, so pcitest bar test can pass as normal. - Other detail change see each patches's change log - Link to v3: https://lore.kernel.org/r/20241015-ep-msi-v3-0-cedc89a16c1a@nxp.com
Change from v2 to v3 - Fixed manivannan's comments - Move common part to pci-ep-msi.c and pci-ep-msi.h - rebase to 6.12-rc1 - use RevID to distingiush old version
mkdir /sys/kernel/config/pci_ep/functions/pci_epf_test/func1 echo 16 > /sys/kernel/config/pci_ep/functions/pci_epf_test/func1/msi_interrupts echo 0x080c > /sys/kernel/config/pci_ep/functions/pci_epf_test/func1/deviceid echo 0x1957 > /sys/kernel/config/pci_ep/functions/pci_epf_test/func1/vendorid echo 1 > /sys/kernel/config/pci_ep/functions/pci_epf_test/func1/revid ^^^^^^ to enable platform msi support. ln -s /sys/kernel/config/pci_ep/functions/pci_epf_test/func1 /sys/kernel/config/pci_ep/controllers/4c380000.pcie-ep
- use new device ID, which identify support doorbell to avoid broken compatility.
Enable doorbell support only for PCI_DEVICE_ID_IMX8_DB, while other devices keep the same behavior as before.
EP side RC with old driver RC with new driver PCI_DEVICE_ID_IMX8_DB no probe doorbell enabled Other device ID doorbell disabled* doorbell disabled*
* Behavior remains unchanged.
Change from v1 to v2 - Add missed patch for endpont/pci-epf-test.c - Move alloc and free to epc driver from epf. - Provide general help function for EPC driver to alloc platform msi irq. - Fixed manivannan's comments.
Signed-off-by: Frank Li Frank.Li@nxp.com --- Frank Li (9): PCI: imx6: Add helper function imx_pcie_add_lut_by_rid() PCI: imx6: Add LUT configuration for MSI/IOMMU in Endpoint mode PCI: endpoint: Add RC-to-EP doorbell support using platform MSI controller PCI: endpoint: pci-ep-msi: Add MSI address/data pair mutable check PCI: endpoint: Add pci_epf_align_inbound_addr() helper for address alignment PCI: endpoint: pci-epf-test: Add doorbell test support misc: pci_endpoint_test: Add doorbell test case selftests: pci_endpoint: Add doorbell test case arm64: dts: imx95: Add msi-map for pci-ep device
Documentation/PCI/endpoint/pci-test-howto.rst | 14 +++ arch/arm64/boot/dts/freescale/imx95.dtsi | 1 + drivers/misc/pci_endpoint_test.c | 85 ++++++++++++- drivers/pci/controller/dwc/pci-imx6.c | 25 ++-- drivers/pci/endpoint/Kconfig | 8 ++ drivers/pci/endpoint/Makefile | 1 + drivers/pci/endpoint/functions/pci-epf-test.c | 136 +++++++++++++++++++++ drivers/pci/endpoint/pci-ep-msi.c | 98 +++++++++++++++ drivers/pci/endpoint/pci-epf-core.c | 44 +++++++ include/linux/pci-ep-msi.h | 28 +++++ include/linux/pci-epf.h | 18 +++ include/uapi/linux/pcitest.h | 1 + .../selftests/pci_endpoint/pci_endpoint_test.c | 28 +++++ 13 files changed, 478 insertions(+), 9 deletions(-) --- base-commit: 19272b37aa4f83ca52bdf9c16d5d81bdd1354494 change-id: 20241010-ep-msi-8b4cab33b1be
Best regards, -- Frank Li Frank.Li@nxp.com
From: Frank Li Frank.Li@nxp.com
Add helper function imx_pcie_add_lut_by_rid(), which will be used for MSI doorbell support in endpoint mode in the future. No functional change.
Signed-off-by: Frank Li Frank.Li@nxp.com --- - change in v20 - update commit message.
change from v14 to v16 - none
change from v13 to v14 - new patch --- drivers/pci/controller/dwc/pci-imx6.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-)
diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c index 5a38cfaf989b1c9606660f2e0bcd10d88fa2d6b1..032b906c44dfaa374a32f511098402a494ef5677 100644 --- a/drivers/pci/controller/dwc/pci-imx6.c +++ b/drivers/pci/controller/dwc/pci-imx6.c @@ -1096,18 +1096,14 @@ static void imx_pcie_remove_lut(struct imx_pcie *imx_pcie, u16 rid) } }
-static int imx_pcie_enable_device(struct pci_host_bridge *bridge, - struct pci_dev *pdev) +static int imx_pcie_add_lut_by_rid(struct imx_pcie *imx_pcie, u32 rid) { - struct imx_pcie *imx_pcie = to_imx_pcie(to_dw_pcie_from_pp(bridge->sysdata)); - u32 sid_i, sid_m, rid = pci_dev_id(pdev); + struct device *dev = imx_pcie->pci->dev; struct device_node *target; - struct device *dev; + u32 sid_i, sid_m; int err_i, err_m; u32 sid = 0;
- dev = imx_pcie->pci->dev; - target = NULL; err_i = of_map_id(dev->of_node, rid, "iommu-map", "iommu-map-mask", &target, &sid_i); @@ -1182,6 +1178,13 @@ static int imx_pcie_enable_device(struct pci_host_bridge *bridge, return imx_pcie_add_lut(imx_pcie, rid, sid); }
+static int imx_pcie_enable_device(struct pci_host_bridge *bridge, struct pci_dev *pdev) +{ + struct imx_pcie *imx_pcie = to_imx_pcie(to_dw_pcie_from_pp(bridge->sysdata)); + + return imx_pcie_add_lut_by_rid(imx_pcie, pci_dev_id(pdev)); +} + static void imx_pcie_disable_device(struct pci_host_bridge *bridge, struct pci_dev *pdev) {
From: Frank Li Frank.Li@nxp.com
Add LUT entry for Endpoint mode by calling imx_pcie_add_lut_by_rid(0), since only one physical function is supported. This sets up a single LUT entry required for MSI or IOMMU.
The Endpoint function can operate without LUT if neither IOMMU nor MSI is used. This LUT setup enables the EP doorbell feature by allowing the Root Complex to trigger the EP’s MSI controller.
Signed-off-by: Frank Li Frank.Li@nxp.com --- change in v20 - update commit message
change from v14 to v16 - none
change from v13 to v14 - new patch --- drivers/pci/controller/dwc/pci-imx6.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/drivers/pci/controller/dwc/pci-imx6.c b/drivers/pci/controller/dwc/pci-imx6.c index 032b906c44dfaa374a32f511098402a494ef5677..3123bf49e209cc0fc448d6e02a472e280d538033 100644 --- a/drivers/pci/controller/dwc/pci-imx6.c +++ b/drivers/pci/controller/dwc/pci-imx6.c @@ -1063,7 +1063,10 @@ static int imx_pcie_add_lut(struct imx_pcie *imx_pcie, u16 rid, u8 sid) data1 |= IMX95_PE0_LUT_VLD; regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA1, data1);
- data2 = IMX95_PE0_LUT_MASK; /* Match all bits of RID */ + if (imx_pcie->drvdata->mode == DW_PCIE_EP_TYPE) + data2 = 0x7; /* EP side's RID from RC, only 'D' is meansful */ + else + data2 = IMX95_PE0_LUT_MASK; /* Match all bits of RID */ data2 |= FIELD_PREP(IMX95_PE0_LUT_REQID, rid); regmap_write(imx_pcie->iomuxc_gpr, IMX95_PE0_LUT_DATA2, data2);
@@ -1767,6 +1770,9 @@ static int imx_pcie_probe(struct platform_device *pdev) ret = imx_add_pcie_ep(imx_pcie, pdev); if (ret < 0) return ret; + + /* Only support one physical function */ + imx_pcie_add_lut_by_rid(imx_pcie, 0); } else { pci->pp.use_atu_msg = true; ret = dw_pcie_host_init(&pci->pp);
From: Frank Li Frank.Li@nxp.com
Doorbell feature is implemented by mapping the EP's MSI interrupt controller message address to a dedicated BAR in the EPC core. It is the responsibility of the EPF driver to pass the actual message data to be written by the host to the doorbell BAR region through its own logic.
Signed-off-by: Frank Li Frank.Li@nxp.com --- change in v20 - remove Niklas Cassel test-by tag - rollback to use epc and only support one epf to void touch epf's of_node - add new config PCI_ENDPOINT_MSI_DOORBELL - error code change to ENODATA - don't overwrite return value of platform_device_msi_init_and_alloc_irqs() - remove comment @name in struct pci_epf_doorbell_msg - Add #include <linux/export.h> to fix kbuild warning
Change from v15 to v16 - fix rebase conflict
Change from v14 to v15 - check CONFIG_GENERIC_MSI
Fix below build error | Reported-by: kernel test robot lkp@intel.com | Closes: https://lore.kernel.org/oe-kbuild-all/202502082204.6PRR3cfG-lkp@intel.com/
All errors (new ones prefixed by >>):
drivers/pci/endpoint/pci-ep-msi.c: In function 'pci_epf_alloc_doorbell':
drivers/pci/endpoint/pci-ep-msi.c:53:15: error: implicit declaration of function 'platform_device_msi_init_and_alloc_irqs' [-Werror=implicit-function-declaration]
53 | ret = platform_device_msi_init_and_alloc_irqs(&epf->dev, num_db, pci_epf_write_msi_msg);
| Reported-by: kernel test robot lkp@intel.com | Closes: https://lore.kernel.org/oe-kbuild-all/202502082242.pOq1hB1d-lkp@intel.com/
All errors (new ones prefixed by >>):
drivers/pci/endpoint/pci-ep-msi.c: In function 'pci_epf_alloc_doorbell':
drivers/pci/endpoint/pci-ep-msi.c:49:14: error: implicit declaration of function 'irq_domain_is_msi_immutable'; did you mean 'irq_domain_is_msi_device'? [-Werror=implicit-function-declaration]
49 | if (!irq_domain_is_msi_immutable(dom)) {
Change from v13 to v14 - basic roll back to v9 - use device:id as msi-map input, its will handle it - use existed platform_device_msi_init_and_alloc_irqs() - pass down epf->dev point, because epf->dev of-node will be the same as epc's parent.
Change from v12 to v13 - Use DOMAIN_BUS_DEVICE_PCI_EP_MSI
Change from v10 to v12 - none
Change from v9 to v10 - Create msi domain for each function device. - Remove only function support limiation. My hardware only support one function, so not test more than one case. - use "msi-map" descript msi information
msi-map = <func_no << 8 | vfunc_no, &its, start_stream_id, size>;
Chagne from v8 to v9 - sort header file - use pci_epc_get(dev_name(msi_desc_to_dev(desc))); - check epf number at pci_epf_alloc_doorbell - Add comments for miss msi-parent case
change from v5 to v8 -none
Change from v4 to v5 - Remove request_irq() in pci_epc_alloc_doorbell() and leave to EP function driver, so ep function driver can register differece call back function for difference doorbell events and set irq affinity to differece CPU core. - Improve error message when MSI allocate failure.
Change from v3 to v4 - msi change to use msi_get_virq() avoid use msi_for_each_desc(). - add new struct for pci_epf_doorbell_msg to msi msg,virq and irq name. - move mutex lock to epc function - initialize variable at declear place. - passdown epf to epc*() function to simplify code. --- drivers/pci/endpoint/Kconfig | 8 ++++ drivers/pci/endpoint/Makefile | 1 + drivers/pci/endpoint/pci-ep-msi.c | 90 +++++++++++++++++++++++++++++++++++++++ include/linux/pci-ep-msi.h | 28 ++++++++++++ include/linux/pci-epf.h | 15 +++++++ 5 files changed, 142 insertions(+)
diff --git a/drivers/pci/endpoint/Kconfig b/drivers/pci/endpoint/Kconfig index 1c5d82eb57d440fb18c788921fc27096b75b061e..72f82ae98f94556ec5cd734e4525b363879aa340 100644 --- a/drivers/pci/endpoint/Kconfig +++ b/drivers/pci/endpoint/Kconfig @@ -28,6 +28,14 @@ config PCI_ENDPOINT_CONFIGFS configure the endpoint function and used to bind the function with an endpoint controller.
+config PCI_ENDPOINT_MSI_DOORBELL + bool "PCI Endpoint MSI Doorbell Support" + depends on PCI_ENDPOINT && GENERIC_MSI_IRQ + help + This enables the EP's MSI interrupt controller to function as a + doorbell. The RC can then notify the EP by writing data to dedicated + BAR, which the EP maps to the controller's message address. + source "drivers/pci/endpoint/functions/Kconfig"
endmenu diff --git a/drivers/pci/endpoint/Makefile b/drivers/pci/endpoint/Makefile index 95b2fe47e3b06be6ecd210e8d834610021a22470..b4869d52053aade87298289b29b3beb547001bfa 100644 --- a/drivers/pci/endpoint/Makefile +++ b/drivers/pci/endpoint/Makefile @@ -6,3 +6,4 @@ obj-$(CONFIG_PCI_ENDPOINT_CONFIGFS) += pci-ep-cfs.o obj-$(CONFIG_PCI_ENDPOINT) += pci-epc-core.o pci-epf-core.o\ pci-epc-mem.o functions/ +obj-$(CONFIG_PCI_ENDPOINT_MSI_DOORBELL) += pci-ep-msi.o diff --git a/drivers/pci/endpoint/pci-ep-msi.c b/drivers/pci/endpoint/pci-ep-msi.c new file mode 100644 index 0000000000000000000000000000000000000000..de610c16f68986eb0a5f4a0f23bd0136aa169f97 --- /dev/null +++ b/drivers/pci/endpoint/pci-ep-msi.c @@ -0,0 +1,90 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * PCI Endpoint *Controller* (EPC) MSI library + * + * Copyright (C) 2025 NXP + * Author: Frank Li Frank.Li@nxp.com + */ + +#include <linux/device.h> +#include <linux/export.h> +#include <linux/irqdomain.h> +#include <linux/module.h> +#include <linux/msi.h> +#include <linux/of_irq.h> +#include <linux/pci-epc.h> +#include <linux/pci-epf.h> +#include <linux/pci-ep-cfs.h> +#include <linux/pci-ep-msi.h> +#include <linux/slab.h> + +static void pci_epf_write_msi_msg(struct msi_desc *desc, struct msi_msg *msg) +{ + struct pci_epc *epc; + struct pci_epf *epf; + + epc = pci_epc_get(dev_name(msi_desc_to_dev(desc))); + if (!epc) + return; + + epf = list_first_entry_or_null(&epc->pci_epf, struct pci_epf, list); + + if (epf && epf->db_msg && desc->msi_index < epf->num_db) + memcpy(&epf->db_msg[desc->msi_index].msg, msg, sizeof(*msg)); + + pci_epc_put(epc); +} + +int pci_epf_alloc_doorbell(struct pci_epf *epf, u16 num_db) +{ + struct pci_epc *epc = epf->epc; + struct device *dev = &epf->dev; + struct irq_domain *dom; + void *msg; + int ret; + int i; + + if (list_first_entry_or_null(&epc->pci_epf, struct pci_epf, list) != epf) { + dev_err(dev, "MSI doorbell only support one endpoint function\n"); + return -EINVAL; + } + + dom = of_msi_map_get_device_domain(epc->dev.parent, 0, DOMAIN_BUS_PLATFORM_MSI); + if (!dom) { + dev_err(dev, "Can't find MSI domain for parent device\n"); + return -ENODEV; + } + + dev_set_msi_domain(epc->dev.parent, dom); + + msg = kcalloc(num_db, sizeof(struct pci_epf_doorbell_msg), GFP_KERNEL); + if (!msg) + return -ENOMEM; + + epf->num_db = num_db; + epf->db_msg = msg; + + ret = platform_device_msi_init_and_alloc_irqs(epc->dev.parent, num_db, + pci_epf_write_msi_msg); + if (ret) { + dev_err(dev, "Failed to allocate MSI\n"); + kfree(msg); + return ret; + } + + for (i = 0; i < num_db; i++) + epf->db_msg[i].virq = msi_get_virq(epc->dev.parent, i); + + return ret; +} +EXPORT_SYMBOL_GPL(pci_epf_alloc_doorbell); + +void pci_epf_free_doorbell(struct pci_epf *epf) +{ + platform_device_msi_free_irqs_all(epf->epc->dev.parent); + + kfree(epf->db_msg); + epf->db_msg = NULL; + epf->num_db = 0; +} +EXPORT_SYMBOL_GPL(pci_epf_free_doorbell); diff --git a/include/linux/pci-ep-msi.h b/include/linux/pci-ep-msi.h new file mode 100644 index 0000000000000000000000000000000000000000..7c5db90f96204559b4be0c8805565f210461ac75 --- /dev/null +++ b/include/linux/pci-ep-msi.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * PCI Endpoint *Function* side MSI header file + * + * Copyright (C) 2024 NXP + * Author: Frank Li Frank.Li@nxp.com + */ + +#ifndef __PCI_EP_MSI__ +#define __PCI_EP_MSI__ + +struct pci_epf; + +#ifdef CONFIG_PCI_ENDPOINT_MSI_DOORBELL +int pci_epf_alloc_doorbell(struct pci_epf *epf, u16 nums); +void pci_epf_free_doorbell(struct pci_epf *epf); +#else +static inline int pci_epf_alloc_doorbell(struct pci_epf *epf, u16 nums) +{ + return -ENODATA; +} + +static inline void pci_epf_free_doorbell(struct pci_epf *epf) +{ +} +#endif /* CONFIG_GENERIC_MSI_IRQ */ + +#endif /* __PCI_EP_MSI__ */ diff --git a/include/linux/pci-epf.h b/include/linux/pci-epf.h index 749cee0bcf2cc0600ad3be1582b0c497c5a21bed..a3b1a1d90d18356e8cda966c7004c3be93962a0d 100644 --- a/include/linux/pci-epf.h +++ b/include/linux/pci-epf.h @@ -12,6 +12,7 @@ #include <linux/configfs.h> #include <linux/device.h> #include <linux/mod_devicetable.h> +#include <linux/msi.h> #include <linux/pci.h>
struct pci_epf; @@ -128,6 +129,16 @@ struct pci_epf_bar { int flags; };
+/** + * struct pci_epf_doorbell_msg - represents doorbell message + * @msi_msg: MSI message + * @virq: irq number of this doorbell MSI message + */ +struct pci_epf_doorbell_msg { + struct msi_msg msg; + int virq; +}; + /** * struct pci_epf - represents the PCI EPF device * @dev: the PCI EPF device @@ -155,6 +166,8 @@ struct pci_epf_bar { * @vfunction_num_map: bitmap to manage virtual function number * @pci_vepf: list of virtual endpoint functions associated with this function * @event_ops: callbacks for capturing the EPC events + * @db_msg: data for MSI from RC side + * @num_db: number of doorbells */ struct pci_epf { struct device dev; @@ -185,6 +198,8 @@ struct pci_epf { unsigned long vfunction_num_map; struct list_head pci_vepf; const struct pci_epc_event_ops *event_ops; + struct pci_epf_doorbell_msg *db_msg; + u16 num_db; };
/**
From: Frank Li Frank.Li@nxp.com
Some MSI controller change address/data pair when irq_set_affinity(). Current PCI endpoint can't support this type MSI controller. Call irq_domain_is_msi_immutable() check if address/data pair immutable.
Also ensure it is parent MSI domains, not device-specific MSI domains, since device MSI domains do not allocate address/data pairs.
Signed-off-by: Frank Li Frank.Li@nxp.com --- change in v20 - update commit message - change return error code to ENODEV
change in v18 - update commit message. remove 'include/linux/msi.h' part.
change from v14 to v17 - none
change from v13 to v14 - bring v10 back
Change from v9 to v10 - new patch --- drivers/pci/endpoint/pci-ep-msi.c | 8 ++++++++ 1 file changed, 8 insertions(+)
diff --git a/drivers/pci/endpoint/pci-ep-msi.c b/drivers/pci/endpoint/pci-ep-msi.c index de610c16f68986eb0a5f4a0f23bd0136aa169f97..2b4405cfeb6f8f3cbe9782c793604a10dcaa4eb7 100644 --- a/drivers/pci/endpoint/pci-ep-msi.c +++ b/drivers/pci/endpoint/pci-ep-msi.c @@ -55,6 +55,14 @@ int pci_epf_alloc_doorbell(struct pci_epf *epf, u16 num_db) return -ENODEV; }
+ if (!irq_domain_is_msi_parent(dom)) + return -ENODEV; + + if (!irq_domain_is_msi_immutable(dom)) { + dev_err(dev, "MSI controller not supported\n"); + return -ENODEV; + } + dev_set_msi_domain(epc->dev.parent, dom);
msg = kcalloc(num_db, sizeof(struct pci_epf_doorbell_msg), GFP_KERNEL);
From: Frank Li Frank.Li@nxp.com
Introduce the helper function pci_epf_align_inbound_addr() to adjust addresses according to PCI BAR alignment requirements, converting addresses into base and offset values.
Signed-off-by: Frank Li Frank.Li@nxp.com --- Changes in V20 - update function kdoc comments. - 128 come from pci_epf_alloc_space() drivers/pci/endpoint/pci-epf-core.c
Change from v15 to v16 - none
Change from v14 to v15 - change out address type to dma_addr_t to fix below build issue
| Reported-by: kernel test robot lkp@intel.com | Closes: https://lore.kernel.org/oe-kbuild-all/202502082311.G1hWGggF-lkp@intel.com/
All errors (new ones prefixed by >>):
drivers/pci/endpoint/functions/pci-epf-test.c: In function 'pci_epf_test_enable_doorbell':
drivers/pci/endpoint/functions/pci-epf-test.c:726:42: error: passing argument 4 of 'pci_epf_align_inbound_addr' from incompatible pointer type [-Werror=incompatible-pointer-types]
726 | &epf_test->db_bar.phys_addr, &offset); | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ | | | dma_addr_t * {aka unsigned int *} In file included from include/linux/pci-epc.h:12,
Change form v9 to v14 - none
change from v8 to v9 - pci_epf_align_inbound_addr(), base and off must be not NULL - rm pci_epf_align_inbound_addr_lo_hi()
change from v7 to v8 - change name to pci_epf_align_inbound_addr() - update comment said only need for memory, which not allocated by pci_epf_alloc_space().
change from v6 to v7 - new patch --- drivers/pci/endpoint/pci-epf-core.c | 44 +++++++++++++++++++++++++++++++++++++ include/linux/pci-epf.h | 3 +++ 2 files changed, 47 insertions(+)
diff --git a/drivers/pci/endpoint/pci-epf-core.c b/drivers/pci/endpoint/pci-epf-core.c index 577a9e490115c9dd5d7fb624c4a3001f39b97e78..6d6911559780a003b8e81c629d7654d6959c7dcf 100644 --- a/drivers/pci/endpoint/pci-epf-core.c +++ b/drivers/pci/endpoint/pci-epf-core.c @@ -477,6 +477,50 @@ struct pci_epf *pci_epf_create(const char *name) } EXPORT_SYMBOL_GPL(pci_epf_create);
+/** + * pci_epf_align_inbound_addr() - Align the given address based on the BAR + * alignment requirement + * @epf: the EPF device + * @addr: inbound address to be aligned + * @bar: the BAR number corresponding to the given addr + * @base: base address matching the @bar alignment requirement. + * @off: offset to be added to the @base address. + * + * Helper function to align input 'addr' to base and offset, which match + * BAR's alignment requirement. + * + * The pci_epf_alloc_space() function already accounts for alignment. This is + * primarily intended for use with other memory regions not allocated by + * pci_epf_alloc_space(), such as peripheral register spaces or the trigger + * address for a platform MSI controller. + */ +int pci_epf_align_inbound_addr(struct pci_epf *epf, enum pci_barno bar, + u64 addr, dma_addr_t *base, size_t *off) +{ + const struct pci_epc_features *epc_features; + u64 align; + + if (!base || !off) + return -EINVAL; + + epc_features = pci_epc_get_features(epf->epc, epf->func_no, epf->vfunc_no); + if (!epc_features) { + dev_err(&epf->dev, "epc_features not implemented\n"); + return -EOPNOTSUPP; + } + + align = epc_features->align; + align = align ? align : 128; + if (epc_features->bar[bar].type == BAR_FIXED) + align = max(epc_features->bar[bar].fixed_size, align); + + *base = round_down(addr, align); + *off = addr & (align - 1); + + return 0; +} +EXPORT_SYMBOL_GPL(pci_epf_align_inbound_addr); + static void pci_epf_dev_release(struct device *dev) { struct pci_epf *epf = to_pci_epf(dev); diff --git a/include/linux/pci-epf.h b/include/linux/pci-epf.h index a3b1a1d90d18356e8cda966c7004c3be93962a0d..0ca08f0d05d7439ef4043a77c204062989a27bd9 100644 --- a/include/linux/pci-epf.h +++ b/include/linux/pci-epf.h @@ -241,6 +241,9 @@ void *pci_epf_alloc_space(struct pci_epf *epf, size_t size, enum pci_barno bar, enum pci_epc_interface_type type); void pci_epf_free_space(struct pci_epf *epf, void *addr, enum pci_barno bar, enum pci_epc_interface_type type); + +int pci_epf_align_inbound_addr(struct pci_epf *epf, enum pci_barno bar, + u64 addr, dma_addr_t *base, size_t *off); int pci_epf_bind(struct pci_epf *epf); void pci_epf_unbind(struct pci_epf *epf); int pci_epf_add_vepf(struct pci_epf *epf_pf, struct pci_epf *epf_vf);
From: Frank Li Frank.Li@nxp.com
Add three registers: doorbell_bar, doorbell_addr, and doorbell_data. Use pci_epf_alloc_doorbell() to allocate a doorbell address space.
Enable the Root Complex (RC) side driver to trigger pci-epc-test's doorbell callback handler by writing doorbell_data to the mapped doorbell_bar's address space.
Set STATUS_DOORBELL_SUCCESS in the doorbell callback to indicate completion.
Avoid breaking compatibility between host and endpoint, add new command COMMAND_ENABLE_DOORBELL and COMMAND_DISABLE_DOORBELL. Host side need send COMMAND_ENABLE_DOORBELL to map one bar's inbound address to MSI space. the command COMMAND_DISABLE_DOORBELL to recovery original inbound address mapping.
Host side new driver Host side old driver
EP: new driver S F EP: old driver F F
S: If EP side support MSI, 'pci_endpoint_test -f pcie_ep_doorbell' return success. If EP side doesn't support MSI, the same to 'F'.
F: 'pci_endpoint_test -f pcie_ep_doorbell' return failure, other case as usual.
Signed-off-by: Frank Li Frank.Li@nxp.com --- changes in v20 - simple error handle by goto - remove unecessary check - remove Tested-by: Niklas Cassel cassel@kernel.org
change from v15 to v16 - use le32 for doorbell_* register and use cpu_to_le32() and le32_to_cpu() when use it.
change from v14 to v15 - none
Change from v9 to v14 - update commit message by use pci_endpoint_test -f pcie_ep_doorbell
Change from v8 to v9 - move pci_epf_alloc_doorbell() into pci_epf_{enable/disable}_doorbell(). - remove doorbell_done in commit message. - rename pci_epf_{enable/disable}_doorbell() to pci_epf_test_{enable/disable}_doorbell() to align corrent code style.
Change from v7 to v8 - rename to pci_epf_align_inbound_addr_lo_hi()
Change from v6 to v7 - use help function pci_epf_align_addr_lo_hi()
Change from v5 to v6 - rename doorbell_addr to doorbell_offset
Chagne from v4 to v5 - Add doorbell free at unbind function. - Move msi irq handler to here to more complex user case, such as differece doorbell can use difference handler function. - Add Niklas's code to handle fixed bar's case. If need add your signed-off tag or co-developer tag, please let me know.
change from v3 to v4 - remove revid requirement - Add command COMMAND_ENABLE_DOORBELL and COMMAND_DISABLE_DOORBELL. - call pci_epc_set_bar() to map inbound address to MSI space only at COMMAND_ENABLE_DOORBELL. --- drivers/pci/endpoint/functions/pci-epf-test.c | 136 ++++++++++++++++++++++++++ 1 file changed, 136 insertions(+)
diff --git a/drivers/pci/endpoint/functions/pci-epf-test.c b/drivers/pci/endpoint/functions/pci-epf-test.c index 50eb4106369f41afa00ed3ae58c84922e0a49e51..dfdd25cfc003948fbb54a4313f1759994e67df99 100644 --- a/drivers/pci/endpoint/functions/pci-epf-test.c +++ b/drivers/pci/endpoint/functions/pci-epf-test.c @@ -11,12 +11,14 @@ #include <linux/dmaengine.h> #include <linux/io.h> #include <linux/module.h> +#include <linux/msi.h> #include <linux/slab.h> #include <linux/pci_ids.h> #include <linux/random.h>
#include <linux/pci-epc.h> #include <linux/pci-epf.h> +#include <linux/pci-ep-msi.h> #include <linux/pci_regs.h>
#define IRQ_TYPE_INTX 0 @@ -29,6 +31,8 @@ #define COMMAND_READ BIT(3) #define COMMAND_WRITE BIT(4) #define COMMAND_COPY BIT(5) +#define COMMAND_ENABLE_DOORBELL BIT(6) +#define COMMAND_DISABLE_DOORBELL BIT(7)
#define STATUS_READ_SUCCESS BIT(0) #define STATUS_READ_FAIL BIT(1) @@ -39,6 +43,11 @@ #define STATUS_IRQ_RAISED BIT(6) #define STATUS_SRC_ADDR_INVALID BIT(7) #define STATUS_DST_ADDR_INVALID BIT(8) +#define STATUS_DOORBELL_SUCCESS BIT(9) +#define STATUS_DOORBELL_ENABLE_SUCCESS BIT(10) +#define STATUS_DOORBELL_ENABLE_FAIL BIT(11) +#define STATUS_DOORBELL_DISABLE_SUCCESS BIT(12) +#define STATUS_DOORBELL_DISABLE_FAIL BIT(13)
#define FLAG_USE_DMA BIT(0)
@@ -66,6 +75,7 @@ struct pci_epf_test { bool dma_supported; bool dma_private; const struct pci_epc_features *epc_features; + struct pci_epf_bar db_bar; };
struct pci_epf_test_reg { @@ -80,6 +90,9 @@ struct pci_epf_test_reg { __le32 irq_number; __le32 flags; __le32 caps; + __le32 doorbell_bar; + __le32 doorbell_offset; + __le32 doorbell_data; } __packed;
static struct pci_epf_header test_header = { @@ -667,6 +680,121 @@ static void pci_epf_test_raise_irq(struct pci_epf_test *epf_test, } }
+static irqreturn_t pci_epf_test_doorbell_handler(int irq, void *data) +{ + struct pci_epf_test *epf_test = data; + enum pci_barno test_reg_bar = epf_test->test_reg_bar; + struct pci_epf_test_reg *reg = epf_test->reg[test_reg_bar]; + u32 status = le32_to_cpu(reg->status); + + status |= STATUS_DOORBELL_SUCCESS; + reg->status = cpu_to_le32(status); + pci_epf_test_raise_irq(epf_test, reg); + + return IRQ_HANDLED; +} + +static void pci_epf_test_doorbell_cleanup(struct pci_epf_test *epf_test) +{ + struct pci_epf_test_reg *reg = epf_test->reg[epf_test->test_reg_bar]; + struct pci_epf *epf = epf_test->epf; + + free_irq(epf->db_msg[0].virq, epf_test); + reg->doorbell_bar = cpu_to_le32(NO_BAR); + + pci_epf_free_doorbell(epf); +} + +static void pci_epf_test_enable_doorbell(struct pci_epf_test *epf_test, + struct pci_epf_test_reg *reg) +{ + u32 status = le32_to_cpu(reg->status); + struct pci_epf *epf = epf_test->epf; + struct pci_epc *epc = epf->epc; + struct msi_msg *msg; + enum pci_barno bar; + size_t offset; + int ret; + + ret = pci_epf_alloc_doorbell(epf, 1); + if (ret) + goto set_status_err; + + msg = &epf->db_msg[0].msg; + bar = pci_epc_get_next_free_bar(epf_test->epc_features, epf_test->test_reg_bar + 1); + if (bar < BAR_0) + goto err_doorbell; + + ret = request_irq(epf->db_msg[0].virq, pci_epf_test_doorbell_handler, 0, + "pci-ep-test-doorbell", epf_test); + if (ret) { + dev_err(&epf->dev, + "Failed to request doorbell IRQ: %d\n", + epf->db_msg[0].virq); + goto err_doorbell; + } + + reg->doorbell_data = cpu_to_le32(msg->data); + reg->doorbell_bar = cpu_to_le32(bar); + + msg = &epf->db_msg[0].msg; + ret = pci_epf_align_inbound_addr(epf, bar, ((u64)msg->address_hi << 32) | msg->address_lo, + &epf_test->db_bar.phys_addr, &offset); + + if (ret) + goto err_doorbell; + + reg->doorbell_offset = cpu_to_le32(offset); + + epf_test->db_bar.barno = bar; + epf_test->db_bar.size = epf->bar[bar].size; + epf_test->db_bar.flags = epf->bar[bar].flags; + + ret = pci_epc_set_bar(epc, epf->func_no, epf->vfunc_no, &epf_test->db_bar); + if (ret) + goto err_doorbell_set_bar; + + status |= STATUS_DOORBELL_ENABLE_SUCCESS; + reg->status = cpu_to_le32(status); + return; + +err_doorbell_set_bar: + pci_epc_set_bar(epc, epf->func_no, epf->vfunc_no, &epf->bar[bar]); +err_doorbell: + pci_epf_test_doorbell_cleanup(epf_test); +set_status_err: + status |= STATUS_DOORBELL_ENABLE_FAIL; + reg->status = cpu_to_le32(status); +} + +static void pci_epf_test_disable_doorbell(struct pci_epf_test *epf_test, + struct pci_epf_test_reg *reg) +{ + enum pci_barno bar = le32_to_cpu(reg->doorbell_bar); + u32 status = le32_to_cpu(reg->status); + struct pci_epf *epf = epf_test->epf; + struct pci_epc *epc = epf->epc; + int ret; + + if (bar < BAR_0) + goto set_status_err; + + pci_epf_test_doorbell_cleanup(epf_test); + + ret = pci_epc_set_bar(epc, epf->func_no, epf->vfunc_no, &epf->bar[bar]); + if (ret) + goto set_status_err; + + status |= STATUS_DOORBELL_DISABLE_SUCCESS; + reg->status = cpu_to_le32(status); + + return; + +set_status_err: + status |= STATUS_DOORBELL_DISABLE_FAIL; + reg->status = cpu_to_le32(status); +} + static void pci_epf_test_cmd_handler(struct work_struct *work) { u32 command; @@ -714,6 +842,14 @@ static void pci_epf_test_cmd_handler(struct work_struct *work) pci_epf_test_copy(epf_test, reg); pci_epf_test_raise_irq(epf_test, reg); break; + case COMMAND_ENABLE_DOORBELL: + pci_epf_test_enable_doorbell(epf_test, reg); + pci_epf_test_raise_irq(epf_test, reg); + break; + case COMMAND_DISABLE_DOORBELL: + pci_epf_test_disable_doorbell(epf_test, reg); + pci_epf_test_raise_irq(epf_test, reg); + break; default: dev_err(dev, "Invalid command 0x%x\n", command); break;
From: Frank Li Frank.Li@nxp.com
Add three registers: PCIE_ENDPOINT_TEST_DB_BAR, PCIE_ENDPOINT_TEST_DB_ADDR, and PCIE_ENDPOINT_TEST_DB_DATA.
Trigger the doorbell by writing data from PCI_ENDPOINT_TEST_DB_DATA to the address provided by PCI_ENDPOINT_TEST_DB_OFFSET and wait for endpoint feedback.
Add two command to COMMAND_ENABLE_DOORBELL and COMMAND_DISABLE_DOORBELL to enable EP side's doorbell support and avoid compatible problem, which host side driver miss-match with endpoint side function driver. See below table:
Host side new driver Host side old driver EP: new driver S F EP: old driver F F
S: If EP side support MSI, 'pci_endpoint_test -f pcie_ep_doorbell' return success. If EP side doesn't support MSI, the same to 'F'.
F: 'pci_endpoint_test -f pcie_ep_doorbell' return failure, other case as usual.
Signed-off-by: Frank Li Frank.Li@nxp.com --- changes in v20 - add return value check of wait_for_completion_timeout() - fix some error message - remove Tested-by: Niklas Cassel cassel@kernel.org
change from v14 to v16 - none
Change from v13 to v14 - update to use pci_endpoint_test -f pcie_ep_doorbell - change ioctrl id to fix conflict
Change from v9 to v13 - none
Change from v8 to v9 - change PCITEST_DOORBELL to 0xa
Change form v6 to v8 - none
Change from v5 to v6 - %s/PCI_ENDPOINT_TEST_DB_ADDR/PCI_ENDPOINT_TEST_DB_OFFSET/g
Change from v4 to v5 - remove unused varible - add irq_type at pci_endpoint_test_doorbell();
change from v3 to v4 - Add COMMAND_ENABLE_DOORBELL and COMMAND_DISABLE_DOORBELL. - Remove new DID requirement. --- drivers/misc/pci_endpoint_test.c | 85 +++++++++++++++++++++++++++++++++++++++- include/uapi/linux/pcitest.h | 1 + 2 files changed, 85 insertions(+), 1 deletion(-)
diff --git a/drivers/misc/pci_endpoint_test.c b/drivers/misc/pci_endpoint_test.c index c4e5e2c977be27adf1451b67ee10cc3ae6fb0cef..254b7f9cbe87b97b4d5d555ffbccc9055fc52330 100644 --- a/drivers/misc/pci_endpoint_test.c +++ b/drivers/misc/pci_endpoint_test.c @@ -37,6 +37,8 @@ #define COMMAND_READ BIT(3) #define COMMAND_WRITE BIT(4) #define COMMAND_COPY BIT(5) +#define COMMAND_ENABLE_DOORBELL BIT(6) +#define COMMAND_DISABLE_DOORBELL BIT(7)
#define PCI_ENDPOINT_TEST_STATUS 0x8 #define STATUS_READ_SUCCESS BIT(0) @@ -48,6 +50,11 @@ #define STATUS_IRQ_RAISED BIT(6) #define STATUS_SRC_ADDR_INVALID BIT(7) #define STATUS_DST_ADDR_INVALID BIT(8) +#define STATUS_DOORBELL_SUCCESS BIT(9) +#define STATUS_DOORBELL_ENABLE_SUCCESS BIT(10) +#define STATUS_DOORBELL_ENABLE_FAIL BIT(11) +#define STATUS_DOORBELL_DISABLE_SUCCESS BIT(12) +#define STATUS_DOORBELL_DISABLE_FAIL BIT(13)
#define PCI_ENDPOINT_TEST_LOWER_SRC_ADDR 0x0c #define PCI_ENDPOINT_TEST_UPPER_SRC_ADDR 0x10 @@ -62,6 +69,7 @@ #define PCI_ENDPOINT_TEST_IRQ_NUMBER 0x28
#define PCI_ENDPOINT_TEST_FLAGS 0x2c + #define FLAG_USE_DMA BIT(0)
#define PCI_ENDPOINT_TEST_CAPS 0x30 @@ -70,6 +78,10 @@ #define CAP_MSIX BIT(2) #define CAP_INTX BIT(3)
+#define PCI_ENDPOINT_TEST_DB_BAR 0x34 +#define PCI_ENDPOINT_TEST_DB_OFFSET 0x38 +#define PCI_ENDPOINT_TEST_DB_DATA 0x3c + #define PCI_DEVICE_ID_TI_AM654 0xb00c #define PCI_DEVICE_ID_TI_J7200 0xb00f #define PCI_DEVICE_ID_TI_AM64 0xb010 @@ -100,6 +112,7 @@ enum pci_barno { BAR_3, BAR_4, BAR_5, + NO_BAR = -1, };
struct pci_endpoint_test { @@ -505,7 +518,7 @@ static int pci_endpoint_test_copy(struct pci_endpoint_test *test,
if (irq_type < PCITEST_IRQ_TYPE_INTX || irq_type > PCITEST_IRQ_TYPE_MSIX) { - dev_err(dev, "Invalid IRQ type option\n"); + dev_err(dev, "Invalid IRQ type\n"); return -EINVAL; }
@@ -841,6 +854,73 @@ static int pci_endpoint_test_set_irq(struct pci_endpoint_test *test, return 0; }
+static int pci_endpoint_test_doorbell(struct pci_endpoint_test *test) +{ + struct pci_dev *pdev = test->pdev; + struct device *dev = &pdev->dev; + int irq_type = test->irq_type; + enum pci_barno bar; + u32 data, status; + u32 addr; + int left; + + if (irq_type < PCITEST_IRQ_TYPE_INTX || + irq_type > PCITEST_IRQ_TYPE_MSIX) { + dev_err(dev, "Invalid IRQ type option\n"); + return -EINVAL; + } + + pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_IRQ_TYPE, irq_type); + pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_IRQ_NUMBER, 1); + pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_COMMAND, + COMMAND_ENABLE_DOORBELL); + + left = wait_for_completion_timeout(&test->irq_raised, msecs_to_jiffies(1000)); + + status = pci_endpoint_test_readl(test, PCI_ENDPOINT_TEST_STATUS); + if (!left || (status & STATUS_DOORBELL_ENABLE_FAIL)) { + dev_err(dev, "Failed to enable doorbell\n"); + return -EINVAL; + } + + data = pci_endpoint_test_readl(test, PCI_ENDPOINT_TEST_DB_DATA); + addr = pci_endpoint_test_readl(test, PCI_ENDPOINT_TEST_DB_OFFSET); + bar = pci_endpoint_test_readl(test, PCI_ENDPOINT_TEST_DB_BAR); + + pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_IRQ_TYPE, irq_type); + pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_IRQ_NUMBER, 1); + + pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_STATUS, 0); + + bar = pci_endpoint_test_readl(test, PCI_ENDPOINT_TEST_DB_BAR); + + writel(data, test->bar[bar] + addr); + + left = wait_for_completion_timeout(&test->irq_raised, msecs_to_jiffies(1000)); + + status = pci_endpoint_test_readl(test, PCI_ENDPOINT_TEST_STATUS); + + if (!left || !(status & STATUS_DOORBELL_SUCCESS)) + dev_err(dev, "Failed to trigger doorbell in endpoint\n"); + + pci_endpoint_test_writel(test, PCI_ENDPOINT_TEST_COMMAND, + COMMAND_DISABLE_DOORBELL); + + wait_for_completion_timeout(&test->irq_raised, msecs_to_jiffies(1000)); + + status |= pci_endpoint_test_readl(test, PCI_ENDPOINT_TEST_STATUS); + + if (status & STATUS_DOORBELL_DISABLE_FAIL) { + dev_err(dev, "Failed to disable doorbell\n"); + return -EINVAL; + } + + if (!(status & STATUS_DOORBELL_SUCCESS)) + return -EINVAL; + + return 0; +} + static long pci_endpoint_test_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -891,6 +971,9 @@ static long pci_endpoint_test_ioctl(struct file *file, unsigned int cmd, case PCITEST_CLEAR_IRQ: ret = pci_endpoint_test_clear_irq(test); break; + case PCITEST_DOORBELL: + ret = pci_endpoint_test_doorbell(test); + break; }
ret: diff --git a/include/uapi/linux/pcitest.h b/include/uapi/linux/pcitest.h index d3aa8715a525ea47485db734a42d13d333399112..d6023a45a9d03416e220c98643c4e86d144898ae 100644 --- a/include/uapi/linux/pcitest.h +++ b/include/uapi/linux/pcitest.h @@ -21,6 +21,7 @@ #define PCITEST_SET_IRQTYPE _IOW('P', 0x8, int) #define PCITEST_GET_IRQTYPE _IO('P', 0x9) #define PCITEST_BARS _IO('P', 0xa) +#define PCITEST_DOORBELL _IO('P', 0xb) #define PCITEST_CLEAR_IRQ _IO('P', 0x10)
#define PCITEST_IRQ_TYPE_UNDEFINED -1
From: Frank Li Frank.Li@nxp.com
Add doorbell test case.
Signed-off-by: Frank Li Frank.Li@nxp.com --- change in v20 - update document
change from v14 to v16 - Add set IRQ type
change from v13 to v14 - merge to selftests framework --- Documentation/PCI/endpoint/pci-test-howto.rst | 14 +++++++++++ .../selftests/pci_endpoint/pci_endpoint_test.c | 28 ++++++++++++++++++++++ 2 files changed, 42 insertions(+)
diff --git a/Documentation/PCI/endpoint/pci-test-howto.rst b/Documentation/PCI/endpoint/pci-test-howto.rst index aafc17ef3fd31f6ada38c82c8df133b87f7a1ae1..0cee35dcb29cc938823de53ef6e1a1cd81d49d52 100644 --- a/Documentation/PCI/endpoint/pci-test-howto.rst +++ b/Documentation/PCI/endpoint/pci-test-howto.rst @@ -203,3 +203,17 @@ controllers, it is advisable to skip this testcase using this command::
# pci_endpoint_test -f pci_ep_bar -f pci_ep_basic -v memcpy -T COPY_TEST -v dma + +Kselftest EP Doorbell +~~~~~~~~~~~~~~~~~~~~~ + +If EP side have MSI controller, run below command to test doorbell function. + + # pci_endpoint_test -f pcie_ep_doorbell + + # Starting 1 tests from 1 test cases. + # RUN pcie_ep_doorbell.DOORBELL_TEST ... + # OK pcie_ep_doorbell.DOORBELL_TEST + ok 1 pcie_ep_doorbell.DOORBELL_TEST + # PASSED: 1 / 1 tests passed. + # Totals: pass:1 fail:0 xfail:0 xpass:0 skip:0 error:0 diff --git a/tools/testing/selftests/pci_endpoint/pci_endpoint_test.c b/tools/testing/selftests/pci_endpoint/pci_endpoint_test.c index ac26481d29d9dce5d07de563048850c55c58c22d..da0db0e7c9693f11b4975ca0e6830174229983ba 100644 --- a/tools/testing/selftests/pci_endpoint/pci_endpoint_test.c +++ b/tools/testing/selftests/pci_endpoint/pci_endpoint_test.c @@ -229,4 +229,32 @@ TEST_F(pci_ep_data_transfer, COPY_TEST) test_size[i]); } } + +FIXTURE(pcie_ep_doorbell) +{ + int fd; +}; + +FIXTURE_SETUP(pcie_ep_doorbell) +{ + self->fd = open(test_device, O_RDWR); + + ASSERT_NE(-1, self->fd) TH_LOG("Can't open PCI Endpoint Test device"); +}; + +FIXTURE_TEARDOWN(pcie_ep_doorbell) +{ + close(self->fd); +}; + +TEST_F(pcie_ep_doorbell, DOORBELL_TEST) +{ + int ret; + + pci_ep_ioctl(PCITEST_SET_IRQTYPE, PCITEST_IRQ_TYPE_AUTO); + ASSERT_EQ(0, ret) TH_LOG("Can't set AUTO IRQ type"); + + pci_ep_ioctl(PCITEST_DOORBELL, 0); + EXPECT_FALSE(ret) TH_LOG("Test failed for Doorbell\n"); +} TEST_HARNESS_MAIN
From: Frank Li Frank.Li@nxp.com
Add msi-map for pci-ep device.
Acked-by: Manivannan Sadhasivam mani@kernel.org Signed-off-by: Frank Li Frank.Li@nxp.com --- change in v20 - add Manivannan's ACK
change from v14 to v16 - none
change from v13 to v14 - new patch --- arch/arm64/boot/dts/freescale/imx95.dtsi | 1 + 1 file changed, 1 insertion(+)
diff --git a/arch/arm64/boot/dts/freescale/imx95.dtsi b/arch/arm64/boot/dts/freescale/imx95.dtsi index 632631a2911224cadc16a943cdb467e091e43384..c59d11eb7a581a500d381ef96f1e44533052c2a2 100644 --- a/arch/arm64/boot/dts/freescale/imx95.dtsi +++ b/arch/arm64/boot/dts/freescale/imx95.dtsi @@ -1797,6 +1797,7 @@ pcie1_ep: pcie-ep@4c380000 { assigned-clock-rates = <3600000000>, <100000000>, <10000000>; assigned-clock-parents = <0>, <0>, <&scmi_clk IMX95_CLK_SYSPLL1_PFD1_DIV2>; + msi-map = <0x0 &its 0x98 0x1>; power-domains = <&scmi_devpd IMX95_PD_HSIO_TOP>; status = "disabled"; };
Hello Frank,
I tested v20 of your series, but unfortunately, it still doesn't work.
When enabling the doorbell, the programming of the inbound iATU fails:
## pci_epf_test_enable_doorbell() ## keeps the BAR size, and BAR type of a BAR that has already been configured, ## but changes the address translation for this BAR to redirect to the GIC ITS ## rather than to the memory allocated by pci_epf_alloc_space() ## (does not free the memory allocated by pci_epf_alloc_space())
[ 39.347502] pci_epf_test_enable_doorbell: msg hi: 0x0 msg low: 0xfe670040 [ 39.348103] pci_epf_test_enable_doorbell: base: 0xfe670000 off: 0x40 [ 39.348658] dw_pcie_ep_inbound_atu index: 1 parent_bus_addr: 0xfe670000 bar: 1 size: 0x100000 [ 39.349403] dw_pcie_prog_ep_inbound_atu parent_bus_addr: 0xfe670000 pci->region_align: 0x10000 IS_ALIGNED: 1 [ 39.350260] dw_pcie_prog_ep_inbound_atu parent_bus_addr: 0xfe670000 size: 0x100000 IS_ALIGNED: 0 [ 39.351028] rockchip-dw-pcie a40000000.pcie-ep: Failed to program IB window
## pci_epf_test_disable_doorbell() ## changes the address translation for this BAR to redirect to the memory ## allocated by pci_epf_alloc_space() (which was never freed when enabling the ## doorbell)
[ 39.351656] dw_pcie_ep_inbound_atu index: 1 parent_bus_addr: 0xa2e00000 bar: 1 size: 0x100000 [ 39.352401] dw_pcie_prog_ep_inbound_atu parent_bus_addr: 0xa2e00000 pci->region_align: 0x10000 IS_ALIGNED: 1 [ 39.353257] dw_pcie_prog_ep_inbound_atu parent_bus_addr: 0xa2e00000 size: 0x100000 IS_ALIGNED: 1
The reason why pci_epf_test_enable_doorbell() fails is because of this check: https://github.com/torvalds/linux/blob/v6.16-rc5/drivers/pci/controller/dwc/...
If you want to understand why this very important check is there, it is because the DWC controller requires that the physical address programmed in the iATU is aligned to the size of the BAR (BAR_MASK+1), see this commit: https://github.com/torvalds/linux/commit/129f6af747b2
Applying the following patch on top of your v20 series makes things work as intended and makes the pcie_ep_doorbell.DOORBELL_TEST selftest pass for me:
diff --git a/drivers/pci/endpoint/functions/pci-epf-test.c b/drivers/pci/endpoint/functions/pci-epf-test.c index dfdd25cfc003..7d356b0201ae 100644 --- a/drivers/pci/endpoint/functions/pci-epf-test.c +++ b/drivers/pci/endpoint/functions/pci-epf-test.c @@ -738,9 +738,9 @@ static void pci_epf_test_enable_doorbell(struct pci_epf_test *epf_test, reg->doorbell_bar = cpu_to_le32(bar);
msg = &epf->db_msg[0].msg; - ret = pci_epf_align_inbound_addr(epf, bar, ((u64)msg->address_hi << 32) | msg->address_lo, + ret = pci_epf_align_inbound_addr(epf, epf->bar[bar].size, + ((u64)msg->address_hi << 32) | msg->address_lo, &epf_test->db_bar.phys_addr, &offset); - if (ret) goto err_doorbell;
diff --git a/drivers/pci/endpoint/pci-epf-core.c b/drivers/pci/endpoint/pci-epf-core.c index c21d8e786eb3..b3d4117182e2 100644 --- a/drivers/pci/endpoint/pci-epf-core.c +++ b/drivers/pci/endpoint/pci-epf-core.c @@ -478,44 +478,36 @@ struct pci_epf *pci_epf_create(const char *name) EXPORT_SYMBOL_GPL(pci_epf_create);
/** - * pci_epf_align_inbound_addr() - Align the given address based on the BAR - * alignment requirement + * pci_epf_align_inbound_addr() - Align the given address based on the BAR size + * * @epf: the EPF device + * @bar_size: the current BAR size * @addr: inbound address to be aligned - * @bar: the BAR number corresponding to the given addr - * @base: base address matching the @bar alignment requirement. + * @base: base address matching the @bar_size alignment requirement. * @off: offset to be added to the @base address. * - * Helper function to align input 'addr' to base and offset, which match - * BAR's alignment requirement. + * Helper function to align input 'addr' to base and offset, when dynamically + * changing a BAR. * * The pci_epf_alloc_space() function already accounts for alignment. This is * primarily intended for use with other memory regions not allocated by * pci_epf_alloc_space(), such as peripheral register spaces or the trigger * address for a platform MSI controller. + * + * Since this function is only used when dynamically changing a BAR (i.e. when + * calling set_bar() twice, without ever calling clear_bar(), as calling + * clear_bar() would clear the BAR's PCI address assigned by the host), this + * function must align to the current BAR size, since we are not clearing the + * BAR configuration. */ -int pci_epf_align_inbound_addr(struct pci_epf *epf, enum pci_barno bar, - u64 addr, dma_addr_t *base, size_t *off) +int pci_epf_align_inbound_addr(struct pci_epf *epf, size_t bar_size, u64 addr, + dma_addr_t *base, size_t *off) { - const struct pci_epc_features *epc_features; - u64 align; - - if (!base || !off) + if (!base || !off || !bar_size) return -EINVAL;
- epc_features = pci_epc_get_features(epf->epc, epf->func_no, epf->vfunc_no); - if (!epc_features) { - dev_err(&epf->dev, "epc_features not implemented\n"); - return -EOPNOTSUPP; - } - - align = epc_features->align; - align = align ? align : 128; - if (epc_features->bar[bar].type == BAR_FIXED) - align = max(epc_features->bar[bar].fixed_size, align); - - *base = round_down(addr, align); - *off = addr & (align - 1); + *base = round_down(addr, bar_size); + *off = addr & (bar_size - 1);
return 0; } diff --git a/include/linux/pci-epf.h b/include/linux/pci-epf.h index 0ca08f0d05d7..bcc8184325d2 100644 --- a/include/linux/pci-epf.h +++ b/include/linux/pci-epf.h @@ -242,8 +242,8 @@ void *pci_epf_alloc_space(struct pci_epf *epf, size_t size, enum pci_barno bar, void pci_epf_free_space(struct pci_epf *epf, void *addr, enum pci_barno bar, enum pci_epc_interface_type type);
-int pci_epf_align_inbound_addr(struct pci_epf *epf, enum pci_barno bar, - u64 addr, dma_addr_t *base, size_t *off); +int pci_epf_align_inbound_addr(struct pci_epf *epf, size_t bar_size, u64 addr, + dma_addr_t *base, size_t *off); int pci_epf_bind(struct pci_epf *epf); void pci_epf_unbind(struct pci_epf *epf); int pci_epf_add_vepf(struct pci_epf *epf_pf, struct pci_epf *epf_vf);
However, the more I think about it, considering that this alignment requirement is inherent to the DWC controller (other controllers might not have this requirement), perhaps pci_epf_align_inbound_addr() should not be a function in pci-epf-core.c, perhaps this function would be better suited to live in drivers/pci/controller/dwc/pcie-designware-ep.c ?
Kind regards, Niklas
On Thu, Jul 10, 2025 at 01:40:25PM +0200, Niklas Cassel wrote:
Hello Frank,
I tested v20 of your series, but unfortunately, it still doesn't work.
When enabling the doorbell, the programming of the inbound iATU fails:
## pci_epf_test_enable_doorbell() ## keeps the BAR size, and BAR type of a BAR that has already been configured, ## but changes the address translation for this BAR to redirect to the GIC ITS ## rather than to the memory allocated by pci_epf_alloc_space() ## (does not free the memory allocated by pci_epf_alloc_space())
[ 39.347502] pci_epf_test_enable_doorbell: msg hi: 0x0 msg low: 0xfe670040 [ 39.348103] pci_epf_test_enable_doorbell: base: 0xfe670000 off: 0x40 [ 39.348658] dw_pcie_ep_inbound_atu index: 1 parent_bus_addr: 0xfe670000 bar: 1 size: 0x100000 [ 39.349403] dw_pcie_prog_ep_inbound_atu parent_bus_addr: 0xfe670000 pci->region_align: 0x10000 IS_ALIGNED: 1 [ 39.350260] dw_pcie_prog_ep_inbound_atu parent_bus_addr: 0xfe670000 size: 0x100000 IS_ALIGNED: 0 [ 39.351028] rockchip-dw-pcie a40000000.pcie-ep: Failed to program IB window
## pci_epf_test_disable_doorbell() ## changes the address translation for this BAR to redirect to the memory ## allocated by pci_epf_alloc_space() (which was never freed when enabling the ## doorbell)
[ 39.351656] dw_pcie_ep_inbound_atu index: 1 parent_bus_addr: 0xa2e00000 bar: 1 size: 0x100000 [ 39.352401] dw_pcie_prog_ep_inbound_atu parent_bus_addr: 0xa2e00000 pci->region_align: 0x10000 IS_ALIGNED: 1 [ 39.353257] dw_pcie_prog_ep_inbound_atu parent_bus_addr: 0xa2e00000 size: 0x100000 IS_ALIGNED: 1
The reason why pci_epf_test_enable_doorbell() fails is because of this check: https://github.com/torvalds/linux/blob/v6.16-rc5/drivers/pci/controller/dwc/...
If you want to understand why this very important check is there, it is because the DWC controller requires that the physical address programmed in the iATU is aligned to the size of the BAR (BAR_MASK+1), see this commit: https://github.com/torvalds/linux/commit/129f6af747b2
Applying the following patch on top of your v20 series makes things work as intended and makes the pcie_ep_doorbell.DOORBELL_TEST selftest pass for me:
diff --git a/drivers/pci/endpoint/functions/pci-epf-test.c b/drivers/pci/endpoint/functions/pci-epf-test.c index dfdd25cfc003..7d356b0201ae 100644 --- a/drivers/pci/endpoint/functions/pci-epf-test.c +++ b/drivers/pci/endpoint/functions/pci-epf-test.c @@ -738,9 +738,9 @@ static void pci_epf_test_enable_doorbell(struct pci_epf_test *epf_test, reg->doorbell_bar = cpu_to_le32(bar);
msg = &epf->db_msg[0].msg;
- ret = pci_epf_align_inbound_addr(epf, bar, ((u64)msg->address_hi << 32) | msg->address_lo,
- ret = pci_epf_align_inbound_addr(epf, epf->bar[bar].size,
((u64)msg->address_hi << 32) | msg->address_lo, &epf_test->db_bar.phys_addr, &offset);
- if (ret) goto err_doorbell;
diff --git a/drivers/pci/endpoint/pci-epf-core.c b/drivers/pci/endpoint/pci-epf-core.c index c21d8e786eb3..b3d4117182e2 100644 --- a/drivers/pci/endpoint/pci-epf-core.c +++ b/drivers/pci/endpoint/pci-epf-core.c @@ -478,44 +478,36 @@ struct pci_epf *pci_epf_create(const char *name) EXPORT_SYMBOL_GPL(pci_epf_create);
/**
- pci_epf_align_inbound_addr() - Align the given address based on the BAR
alignment requirement
- pci_epf_align_inbound_addr() - Align the given address based on the BAR size
- @epf: the EPF device
- @bar_size: the current BAR size
- @addr: inbound address to be aligned
- @bar: the BAR number corresponding to the given addr
- @base: base address matching the @bar alignment requirement.
- @base: base address matching the @bar_size alignment requirement.
- @off: offset to be added to the @base address.
- Helper function to align input 'addr' to base and offset, which match
- BAR's alignment requirement.
- Helper function to align input 'addr' to base and offset, when dynamically
- changing a BAR.
- The pci_epf_alloc_space() function already accounts for alignment. This is
- primarily intended for use with other memory regions not allocated by
- pci_epf_alloc_space(), such as peripheral register spaces or the trigger
- address for a platform MSI controller.
- Since this function is only used when dynamically changing a BAR (i.e. when
- calling set_bar() twice, without ever calling clear_bar(), as calling
- clear_bar() would clear the BAR's PCI address assigned by the host), this
- function must align to the current BAR size, since we are not clearing the
*/
- BAR configuration.
-int pci_epf_align_inbound_addr(struct pci_epf *epf, enum pci_barno bar,
u64 addr, dma_addr_t *base, size_t *off)+int pci_epf_align_inbound_addr(struct pci_epf *epf, size_t bar_size, u64 addr,
dma_addr_t *base, size_t *off){
- const struct pci_epc_features *epc_features;
- u64 align;
- if (!base || !off)
- if (!base || !off || !bar_size) return -EINVAL;
- epc_features = pci_epc_get_features(epf->epc, epf->func_no, epf->vfunc_no);
- if (!epc_features) {
dev_err(&epf->dev, "epc_features not implemented\n");return -EOPNOTSUPP;- }
- align = epc_features->align;
- align = align ? align : 128;
- if (epc_features->bar[bar].type == BAR_FIXED)
align = max(epc_features->bar[bar].fixed_size, align);- *base = round_down(addr, align);
- *off = addr & (align - 1);
*base = round_down(addr, bar_size);
*off = addr & (bar_size - 1);
return 0;
} diff --git a/include/linux/pci-epf.h b/include/linux/pci-epf.h index 0ca08f0d05d7..bcc8184325d2 100644 --- a/include/linux/pci-epf.h +++ b/include/linux/pci-epf.h @@ -242,8 +242,8 @@ void *pci_epf_alloc_space(struct pci_epf *epf, size_t size, enum pci_barno bar, void pci_epf_free_space(struct pci_epf *epf, void *addr, enum pci_barno bar, enum pci_epc_interface_type type);
-int pci_epf_align_inbound_addr(struct pci_epf *epf, enum pci_barno bar,
u64 addr, dma_addr_t *base, size_t *off);+int pci_epf_align_inbound_addr(struct pci_epf *epf, size_t bar_size, u64 addr,
dma_addr_t *base, size_t *off);int pci_epf_bind(struct pci_epf *epf); void pci_epf_unbind(struct pci_epf *epf); int pci_epf_add_vepf(struct pci_epf *epf_pf, struct pci_epf *epf_vf);
However, the more I think about it, considering that this alignment requirement is inherent to the DWC controller (other controllers might not have this requirement), perhaps pci_epf_align_inbound_addr() should not be a function in pci-epf-core.c, perhaps this function would be better suited to live in drivers/pci/controller/dwc/pcie-designware-ep.c ?
I think align to bar size is quite make sense now. Even other controllers have not such requirement, align to bar size still work.
Anyways, it need be align. We still pass down bar number, get bar size in pci_epf_align_inbound_addr(). So we can fine tune algorithm in future, such as add field in epc_features.
Frank
Kind regards, Niklas
linux-kselftest-mirror@lists.linaro.org
-
Frank Li -
Frank Li via B4 Relay -
Niklas Cassel