From the functionality point of view this series may be split into the
following logic parts: 1. New ECAM API and update for users of the pci-host-common API 2. Necessary fixes as the preparation for using driver on ARM64. 3. Use new MCFG interface and implement generic ACPI based PCI host controller driver. 4. Enable above driver on ARM64
Patches has been built on top of 4.6-rc7 and can be found here: git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v7)
This has been tested on Cavium ThunderX server. Any help in reviewing and testing is very appreciated.
v6 -> v7 - drop quirks handling - changes for ACPI companion and domain number assignment approach - implement arch pcibios_{add|remove}_bus and call acpi_pci_{add|remove}_bus from there - cleanups around nomenclature - use resources oriented API for ECAM - fix for based address calculation before mapping ECAM region - remove useless lock for MCFG lookup - move MCFG stuff to separated file pci_mcfg.c - drop MCFG entries caching - rebase against 4.6-rc7
v5 -> v6 - drop idea of x86 MMCONFIG code refactoring - integrate JC's patches which introduce new ECAM API: https://lkml.org/lkml/2016/4/11/907 git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3) - integrate Sinan's fix for releasing IO resources, see patch [06/13] - added ACPI support for ThunderX ECAM and PEM drivers - rebase against 4.6-rc2
v4 -> v5 - drop MCFG refactoring group patches 1-6 from series v4 and integrate Jayachandran's patch https://patchwork.ozlabs.org/patch/575525/ - rewrite PCI legacy IRQs allocation - squash two patches 11 and 12 from series v4, fixed bisection issue - changelog improvements - rebase against 4.5-rc3
v3 -> v4 - drop Jiang's fix http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04318.html - add Lorenzo's fix patch 19/24 - ACPI PCI bus domain number assigning cleanup - change resource management, we now claim and reassign resources - improvements for applying quirks - drop Matthew's http://www.spinics.net/lists/linux-pci/msg45950.html dependency - rebase against 4.5-rc1
v2 -> v3 - fix legacy IRQ assigning and IO ports registration - remove reference to arch specific companion device for ia64 - move ACPI PCI host controller driver to pci_root.c - drop generic domain assignment for x86 and ia64 as I am not able to run all necessary test variants - drop patch which cleaned legacy IRQ assignment since it belongs to Mathew's series: https://patchwork.ozlabs.org/patch/557504/ - extend MCFG quirk code - rebase against 4.4
v1 -> v2 - move non-arch specific piece of code to dirver/acpi/ directory - fix IO resource handling - introduce PCI config accessors quirks matching - moved ACPI_COMPANION_SET to generic code
v1 - https://lkml.org/lkml/2015/10/27/504 v2 - https://lkml.org/lkml/2015/12/16/246 v3 - http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04308.html v4 - https://lkml.org/lkml/2016/2/4/646 v5 - https://lkml.org/lkml/2016/2/16/426 v6 - https://lkml.org/lkml/2016/4/15/594
Jayachandran C (2): PCI: Provide common functions for ECAM mapping PCI: generic, thunder: update to use generic ECAM API
Tomasz Nowicki (9): pci, of: Move the PCI I/O space management to PCI core code. pci: Add new function to unmap IO resources. acpi, pci: Support IO resources when parsing PCI host bridge resources. pci, acpi: Provide a way to assign bus domain number. pci, acpi: Handle ACPI companion assignment. pci, acpi: Support for ACPI based generic PCI host controller arm64, pci, acpi: ACPI support for legacy IRQs parsing and consolidation with DT code. arm64, pci, acpi: Provide ACPI-specific prerequisites for PCI bus enumeration. arm64, pci, acpi: Start using ACPI based PCI host controller driver for ARM64.
arch/arm64/Kconfig | 1 + arch/arm64/kernel/pci.c | 34 +++----- drivers/acpi/Kconfig | 8 ++ drivers/acpi/Makefile | 1 + drivers/acpi/pci_mcfg.c | 97 ++++++++++++++++++++++ drivers/acpi/pci_root.c | 33 ++++++++ drivers/acpi/pci_root_generic.c | 149 +++++++++++++++++++++++++++++++++ drivers/of/address.c | 116 +------------------------- drivers/pci/Kconfig | 3 + drivers/pci/Makefile | 2 + drivers/pci/ecam.c | 161 ++++++++++++++++++++++++++++++++++++ drivers/pci/ecam.h | 72 ++++++++++++++++ drivers/pci/host/Kconfig | 1 + drivers/pci/host/pci-host-common.c | 114 +++++++++++-------------- drivers/pci/host/pci-host-common.h | 47 ----------- drivers/pci/host/pci-host-generic.c | 52 +++--------- drivers/pci/host/pci-thunder-ecam.c | 39 ++------- drivers/pci/host/pci-thunder-pem.c | 92 ++++++++++----------- drivers/pci/pci.c | 150 ++++++++++++++++++++++++++++++++- drivers/pci/probe.c | 2 + include/linux/of_address.h | 9 -- include/linux/pci-acpi.h | 14 ++++ include/linux/pci.h | 11 ++- 23 files changed, 823 insertions(+), 385 deletions(-) create mode 100644 drivers/acpi/pci_mcfg.c create mode 100644 drivers/acpi/pci_root_generic.c create mode 100644 drivers/pci/ecam.c create mode 100644 drivers/pci/ecam.h delete mode 100644 drivers/pci/host/pci-host-common.h
From: Jayachandran C jchandra@broadcom.com
Add config option PCI_ECAM and file drivers/pci/ecam.c to provide generic functions for accessing memory mapped PCI config space.
The API is defined in drivers/pci/ecam.h and is written to replace the API in drivers/pci/host/pci-host-common.h. The file defines a new 'struct pci_config_window' to hold the information related to a PCI config area and its mapping. This structure is expected to be used as sysdata for controllers that have ECAM based mapping.
Helper functions are provided to setup the mapping, free the mapping and to implement the map_bus method in 'struct pci_ops'
Signed-off-by: Jayachandran C jchandra@broadcom.com --- drivers/pci/Kconfig | 3 + drivers/pci/Makefile | 2 + drivers/pci/ecam.c | 161 +++++++++++++++++++++++++++++++++++++++++++++++++++ drivers/pci/ecam.h | 62 ++++++++++++++++++++ 4 files changed, 228 insertions(+) create mode 100644 drivers/pci/ecam.c create mode 100644 drivers/pci/ecam.h
diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig index 209292e..56389be 100644 --- a/drivers/pci/Kconfig +++ b/drivers/pci/Kconfig @@ -83,6 +83,9 @@ config HT_IRQ config PCI_ATS bool
+config PCI_ECAM + bool + config PCI_IOV bool "PCI IOV support" depends on PCI diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile index 2154092..1fa6925 100644 --- a/drivers/pci/Makefile +++ b/drivers/pci/Makefile @@ -55,6 +55,8 @@ obj-$(CONFIG_PCI_SYSCALL) += syscall.o
obj-$(CONFIG_PCI_STUB) += pci-stub.o
+obj-$(CONFIG_PCI_ECAM) += ecam.o + obj-$(CONFIG_XEN_PCIDEV_FRONTEND) += xen-pcifront.o
obj-$(CONFIG_OF) += of.o diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c new file mode 100644 index 0000000..3d52005 --- /dev/null +++ b/drivers/pci/ecam.c @@ -0,0 +1,161 @@ +/* + * Copyright 2016 Broadcom + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License, version 2, as + * published by the Free Software Foundation (the "GPL"). + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * General Public License version 2 (GPLv2) for more details. + * + * You should have received a copy of the GNU General Public License + * version 2 (GPLv2) along with this source code. + */ + +#include <linux/device.h> +#include <linux/io.h> +#include <linux/kernel.h> +#include <linux/module.h> +#include <linux/pci.h> +#include <linux/slab.h> + +#include "ecam.h" + +/* + * On 64 bit systems, we do a single ioremap for the whole config space + * since we have enough virtual address range available. On 32 bit, do an + * ioremap per bus. + */ +static const bool per_bus_mapping = !config_enabled(CONFIG_64BIT); + +/* + * Create a PCI config space window + * - reserve mem region + * - alloc struct pci_config_window with space for all mappings + * - ioremap the config space + */ +struct pci_config_window *pci_ecam_create(struct device *dev, + struct resource *cfgres, struct resource *busr, + struct pci_ecam_ops *ops) +{ + struct pci_config_window *cfg; + unsigned int bus_range, bus_range_max, bsz; + int i, err; + + if (busr->start > busr->end) + return ERR_PTR(-EINVAL); + + cfg = kzalloc(sizeof(*cfg), GFP_KERNEL); + if (!cfg) + return ERR_PTR(-ENOMEM); + + cfg->ops = ops; + cfg->busr.start = busr->start; + cfg->busr.end = busr->end; + cfg->busr.flags = IORESOURCE_BUS; + bus_range = resource_size(&cfg->busr); + bus_range_max = resource_size(cfgres) >> ops->bus_shift; + if (bus_range > bus_range_max) { + dev_warn(dev, "bus max %#x reduced to %#x", + bus_range, bus_range_max); + bus_range = bus_range_max; + cfg->busr.end = busr->start + bus_range - 1; + } + bsz = 1 << ops->bus_shift; + + cfg->res.start = cfgres->start; + cfg->res.end = cfgres->end; + cfg->res.flags = IORESOURCE_MEM | IORESOURCE_BUSY; + cfg->res.name = "PCI ECAM"; + + err = request_resource(&iomem_resource, &cfg->res); + if (err) { + dev_err(dev, "request ECAM res %pR failed\n", &cfg->res); + goto err_exit; + } + + if (per_bus_mapping) { + cfg->winp = kcalloc(bus_range, sizeof(*cfg->winp), GFP_KERNEL); + if (!cfg->winp) + goto err_exit_malloc; + for (i = 0; i < bus_range; i++) { + cfg->winp[i] = ioremap(cfgres->start + i * bsz, bsz); + if (!cfg->winp[i]) + goto err_exit_iomap; + } + } else { + cfg->win = ioremap(cfgres->start, bus_range * bsz); + if (!cfg->win) + goto err_exit_iomap; + } + + if (ops->init) { + err = ops->init(dev, cfg); + if (err) + goto err_exit; + } + dev_info(dev, "ECAM at %pR for %pR\n", &cfg->res, &cfg->busr); + return cfg; + +err_exit_iomap: + dev_err(dev, "ECAM ioremap failed\n"); +err_exit_malloc: + err = -ENOMEM; +err_exit: + pci_ecam_free(cfg); + return ERR_PTR(err); +} + +void pci_ecam_free(struct pci_config_window *cfg) +{ + int i; + + if (per_bus_mapping) { + if (cfg->winp) { + for (i = 0; i < resource_size(&cfg->busr); i++) + if (cfg->winp[i]) + iounmap(cfg->winp[i]); + kfree(cfg->winp); + } + } else { + if (cfg->win) + iounmap(cfg->win); + } + if (cfg->res.parent) + release_resource(&cfg->res); + kfree(cfg); +} + +/* + * Function to implement the pci_ops ->map_bus method + */ +void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn, + int where) +{ + struct pci_config_window *cfg = bus->sysdata; + unsigned int devfn_shift = cfg->ops->bus_shift - 8; + unsigned int busn = bus->number; + void __iomem *base; + + if (busn < cfg->busr.start || busn > cfg->busr.end) + return NULL; + + busn -= cfg->busr.start; + if (per_bus_mapping) + base = cfg->winp[busn]; + else + base = cfg->win + (busn << cfg->ops->bus_shift); + return base + (devfn << devfn_shift) + where; +} + +/* ECAM ops */ +struct pci_ecam_ops pci_generic_ecam_ops = { + .bus_shift = 20, + .pci_ops = { + .map_bus = pci_ecam_map_bus, + .read = pci_generic_config_read, + .write = pci_generic_config_write, + } +}; diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h new file mode 100644 index 0000000..57de00d --- /dev/null +++ b/drivers/pci/ecam.h @@ -0,0 +1,62 @@ +/* + * Copyright 2016 Broadcom + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License, version 2, as + * published by the Free Software Foundation (the "GPL"). + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * General Public License version 2 (GPLv2) for more details. + * + * You should have received a copy of the GNU General Public License + * version 2 (GPLv2) along with this source code. + */ +#ifndef DRIVERS_PCI_ECAM_H +#define DRIVERS_PCI_ECAM_H + +#include <linux/kernel.h> +#include <linux/platform_device.h> + +/* + * struct to hold pci ops and bus shift of the config window + * for a PCI controller. + */ +struct pci_config_window; +struct pci_ecam_ops { + unsigned int bus_shift; + struct pci_ops pci_ops; + int (*init)(struct device *, + struct pci_config_window *); +}; + +/* + * struct to hold the mappings of a config space window. This + * is expected to be used as sysdata for PCI controlllers which + * use ECAM. + */ +struct pci_config_window { + struct resource res; + struct resource busr; + void *priv; + struct pci_ecam_ops *ops; + union { + void __iomem *win; /* 64-bit single mapping */ + void __iomem **winp; /* 32-bit per bus mapping */ + }; +}; + +/* create and free for pci_config_window */ +struct pci_config_window *pci_ecam_create(struct device *dev, + struct resource *cfgres, struct resource *busr, + struct pci_ecam_ops *ops); +void pci_ecam_free(struct pci_config_window *cfg); + +/* map_bus when ->sysdata is an instance of pci_config_window */ +void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn, + int where); +/* default ECAM ops, bus shift 20, generic read and write */ +extern struct pci_ecam_ops pci_generic_ecam_ops; + +#endif
From: Jayachandran C jchandra@broadcom.com
Use functions provided by drivers/pci/ecam.h for mapping the config space in drivers/pci/host/pci-host-common.c, and update its users to use 'struct pci_config_window' and 'struct pci_ecam_ops'
The changes are mostly to use 'struct pci_config_window' in place of 'struct gen_pci'. Some of the fields of gen_pci were only used temporarily and can be eliminated by using local variables or function arguments, these are not carried over to struct pci_config_window.
pci-thunder-ecam.c and pci-thunder-pem.c are the only users of the pci_host_common_probe function and the gen_pci structure, these have been updated to use the new API as well.
The patch does not introduce any functional changes other than a very minor one: with the new code, on 64-bit platforms, we do just a single ioremap for the whole config space.
Signed-off-by: Jayachandran C jchandra@broadcom.com --- drivers/pci/ecam.h | 5 ++ drivers/pci/host/Kconfig | 1 + drivers/pci/host/pci-host-common.c | 114 +++++++++++++++--------------------- drivers/pci/host/pci-host-common.h | 47 --------------- drivers/pci/host/pci-host-generic.c | 52 ++++------------ drivers/pci/host/pci-thunder-ecam.c | 39 +++--------- drivers/pci/host/pci-thunder-pem.c | 92 ++++++++++++++--------------- 7 files changed, 113 insertions(+), 237 deletions(-) delete mode 100644 drivers/pci/host/pci-host-common.h
diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h index 57de00d..1ad2176 100644 --- a/drivers/pci/ecam.h +++ b/drivers/pci/ecam.h @@ -59,4 +59,9 @@ void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn, /* default ECAM ops, bus shift 20, generic read and write */ extern struct pci_ecam_ops pci_generic_ecam_ops;
+#ifdef CONFIG_PCI_HOST_GENERIC +/* for DT based pci controllers that support ECAM */ +int pci_host_common_probe(struct platform_device *pdev, + struct pci_ecam_ops *ops); +#endif #endif diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig index 7a0780d..963300d 100644 --- a/drivers/pci/host/Kconfig +++ b/drivers/pci/host/Kconfig @@ -77,6 +77,7 @@ config PCI_RCAR_GEN2_PCIE
config PCI_HOST_COMMON bool + select PCI_ECAM
config PCI_HOST_GENERIC bool "Generic PCI host controller" diff --git a/drivers/pci/host/pci-host-common.c b/drivers/pci/host/pci-host-common.c index e9f850f..8cba7ab 100644 --- a/drivers/pci/host/pci-host-common.c +++ b/drivers/pci/host/pci-host-common.c @@ -22,27 +22,21 @@ #include <linux/of_pci.h> #include <linux/platform_device.h>
-#include "pci-host-common.h" +#include "../ecam.h"
-static void gen_pci_release_of_pci_ranges(struct gen_pci *pci) -{ - pci_free_resource_list(&pci->resources); -} - -static int gen_pci_parse_request_of_pci_ranges(struct gen_pci *pci) +static int gen_pci_parse_request_of_pci_ranges(struct device *dev, + struct list_head *resources, struct resource **bus_range) { int err, res_valid = 0; - struct device *dev = pci->host.dev.parent; struct device_node *np = dev->of_node; resource_size_t iobase; struct resource_entry *win;
- err = of_pci_get_host_bridge_resources(np, 0, 0xff, &pci->resources, - &iobase); + err = of_pci_get_host_bridge_resources(np, 0, 0xff, resources, &iobase); if (err) return err;
- resource_list_for_each_entry(win, &pci->resources) { + resource_list_for_each_entry(win, resources) { struct resource *parent, *res = win->res;
switch (resource_type(res)) { @@ -60,7 +54,7 @@ static int gen_pci_parse_request_of_pci_ranges(struct gen_pci *pci) res_valid |= !(res->flags & IORESOURCE_PREFETCH); break; case IORESOURCE_BUS: - pci->cfg.bus_range = res; + *bus_range = res; default: continue; } @@ -79,65 +73,60 @@ static int gen_pci_parse_request_of_pci_ranges(struct gen_pci *pci) return 0;
out_release_res: - gen_pci_release_of_pci_ranges(pci); return err; }
-static int gen_pci_parse_map_cfg_windows(struct gen_pci *pci) +static void gen_pci_unmap_cfg(void *ptr) +{ + pci_ecam_free((struct pci_config_window *)ptr); +} + +static struct pci_config_window *gen_pci_init(struct device *dev, + struct list_head *resources, struct pci_ecam_ops *ops) { int err; - u8 bus_max; - resource_size_t busn; - struct resource *bus_range; - struct device *dev = pci->host.dev.parent; - struct device_node *np = dev->of_node; - u32 sz = 1 << pci->cfg.ops->bus_shift; + struct resource cfgres; + struct resource *bus_range = NULL; + struct pci_config_window *cfg;
- err = of_address_to_resource(np, 0, &pci->cfg.res); + /* Parse our PCI ranges and request their resources */ + err = gen_pci_parse_request_of_pci_ranges(dev, resources, &bus_range); + if (err) + goto err_out; + + err = of_address_to_resource(dev->of_node, 0, &cfgres); if (err) { dev_err(dev, "missing "reg" property\n"); - return err; + goto err_out; }
- /* Limit the bus-range to fit within reg */ - bus_max = pci->cfg.bus_range->start + - (resource_size(&pci->cfg.res) >> pci->cfg.ops->bus_shift) - 1; - pci->cfg.bus_range->end = min_t(resource_size_t, - pci->cfg.bus_range->end, bus_max); - - pci->cfg.win = devm_kcalloc(dev, resource_size(pci->cfg.bus_range), - sizeof(*pci->cfg.win), GFP_KERNEL); - if (!pci->cfg.win) - return -ENOMEM; - - /* Map our Configuration Space windows */ - if (!devm_request_mem_region(dev, pci->cfg.res.start, - resource_size(&pci->cfg.res), - "Configuration Space")) - return -ENOMEM; - - bus_range = pci->cfg.bus_range; - for (busn = bus_range->start; busn <= bus_range->end; ++busn) { - u32 idx = busn - bus_range->start; - - pci->cfg.win[idx] = devm_ioremap(dev, - pci->cfg.res.start + idx * sz, - sz); - if (!pci->cfg.win[idx]) - return -ENOMEM; + cfg = pci_ecam_create(dev, &cfgres, bus_range, ops); + if (IS_ERR(cfg)) { + err = PTR_ERR(cfg); + goto err_out; }
- return 0; + err = devm_add_action(dev, gen_pci_unmap_cfg, cfg); + if (err) { + gen_pci_unmap_cfg(cfg); + goto err_out; + } + return cfg; + +err_out: + pci_free_resource_list(resources); + return ERR_PTR(err); }
int pci_host_common_probe(struct platform_device *pdev, - struct gen_pci *pci) + struct pci_ecam_ops *ops) { - int err; const char *type; struct device *dev = &pdev->dev; struct device_node *np = dev->of_node; struct pci_bus *bus, *child; + struct pci_config_window *cfg; + struct list_head resources;
type = of_get_property(np, "device_type", NULL); if (!type || strcmp(type, "pci")) { @@ -147,29 +136,18 @@ int pci_host_common_probe(struct platform_device *pdev,
of_pci_check_probe_only();
- pci->host.dev.parent = dev; - INIT_LIST_HEAD(&pci->host.windows); - INIT_LIST_HEAD(&pci->resources); - - /* Parse our PCI ranges and request their resources */ - err = gen_pci_parse_request_of_pci_ranges(pci); - if (err) - return err; - /* Parse and map our Configuration Space windows */ - err = gen_pci_parse_map_cfg_windows(pci); - if (err) { - gen_pci_release_of_pci_ranges(pci); - return err; - } + INIT_LIST_HEAD(&resources); + cfg = gen_pci_init(dev, &resources, ops); + if (IS_ERR(cfg)) + return PTR_ERR(cfg);
/* Do not reassign resources if probe only */ if (!pci_has_flag(PCI_PROBE_ONLY)) pci_add_flags(PCI_REASSIGN_ALL_RSRC | PCI_REASSIGN_ALL_BUS);
- - bus = pci_scan_root_bus(dev, pci->cfg.bus_range->start, - &pci->cfg.ops->ops, pci, &pci->resources); + bus = pci_scan_root_bus(dev, cfg->busr.start, &ops->pci_ops, cfg, + &resources); if (!bus) { dev_err(dev, "Scanning rootbus failed"); return -ENODEV; diff --git a/drivers/pci/host/pci-host-common.h b/drivers/pci/host/pci-host-common.h deleted file mode 100644 index 09f3fa0..0000000 --- a/drivers/pci/host/pci-host-common.h +++ /dev/null @@ -1,47 +0,0 @@ -/* - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see http://www.gnu.org/licenses/. - * - * Copyright (C) 2014 ARM Limited - * - * Author: Will Deacon will.deacon@arm.com - */ - -#ifndef _PCI_HOST_COMMON_H -#define _PCI_HOST_COMMON_H - -#include <linux/kernel.h> -#include <linux/platform_device.h> - -struct gen_pci_cfg_bus_ops { - u32 bus_shift; - struct pci_ops ops; -}; - -struct gen_pci_cfg_windows { - struct resource res; - struct resource *bus_range; - void __iomem **win; - - struct gen_pci_cfg_bus_ops *ops; -}; - -struct gen_pci { - struct pci_host_bridge host; - struct gen_pci_cfg_windows cfg; - struct list_head resources; -}; - -int pci_host_common_probe(struct platform_device *pdev, - struct gen_pci *pci); - -#endif /* _PCI_HOST_COMMON_H */ diff --git a/drivers/pci/host/pci-host-generic.c b/drivers/pci/host/pci-host-generic.c index e8aa78f..6eaceab 100644 --- a/drivers/pci/host/pci-host-generic.c +++ b/drivers/pci/host/pci-host-generic.c @@ -25,41 +25,12 @@ #include <linux/of_pci.h> #include <linux/platform_device.h>
-#include "pci-host-common.h" +#include "../ecam.h"
-static void __iomem *gen_pci_map_cfg_bus_cam(struct pci_bus *bus, - unsigned int devfn, - int where) -{ - struct gen_pci *pci = bus->sysdata; - resource_size_t idx = bus->number - pci->cfg.bus_range->start; - - return pci->cfg.win[idx] + ((devfn << 8) | where); -} - -static struct gen_pci_cfg_bus_ops gen_pci_cfg_cam_bus_ops = { +static struct pci_ecam_ops gen_pci_cfg_cam_bus_ops = { .bus_shift = 16, - .ops = { - .map_bus = gen_pci_map_cfg_bus_cam, - .read = pci_generic_config_read, - .write = pci_generic_config_write, - } -}; - -static void __iomem *gen_pci_map_cfg_bus_ecam(struct pci_bus *bus, - unsigned int devfn, - int where) -{ - struct gen_pci *pci = bus->sysdata; - resource_size_t idx = bus->number - pci->cfg.bus_range->start; - - return pci->cfg.win[idx] + ((devfn << 12) | where); -} - -static struct gen_pci_cfg_bus_ops gen_pci_cfg_ecam_bus_ops = { - .bus_shift = 20, - .ops = { - .map_bus = gen_pci_map_cfg_bus_ecam, + .pci_ops = { + .map_bus = pci_ecam_map_bus, .read = pci_generic_config_read, .write = pci_generic_config_write, } @@ -70,25 +41,22 @@ static const struct of_device_id gen_pci_of_match[] = { .data = &gen_pci_cfg_cam_bus_ops },
{ .compatible = "pci-host-ecam-generic", - .data = &gen_pci_cfg_ecam_bus_ops }, + .data = &pci_generic_ecam_ops },
{ }, }; + MODULE_DEVICE_TABLE(of, gen_pci_of_match);
static int gen_pci_probe(struct platform_device *pdev) { - struct device *dev = &pdev->dev; const struct of_device_id *of_id; - struct gen_pci *pci = devm_kzalloc(dev, sizeof(*pci), GFP_KERNEL); - - if (!pci) - return -ENOMEM; + struct pci_ecam_ops *ops;
- of_id = of_match_node(gen_pci_of_match, dev->of_node); - pci->cfg.ops = (struct gen_pci_cfg_bus_ops *)of_id->data; + of_id = of_match_node(gen_pci_of_match, pdev->dev.of_node); + ops = (struct pci_ecam_ops *)of_id->data;
- return pci_host_common_probe(pdev, pci); + return pci_host_common_probe(pdev, ops); }
static struct platform_driver gen_pci_driver = { diff --git a/drivers/pci/host/pci-thunder-ecam.c b/drivers/pci/host/pci-thunder-ecam.c index d71935cb..540d030 100644 --- a/drivers/pci/host/pci-thunder-ecam.c +++ b/drivers/pci/host/pci-thunder-ecam.c @@ -13,18 +13,7 @@ #include <linux/of.h> #include <linux/platform_device.h>
-#include "pci-host-common.h" - -/* Mapping is standard ECAM */ -static void __iomem *thunder_ecam_map_bus(struct pci_bus *bus, - unsigned int devfn, - int where) -{ - struct gen_pci *pci = bus->sysdata; - resource_size_t idx = bus->number - pci->cfg.bus_range->start; - - return pci->cfg.win[idx] + ((devfn << 12) | where); -} +#include "../ecam.h"
static void set_val(u32 v, int where, int size, u32 *val) { @@ -99,7 +88,7 @@ static int handle_ea_bar(u32 e0, int bar, struct pci_bus *bus, static int thunder_ecam_p2_config_read(struct pci_bus *bus, unsigned int devfn, int where, int size, u32 *val) { - struct gen_pci *pci = bus->sysdata; + struct pci_config_window *cfg = bus->sysdata; int where_a = where & ~3; void __iomem *addr; u32 node_bits; @@ -129,7 +118,7 @@ static int thunder_ecam_p2_config_read(struct pci_bus *bus, unsigned int devfn, * the config space access window. Since we are working with * the high-order 32 bits, shift everything down by 32 bits. */ - node_bits = (pci->cfg.res.start >> 32) & (1 << 12); + node_bits = (cfg->res.start >> 32) & (1 << 12);
v |= node_bits; set_val(v, where, size, val); @@ -358,36 +347,24 @@ static int thunder_ecam_config_write(struct pci_bus *bus, unsigned int devfn, return pci_generic_config_write(bus, devfn, where, size, val); }
-static struct gen_pci_cfg_bus_ops thunder_ecam_bus_ops = { +static struct pci_ecam_ops pci_thunder_ecam_ops = { .bus_shift = 20, - .ops = { - .map_bus = thunder_ecam_map_bus, + .pci_ops = { + .map_bus = pci_ecam_map_bus, .read = thunder_ecam_config_read, .write = thunder_ecam_config_write, } };
static const struct of_device_id thunder_ecam_of_match[] = { - { .compatible = "cavium,pci-host-thunder-ecam", - .data = &thunder_ecam_bus_ops }, - + { .compatible = "cavium,pci-host-thunder-ecam" }, { }, }; MODULE_DEVICE_TABLE(of, thunder_ecam_of_match);
static int thunder_ecam_probe(struct platform_device *pdev) { - struct device *dev = &pdev->dev; - const struct of_device_id *of_id; - struct gen_pci *pci = devm_kzalloc(dev, sizeof(*pci), GFP_KERNEL); - - if (!pci) - return -ENOMEM; - - of_id = of_match_node(thunder_ecam_of_match, dev->of_node); - pci->cfg.ops = (struct gen_pci_cfg_bus_ops *)of_id->data; - - return pci_host_common_probe(pdev, pci); + return pci_host_common_probe(pdev, &pci_thunder_ecam_ops); }
static struct platform_driver thunder_ecam_driver = { diff --git a/drivers/pci/host/pci-thunder-pem.c b/drivers/pci/host/pci-thunder-pem.c index cabb92a..a7b61ce 100644 --- a/drivers/pci/host/pci-thunder-pem.c +++ b/drivers/pci/host/pci-thunder-pem.c @@ -20,34 +20,22 @@ #include <linux/of_pci.h> #include <linux/platform_device.h>
-#include "pci-host-common.h" +#include "../ecam.h"
#define PEM_CFG_WR 0x28 #define PEM_CFG_RD 0x30
struct thunder_pem_pci { - struct gen_pci gen_pci; u32 ea_entry[3]; void __iomem *pem_reg_base; };
-static void __iomem *thunder_pem_map_bus(struct pci_bus *bus, - unsigned int devfn, int where) -{ - struct gen_pci *pci = bus->sysdata; - resource_size_t idx = bus->number - pci->cfg.bus_range->start; - - return pci->cfg.win[idx] + ((devfn << 16) | where); -} - static int thunder_pem_bridge_read(struct pci_bus *bus, unsigned int devfn, int where, int size, u32 *val) { u64 read_val; - struct thunder_pem_pci *pem_pci; - struct gen_pci *pci = bus->sysdata; - - pem_pci = container_of(pci, struct thunder_pem_pci, gen_pci); + struct pci_config_window *cfg = bus->sysdata; + struct thunder_pem_pci *pem_pci = (struct thunder_pem_pci *)cfg->priv;
if (devfn != 0 || where >= 2048) { *val = ~0; @@ -132,17 +120,17 @@ static int thunder_pem_bridge_read(struct pci_bus *bus, unsigned int devfn, static int thunder_pem_config_read(struct pci_bus *bus, unsigned int devfn, int where, int size, u32 *val) { - struct gen_pci *pci = bus->sysdata; + struct pci_config_window *cfg = bus->sysdata;
- if (bus->number < pci->cfg.bus_range->start || - bus->number > pci->cfg.bus_range->end) + if (bus->number < cfg->busr.start || + bus->number > cfg->busr.end) return PCIBIOS_DEVICE_NOT_FOUND;
/* * The first device on the bus is the PEM PCIe bridge. * Special case its config access. */ - if (bus->number == pci->cfg.bus_range->start) + if (bus->number == cfg->busr.start) return thunder_pem_bridge_read(bus, devfn, where, size, val);
return pci_generic_config_read(bus, devfn, where, size, val); @@ -187,12 +175,11 @@ static u32 thunder_pem_bridge_w1c_bits(int where) static int thunder_pem_bridge_write(struct pci_bus *bus, unsigned int devfn, int where, int size, u32 val) { - struct gen_pci *pci = bus->sysdata; - struct thunder_pem_pci *pem_pci; + struct pci_config_window *cfg = bus->sysdata; + struct thunder_pem_pci *pem_pci = (struct thunder_pem_pci *)cfg->priv; u64 write_val, read_val; u32 mask = 0;
- pem_pci = container_of(pci, struct thunder_pem_pci, gen_pci);
if (devfn != 0 || where >= 2048) return PCIBIOS_DEVICE_NOT_FOUND; @@ -256,53 +243,38 @@ static int thunder_pem_bridge_write(struct pci_bus *bus, unsigned int devfn, static int thunder_pem_config_write(struct pci_bus *bus, unsigned int devfn, int where, int size, u32 val) { - struct gen_pci *pci = bus->sysdata; + struct pci_config_window *cfg = bus->sysdata;
- if (bus->number < pci->cfg.bus_range->start || - bus->number > pci->cfg.bus_range->end) + if (bus->number < cfg->busr.start || + bus->number > cfg->busr.end) return PCIBIOS_DEVICE_NOT_FOUND; /* * The first device on the bus is the PEM PCIe bridge. * Special case its config access. */ - if (bus->number == pci->cfg.bus_range->start) + if (bus->number == cfg->busr.start) return thunder_pem_bridge_write(bus, devfn, where, size, val);
return pci_generic_config_write(bus, devfn, where, size, val); }
-static struct gen_pci_cfg_bus_ops thunder_pem_bus_ops = { - .bus_shift = 24, - .ops = { - .map_bus = thunder_pem_map_bus, - .read = thunder_pem_config_read, - .write = thunder_pem_config_write, - } -}; - -static const struct of_device_id thunder_pem_of_match[] = { - { .compatible = "cavium,pci-host-thunder-pem", - .data = &thunder_pem_bus_ops }, - - { }, -}; -MODULE_DEVICE_TABLE(of, thunder_pem_of_match); - -static int thunder_pem_probe(struct platform_device *pdev) +static int thunder_pem_init(struct device *dev, struct pci_config_window *cfg) { - struct device *dev = &pdev->dev; - const struct of_device_id *of_id; resource_size_t bar4_start; struct resource *res_pem; struct thunder_pem_pci *pem_pci; + struct platform_device *pdev; + + /* Only OF support for now */ + if (!dev->of_node) + return -EINVAL;
pem_pci = devm_kzalloc(dev, sizeof(*pem_pci), GFP_KERNEL); if (!pem_pci) return -ENOMEM;
- of_id = of_match_node(thunder_pem_of_match, dev->of_node); - pem_pci->gen_pci.cfg.ops = (struct gen_pci_cfg_bus_ops *)of_id->data; + pdev = to_platform_device(dev);
/* * The second register range is the PEM bridge to the PCIe @@ -330,7 +302,29 @@ static int thunder_pem_probe(struct platform_device *pdev) pem_pci->ea_entry[1] = (u32)(res_pem->end - bar4_start) & ~3u; pem_pci->ea_entry[2] = (u32)(bar4_start >> 32);
- return pci_host_common_probe(pdev, &pem_pci->gen_pci); + cfg->priv = pem_pci; + return 0; +} + +static struct pci_ecam_ops pci_thunder_pem_ops = { + .bus_shift = 24, + .init = thunder_pem_init, + .pci_ops = { + .map_bus = pci_ecam_map_bus, + .read = thunder_pem_config_read, + .write = thunder_pem_config_write, + } +}; + +static const struct of_device_id thunder_pem_of_match[] = { + { .compatible = "cavium,pci-host-thunder-pem" }, + { }, +}; +MODULE_DEVICE_TABLE(of, thunder_pem_of_match); + +static int thunder_pem_probe(struct platform_device *pdev) +{ + return pci_host_common_probe(pdev, &pci_thunder_pem_ops); }
static struct platform_driver thunder_pem_driver = {
No functional changes in this patch.
PCI I/O space mapping code does not depend on OF, therefore it can be moved to PCI core code. This way we will be able to use it e.g. in ACPI PCI code.
Suggested-by: Lorenzo Pieralisi lorenzo.pieralisi@arm.com Signed-off-by: Tomasz Nowicki tn@semihalf.com CC: Arnd Bergmann arnd@arndb.de CC: Liviu Dudau Liviu.Dudau@arm.com --- drivers/of/address.c | 116 +-------------------------------------------- drivers/pci/pci.c | 115 ++++++++++++++++++++++++++++++++++++++++++++ include/linux/of_address.h | 9 ---- include/linux/pci.h | 5 ++ 4 files changed, 121 insertions(+), 124 deletions(-)
diff --git a/drivers/of/address.c b/drivers/of/address.c index 91a469d..0a553c0 100644 --- a/drivers/of/address.c +++ b/drivers/of/address.c @@ -4,6 +4,7 @@ #include <linux/ioport.h> #include <linux/module.h> #include <linux/of_address.h> +#include <linux/pci.h> #include <linux/pci_regs.h> #include <linux/sizes.h> #include <linux/slab.h> @@ -673,121 +674,6 @@ const __be32 *of_get_address(struct device_node *dev, int index, u64 *size, } EXPORT_SYMBOL(of_get_address);
-#ifdef PCI_IOBASE -struct io_range { - struct list_head list; - phys_addr_t start; - resource_size_t size; -}; - -static LIST_HEAD(io_range_list); -static DEFINE_SPINLOCK(io_range_lock); -#endif - -/* - * Record the PCI IO range (expressed as CPU physical address + size). - * Return a negative value if an error has occured, zero otherwise - */ -int __weak pci_register_io_range(phys_addr_t addr, resource_size_t size) -{ - int err = 0; - -#ifdef PCI_IOBASE - struct io_range *range; - resource_size_t allocated_size = 0; - - /* check if the range hasn't been previously recorded */ - spin_lock(&io_range_lock); - list_for_each_entry(range, &io_range_list, list) { - if (addr >= range->start && addr + size <= range->start + size) { - /* range already registered, bail out */ - goto end_register; - } - allocated_size += range->size; - } - - /* range not registed yet, check for available space */ - if (allocated_size + size - 1 > IO_SPACE_LIMIT) { - /* if it's too big check if 64K space can be reserved */ - if (allocated_size + SZ_64K - 1 > IO_SPACE_LIMIT) { - err = -E2BIG; - goto end_register; - } - - size = SZ_64K; - pr_warn("Requested IO range too big, new size set to 64K\n"); - } - - /* add the range to the list */ - range = kzalloc(sizeof(*range), GFP_ATOMIC); - if (!range) { - err = -ENOMEM; - goto end_register; - } - - range->start = addr; - range->size = size; - - list_add_tail(&range->list, &io_range_list); - -end_register: - spin_unlock(&io_range_lock); -#endif - - return err; -} - -phys_addr_t pci_pio_to_address(unsigned long pio) -{ - phys_addr_t address = (phys_addr_t)OF_BAD_ADDR; - -#ifdef PCI_IOBASE - struct io_range *range; - resource_size_t allocated_size = 0; - - if (pio > IO_SPACE_LIMIT) - return address; - - spin_lock(&io_range_lock); - list_for_each_entry(range, &io_range_list, list) { - if (pio >= allocated_size && pio < allocated_size + range->size) { - address = range->start + pio - allocated_size; - break; - } - allocated_size += range->size; - } - spin_unlock(&io_range_lock); -#endif - - return address; -} - -unsigned long __weak pci_address_to_pio(phys_addr_t address) -{ -#ifdef PCI_IOBASE - struct io_range *res; - resource_size_t offset = 0; - unsigned long addr = -1; - - spin_lock(&io_range_lock); - list_for_each_entry(res, &io_range_list, list) { - if (address >= res->start && address < res->start + res->size) { - addr = address - res->start + offset; - break; - } - offset += res->size; - } - spin_unlock(&io_range_lock); - - return addr; -#else - if (address > IO_SPACE_LIMIT) - return (unsigned long)-1; - - return (unsigned long) address; -#endif -} - static int __of_address_to_resource(struct device_node *dev, const __be32 *addrp, u64 size, unsigned int flags, const char *name, struct resource *r) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 25e0327..bc0c914 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -3021,6 +3021,121 @@ int pci_request_regions_exclusive(struct pci_dev *pdev, const char *res_name) } EXPORT_SYMBOL(pci_request_regions_exclusive);
+#ifdef PCI_IOBASE +struct io_range { + struct list_head list; + phys_addr_t start; + resource_size_t size; +}; + +static LIST_HEAD(io_range_list); +static DEFINE_SPINLOCK(io_range_lock); +#endif + +/* + * Record the PCI IO range (expressed as CPU physical address + size). + * Return a negative value if an error has occured, zero otherwise + */ +int __weak pci_register_io_range(phys_addr_t addr, resource_size_t size) +{ + int err = 0; + +#ifdef PCI_IOBASE + struct io_range *range; + resource_size_t allocated_size = 0; + + /* check if the range hasn't been previously recorded */ + spin_lock(&io_range_lock); + list_for_each_entry(range, &io_range_list, list) { + if (addr >= range->start && addr + size <= range->start + size) { + /* range already registered, bail out */ + goto end_register; + } + allocated_size += range->size; + } + + /* range not registed yet, check for available space */ + if (allocated_size + size - 1 > IO_SPACE_LIMIT) { + /* if it's too big check if 64K space can be reserved */ + if (allocated_size + SZ_64K - 1 > IO_SPACE_LIMIT) { + err = -E2BIG; + goto end_register; + } + + size = SZ_64K; + pr_warn("Requested IO range too big, new size set to 64K\n"); + } + + /* add the range to the list */ + range = kzalloc(sizeof(*range), GFP_ATOMIC); + if (!range) { + err = -ENOMEM; + goto end_register; + } + + range->start = addr; + range->size = size; + + list_add_tail(&range->list, &io_range_list); + +end_register: + spin_unlock(&io_range_lock); +#endif + + return err; +} + +phys_addr_t pci_pio_to_address(unsigned long pio) +{ + phys_addr_t address = (phys_addr_t)OF_BAD_ADDR; + +#ifdef PCI_IOBASE + struct io_range *range; + resource_size_t allocated_size = 0; + + if (pio > IO_SPACE_LIMIT) + return address; + + spin_lock(&io_range_lock); + list_for_each_entry(range, &io_range_list, list) { + if (pio >= allocated_size && pio < allocated_size + range->size) { + address = range->start + pio - allocated_size; + break; + } + allocated_size += range->size; + } + spin_unlock(&io_range_lock); +#endif + + return address; +} + +unsigned long __weak pci_address_to_pio(phys_addr_t address) +{ +#ifdef PCI_IOBASE + struct io_range *res; + resource_size_t offset = 0; + unsigned long addr = -1; + + spin_lock(&io_range_lock); + list_for_each_entry(res, &io_range_list, list) { + if (address >= res->start && address < res->start + res->size) { + addr = address - res->start + offset; + break; + } + offset += res->size; + } + spin_unlock(&io_range_lock); + + return addr; +#else + if (address > IO_SPACE_LIMIT) + return (unsigned long)-1; + + return (unsigned long) address; +#endif +} + /** * pci_remap_iospace - Remap the memory mapped I/O space * @res: Resource describing the I/O space diff --git a/include/linux/of_address.h b/include/linux/of_address.h index 01c0a55..3786473 100644 --- a/include/linux/of_address.h +++ b/include/linux/of_address.h @@ -47,10 +47,6 @@ void __iomem *of_io_request_and_map(struct device_node *device, extern const __be32 *of_get_address(struct device_node *dev, int index, u64 *size, unsigned int *flags);
-extern int pci_register_io_range(phys_addr_t addr, resource_size_t size); -extern unsigned long pci_address_to_pio(phys_addr_t addr); -extern phys_addr_t pci_pio_to_address(unsigned long pio); - extern int of_pci_range_parser_init(struct of_pci_range_parser *parser, struct device_node *node); extern struct of_pci_range *of_pci_range_parser_one( @@ -86,11 +82,6 @@ static inline const __be32 *of_get_address(struct device_node *dev, int index, return NULL; }
-static inline phys_addr_t pci_pio_to_address(unsigned long pio) -{ - return 0; -} - static inline int of_pci_range_parser_init(struct of_pci_range_parser *parser, struct device_node *node) { diff --git a/include/linux/pci.h b/include/linux/pci.h index 932ec74..d2a57f3 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1165,6 +1165,9 @@ int __must_check pci_bus_alloc_resource(struct pci_bus *bus, void *alignf_data);
+int pci_register_io_range(phys_addr_t addr, resource_size_t size); +unsigned long pci_address_to_pio(phys_addr_t addr); +phys_addr_t pci_pio_to_address(unsigned long pio); int pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr);
static inline pci_bus_addr_t pci_bus_address(struct pci_dev *pdev, int bar) @@ -1481,6 +1484,8 @@ static inline int pci_request_regions(struct pci_dev *dev, const char *res_name) { return -EIO; } static inline void pci_release_regions(struct pci_dev *dev) { }
+static inline unsigned long pci_address_to_pio(phys_addr_t addr) { return -1; } + static inline void pci_block_cfg_access(struct pci_dev *dev) { } static inline int pci_block_cfg_access_in_atomic(struct pci_dev *dev) { return 0; }
On Tue, May 10, 2016 at 5:19 PM, Tomasz Nowicki tn@semihalf.com wrote:
No functional changes in this patch.
PCI I/O space mapping code does not depend on OF, therefore it can be moved to PCI core code. This way we will be able to use it e.g. in ACPI PCI code.
Suggested-by: Lorenzo Pieralisi lorenzo.pieralisi@arm.com Signed-off-by: Tomasz Nowicki tn@semihalf.com CC: Arnd Bergmann arnd@arndb.de CC: Liviu Dudau Liviu.Dudau@arm.com
drivers/of/address.c | 116 +-------------------------------------------- drivers/pci/pci.c | 115 ++++++++++++++++++++++++++++++++++++++++++++ include/linux/of_address.h | 9 ---- include/linux/pci.h | 5 ++ 4 files changed, 121 insertions(+), 124 deletions(-)
diff --git a/drivers/of/address.c b/drivers/of/address.c index 91a469d..0a553c0 100644 --- a/drivers/of/address.c +++ b/drivers/of/address.c @@ -4,6 +4,7 @@ #include <linux/ioport.h> #include <linux/module.h> #include <linux/of_address.h> +#include <linux/pci.h> #include <linux/pci_regs.h> #include <linux/sizes.h> #include <linux/slab.h> @@ -673,121 +674,6 @@ const __be32 *of_get_address(struct device_node *dev, int index, u64 *size, } EXPORT_SYMBOL(of_get_address);
-#ifdef PCI_IOBASE -struct io_range {
struct list_head list;phys_addr_t start;resource_size_t size;-};
-static LIST_HEAD(io_range_list); -static DEFINE_SPINLOCK(io_range_lock); -#endif
-/*
- Record the PCI IO range (expressed as CPU physical address + size).
- Return a negative value if an error has occured, zero otherwise
- */
-int __weak pci_register_io_range(phys_addr_t addr, resource_size_t size) -{
int err = 0;-#ifdef PCI_IOBASE
struct io_range *range;resource_size_t allocated_size = 0;/* check if the range hasn't been previously recorded */spin_lock(&io_range_lock);list_for_each_entry(range, &io_range_list, list) {if (addr >= range->start && addr + size <= range->start + size) {/* range already registered, bail out */goto end_register;}allocated_size += range->size;}/* range not registed yet, check for available space */if (allocated_size + size - 1 > IO_SPACE_LIMIT) {/* if it's too big check if 64K space can be reserved */if (allocated_size + SZ_64K - 1 > IO_SPACE_LIMIT) {err = -E2BIG;goto end_register;}size = SZ_64K;pr_warn("Requested IO range too big, new size set to 64K\n");}/* add the range to the list */range = kzalloc(sizeof(*range), GFP_ATOMIC);if (!range) {err = -ENOMEM;goto end_register;}range->start = addr;range->size = size;list_add_tail(&range->list, &io_range_list);-end_register:
spin_unlock(&io_range_lock);-#endif
return err;-}
-phys_addr_t pci_pio_to_address(unsigned long pio) -{
phys_addr_t address = (phys_addr_t)OF_BAD_ADDR;-#ifdef PCI_IOBASE
struct io_range *range;resource_size_t allocated_size = 0;if (pio > IO_SPACE_LIMIT)return address;spin_lock(&io_range_lock);list_for_each_entry(range, &io_range_list, list) {if (pio >= allocated_size && pio < allocated_size + range->size) {address = range->start + pio - allocated_size;break;}allocated_size += range->size;}spin_unlock(&io_range_lock);-#endif
return address;-}
-unsigned long __weak pci_address_to_pio(phys_addr_t address) -{ -#ifdef PCI_IOBASE
struct io_range *res;resource_size_t offset = 0;unsigned long addr = -1;spin_lock(&io_range_lock);list_for_each_entry(res, &io_range_list, list) {if (address >= res->start && address < res->start + res->size) {addr = address - res->start + offset;break;}offset += res->size;}spin_unlock(&io_range_lock);return addr;-#else
if (address > IO_SPACE_LIMIT)return (unsigned long)-1;return (unsigned long) address;-#endif -}
static int __of_address_to_resource(struct device_node *dev, const __be32 *addrp, u64 size, unsigned int flags, const char *name, struct resource *r) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 25e0327..bc0c914 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -3021,6 +3021,121 @@ int pci_request_regions_exclusive(struct pci_dev *pdev, const char *res_name) } EXPORT_SYMBOL(pci_request_regions_exclusive);
+#ifdef PCI_IOBASE +struct io_range {
struct list_head list;phys_addr_t start;resource_size_t size;+};
+static LIST_HEAD(io_range_list); +static DEFINE_SPINLOCK(io_range_lock); +#endif
+/*
- Record the PCI IO range (expressed as CPU physical address + size).
- Return a negative value if an error has occured, zero otherwise
- */
+int __weak pci_register_io_range(phys_addr_t addr, resource_size_t size) +{
int err = 0;+#ifdef PCI_IOBASE
I understand that this moves code around, but those in-function #ifdefs aren't nice. Any chance to get rid of them but putting whole functions under the #ifdef?
On 10.05.2016 19:59, Rafael J. Wysocki wrote:
On Tue, May 10, 2016 at 5:19 PM, Tomasz Nowicki tn@semihalf.com wrote:
No functional changes in this patch.
PCI I/O space mapping code does not depend on OF, therefore it can be moved to PCI core code. This way we will be able to use it e.g. in ACPI PCI code.
Suggested-by: Lorenzo Pieralisi lorenzo.pieralisi@arm.com Signed-off-by: Tomasz Nowicki tn@semihalf.com CC: Arnd Bergmann arnd@arndb.de CC: Liviu Dudau Liviu.Dudau@arm.com
drivers/of/address.c | 116 +-------------------------------------------- drivers/pci/pci.c | 115 ++++++++++++++++++++++++++++++++++++++++++++ include/linux/of_address.h | 9 ---- include/linux/pci.h | 5 ++ 4 files changed, 121 insertions(+), 124 deletions(-)
diff --git a/drivers/of/address.c b/drivers/of/address.c index 91a469d..0a553c0 100644 --- a/drivers/of/address.c +++ b/drivers/of/address.c @@ -4,6 +4,7 @@ #include <linux/ioport.h> #include <linux/module.h> #include <linux/of_address.h> +#include <linux/pci.h> #include <linux/pci_regs.h> #include <linux/sizes.h> #include <linux/slab.h> @@ -673,121 +674,6 @@ const __be32 *of_get_address(struct device_node *dev, int index, u64 *size, } EXPORT_SYMBOL(of_get_address);
-#ifdef PCI_IOBASE -struct io_range {
struct list_head list;phys_addr_t start;resource_size_t size;-};
-static LIST_HEAD(io_range_list); -static DEFINE_SPINLOCK(io_range_lock); -#endif
-/*
- Record the PCI IO range (expressed as CPU physical address + size).
- Return a negative value if an error has occured, zero otherwise
- */
-int __weak pci_register_io_range(phys_addr_t addr, resource_size_t size) -{
int err = 0;-#ifdef PCI_IOBASE
struct io_range *range;resource_size_t allocated_size = 0;/* check if the range hasn't been previously recorded */spin_lock(&io_range_lock);list_for_each_entry(range, &io_range_list, list) {if (addr >= range->start && addr + size <= range->start + size) {/* range already registered, bail out */goto end_register;}allocated_size += range->size;}/* range not registed yet, check for available space */if (allocated_size + size - 1 > IO_SPACE_LIMIT) {/* if it's too big check if 64K space can be reserved */if (allocated_size + SZ_64K - 1 > IO_SPACE_LIMIT) {err = -E2BIG;goto end_register;}size = SZ_64K;pr_warn("Requested IO range too big, new size set to 64K\n");}/* add the range to the list */range = kzalloc(sizeof(*range), GFP_ATOMIC);if (!range) {err = -ENOMEM;goto end_register;}range->start = addr;range->size = size;list_add_tail(&range->list, &io_range_list);-end_register:
spin_unlock(&io_range_lock);-#endif
return err;-}
-phys_addr_t pci_pio_to_address(unsigned long pio) -{
phys_addr_t address = (phys_addr_t)OF_BAD_ADDR;-#ifdef PCI_IOBASE
struct io_range *range;resource_size_t allocated_size = 0;if (pio > IO_SPACE_LIMIT)return address;spin_lock(&io_range_lock);list_for_each_entry(range, &io_range_list, list) {if (pio >= allocated_size && pio < allocated_size + range->size) {address = range->start + pio - allocated_size;break;}allocated_size += range->size;}spin_unlock(&io_range_lock);-#endif
return address;-}
-unsigned long __weak pci_address_to_pio(phys_addr_t address) -{ -#ifdef PCI_IOBASE
struct io_range *res;resource_size_t offset = 0;unsigned long addr = -1;spin_lock(&io_range_lock);list_for_each_entry(res, &io_range_list, list) {if (address >= res->start && address < res->start + res->size) {addr = address - res->start + offset;break;}offset += res->size;}spin_unlock(&io_range_lock);return addr;-#else
if (address > IO_SPACE_LIMIT)return (unsigned long)-1;return (unsigned long) address;-#endif -}
- static int __of_address_to_resource(struct device_node *dev, const __be32 *addrp, u64 size, unsigned int flags, const char *name, struct resource *r)
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 25e0327..bc0c914 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -3021,6 +3021,121 @@ int pci_request_regions_exclusive(struct pci_dev *pdev, const char *res_name) } EXPORT_SYMBOL(pci_request_regions_exclusive);
+#ifdef PCI_IOBASE +struct io_range {
struct list_head list;phys_addr_t start;resource_size_t size;+};
+static LIST_HEAD(io_range_list); +static DEFINE_SPINLOCK(io_range_lock); +#endif
+/*
- Record the PCI IO range (expressed as CPU physical address + size).
- Return a negative value if an error has occured, zero otherwise
- */
+int __weak pci_register_io_range(phys_addr_t addr, resource_size_t size) +{
int err = 0;+#ifdef PCI_IOBASE
I understand that this moves code around, but those in-function #ifdefs aren't nice. Any chance to get rid of them but putting whole functions under the #ifdef?
This is a __weak implementation, so assuming I would move #ifdef out of function I need to provide another empty __weak stub. I do not know which solution is more ugly. In any case we can do that cleanup separately.
Tomasz
On Wednesday 11 May 2016 09:36:58 Tomasz Nowicki wrote:
I understand that this moves code around, but those in-function #ifdefs aren't nice. Any chance to get rid of them but putting whole functions under the #ifdef?
This is a __weak implementation, so assuming I would move #ifdef out of function I need to provide another empty __weak stub. I do not know which solution is more ugly. In any case we can do that cleanup separately.
I'd vote for just dropping the __weak here, given that there is no non-weak implementation. If we end up needing to override this for some architecture or host bridge in the future, we can think about how to best do that then.
I agree that should be a separate patch, this one should only move code from one file to another.
Arnd
It is very useful to release I/O resources so that the same I/O resources can be allocated again (pci_remap_iospace), like in PCI hotplug removal scenario. Therefore this patch implements new pci_unmap_iospace call which unmaps I/O space as the symmetry to pci_remap_iospace.
Signed-off-by: Sinan Kaya okaya@codeaurora.org Signed-off-by: Tomasz Nowicki tn@semihalf.com --- drivers/pci/pci.c | 24 ++++++++++++++++++++++++ include/linux/pci.h | 1 + 2 files changed, 25 insertions(+)
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index bc0c914..ff97a0b 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -25,6 +25,7 @@ #include <linux/device.h> #include <linux/pm_runtime.h> #include <linux/pci_hotplug.h> +#include <linux/vmalloc.h> #include <asm/setup.h> #include <linux/aer.h> #include "pci.h" @@ -3167,6 +3168,29 @@ int __weak pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr) #endif }
+/** + * pci_unmap_iospace - Unmap the memory mapped I/O space + * @res: resource to be unmapped + * + * Unmap the CPU virtual address @res from virtual address space. + * Only architectures that have memory mapped IO functions defined + * (and the PCI_IOBASE value defined) should call this function. + */ +void pci_unmap_iospace(struct resource *res) +{ +#if defined(PCI_IOBASE) && defined(CONFIG_MMU) + unsigned long vaddr = (unsigned long)PCI_IOBASE + res->start; + + unmap_kernel_range(vaddr, resource_size(res)); +#else + /* + * This architecture does not have memory mapped I/O space, + * so this function should never be called. + */ + WARN_ONCE(1, "This architecture does not support memory mapped I/O\n"); +#endif +} + static void __pci_set_master(struct pci_dev *dev, bool enable) { u16 old_cmd, cmd; diff --git a/include/linux/pci.h b/include/linux/pci.h index d2a57f3..d6ea6ce 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1169,6 +1169,7 @@ int pci_register_io_range(phys_addr_t addr, resource_size_t size); unsigned long pci_address_to_pio(phys_addr_t addr); phys_addr_t pci_pio_to_address(unsigned long pio); int pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr); +void pci_unmap_iospace(struct resource *res);
static inline pci_bus_addr_t pci_bus_address(struct pci_dev *pdev, int bar) {
On Tue, May 10, 2016 at 8:49 PM, Tomasz Nowicki tn@semihalf.com wrote:
It is very useful to release I/O resources so that the same I/O resources can be allocated again (pci_remap_iospace), like in PCI hotplug removal scenario. Therefore this patch implements new pci_unmap_iospace call which unmaps I/O space as the symmetry to pci_remap_iospace.
Signed-off-by: Sinan Kaya okaya@codeaurora.org Signed-off-by: Tomasz Nowicki tn@semihalf.com
drivers/pci/pci.c | 24 ++++++++++++++++++++++++ include/linux/pci.h | 1 + 2 files changed, 25 insertions(+)
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index bc0c914..ff97a0b 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -25,6 +25,7 @@ #include <linux/device.h> #include <linux/pm_runtime.h> #include <linux/pci_hotplug.h> +#include <linux/vmalloc.h> #include <asm/setup.h> #include <linux/aer.h> #include "pci.h" @@ -3167,6 +3168,29 @@ int __weak pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr) #endif }
+/**
pci_unmap_iospace - Unmap the memory mapped I/O space
@res: resource to be unmapped
Unmap the CPU virtual address @res from virtual address space.
Only architectures that have memory mapped IO functions defined
(and the PCI_IOBASE value defined) should call this function.- */
+void pci_unmap_iospace(struct resource *res) +{ +#if defined(PCI_IOBASE) && defined(CONFIG_MMU)
unsigned long vaddr = (unsigned long)PCI_IOBASE + res->start;unmap_kernel_range(vaddr, resource_size(res));+#else
/** This architecture does not have memory mapped I/O space,* so this function should never be called.*/WARN_ONCE(1, "This architecture does not support memory mapped I/O\n");+#endif +}
WARN is not needed here, since we would have already done it in pci_remap_iospace.
Ideally, we should undo the pci_register_io_range as well, but re-registering the same range seems to be fine.
JC.
Platforms that have memory mapped IO port (such as ARM64) need special handling for PCI I/O resources. For host bridge's resource probing case these resources need to be fixed up with pci_register_io_range/pci_remap_iospace etc.
The same I/O resources need to be released after hotplug removal so that it can be re-added back by the pci_remap_iospace function during insertion. As a consequence we unmap I/O resources with pci_unmap_iospace when we release host bridge resources.
Signed-off-by: Jayachandran C jchandra@broadcom.com Signed-off-by: Sinan Kaya okaya@codeaurora.org Signed-off-by: Tomasz Nowicki tn@semihalf.com --- drivers/acpi/pci_root.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+)
diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c index ae3fe4e..cb3071d 100644 --- a/drivers/acpi/pci_root.c +++ b/drivers/acpi/pci_root.c @@ -719,6 +719,34 @@ next: resource_list_add_tail(entry, resources); } } +static void acpi_pci_root_remap_iospace(struct resource_entry *entry) +{ +#ifdef PCI_IOBASE + struct resource *res = entry->res; + resource_size_t cpu_addr = res->start; + resource_size_t pci_addr = cpu_addr - entry->offset; + resource_size_t length = resource_size(res); + unsigned long port; + + if (pci_register_io_range(cpu_addr, length)) + goto err; + + port = pci_address_to_pio(cpu_addr); + if (port == (unsigned long)-1) + goto err; + + res->start = port; + res->end = port + length - 1; + entry->offset = port - pci_addr; + + if (pci_remap_iospace(res, cpu_addr) < 0) + goto err; + pr_info("Remapped I/O %pa to %pR\n", &cpu_addr, res); + return; +err: + res->flags |= IORESOURCE_DISABLED; +#endif +}
int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info) { @@ -740,6 +768,9 @@ int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info) "no IO and memory resources present in _CRS\n"); else { resource_list_for_each_entry_safe(entry, tmp, list) { + if (entry->res->flags & IORESOURCE_IO) + acpi_pci_root_remap_iospace(entry); + if (entry->res->flags & IORESOURCE_DISABLED) resource_list_destroy_entry(entry); else @@ -811,6 +842,8 @@ static void acpi_pci_root_release_info(struct pci_host_bridge *bridge)
resource_list_for_each_entry(entry, &bridge->windows) { res = entry->res; + if (res->flags & IORESOURCE_IO) + pci_unmap_iospace(res); if (res->parent && (res->flags & (IORESOURCE_MEM | IORESOURCE_IO))) release_resource(res);
On Tue, May 10, 2016 at 5:19 PM, Tomasz Nowicki tn@semihalf.com wrote:
Platforms that have memory mapped IO port (such as ARM64) need special handling for PCI I/O resources. For host bridge's resource probing case these resources need to be fixed up with pci_register_io_range/pci_remap_iospace etc.
The same I/O resources need to be released after hotplug removal so that it can be re-added back by the pci_remap_iospace function during insertion. As a consequence we unmap I/O resources with pci_unmap_iospace when we release host bridge resources.
Signed-off-by: Jayachandran C jchandra@broadcom.com Signed-off-by: Sinan Kaya okaya@codeaurora.org Signed-off-by: Tomasz Nowicki tn@semihalf.com
drivers/acpi/pci_root.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+)
diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c index ae3fe4e..cb3071d 100644 --- a/drivers/acpi/pci_root.c +++ b/drivers/acpi/pci_root.c @@ -719,6 +719,34 @@ next: resource_list_add_tail(entry, resources); } } +static void acpi_pci_root_remap_iospace(struct resource_entry *entry) +{ +#ifdef PCI_IOBASE
Same comment about the #ifdefs as in the other patch.
struct resource *res = entry->res;resource_size_t cpu_addr = res->start;resource_size_t pci_addr = cpu_addr - entry->offset;resource_size_t length = resource_size(res);unsigned long port;if (pci_register_io_range(cpu_addr, length))goto err;port = pci_address_to_pio(cpu_addr);if (port == (unsigned long)-1)goto err;res->start = port;res->end = port + length - 1;entry->offset = port - pci_addr;if (pci_remap_iospace(res, cpu_addr) < 0)goto err;
An empty line here?
pr_info("Remapped I/O %pa to %pR\n", &cpu_addr, res);return;+err:
res->flags |= IORESOURCE_DISABLED;+#endif +}
int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info) { @@ -740,6 +768,9 @@ int acpi_pci_probe_root_resources(struct acpi_pci_root_info *info) "no IO and memory resources present in _CRS\n"); else { resource_list_for_each_entry_safe(entry, tmp, list) {
if (entry->res->flags & IORESOURCE_IO)acpi_pci_root_remap_iospace(entry);if (entry->res->flags & IORESOURCE_DISABLED) resource_list_destroy_entry(entry); else@@ -811,6 +842,8 @@ static void acpi_pci_root_release_info(struct pci_host_bridge *bridge)
resource_list_for_each_entry(entry, &bridge->windows) { res = entry->res;
if (res->flags & IORESOURCE_IO)pci_unmap_iospace(res); if (res->parent && (res->flags & (IORESOURCE_MEM | IORESOURCE_IO))) release_resource(res);--
On 10.05.2016 20:20, Rafael J. Wysocki wrote:
On Tue, May 10, 2016 at 5:19 PM, Tomasz Nowicki tn@semihalf.com wrote:
Platforms that have memory mapped IO port (such as ARM64) need special handling for PCI I/O resources. For host bridge's resource probing case these resources need to be fixed up with pci_register_io_range/pci_remap_iospace etc.
The same I/O resources need to be released after hotplug removal so that it can be re-added back by the pci_remap_iospace function during insertion. As a consequence we unmap I/O resources with pci_unmap_iospace when we release host bridge resources.
Signed-off-by: Jayachandran C jchandra@broadcom.com Signed-off-by: Sinan Kaya okaya@codeaurora.org Signed-off-by: Tomasz Nowicki tn@semihalf.com
drivers/acpi/pci_root.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+)
diff --git a/drivers/acpi/pci_root.c b/drivers/acpi/pci_root.c index ae3fe4e..cb3071d 100644 --- a/drivers/acpi/pci_root.c +++ b/drivers/acpi/pci_root.c @@ -719,6 +719,34 @@ next: resource_list_add_tail(entry, resources); } } +static void acpi_pci_root_remap_iospace(struct resource_entry *entry) +{ +#ifdef PCI_IOBASE
Same comment about the #ifdefs as in the other patch.
OK
struct resource *res = entry->res;resource_size_t cpu_addr = res->start;resource_size_t pci_addr = cpu_addr - entry->offset;resource_size_t length = resource_size(res);unsigned long port;if (pci_register_io_range(cpu_addr, length))goto err;port = pci_address_to_pio(cpu_addr);if (port == (unsigned long)-1)goto err;res->start = port;res->end = port + length - 1;entry->offset = port - pci_addr;if (pci_remap_iospace(res, cpu_addr) < 0)goto err;An empty line here?
yes, empty line would be nice here.
Tomasz
This patch provides a way to set the ACPI domain number in PCI code. pci_create_root_bus is called with NULL as parent in ACPI. This ends up calling pci_bus_assign_domain_nr with a NULL parent. So we define acpi_pci_bus_domain_nr() which is meant to retrieve the PCI domain number based on 'struct pci_bus' in the ACPI way. pci_bus_assign_domain_nr() is updated to call acpi_pci_bus_domain_nr() and assign domain number on the root bus, in case of ACPI.
acpi_pci_bus_domain_nr function is stub for now.
While at it, for the sake of code clarity we put ACPI and DT domain assign methods into the corresponding helpers.
Signed-off-by: Jayachandran C jchandra@broadcom.com Signed-off-by: Tomasz Nowicki tn@semihalf.com --- drivers/pci/pci.c | 11 +++++++++-- include/linux/pci-acpi.h | 5 +++++ 2 files changed, 14 insertions(+), 2 deletions(-)
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index ff97a0b..a1d7bcf 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -19,6 +19,7 @@ #include <linux/spinlock.h> #include <linux/string.h> #include <linux/log2.h> +#include <linux/pci-acpi.h> #include <linux/pci-aspm.h> #include <linux/pm_wakeup.h> #include <linux/interrupt.h> @@ -4918,7 +4919,7 @@ int pci_get_new_domain_nr(void) }
#ifdef CONFIG_PCI_DOMAINS_GENERIC -void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent) +static int of_pci_bus_domain_nr(struct device *parent) { static int use_dt_domains = -1; int domain = -1; @@ -4962,7 +4963,13 @@ void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent) domain = -1; }
- bus->domain_nr = domain; + return domain; +} + +void pci_bus_assign_domain_nr(struct pci_bus *bus, struct device *parent) +{ + bus->domain_nr = acpi_disabled ? of_pci_bus_domain_nr(parent) : + acpi_pci_bus_domain_nr(bus); } #endif #endif diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h index 89ab057..09f9f02 100644 --- a/include/linux/pci-acpi.h +++ b/include/linux/pci-acpi.h @@ -111,6 +111,11 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { } static inline void acpi_pci_remove_bus(struct pci_bus *bus) { } #endif /* CONFIG_ACPI */
+static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus) +{ + return 0; +} + #ifdef CONFIG_ACPI_APEI extern bool aer_acpi_firmware_first(void); #else
This patch provides a way to set the ACPI companion in PCI code. We define acpi_pci_set_companion() to set the ACPI companion pointer and call it from PCI core code. The function is stub for now.
Signed-off-by: Jayachandran C jchandra@broadcom.com Signed-off-by: Tomasz Nowicki tn@semihalf.com --- drivers/pci/probe.c | 2 ++ include/linux/pci-acpi.h | 4 ++++ 2 files changed, 6 insertions(+)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 8004f67..fb0b752 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -12,6 +12,7 @@ #include <linux/slab.h> #include <linux/module.h> #include <linux/cpumask.h> +#include <linux/pci-acpi.h> #include <linux/pci-aspm.h> #include <linux/aer.h> #include <linux/acpi.h> @@ -2141,6 +2142,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus, bridge->dev.parent = parent; bridge->dev.release = pci_release_host_bridge_dev; dev_set_name(&bridge->dev, "pci%04x:%02x", pci_domain_nr(b), bus); + acpi_pci_set_companion(bridge); error = pcibios_root_bridge_prepare(bridge); if (error) { kfree(bridge); diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h index 09f9f02..1baa515 100644 --- a/include/linux/pci-acpi.h +++ b/include/linux/pci-acpi.h @@ -111,6 +111,10 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { } static inline void acpi_pci_remove_bus(struct pci_bus *bus) { } #endif /* CONFIG_ACPI */
+static inline void acpi_pci_set_companion(struct pci_host_bridge *bridge) +{ +} + static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus) { return 0;
On Tue, May 10, 2016 at 5:19 PM, Tomasz Nowicki tn@semihalf.com wrote:
This patch provides a way to set the ACPI companion in PCI code. We define acpi_pci_set_companion() to set the ACPI companion pointer and call it from PCI core code. The function is stub for now.
Signed-off-by: Jayachandran C jchandra@broadcom.com Signed-off-by: Tomasz Nowicki tn@semihalf.com
drivers/pci/probe.c | 2 ++ include/linux/pci-acpi.h | 4 ++++ 2 files changed, 6 insertions(+)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 8004f67..fb0b752 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -12,6 +12,7 @@ #include <linux/slab.h> #include <linux/module.h> #include <linux/cpumask.h> +#include <linux/pci-acpi.h> #include <linux/pci-aspm.h> #include <linux/aer.h> #include <linux/acpi.h> @@ -2141,6 +2142,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus, bridge->dev.parent = parent; bridge->dev.release = pci_release_host_bridge_dev; dev_set_name(&bridge->dev, "pci%04x:%02x", pci_domain_nr(b), bus);
acpi_pci_set_companion(bridge);
Yes, we'll probably add something similar here.
Do I think now is the right time to do that? No.
error = pcibios_root_bridge_prepare(bridge); if (error) { kfree(bridge);diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h index 09f9f02..1baa515 100644 --- a/include/linux/pci-acpi.h +++ b/include/linux/pci-acpi.h @@ -111,6 +111,10 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { } static inline void acpi_pci_remove_bus(struct pci_bus *bus) { } #endif /* CONFIG_ACPI */
+static inline void acpi_pci_set_companion(struct pci_host_bridge *bridge) +{ +}
static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus) { return 0; --
Honestly, to me it looks like this series is trying very hard to avoid doing any PCI host bridge configuration stuff from arch/arm64/ although (a) that might be simpler and (b) it would allow us to identify the code that's common between *all* architectures using ACPI support for host bridge configuration and to move *that* to a common place later. As done here it seems to be following the "ARM64 is generic and the rest of the world is special" line which isn't really helpful.
On Tue, May 10, 2016 at 8:37 PM, Rafael J. Wysocki rafael@kernel.org wrote:
On Tue, May 10, 2016 at 5:19 PM, Tomasz Nowicki tn@semihalf.com wrote:
This patch provides a way to set the ACPI companion in PCI code. We define acpi_pci_set_companion() to set the ACPI companion pointer and call it from PCI core code. The function is stub for now.
Signed-off-by: Jayachandran C jchandra@broadcom.com Signed-off-by: Tomasz Nowicki tn@semihalf.com
drivers/pci/probe.c | 2 ++ include/linux/pci-acpi.h | 4 ++++ 2 files changed, 6 insertions(+)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 8004f67..fb0b752 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -12,6 +12,7 @@ #include <linux/slab.h> #include <linux/module.h> #include <linux/cpumask.h> +#include <linux/pci-acpi.h> #include <linux/pci-aspm.h> #include <linux/aer.h> #include <linux/acpi.h> @@ -2141,6 +2142,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus, bridge->dev.parent = parent; bridge->dev.release = pci_release_host_bridge_dev; dev_set_name(&bridge->dev, "pci%04x:%02x", pci_domain_nr(b), bus);
acpi_pci_set_companion(bridge);Yes, we'll probably add something similar here.
Do I think now is the right time to do that? No.
error = pcibios_root_bridge_prepare(bridge); if (error) { kfree(bridge);diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h index 09f9f02..1baa515 100644 --- a/include/linux/pci-acpi.h +++ b/include/linux/pci-acpi.h @@ -111,6 +111,10 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { } static inline void acpi_pci_remove_bus(struct pci_bus *bus) { } #endif /* CONFIG_ACPI */
+static inline void acpi_pci_set_companion(struct pci_host_bridge *bridge) +{ +}
static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus) { return 0; --
Honestly, to me it looks like this series is trying very hard to avoid doing any PCI host bridge configuration stuff from arch/arm64/ although (a) that might be simpler and (b) it would allow us to identify the code that's common between *all* architectures using ACPI support for host bridge configuration and to move *that* to a common place later. As done here it seems to be following the "ARM64 is generic and the rest of the world is special" line which isn't really helpful.
Speaking of which, at least one of the reasons why the ACPI PCI host bridge thing on x86 and ia64 went to the arch code was to avoid explicit references to ACPI-specific data types and related #ifdeffery in the generic PCI code and data structures. If you are going to add those references now anyway, that reason is not relevant any more and all of that can just be reworked to refer to ACPI explicitly.
On Tue, May 10, 2016 at 08:37:00PM +0200, Rafael J. Wysocki wrote:
On Tue, May 10, 2016 at 5:19 PM, Tomasz Nowicki tn@semihalf.com wrote:
This patch provides a way to set the ACPI companion in PCI code. We define acpi_pci_set_companion() to set the ACPI companion pointer and call it from PCI core code. The function is stub for now.
Signed-off-by: Jayachandran C jchandra@broadcom.com Signed-off-by: Tomasz Nowicki tn@semihalf.com
drivers/pci/probe.c | 2 ++ include/linux/pci-acpi.h | 4 ++++ 2 files changed, 6 insertions(+)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 8004f67..fb0b752 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -12,6 +12,7 @@ #include <linux/slab.h> #include <linux/module.h> #include <linux/cpumask.h> +#include <linux/pci-acpi.h> #include <linux/pci-aspm.h> #include <linux/aer.h> #include <linux/acpi.h> @@ -2141,6 +2142,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus, bridge->dev.parent = parent; bridge->dev.release = pci_release_host_bridge_dev; dev_set_name(&bridge->dev, "pci%04x:%02x", pci_domain_nr(b), bus);
acpi_pci_set_companion(bridge);Yes, we'll probably add something similar here.
Do I think now is the right time to do that? No.
error = pcibios_root_bridge_prepare(bridge); if (error) { kfree(bridge);diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h index 09f9f02..1baa515 100644 --- a/include/linux/pci-acpi.h +++ b/include/linux/pci-acpi.h @@ -111,6 +111,10 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { } static inline void acpi_pci_remove_bus(struct pci_bus *bus) { } #endif /* CONFIG_ACPI */
+static inline void acpi_pci_set_companion(struct pci_host_bridge *bridge) +{ +}
static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus) { return 0; --
Honestly, to me it looks like this series is trying very hard to avoid doing any PCI host bridge configuration stuff from arch/arm64/ although (a) that might be simpler and (b) it would allow us to identify the code that's common between *all* architectures using ACPI support for host bridge configuration and to move *that* to a common place later. As done here it seems to be following the "ARM64 is generic and the rest of the world is special" line which isn't really helpful.
I think patch [1-2] should be merged regardless (they may require minor tweaks if we decide to move pci_acpi_scan_root() to arch/arm64 though, for include files location). I guess you are referring to patch 8 in your comments above, which boils down to deciding whether:
- pci_acpi_scan_root() (and unfortunately all the MCFG/ECAM handling that goes with it) should live in arch/arm64 or drivers/acpi
acpi_pci_bus_domain_nr() is a bit more problematic since it is meant to be called from PCI core code (ARM64 selects PCI_DOMAINS_GENERIC for DT and same kernel has to work with OF and ACPI selected) and it is arch specific (because what we have in bus->sysdata is arch specific, waiting for the domain number to be embedded in struct pci_host_bridge).
Your point is fair, I am not sure that moving the pci_acpi_scan_root() to arch/arm64 would make things much simpler though, it is just a matter of deciding where that code has to live.
How do you want us to proceed ?
Thanks, Lorenzo
On Wed, May 11, 2016 at 12:11 PM, Lorenzo Pieralisi lorenzo.pieralisi@arm.com wrote:
On Tue, May 10, 2016 at 08:37:00PM +0200, Rafael J. Wysocki wrote:
On Tue, May 10, 2016 at 5:19 PM, Tomasz Nowicki tn@semihalf.com wrote:
This patch provides a way to set the ACPI companion in PCI code. We define acpi_pci_set_companion() to set the ACPI companion pointer and call it from PCI core code. The function is stub for now.
Signed-off-by: Jayachandran C jchandra@broadcom.com Signed-off-by: Tomasz Nowicki tn@semihalf.com
drivers/pci/probe.c | 2 ++ include/linux/pci-acpi.h | 4 ++++ 2 files changed, 6 insertions(+)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 8004f67..fb0b752 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -12,6 +12,7 @@ #include <linux/slab.h> #include <linux/module.h> #include <linux/cpumask.h> +#include <linux/pci-acpi.h> #include <linux/pci-aspm.h> #include <linux/aer.h> #include <linux/acpi.h> @@ -2141,6 +2142,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus, bridge->dev.parent = parent; bridge->dev.release = pci_release_host_bridge_dev; dev_set_name(&bridge->dev, "pci%04x:%02x", pci_domain_nr(b), bus);
acpi_pci_set_companion(bridge);Yes, we'll probably add something similar here.
Do I think now is the right time to do that? No.
error = pcibios_root_bridge_prepare(bridge); if (error) { kfree(bridge);diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h index 09f9f02..1baa515 100644 --- a/include/linux/pci-acpi.h +++ b/include/linux/pci-acpi.h @@ -111,6 +111,10 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { } static inline void acpi_pci_remove_bus(struct pci_bus *bus) { } #endif /* CONFIG_ACPI */
+static inline void acpi_pci_set_companion(struct pci_host_bridge *bridge) +{ +}
static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus) { return 0; --
Honestly, to me it looks like this series is trying very hard to avoid doing any PCI host bridge configuration stuff from arch/arm64/ although (a) that might be simpler and (b) it would allow us to identify the code that's common between *all* architectures using ACPI support for host bridge configuration and to move *that* to a common place later. As done here it seems to be following the "ARM64 is generic and the rest of the world is special" line which isn't really helpful.
I think patch [1-2] should be merged regardless (they may require minor tweaks if we decide to move pci_acpi_scan_root() to arch/arm64 though, for include files location). I guess you are referring to patch 8 in your comments above, which boils down to deciding whether:
- pci_acpi_scan_root() (and unfortunately all the MCFG/ECAM handling that goes with it) should live in arch/arm64 or drivers/acpi
To be precise, everything under #ifdef CONFIG_ACPI_PCI_HOST_GENERIC or equivalent is de facto ARM64-specific, because (as it stands in the patch series) ARM64 is the only architecture that will select that option. Unless you are aware of any more architectures planning to use ACPI (and I'm not aware of any), it will stay the only architecture selecting it in the foreseeable future.
Therefore you could replace CONFIG_ACPI_PCI_HOST_GENERIC with CONFIG_ARM64 everywhere in that code which is why in my opinion the code should live somewhere under arch/arm64/.
Going forward, it should be possible to identify common parts of the PCI host bridge configuration code in arch/ and move it to drivers/acpi/ or drivers/pci/, but I bet that won't be the entire code this series puts under CONFIG_ACPI_PCI_HOST_GENERIC.
The above leads to a quite straightforward conclusion about the order in which to do things: I'd add ACPI support for PCI host bridge on ARM64 following what's been done on ia64 (as x86 is more quirky and kludgy overall) as far as reasonably possible first and then think about moving common stuff to a common place.
acpi_pci_bus_domain_nr() is a bit more problematic since it is meant to be called from PCI core code (ARM64 selects PCI_DOMAINS_GENERIC for DT and same kernel has to work with OF and ACPI selected) and it is arch specific (because what we have in bus->sysdata is arch specific, waiting for the domain number to be embedded in struct pci_host_bridge).
Your point is fair, I am not sure that moving the pci_acpi_scan_root() to arch/arm64 would make things much simpler though, it is just a matter of deciding where that code has to live.
How do you want us to proceed ?
Pretty much as stated above. :-)
Thanks, Rafael
On Wed, May 11, 2016 at 10:30:51PM +0200, Rafael J. Wysocki wrote:
On Wed, May 11, 2016 at 12:11 PM, Lorenzo Pieralisi lorenzo.pieralisi@arm.com wrote:
On Tue, May 10, 2016 at 08:37:00PM +0200, Rafael J. Wysocki wrote:
On Tue, May 10, 2016 at 5:19 PM, Tomasz Nowicki tn@semihalf.com wrote:
This patch provides a way to set the ACPI companion in PCI code. We define acpi_pci_set_companion() to set the ACPI companion pointer and call it from PCI core code. The function is stub for now.
Signed-off-by: Jayachandran C jchandra@broadcom.com Signed-off-by: Tomasz Nowicki tn@semihalf.com
drivers/pci/probe.c | 2 ++ include/linux/pci-acpi.h | 4 ++++ 2 files changed, 6 insertions(+)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 8004f67..fb0b752 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -12,6 +12,7 @@ #include <linux/slab.h> #include <linux/module.h> #include <linux/cpumask.h> +#include <linux/pci-acpi.h> #include <linux/pci-aspm.h> #include <linux/aer.h> #include <linux/acpi.h> @@ -2141,6 +2142,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus, bridge->dev.parent = parent; bridge->dev.release = pci_release_host_bridge_dev; dev_set_name(&bridge->dev, "pci%04x:%02x", pci_domain_nr(b), bus);
acpi_pci_set_companion(bridge);Yes, we'll probably add something similar here.
Do I think now is the right time to do that? No.
error = pcibios_root_bridge_prepare(bridge); if (error) { kfree(bridge);diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h index 09f9f02..1baa515 100644 --- a/include/linux/pci-acpi.h +++ b/include/linux/pci-acpi.h @@ -111,6 +111,10 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { } static inline void acpi_pci_remove_bus(struct pci_bus *bus) { } #endif /* CONFIG_ACPI */
+static inline void acpi_pci_set_companion(struct pci_host_bridge *bridge) +{ +}
static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus) { return 0; --
Honestly, to me it looks like this series is trying very hard to avoid doing any PCI host bridge configuration stuff from arch/arm64/ although (a) that might be simpler and (b) it would allow us to identify the code that's common between *all* architectures using ACPI support for host bridge configuration and to move *that* to a common place later. As done here it seems to be following the "ARM64 is generic and the rest of the world is special" line which isn't really helpful.
I think patch [1-2] should be merged regardless (they may require minor tweaks if we decide to move pci_acpi_scan_root() to arch/arm64 though, for include files location). I guess you are referring to patch 8 in your comments above, which boils down to deciding whether:
- pci_acpi_scan_root() (and unfortunately all the MCFG/ECAM handling that goes with it) should live in arch/arm64 or drivers/acpi
To be precise, everything under #ifdef CONFIG_ACPI_PCI_HOST_GENERIC or equivalent is de facto ARM64-specific, because (as it stands in the patch series) ARM64 is the only architecture that will select that option. Unless you are aware of any more architectures planning to use ACPI (and I'm not aware of any), it will stay the only architecture selecting it in the foreseeable future.
Therefore you could replace CONFIG_ACPI_PCI_HOST_GENERIC with CONFIG_ARM64 everywhere in that code which is why in my opinion the code should live somewhere under arch/arm64/.
Going forward, it should be possible to identify common parts of the PCI host bridge configuration code in arch/ and move it to drivers/acpi/ or drivers/pci/, but I bet that won't be the entire code this series puts under CONFIG_ACPI_PCI_HOST_GENERIC.
The above leads to a quite straightforward conclusion about the order in which to do things: I'd add ACPI support for PCI host bridge on ARM64 following what's been done on ia64 (as x86 is more quirky and kludgy overall) as far as reasonably possible first and then think about moving common stuff to a common place.
That does seem like a reasonable approach. I had hoped to get more of this in for v4.7, but we don't have much time left. Maybe some of Rafael's comments can be addressed by moving and slight restructuring and we can still squeeze it in.
The first three patches:
PCI: Provide common functions for ECAM mapping PCI: generic, thunder: Use generic ECAM API PCI, of: Move PCI I/O space management to PCI core code
seem relatively straightforward, and I applied them to pci/arm64 with the intent of merging them unless there are objections. I made the following tweaks, mainly to try to improve some error messages:
diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c index 3d52005..e1add01 100644 --- a/drivers/pci/ecam.c +++ b/drivers/pci/ecam.c @@ -24,9 +24,9 @@ #include "ecam.h"
/* - * On 64 bit systems, we do a single ioremap for the whole config space - * since we have enough virtual address range available. On 32 bit, do an - * ioremap per bus. + * On 64-bit systems, we do a single ioremap for the whole config space + * since we have enough virtual address range available. On 32-bit, we + * ioremap the config space for each bus individually. */ static const bool per_bus_mapping = !config_enabled(CONFIG_64BIT);
@@ -42,6 +42,7 @@ struct pci_config_window *pci_ecam_create(struct device *dev, { struct pci_config_window *cfg; unsigned int bus_range, bus_range_max, bsz; + struct resource *conflict; int i, err;
if (busr->start > busr->end) @@ -58,10 +59,10 @@ struct pci_config_window *pci_ecam_create(struct device *dev, bus_range = resource_size(&cfg->busr); bus_range_max = resource_size(cfgres) >> ops->bus_shift; if (bus_range > bus_range_max) { - dev_warn(dev, "bus max %#x reduced to %#x", - bus_range, bus_range_max); bus_range = bus_range_max; cfg->busr.end = busr->start + bus_range - 1; + dev_warn(dev, "ECAM area %pR can only accommodate %pR (reduced from %pR desired)\n", + cfgres, &cfg->busr, busr); } bsz = 1 << ops->bus_shift;
@@ -70,9 +71,11 @@ struct pci_config_window *pci_ecam_create(struct device *dev, cfg->res.flags = IORESOURCE_MEM | IORESOURCE_BUSY; cfg->res.name = "PCI ECAM";
- err = request_resource(&iomem_resource, &cfg->res); - if (err) { - dev_err(dev, "request ECAM res %pR failed\n", &cfg->res); + conflict = request_resource(&iomem_resource, &cfg->res); + if (conflict) { + err = -EBUSY; + dev_err(dev, "can't claim ECAM area %pR: address conflict with %s %pR\n", + &cfg->res, conflict->name, conflict); goto err_exit; }
diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h index 1ad2176..9878beb 100644 --- a/drivers/pci/ecam.h +++ b/drivers/pci/ecam.h @@ -33,7 +33,7 @@ struct pci_ecam_ops {
/* * struct to hold the mappings of a config space window. This - * is expected to be used as sysdata for PCI controlllers which + * is expected to be used as sysdata for PCI controllers that * use ECAM. */ struct pci_config_window { @@ -43,11 +43,11 @@ struct pci_config_window { struct pci_ecam_ops *ops; union { void __iomem *win; /* 64-bit single mapping */ - void __iomem **winp; /* 32-bit per bus mapping */ + void __iomem **winp; /* 32-bit per-bus mapping */ }; };
-/* create and free for pci_config_window */ +/* create and free pci_config_window */ struct pci_config_window *pci_ecam_create(struct device *dev, struct resource *cfgres, struct resource *busr, struct pci_ecam_ops *ops); @@ -56,11 +56,11 @@ void pci_ecam_free(struct pci_config_window *cfg); /* map_bus when ->sysdata is an instance of pci_config_window */ void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn, int where); -/* default ECAM ops, bus shift 20, generic read and write */ +/* default ECAM ops */ extern struct pci_ecam_ops pci_generic_ecam_ops;
#ifdef CONFIG_PCI_HOST_GENERIC -/* for DT based pci controllers that support ECAM */ +/* for DT-based PCI controllers that support ECAM */ int pci_host_common_probe(struct platform_device *pdev, struct pci_ecam_ops *ops); #endif
On Wed, May 11, 2016 at 05:43:14PM -0500, Bjorn Helgaas wrote:
On Wed, May 11, 2016 at 10:30:51PM +0200, Rafael J. Wysocki wrote:
On Wed, May 11, 2016 at 12:11 PM, Lorenzo Pieralisi lorenzo.pieralisi@arm.com wrote:
On Tue, May 10, 2016 at 08:37:00PM +0200, Rafael J. Wysocki wrote:
On Tue, May 10, 2016 at 5:19 PM, Tomasz Nowicki tn@semihalf.com wrote:
This patch provides a way to set the ACPI companion in PCI code. We define acpi_pci_set_companion() to set the ACPI companion pointer and call it from PCI core code. The function is stub for now.
Signed-off-by: Jayachandran C jchandra@broadcom.com Signed-off-by: Tomasz Nowicki tn@semihalf.com
drivers/pci/probe.c | 2 ++ include/linux/pci-acpi.h | 4 ++++ 2 files changed, 6 insertions(+)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 8004f67..fb0b752 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -12,6 +12,7 @@ #include <linux/slab.h> #include <linux/module.h> #include <linux/cpumask.h> +#include <linux/pci-acpi.h> #include <linux/pci-aspm.h> #include <linux/aer.h> #include <linux/acpi.h> @@ -2141,6 +2142,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus, bridge->dev.parent = parent; bridge->dev.release = pci_release_host_bridge_dev; dev_set_name(&bridge->dev, "pci%04x:%02x", pci_domain_nr(b), bus);
acpi_pci_set_companion(bridge);Yes, we'll probably add something similar here.
Do I think now is the right time to do that? No.
error = pcibios_root_bridge_prepare(bridge); if (error) { kfree(bridge);diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h index 09f9f02..1baa515 100644 --- a/include/linux/pci-acpi.h +++ b/include/linux/pci-acpi.h @@ -111,6 +111,10 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { } static inline void acpi_pci_remove_bus(struct pci_bus *bus) { } #endif /* CONFIG_ACPI */
+static inline void acpi_pci_set_companion(struct pci_host_bridge *bridge) +{ +}
static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus) { return 0; --
Honestly, to me it looks like this series is trying very hard to avoid doing any PCI host bridge configuration stuff from arch/arm64/ although (a) that might be simpler and (b) it would allow us to identify the code that's common between *all* architectures using ACPI support for host bridge configuration and to move *that* to a common place later. As done here it seems to be following the "ARM64 is generic and the rest of the world is special" line which isn't really helpful.
I think patch [1-2] should be merged regardless (they may require minor tweaks if we decide to move pci_acpi_scan_root() to arch/arm64 though, for include files location). I guess you are referring to patch 8 in your comments above, which boils down to deciding whether:
- pci_acpi_scan_root() (and unfortunately all the MCFG/ECAM handling that goes with it) should live in arch/arm64 or drivers/acpi
To be precise, everything under #ifdef CONFIG_ACPI_PCI_HOST_GENERIC or equivalent is de facto ARM64-specific, because (as it stands in the patch series) ARM64 is the only architecture that will select that option. Unless you are aware of any more architectures planning to use ACPI (and I'm not aware of any), it will stay the only architecture selecting it in the foreseeable future.
Therefore you could replace CONFIG_ACPI_PCI_HOST_GENERIC with CONFIG_ARM64 everywhere in that code which is why in my opinion the code should live somewhere under arch/arm64/.
Going forward, it should be possible to identify common parts of the PCI host bridge configuration code in arch/ and move it to drivers/acpi/ or drivers/pci/, but I bet that won't be the entire code this series puts under CONFIG_ACPI_PCI_HOST_GENERIC.
The above leads to a quite straightforward conclusion about the order in which to do things: I'd add ACPI support for PCI host bridge on ARM64 following what's been done on ia64 (as x86 is more quirky and kludgy overall) as far as reasonably possible first and then think about moving common stuff to a common place.
That does seem like a reasonable approach. I had hoped to get more of this in for v4.7, but we don't have much time left. Maybe some of Rafael's comments can be addressed by moving and slight restructuring and we can still squeeze it in.
Yes, it seems like a reasonable approach, as long as we accept that part of this series has to live in arch/arm64 otherwise we are going round in circles (because that's the gist of this discussion, to decide where this code has to live, I do not think there is any objection to the code per-se anymore).
I suggest we post a v8 (with code move to arch/arm64) end of merge window (or you prefer seeing patches now to prevent any additional changes later ?), my aim is to get this into -next (whether via arm64 or pci tree it has to be decided) as early as possible for next cycle (-rc1) so that it can get exposure and testing, I do not think that missing the merge window is a big issue if we agree that the code is ready to go.
The first three patches:
PCI: Provide common functions for ECAM mapping PCI: generic, thunder: Use generic ECAM API PCI, of: Move PCI I/O space management to PCI core code
seem relatively straightforward, and I applied them to pci/arm64 with the intent of merging them unless there are objections. I made the following tweaks, mainly to try to improve some error messages:
Ok, thanks a lot !
Lorenzo
diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c index 3d52005..e1add01 100644 --- a/drivers/pci/ecam.c +++ b/drivers/pci/ecam.c @@ -24,9 +24,9 @@ #include "ecam.h" /*
- On 64 bit systems, we do a single ioremap for the whole config space
- since we have enough virtual address range available. On 32 bit, do an
- ioremap per bus.
- On 64-bit systems, we do a single ioremap for the whole config space
- since we have enough virtual address range available. On 32-bit, we
*/
- ioremap the config space for each bus individually.
static const bool per_bus_mapping = !config_enabled(CONFIG_64BIT); @@ -42,6 +42,7 @@ struct pci_config_window *pci_ecam_create(struct device *dev, { struct pci_config_window *cfg; unsigned int bus_range, bus_range_max, bsz;
- struct resource *conflict; int i, err;
if (busr->start > busr->end) @@ -58,10 +59,10 @@ struct pci_config_window *pci_ecam_create(struct device *dev, bus_range = resource_size(&cfg->busr); bus_range_max = resource_size(cfgres) >> ops->bus_shift; if (bus_range > bus_range_max) {
dev_warn(dev, "bus max %#x reduced to %#x",bus_range, bus_range_max);
dev_warn(dev, "ECAM area %pR can only accommodate %pR (reduced from %pR desired)\n",cfgres, &cfg->busr, busr);@@ -70,9 +71,11 @@ struct pci_config_window *pci_ecam_create(struct device *dev, cfg->res.flags = IORESOURCE_MEM | IORESOURCE_BUSY; cfg->res.name = "PCI ECAM";
- err = request_resource(&iomem_resource, &cfg->res);
- if (err) {
dev_err(dev, "request ECAM res %pR failed\n", &cfg->res);
- conflict = request_resource(&iomem_resource, &cfg->res);
- if (conflict) {
err = -EBUSY;dev_err(dev, "can't claim ECAM area %pR: address conflict with %s %pR\n",&cfg->res, conflict->name, conflict);diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h index 1ad2176..9878beb 100644 --- a/drivers/pci/ecam.h +++ b/drivers/pci/ecam.h @@ -33,7 +33,7 @@ struct pci_ecam_ops { /*
- struct to hold the mappings of a config space window. This
- is expected to be used as sysdata for PCI controlllers which
*/
- is expected to be used as sysdata for PCI controllers that
- use ECAM.
struct pci_config_window { @@ -43,11 +43,11 @@ struct pci_config_window { struct pci_ecam_ops *ops; union { void __iomem *win; /* 64-bit single mapping */
void __iomem **winp; /* 32-bit per bus mapping */
void __iomem **winp; /* 32-bit per-bus mapping */}; -/* create and free for pci_config_window */ +/* create and free pci_config_window */ struct pci_config_window *pci_ecam_create(struct device *dev, struct resource *cfgres, struct resource *busr, struct pci_ecam_ops *ops); @@ -56,11 +56,11 @@ void pci_ecam_free(struct pci_config_window *cfg); /* map_bus when ->sysdata is an instance of pci_config_window */ void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn, int where); -/* default ECAM ops, bus shift 20, generic read and write */ +/* default ECAM ops */ extern struct pci_ecam_ops pci_generic_ecam_ops; #ifdef CONFIG_PCI_HOST_GENERIC -/* for DT based pci controllers that support ECAM */ +/* for DT-based PCI controllers that support ECAM */ int pci_host_common_probe(struct platform_device *pdev, struct pci_ecam_ops *ops); #endif
On Thu, May 12, 2016 at 4:13 AM, Bjorn Helgaas helgaas@kernel.org wrote:
On Wed, May 11, 2016 at 10:30:51PM +0200, Rafael J. Wysocki wrote:
On Wed, May 11, 2016 at 12:11 PM, Lorenzo Pieralisi lorenzo.pieralisi@arm.com wrote:
On Tue, May 10, 2016 at 08:37:00PM +0200, Rafael J. Wysocki wrote:
On Tue, May 10, 2016 at 5:19 PM, Tomasz Nowicki tn@semihalf.com wrote:
This patch provides a way to set the ACPI companion in PCI code. We define acpi_pci_set_companion() to set the ACPI companion pointer and call it from PCI core code. The function is stub for now.
Signed-off-by: Jayachandran C jchandra@broadcom.com Signed-off-by: Tomasz Nowicki tn@semihalf.com
drivers/pci/probe.c | 2 ++ include/linux/pci-acpi.h | 4 ++++ 2 files changed, 6 insertions(+)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 8004f67..fb0b752 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -12,6 +12,7 @@ #include <linux/slab.h> #include <linux/module.h> #include <linux/cpumask.h> +#include <linux/pci-acpi.h> #include <linux/pci-aspm.h> #include <linux/aer.h> #include <linux/acpi.h> @@ -2141,6 +2142,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus, bridge->dev.parent = parent; bridge->dev.release = pci_release_host_bridge_dev; dev_set_name(&bridge->dev, "pci%04x:%02x", pci_domain_nr(b), bus);
acpi_pci_set_companion(bridge);Yes, we'll probably add something similar here.
Do I think now is the right time to do that? No.
error = pcibios_root_bridge_prepare(bridge); if (error) { kfree(bridge);diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h index 09f9f02..1baa515 100644 --- a/include/linux/pci-acpi.h +++ b/include/linux/pci-acpi.h @@ -111,6 +111,10 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { } static inline void acpi_pci_remove_bus(struct pci_bus *bus) { } #endif /* CONFIG_ACPI */
+static inline void acpi_pci_set_companion(struct pci_host_bridge *bridge) +{ +}
static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus) { return 0; --
Honestly, to me it looks like this series is trying very hard to avoid doing any PCI host bridge configuration stuff from arch/arm64/ although (a) that might be simpler and (b) it would allow us to identify the code that's common between *all* architectures using ACPI support for host bridge configuration and to move *that* to a common place later. As done here it seems to be following the "ARM64 is generic and the rest of the world is special" line which isn't really helpful.
I think patch [1-2] should be merged regardless (they may require minor tweaks if we decide to move pci_acpi_scan_root() to arch/arm64 though, for include files location). I guess you are referring to patch 8 in your comments above, which boils down to deciding whether:
- pci_acpi_scan_root() (and unfortunately all the MCFG/ECAM handling that goes with it) should live in arch/arm64 or drivers/acpi
To be precise, everything under #ifdef CONFIG_ACPI_PCI_HOST_GENERIC or equivalent is de facto ARM64-specific, because (as it stands in the patch series) ARM64 is the only architecture that will select that option. Unless you are aware of any more architectures planning to use ACPI (and I'm not aware of any), it will stay the only architecture selecting it in the foreseeable future.
Therefore you could replace CONFIG_ACPI_PCI_HOST_GENERIC with CONFIG_ARM64 everywhere in that code which is why in my opinion the code should live somewhere under arch/arm64/.
Going forward, it should be possible to identify common parts of the PCI host bridge configuration code in arch/ and move it to drivers/acpi/ or drivers/pci/, but I bet that won't be the entire code this series puts under CONFIG_ACPI_PCI_HOST_GENERIC.
The above leads to a quite straightforward conclusion about the order in which to do things: I'd add ACPI support for PCI host bridge on ARM64 following what's been done on ia64 (as x86 is more quirky and kludgy overall) as far as reasonably possible first and then think about moving common stuff to a common place.
That does seem like a reasonable approach. I had hoped to get more of this in for v4.7, but we don't have much time left. Maybe some of Rafael's comments can be addressed by moving and slight restructuring and we can still squeeze it in.
The first three patches:
PCI: Provide common functions for ECAM mapping PCI: generic, thunder: Use generic ECAM API PCI, of: Move PCI I/O space management to PCI core code
seem relatively straightforward, and I applied them to pci/arm64 with the intent of merging them unless there are objections. I made the following tweaks, mainly to try to improve some error messages:
diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c index 3d52005..e1add01 100644 --- a/drivers/pci/ecam.c +++ b/drivers/pci/ecam.c @@ -24,9 +24,9 @@ #include "ecam.h"
/*
- On 64 bit systems, we do a single ioremap for the whole config space
- since we have enough virtual address range available. On 32 bit, do an
- ioremap per bus.
- On 64-bit systems, we do a single ioremap for the whole config space
- since we have enough virtual address range available. On 32-bit, we
*/
- ioremap the config space for each bus individually.
static const bool per_bus_mapping = !config_enabled(CONFIG_64BIT);
@@ -42,6 +42,7 @@ struct pci_config_window *pci_ecam_create(struct device *dev, { struct pci_config_window *cfg; unsigned int bus_range, bus_range_max, bsz;
struct resource *conflict; int i, err; if (busr->start > busr->end)@@ -58,10 +59,10 @@ struct pci_config_window *pci_ecam_create(struct device *dev, bus_range = resource_size(&cfg->busr); bus_range_max = resource_size(cfgres) >> ops->bus_shift; if (bus_range > bus_range_max) {
dev_warn(dev, "bus max %#x reduced to %#x",bus_range, bus_range_max); bus_range = bus_range_max; cfg->busr.end = busr->start + bus_range - 1;
dev_warn(dev, "ECAM area %pR can only accommodate %pR (reduced from %pR desired)\n",cfgres, &cfg->busr, busr); } bsz = 1 << ops->bus_shift;@@ -70,9 +71,11 @@ struct pci_config_window *pci_ecam_create(struct device *dev, cfg->res.flags = IORESOURCE_MEM | IORESOURCE_BUSY; cfg->res.name = "PCI ECAM";
err = request_resource(&iomem_resource, &cfg->res);if (err) {dev_err(dev, "request ECAM res %pR failed\n", &cfg->res);
conflict = request_resource(&iomem_resource, &cfg->res);if (conflict) {err = -EBUSY;dev_err(dev, "can't claim ECAM area %pR: address conflict with %s %pR\n",&cfg->res, conflict->name, conflict); goto err_exit; }diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h index 1ad2176..9878beb 100644 --- a/drivers/pci/ecam.h +++ b/drivers/pci/ecam.h @@ -33,7 +33,7 @@ struct pci_ecam_ops {
/*
- struct to hold the mappings of a config space window. This
- is expected to be used as sysdata for PCI controlllers which
*/
- is expected to be used as sysdata for PCI controllers that
- use ECAM.
struct pci_config_window { @@ -43,11 +43,11 @@ struct pci_config_window { struct pci_ecam_ops *ops; union { void __iomem *win; /* 64-bit single mapping */
void __iomem **winp; /* 32-bit per bus mapping */
void __iomem **winp; /* 32-bit per-bus mapping */ };};
-/* create and free for pci_config_window */ +/* create and free pci_config_window */ struct pci_config_window *pci_ecam_create(struct device *dev, struct resource *cfgres, struct resource *busr, struct pci_ecam_ops *ops); @@ -56,11 +56,11 @@ void pci_ecam_free(struct pci_config_window *cfg); /* map_bus when ->sysdata is an instance of pci_config_window */ void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn, int where); -/* default ECAM ops, bus shift 20, generic read and write */ +/* default ECAM ops */ extern struct pci_ecam_ops pci_generic_ecam_ops;
#ifdef CONFIG_PCI_HOST_GENERIC -/* for DT based pci controllers that support ECAM */ +/* for DT-based PCI controllers that support ECAM */ int pci_host_common_probe(struct platform_device *pdev, struct pci_ecam_ops *ops); #endif
If we are moving the ACPI/PCI code from drivers/acpi to arch/arm64/ , there is an issue in having the header file ecam.h in drivers/pci
The current include of "../pci/ecam.h" is slightly ugly (Arnd and David had already noted this), but including the driver header from arch code would be even worse.
I can either merge ecam.h into include/linux/pci.h or move it to a new file include/linux/pci-ecam.h, any suggestion on which is preferable?
JC.
On Thu, May 12, 2016 at 12:43 PM, Jayachandran C jchandra@broadcom.com wrote:
On Thu, May 12, 2016 at 4:13 AM, Bjorn Helgaas helgaas@kernel.org wrote:
On Wed, May 11, 2016 at 10:30:51PM +0200, Rafael J. Wysocki wrote:
On Wed, May 11, 2016 at 12:11 PM, Lorenzo Pieralisi lorenzo.pieralisi@arm.com wrote:
On Tue, May 10, 2016 at 08:37:00PM +0200, Rafael J. Wysocki wrote:
On Tue, May 10, 2016 at 5:19 PM, Tomasz Nowicki tn@semihalf.com wrote:
[cut]
If we are moving the ACPI/PCI code from drivers/acpi to arch/arm64/ , there is an issue in having the header file ecam.h in drivers/pci
The current include of "../pci/ecam.h" is slightly ugly (Arnd and David had already noted this), but including the driver header from arch code would be even worse.
I can either merge ecam.h into include/linux/pci.h or move it to a new file include/linux/pci-ecam.h, any suggestion on which is preferable?
My preference would be pci-ecam.h as we did a similar thing for pci-dma.h, for example, but basically this is up to Bjorn.
On Thu, May 12, 2016 at 01:27:23PM +0200, Rafael J. Wysocki wrote:
On Thu, May 12, 2016 at 12:43 PM, Jayachandran C jchandra@broadcom.com wrote:
On Thu, May 12, 2016 at 4:13 AM, Bjorn Helgaas helgaas@kernel.org wrote:
On Wed, May 11, 2016 at 10:30:51PM +0200, Rafael J. Wysocki wrote:
On Wed, May 11, 2016 at 12:11 PM, Lorenzo Pieralisi lorenzo.pieralisi@arm.com wrote:
On Tue, May 10, 2016 at 08:37:00PM +0200, Rafael J. Wysocki wrote:
On Tue, May 10, 2016 at 5:19 PM, Tomasz Nowicki tn@semihalf.com wrote:
[cut]
If we are moving the ACPI/PCI code from drivers/acpi to arch/arm64/ , there is an issue in having the header file ecam.h in drivers/pci
The current include of "../pci/ecam.h" is slightly ugly (Arnd and David had already noted this), but including the driver header from arch code would be even worse.
I can either merge ecam.h into include/linux/pci.h or move it to a new file include/linux/pci-ecam.h, any suggestion on which is preferable?
My preference would be pci-ecam.h as we did a similar thing for pci-dma.h, for example, but basically this is up to Bjorn.
A word of caution for all interested parties, what we may move to arch/arm64 (if Catalin and Will are ok with that) here is content of drivers/acpi/pci_root_generic.c, not drivers/acpi/pci_mcfg.c (and definitely not the MCFG quirks handling that is coming up next on top of this series).
I just wanted to make sure we understand that MCFG quirks handling like eg:
https://lkml.org/lkml/2016/4/28/790
that is coming up following this series has no chance whatsoever to be handled within arch/arm64, it is just not going to happen.
Maybe I am jumping the gun, I just want to make sure that everyone is aware that moving part of this series to arch/arm64 has implications, (and that's why I said that moving part of this code to arch/arm64 is not as simple as it looks) it may be ok to have an ACPI PCI implementation that is arch/arm64 specific (mostly for IO space and PCI resources assignment handling that unfortunately is not uniform across X86, IA64 and ARM64), but MCFG quirks and related platform code stay out of arch/arm64 I guess we are all aware of that, just wanted to make sure :)
Lorenzo
On 12.05.2016 00:43, Bjorn Helgaas wrote:
On Wed, May 11, 2016 at 10:30:51PM +0200, Rafael J. Wysocki wrote:
On Wed, May 11, 2016 at 12:11 PM, Lorenzo Pieralisi lorenzo.pieralisi@arm.com wrote:
On Tue, May 10, 2016 at 08:37:00PM +0200, Rafael J. Wysocki wrote:
On Tue, May 10, 2016 at 5:19 PM, Tomasz Nowicki tn@semihalf.com wrote:
This patch provides a way to set the ACPI companion in PCI code. We define acpi_pci_set_companion() to set the ACPI companion pointer and call it from PCI core code. The function is stub for now.
Signed-off-by: Jayachandran C jchandra@broadcom.com Signed-off-by: Tomasz Nowicki tn@semihalf.com
drivers/pci/probe.c | 2 ++ include/linux/pci-acpi.h | 4 ++++ 2 files changed, 6 insertions(+)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 8004f67..fb0b752 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -12,6 +12,7 @@ #include <linux/slab.h> #include <linux/module.h> #include <linux/cpumask.h> +#include <linux/pci-acpi.h> #include <linux/pci-aspm.h> #include <linux/aer.h> #include <linux/acpi.h> @@ -2141,6 +2142,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus, bridge->dev.parent = parent; bridge->dev.release = pci_release_host_bridge_dev; dev_set_name(&bridge->dev, "pci%04x:%02x", pci_domain_nr(b), bus);
acpi_pci_set_companion(bridge);Yes, we'll probably add something similar here.
Do I think now is the right time to do that? No.
error = pcibios_root_bridge_prepare(bridge); if (error) { kfree(bridge);diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h index 09f9f02..1baa515 100644 --- a/include/linux/pci-acpi.h +++ b/include/linux/pci-acpi.h @@ -111,6 +111,10 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { } static inline void acpi_pci_remove_bus(struct pci_bus *bus) { } #endif /* CONFIG_ACPI */
+static inline void acpi_pci_set_companion(struct pci_host_bridge *bridge) +{ +}
- static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus) { return 0;
--
Honestly, to me it looks like this series is trying very hard to avoid doing any PCI host bridge configuration stuff from arch/arm64/ although (a) that might be simpler and (b) it would allow us to identify the code that's common between *all* architectures using ACPI support for host bridge configuration and to move *that* to a common place later. As done here it seems to be following the "ARM64 is generic and the rest of the world is special" line which isn't really helpful.
I think patch [1-2] should be merged regardless (they may require minor tweaks if we decide to move pci_acpi_scan_root() to arch/arm64 though, for include files location). I guess you are referring to patch 8 in your comments above, which boils down to deciding whether:
- pci_acpi_scan_root() (and unfortunately all the MCFG/ECAM handling that goes with it) should live in arch/arm64 or drivers/acpi
To be precise, everything under #ifdef CONFIG_ACPI_PCI_HOST_GENERIC or equivalent is de facto ARM64-specific, because (as it stands in the patch series) ARM64 is the only architecture that will select that option. Unless you are aware of any more architectures planning to use ACPI (and I'm not aware of any), it will stay the only architecture selecting it in the foreseeable future.
Therefore you could replace CONFIG_ACPI_PCI_HOST_GENERIC with CONFIG_ARM64 everywhere in that code which is why in my opinion the code should live somewhere under arch/arm64/.
Going forward, it should be possible to identify common parts of the PCI host bridge configuration code in arch/ and move it to drivers/acpi/ or drivers/pci/, but I bet that won't be the entire code this series puts under CONFIG_ACPI_PCI_HOST_GENERIC.
The above leads to a quite straightforward conclusion about the order in which to do things: I'd add ACPI support for PCI host bridge on ARM64 following what's been done on ia64 (as x86 is more quirky and kludgy overall) as far as reasonably possible first and then think about moving common stuff to a common place.
That does seem like a reasonable approach. I had hoped to get more of this in for v4.7, but we don't have much time left. Maybe some of Rafael's comments can be addressed by moving and slight restructuring and we can still squeeze it in.
The first three patches:
PCI: Provide common functions for ECAM mapping PCI: generic, thunder: Use generic ECAM API PCI, of: Move PCI I/O space management to PCI core code
seem relatively straightforward, and I applied them to pci/arm64 with the intent of merging them unless there are objections. I made the following tweaks, mainly to try to improve some error messages:
diff --git a/drivers/pci/ecam.c b/drivers/pci/ecam.c index 3d52005..e1add01 100644 --- a/drivers/pci/ecam.c +++ b/drivers/pci/ecam.c @@ -24,9 +24,9 @@ #include "ecam.h"
/*
- On 64 bit systems, we do a single ioremap for the whole config space
- since we have enough virtual address range available. On 32 bit, do an
- ioremap per bus.
- On 64-bit systems, we do a single ioremap for the whole config space
- since we have enough virtual address range available. On 32-bit, we
*/ static const bool per_bus_mapping = !config_enabled(CONFIG_64BIT);
- ioremap the config space for each bus individually.
@@ -42,6 +42,7 @@ struct pci_config_window *pci_ecam_create(struct device *dev, { struct pci_config_window *cfg; unsigned int bus_range, bus_range_max, bsz;
struct resource *conflict; int i, err;
if (busr->start > busr->end)
@@ -58,10 +59,10 @@ struct pci_config_window *pci_ecam_create(struct device *dev, bus_range = resource_size(&cfg->busr); bus_range_max = resource_size(cfgres) >> ops->bus_shift; if (bus_range > bus_range_max) {
dev_warn(dev, "bus max %#x reduced to %#x",bus_range, bus_range_max);
dev_warn(dev, "ECAM area %pR can only accommodate %pR (reduced from %pR desired)\n",cfgres, &cfg->busr, busr);@@ -70,9 +71,11 @@ struct pci_config_window *pci_ecam_create(struct device *dev, cfg->res.flags = IORESOURCE_MEM | IORESOURCE_BUSY; cfg->res.name = "PCI ECAM";
- err = request_resource(&iomem_resource, &cfg->res);
- if (err) {
dev_err(dev, "request ECAM res %pR failed\n", &cfg->res);
- conflict = request_resource(&iomem_resource, &cfg->res);
We need request_resource_conflict here then: - conflict = request_resource(&iomem_resource, &cfg->res); + conflict = request_resource_conflict(&iomem_resource, &cfg->res);
Thanks, Tomasz
On Thu, May 12, 2016 at 12:50:07PM +0200, Tomasz Nowicki wrote:
On 12.05.2016 00:43, Bjorn Helgaas wrote:
@@ -70,9 +71,11 @@ struct pci_config_window *pci_ecam_create(struct device *dev, cfg->res.flags = IORESOURCE_MEM | IORESOURCE_BUSY; cfg->res.name = "PCI ECAM";
- err = request_resource(&iomem_resource, &cfg->res);
- if (err) {
dev_err(dev, "request ECAM res %pR failed\n", &cfg->res);
- conflict = request_resource(&iomem_resource, &cfg->res);
We need request_resource_conflict here then:
- conflict = request_resource(&iomem_resource, &cfg->res);
- conflict = request_resource_conflict(&iomem_resource, &cfg->res);
Whoops, fixed, thanks!
Hi Tomasz
I used the patchset and added "PATCH V6 11/13 specic quirks", tested on HiSilicon D02 board but met the below problem.
[ 2.614115] [<ffffff80083b13bc>] hisi_pcie_init+0x6c/0x1ec [ 2.619571] [<ffffff80083ab060>] pci_ecam_create+0x130/0x1ec [ 2.625209] [<ffffff80083f3764>] pci_acpi_scan_root+0x160/0x218 [ 2.631096] [<ffffff80083d1f6c>] acpi_pci_root_add+0x36c/0x42c [ 2.636897] [<ffffff80083ce36c>] acpi_bus_attach+0xe4/0x1a8 [ 2.642438] [<ffffff80083ce3d8>] acpi_bus_attach+0x150/0x1a8 [ 2.648066] [<ffffff80083ce3d8>] acpi_bus_attach+0x150/0x1a8 [ 2.653693] [<ffffff80083ce55c>] acpi_bus_scan+0x64/0x74 [ 2.658975] [<ffffff8008ae665c>] acpi_scan_init+0x5c/0x19c [ 2.664429] [<ffffff8008ae6408>] acpi_init+0x280/0x2a4 [ 2.669538] [<ffffff80080829e8>] do_one_initcall+0x8c/0x19c [ 2.675080] [<ffffff8008ac3af8>] kernel_init_freeable+0x14c/0x1ec [ 2.681139] [<ffffff80087a8438>] kernel_init+0x10/0xfc [ 2.686248] [<ffffff8008085e10>] ret_from_fork+0x10/0x40
In hisi_pcie_init, I used "struct acpi_device *device = ACPI_COMPANION(dev);". I found the reason is V7 lack the below code. I added the below code, it worked ok.
[PATCH V6 01/13] pci, acpi, x86, ia64: Move ACPI host bridge device companion assignment to core code. --- a/drivers/acpi/pci_root.c +++ b/drivers/acpi/pci_root.c @@ -564,6 +564,11 @@ static int acpi_pci_root_add(struct acpi_device *device, } }
+ /* + * pci_create_root_bus() needs to detect the parent device type, + * so initialize its companion data accordingly. + */ + ACPI_COMPANION_SET(&device->dev, device);
This code will be upstreamed with the "PATCH V6 11/13 specic quirks" in next time after the patchset is accepted. Right ?
在 2016/5/10 23:19, Tomasz Nowicki 写道:
This patch provides a way to set the ACPI companion in PCI code. We define acpi_pci_set_companion() to set the ACPI companion pointer and call it from PCI core code. The function is stub for now.
Signed-off-by: Jayachandran C jchandra@broadcom.com Signed-off-by: Tomasz Nowicki tn@semihalf.com
drivers/pci/probe.c | 2 ++ include/linux/pci-acpi.h | 4 ++++ 2 files changed, 6 insertions(+)
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 8004f67..fb0b752 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -12,6 +12,7 @@ #include <linux/slab.h> #include <linux/module.h> #include <linux/cpumask.h> +#include <linux/pci-acpi.h> #include <linux/pci-aspm.h> #include <linux/aer.h> #include <linux/acpi.h> @@ -2141,6 +2142,7 @@ struct pci_bus *pci_create_root_bus(struct device *parent, int bus, bridge->dev.parent = parent; bridge->dev.release = pci_release_host_bridge_dev; dev_set_name(&bridge->dev, "pci%04x:%02x", pci_domain_nr(b), bus);
- acpi_pci_set_companion(bridge); error = pcibios_root_bridge_prepare(bridge); if (error) { kfree(bridge);
diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h index 09f9f02..1baa515 100644 --- a/include/linux/pci-acpi.h +++ b/include/linux/pci-acpi.h @@ -111,6 +111,10 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { } static inline void acpi_pci_remove_bus(struct pci_bus *bus) { } #endif /* CONFIG_ACPI */
+static inline void acpi_pci_set_companion(struct pci_host_bridge *bridge) +{ +}
- static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus) { return 0;
On 17.05.2016 05:11, Dongdong Liu wrote:
Hi Tomasz
I used the patchset and added "PATCH V6 11/13 specic quirks", tested on HiSilicon D02 board but met the below problem.
[ 2.614115] [<ffffff80083b13bc>] hisi_pcie_init+0x6c/0x1ec [ 2.619571] [<ffffff80083ab060>] pci_ecam_create+0x130/0x1ec [ 2.625209] [<ffffff80083f3764>] pci_acpi_scan_root+0x160/0x218 [ 2.631096] [<ffffff80083d1f6c>] acpi_pci_root_add+0x36c/0x42c [ 2.636897] [<ffffff80083ce36c>] acpi_bus_attach+0xe4/0x1a8 [ 2.642438] [<ffffff80083ce3d8>] acpi_bus_attach+0x150/0x1a8 [ 2.648066] [<ffffff80083ce3d8>] acpi_bus_attach+0x150/0x1a8 [ 2.653693] [<ffffff80083ce55c>] acpi_bus_scan+0x64/0x74 [ 2.658975] [<ffffff8008ae665c>] acpi_scan_init+0x5c/0x19c [ 2.664429] [<ffffff8008ae6408>] acpi_init+0x280/0x2a4 [ 2.669538] [<ffffff80080829e8>] do_one_initcall+0x8c/0x19c [ 2.675080] [<ffffff8008ac3af8>] kernel_init_freeable+0x14c/0x1ec [ 2.681139] [<ffffff80087a8438>] kernel_init+0x10/0xfc [ 2.686248] [<ffffff8008085e10>] ret_from_fork+0x10/0x40
In hisi_pcie_init, I used "struct acpi_device *device = ACPI_COMPANION(dev);". I found the reason is V7 lack the below code. I added the below code, it worked ok.
[PATCH V6 01/13] pci, acpi, x86, ia64: Move ACPI host bridge device companion assignment to core code. --- a/drivers/acpi/pci_root.c +++ b/drivers/acpi/pci_root.c @@ -564,6 +564,11 @@ static int acpi_pci_root_add(struct acpi_device *device, } }
- /*
* pci_create_root_bus() needs to detect the parent device type,* so initialize its companion data accordingly.*/- ACPI_COMPANION_SET(&device->dev, device);
This code will be upstreamed with the "PATCH V6 11/13 specic quirks" in next time after the patchset is accepted. Right ?
We had that patch in previous series to retrieve PCI domain nicely. But that has bad implication to userspace. See: https://lkml.org/lkml/2016/5/9/918
I understand that: [PATCH V6 01/13] pci, acpi, x86, ia64: Move ACPI host bridge device companion assignment to core code. helps to get firmware specific info in hisi_pcie_init but we need to figure out something better for quirk handling too.
Tomasz
This patch is going to implement generic PCI host controller for ACPI world, similar to what pci-host-generic.c driver does for DT world.
All such drivers, which we have seen so far, were implemented within arch/ directory since they had some arch assumptions (x86 and ia64). However, they all are doing similar thing, so it makes sense to find some common code and abstract it into the generic driver.
In order to handle PCI config space regions properly, we define new MCFG interface which does sanity checks on MCFG table and keeps its root pointer. User is able to lookup MCFG regions based on that root pointer and specified domain:bus_start:bus_end touple. We are using pci_mmcfg_late_init old prototype to avoid another function name.
The implementation of pci_acpi_scan_root() looks up the MCFG entries and sets up a new mapping (regions are not mapped until host controller ask for it). Generic PCI functions are used for accessing config space. Driver selects PCI_ECAM and uses functions from drivers/pci/ecam.h to create and access ECAM mappings.
As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice should be made on a per-architecture basis.
Signed-off-by: Tomasz Nowicki tn@semihalf.com Signed-off-by: Jayachandran C jchandra@broadcom.com --- drivers/acpi/Kconfig | 8 +++ drivers/acpi/Makefile | 1 + drivers/acpi/pci_mcfg.c | 97 ++++++++++++++++++++++++++ drivers/acpi/pci_root_generic.c | 149 ++++++++++++++++++++++++++++++++++++++++ drivers/pci/ecam.h | 5 ++ include/linux/pci-acpi.h | 5 ++ include/linux/pci.h | 5 +- 7 files changed, 269 insertions(+), 1 deletion(-) create mode 100644 drivers/acpi/pci_mcfg.c create mode 100644 drivers/acpi/pci_root_generic.c
diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig index 183ffa3..44afc76 100644 --- a/drivers/acpi/Kconfig +++ b/drivers/acpi/Kconfig @@ -346,6 +346,14 @@ config ACPI_PCI_SLOT i.e., segment/bus/device/function tuples, with physical slots in the system. If you are unsure, say N.
+config ACPI_PCI_HOST_GENERIC + bool + select PCI_ECAM + help + Select this config option from the architecture Kconfig, + if it is preferred to enable ACPI PCI host controller driver which + has no arch-specific assumptions. + config X86_PM_TIMER bool "Power Management Timer Support" if EXPERT depends on X86 diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile index 81e5cbc..627a2b7 100644 --- a/drivers/acpi/Makefile +++ b/drivers/acpi/Makefile @@ -40,6 +40,7 @@ acpi-$(CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC) += processor_pdc.o acpi-y += ec.o acpi-$(CONFIG_ACPI_DOCK) += dock.o acpi-y += pci_root.o pci_link.o pci_irq.o +obj-$(CONFIG_ACPI_PCI_HOST_GENERIC) += pci_root_generic.o pci_mcfg.o acpi-y += acpi_lpss.o acpi_apd.o acpi-y += acpi_platform.o acpi-y += acpi_pnp.o diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c new file mode 100644 index 0000000..373d079 --- /dev/null +++ b/drivers/acpi/pci_mcfg.c @@ -0,0 +1,97 @@ +/* + * Copyright (C) 2016 Broadcom + * Author: Jayachandran C jchandra@broadcom.com + * Copyright (C) 2016 Semihalf + * Author: Tomasz Nowicki tn@semihalf.com + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License, version 2, as + * published by the Free Software Foundation (the "GPL"). + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * General Public License version 2 (GPLv2) for more details. + * + * You should have received a copy of the GNU General Public License + * version 2 (GPLv2) along with this source code. + */ +#include <linux/kernel.h> +#include <linux/pci.h> +#include <linux/pci-acpi.h> + +#define PREFIX "ACPI: " + +/* Root pointer to the mapped MCFG table */ +static struct acpi_table_mcfg *mcfg_table; + +#define MCFG_ENTRIES(mcfg_ptr) (((mcfg_ptr)->header.length - \ + sizeof(struct acpi_table_mcfg)) / \ + sizeof(struct acpi_mcfg_allocation)) + +static phys_addr_t pci_mcfg_lookup_static(u16 seg, u8 bus_start, u8 bus_end) +{ + struct acpi_mcfg_allocation *mptr; + int i; + + if (!mcfg_table) { + pr_err(PREFIX "MCFG table not available, lookup failed\n"); + return -ENXIO; + } + + mptr = (struct acpi_mcfg_allocation *) &mcfg_table[1]; + + /* + * We expect exact match, unless MCFG entry end bus covers more than + * specified by caller. + */ + for (i = 0; i < MCFG_ENTRIES(mcfg_table); i++, mptr++) { + if (mptr->pci_segment == seg && + mptr->start_bus_number == bus_start && + mptr->end_bus_number >= bus_end) { + return mptr->address; + } + } + + return -ENXIO; +} + +phys_addr_t pci_mcfg_lookup(struct acpi_device *device, u16 seg, + struct resource *bus_res) +{ + phys_addr_t addr; + + addr = acpi_pci_root_get_mcfg_addr(device->handle); + if (addr) + return addr; + + return pci_mcfg_lookup_static(seg, bus_res->start, bus_res->end); +} + +static __init int pci_mcfg_parse(struct acpi_table_header *header) +{ + struct acpi_table_mcfg *mcfg; + int n; + + if (!header) + return -EINVAL; + + mcfg = (struct acpi_table_mcfg *)header; + n = MCFG_ENTRIES(mcfg); + if (n <= 0 || n > 255) { + pr_err(PREFIX "MCFG has incorrect entries (%d).\n", n); + return -EINVAL; + } + + mcfg_table = mcfg; + pr_info(PREFIX "MCFG table loaded, %d entries detected\n", n); + return 0; +} + +/* Interface called by ACPI - parse and save MCFG table */ +void __init pci_mmcfg_late_init(void) +{ + int err = acpi_table_parse(ACPI_SIG_MCFG, pci_mcfg_parse); + if (err) + pr_err(PREFIX "Failed to parse MCFG (%d)\n", err); +} diff --git a/drivers/acpi/pci_root_generic.c b/drivers/acpi/pci_root_generic.c new file mode 100644 index 0000000..6f4940a --- /dev/null +++ b/drivers/acpi/pci_root_generic.c @@ -0,0 +1,149 @@ +/* + * Copyright (C) 2016 Broadcom + * Author: Jayachandran C jchandra@broadcom.com + * Copyright (C) 2016 Semihalf + * Author: Tomasz Nowicki tn@semihalf.com + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License, version 2, as + * published by the Free Software Foundation (the "GPL"). + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * General Public License version 2 (GPLv2) for more details. + * + * You should have received a copy of the GNU General Public License + * version 2 (GPLv2) along with this source code. + */ +#include <linux/kernel.h> +#include <linux/pci.h> +#include <linux/pci-acpi.h> +#include <linux/slab.h> + +#include "../pci/ecam.h" + +#define PREFIX "ACPI PCI: " + +/* ACPI info for generic ACPI PCI controller */ +struct acpi_pci_generic_root_info { + struct acpi_pci_root_info common; + struct pci_config_window *cfg; /* config space mapping */ +}; + +void acpi_pci_set_companion(struct pci_host_bridge *bridge) +{ + struct pci_config_window *cfg = bridge->bus->sysdata; + + ACPI_COMPANION_SET(&bridge->dev, cfg->companion); +} + +int acpi_pci_bus_domain_nr(struct pci_bus *bus) +{ + struct pci_config_window *cfg = bus->sysdata; + + return cfg->domain; +} + +/* + * Lookup the bus range for the domain in MCFG, and set up config space + * mapping. + */ +static int pci_acpi_setup_ecam_mapping(struct acpi_pci_root *root, + struct acpi_pci_generic_root_info *ri) +{ + struct resource *bus_res = &root->secondary; + u16 seg = root->segment; + struct pci_config_window *cfg; + struct resource cfgres; + unsigned int bsz; + phys_addr_t addr; + + addr = pci_mcfg_lookup(root->device, seg, bus_res); + if (IS_ERR_VALUE(addr)) { + pr_err(PREFIX"%04x:%pR MCFG region not found\n", seg, bus_res); + return addr; + } + + bsz = 1 << pci_generic_ecam_ops.bus_shift; + cfgres.start = addr + bus_res->start * bsz; + cfgres.end = addr + (bus_res->end + 1) * bsz - 1; + cfgres.flags = IORESOURCE_MEM; + cfg = pci_ecam_create(&root->device->dev, &cfgres, bus_res, + &pci_generic_ecam_ops); + if (IS_ERR(cfg)) { + pr_err("%04x:%pR error %ld mapping CAM\n", seg, bus_res, + PTR_ERR(cfg)); + return PTR_ERR(cfg); + } + + cfg->domain = seg; + cfg->companion = root->device; + ri->cfg = cfg; + return 0; +} + +/* release_info: free resrouces allocated by init_info */ +static void pci_acpi_generic_release_info(struct acpi_pci_root_info *ci) +{ + struct acpi_pci_generic_root_info *ri; + + ri = container_of(ci, struct acpi_pci_generic_root_info, common); + pci_ecam_free(ri->cfg); + kfree(ri); +} + +static struct acpi_pci_root_ops acpi_pci_root_ops = { + .release_info = pci_acpi_generic_release_info, +}; + +/* Interface called from ACPI code to setup PCI host controller */ +struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root) +{ + int node = acpi_get_node(root->device->handle); + struct acpi_pci_generic_root_info *ri; + struct pci_bus *bus, *child; + int err; + + ri = kzalloc_node(sizeof(*ri), GFP_KERNEL, node); + if (!ri) + return NULL; + + err = pci_acpi_setup_ecam_mapping(root, ri); + if (err) + return NULL; + + acpi_pci_root_ops.pci_ops = &ri->cfg->ops->pci_ops; + bus = acpi_pci_root_create(root, &acpi_pci_root_ops, &ri->common, + ri->cfg); + if (!bus) + return NULL; + + pci_bus_size_bridges(bus); + pci_bus_assign_resources(bus); + + list_for_each_entry(child, &bus->children, node) + pcie_bus_configure_settings(child); + + return bus; +} + +int raw_pci_read(unsigned int domain, unsigned int busn, unsigned int devfn, + int reg, int len, u32 *val) +{ + struct pci_bus *bus = pci_find_bus(domain, busn); + + if (!bus) + return PCIBIOS_DEVICE_NOT_FOUND; + return bus->ops->read(bus, devfn, reg, len, val); +} + +int raw_pci_write(unsigned int domain, unsigned int busn, unsigned int devfn, + int reg, int len, u32 val) +{ + struct pci_bus *bus = pci_find_bus(domain, busn); + + if (!bus) + return PCIBIOS_DEVICE_NOT_FOUND; + return bus->ops->write(bus, devfn, reg, len, val); +} diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h index 1ad2176..1cccf57 100644 --- a/drivers/pci/ecam.h +++ b/drivers/pci/ecam.h @@ -45,6 +45,11 @@ struct pci_config_window { void __iomem *win; /* 64-bit single mapping */ void __iomem **winp; /* 32-bit per bus mapping */ }; +#ifdef CONFIG_ACPI_PCI_HOST_GENERIC + struct acpi_device *companion; /* ACPI companion device */ +#endif + int domain; + };
/* create and free for pci_config_window */ diff --git a/include/linux/pci-acpi.h b/include/linux/pci-acpi.h index 1baa515..42ff844 100644 --- a/include/linux/pci-acpi.h +++ b/include/linux/pci-acpi.h @@ -111,6 +111,10 @@ static inline void acpi_pci_add_bus(struct pci_bus *bus) { } static inline void acpi_pci_remove_bus(struct pci_bus *bus) { } #endif /* CONFIG_ACPI */
+#ifdef CONFIG_ACPI_PCI_HOST_GENERIC +void acpi_pci_set_companion(struct pci_host_bridge *bridge); +int acpi_pci_bus_domain_nr(struct pci_bus *bus); +#else static inline void acpi_pci_set_companion(struct pci_host_bridge *bridge) { } @@ -119,6 +123,7 @@ static inline int acpi_pci_bus_domain_nr(struct pci_bus *bus) { return 0; } +#endif
#ifdef CONFIG_ACPI_APEI extern bool aer_acpi_firmware_first(void); diff --git a/include/linux/pci.h b/include/linux/pci.h index d6ea6ce..b2e8886 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1722,7 +1722,10 @@ void pcibios_free_irq(struct pci_dev *dev); extern struct dev_pm_ops pcibios_pm_ops; #endif
-#ifdef CONFIG_PCI_MMCONFIG +#if defined(CONFIG_PCI_MMCONFIG) || defined(CONFIG_ACPI_PCI_HOST_GENERIC) +struct acpi_device; +phys_addr_t pci_mcfg_lookup(struct acpi_device *device, u16 seg, + struct resource *bus_res); void __init pci_mmcfg_early_init(void); void __init pci_mmcfg_late_init(void); #else
On Tue, May 10, 2016 at 5:19 PM, Tomasz Nowicki tn@semihalf.com wrote:
This patch is going to implement generic PCI host controller for ACPI world, similar to what pci-host-generic.c driver does for DT world.
All such drivers, which we have seen so far, were implemented within arch/ directory since they had some arch assumptions (x86 and ia64). However, they all are doing similar thing, so it makes sense to find some common code and abstract it into the generic driver.
In order to handle PCI config space regions properly, we define new MCFG interface which does sanity checks on MCFG table and keeps its root pointer. User is able to lookup MCFG regions based on that root pointer and specified domain:bus_start:bus_end touple. We are using pci_mmcfg_late_init old prototype to avoid another function name.
The implementation of pci_acpi_scan_root() looks up the MCFG entries and sets up a new mapping (regions are not mapped until host controller ask for it). Generic PCI functions are used for accessing config space. Driver selects PCI_ECAM and uses functions from drivers/pci/ecam.h to create and access ECAM mappings.
As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice should be made on a per-architecture basis.
Signed-off-by: Tomasz Nowicki tn@semihalf.com Signed-off-by: Jayachandran C jchandra@broadcom.com
drivers/acpi/Kconfig | 8 +++ drivers/acpi/Makefile | 1 + drivers/acpi/pci_mcfg.c | 97 ++++++++++++++++++++++++++ drivers/acpi/pci_root_generic.c | 149 ++++++++++++++++++++++++++++++++++++++++
Why do we need a new file? Would there be any problem with adding that code to pci_root.c?
On Tue, May 10, 2016 at 5:19 PM, Tomasz Nowicki tn@semihalf.com wrote:
This patch is going to implement generic PCI host controller for ACPI world, similar to what pci-host-generic.c driver does for DT world.
All such drivers, which we have seen so far, were implemented within arch/ directory since they had some arch assumptions (x86 and ia64). However, they all are doing similar thing, so it makes sense to find some common code and abstract it into the generic driver.
Does it mean x86 and ia64 will now be able to use this code too?
In order to handle PCI config space regions properly, we define new MCFG interface which does sanity checks on MCFG table and keeps its root pointer. User is able to lookup MCFG regions based on that root pointer and specified domain:bus_start:bus_end touple. We are using pci_mmcfg_late_init old prototype to avoid another function name.
The implementation of pci_acpi_scan_root() looks up the MCFG entries and sets up a new mapping (regions are not mapped until host controller ask for it). Generic PCI functions are used for accessing config space. Driver selects PCI_ECAM and uses functions from drivers/pci/ecam.h to create and access ECAM mappings.
As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice should be made on a per-architecture basis.
If that code really is generic and there will be more than one architecture using it ever, I think it'll be better for the architectures that don't use it to set something like ARCH_ACPI_PCI_HOST and whoever doesn't set that will use the generic thing. That'd be more logical at least IMO.
Signed-off-by: Tomasz Nowicki tn@semihalf.com Signed-off-by: Jayachandran C jchandra@broadcom.com
drivers/acpi/Kconfig | 8 +++ drivers/acpi/Makefile | 1 + drivers/acpi/pci_mcfg.c | 97 ++++++++++++++++++++++++++ drivers/acpi/pci_root_generic.c | 149 ++++++++++++++++++++++++++++++++++++++++ drivers/pci/ecam.h | 5 ++ include/linux/pci-acpi.h | 5 ++ include/linux/pci.h | 5 +- 7 files changed, 269 insertions(+), 1 deletion(-) create mode 100644 drivers/acpi/pci_mcfg.c create mode 100644 drivers/acpi/pci_root_generic.c
diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig index 183ffa3..44afc76 100644 --- a/drivers/acpi/Kconfig +++ b/drivers/acpi/Kconfig @@ -346,6 +346,14 @@ config ACPI_PCI_SLOT i.e., segment/bus/device/function tuples, with physical slots in the system. If you are unsure, say N.
+config ACPI_PCI_HOST_GENERIC
boolselect PCI_ECAMhelpSelect this config option from the architecture Kconfig,if it is preferred to enable ACPI PCI host controller driver whichhas no arch-specific assumptions.config X86_PM_TIMER bool "Power Management Timer Support" if EXPERT depends on X86 diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile index 81e5cbc..627a2b7 100644 --- a/drivers/acpi/Makefile +++ b/drivers/acpi/Makefile @@ -40,6 +40,7 @@ acpi-$(CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC) += processor_pdc.o acpi-y += ec.o acpi-$(CONFIG_ACPI_DOCK) += dock.o acpi-y += pci_root.o pci_link.o pci_irq.o +obj-$(CONFIG_ACPI_PCI_HOST_GENERIC) += pci_root_generic.o pci_mcfg.o acpi-y += acpi_lpss.o acpi_apd.o acpi-y += acpi_platform.o acpi-y += acpi_pnp.o diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c new file mode 100644 index 0000000..373d079 --- /dev/null +++ b/drivers/acpi/pci_mcfg.c @@ -0,0 +1,97 @@ +/*
- Copyright (C) 2016 Broadcom
Author: Jayachandran C <jchandra@broadcom.com>
- Copyright (C) 2016 Semihalf
Author: Tomasz Nowicki <tn@semihalf.com>
- This program is free software; you can redistribute it and/or modify
- it under the terms of the GNU General Public License, version 2, as
- published by the Free Software Foundation (the "GPL").
- This program is distributed in the hope that it will be useful, but
- WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- General Public License version 2 (GPLv2) for more details.
- You should have received a copy of the GNU General Public License
- version 2 (GPLv2) along with this source code.
- */
+#include <linux/kernel.h> +#include <linux/pci.h> +#include <linux/pci-acpi.h>
+#define PREFIX "ACPI: "
If that is a new file (and I'm totally unconvinced about the need for it), can we simply define a pr_fmt() here as all messages in it seem to be printed by the pr_* functions?
On Tue, May 10, 2016 at 8:49 PM, Tomasz Nowicki tn@semihalf.com wrote:
This patch is going to implement generic PCI host controller for ACPI world, similar to what pci-host-generic.c driver does for DT world.
All such drivers, which we have seen so far, were implemented within arch/ directory since they had some arch assumptions (x86 and ia64). However, they all are doing similar thing, so it makes sense to find some common code and abstract it into the generic driver.
In order to handle PCI config space regions properly, we define new MCFG interface which does sanity checks on MCFG table and keeps its root pointer. User is able to lookup MCFG regions based on that root pointer and specified domain:bus_start:bus_end touple. We are using pci_mmcfg_late_init old prototype to avoid another function name.
The implementation of pci_acpi_scan_root() looks up the MCFG entries and sets up a new mapping (regions are not mapped until host controller ask for it). Generic PCI functions are used for accessing config space. Driver selects PCI_ECAM and uses functions from drivers/pci/ecam.h to create and access ECAM mappings.
As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice should be made on a per-architecture basis.
Signed-off-by: Tomasz Nowicki tn@semihalf.com Signed-off-by: Jayachandran C jchandra@broadcom.com
[....]
diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h index 1ad2176..1cccf57 100644 --- a/drivers/pci/ecam.h +++ b/drivers/pci/ecam.h @@ -45,6 +45,11 @@ struct pci_config_window { void __iomem *win; /* 64-bit single mapping */ void __iomem **winp; /* 32-bit per bus mapping */ }; +#ifdef CONFIG_ACPI_PCI_HOST_GENERIC
struct acpi_device *companion; /* ACPI companion device */+#endif
int domain;};
Using struct pci_config_window to pass along domain and companion looks bad. I think there are two possible options to do this better:
1. add a 'struct fwnode_handle *' or 'struct device *parent_dev' instead of the companion and domain fields above. In case of ACPI either of them can be used to get the acpi_device and both domain and companion can be set from that.
2. make pci_config_window fully embeddable by moving allocation out of pci_ecam_create to its callers. Then it can be embedded into acpi_pci_generic_root_info, and container_of can be used to get acpi info from ->sysdata.
The first option should be easier to implement but the second may be better on long run. I would leave it to the Bjorn or Rafael to suggest which is preferred.
JC.
On Fri, May 13, 2016 at 1:25 PM, Jayachandran C jchandra@broadcom.com wrote:
On Tue, May 10, 2016 at 8:49 PM, Tomasz Nowicki tn@semihalf.com wrote:
This patch is going to implement generic PCI host controller for ACPI world, similar to what pci-host-generic.c driver does for DT world.
All such drivers, which we have seen so far, were implemented within arch/ directory since they had some arch assumptions (x86 and ia64). However, they all are doing similar thing, so it makes sense to find some common code and abstract it into the generic driver.
In order to handle PCI config space regions properly, we define new MCFG interface which does sanity checks on MCFG table and keeps its root pointer. User is able to lookup MCFG regions based on that root pointer and specified domain:bus_start:bus_end touple. We are using pci_mmcfg_late_init old prototype to avoid another function name.
The implementation of pci_acpi_scan_root() looks up the MCFG entries and sets up a new mapping (regions are not mapped until host controller ask for it). Generic PCI functions are used for accessing config space. Driver selects PCI_ECAM and uses functions from drivers/pci/ecam.h to create and access ECAM mappings.
As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice should be made on a per-architecture basis.
Signed-off-by: Tomasz Nowicki tn@semihalf.com Signed-off-by: Jayachandran C jchandra@broadcom.com
[....]
diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h index 1ad2176..1cccf57 100644 --- a/drivers/pci/ecam.h +++ b/drivers/pci/ecam.h @@ -45,6 +45,11 @@ struct pci_config_window { void __iomem *win; /* 64-bit single mapping */ void __iomem **winp; /* 32-bit per bus mapping */ }; +#ifdef CONFIG_ACPI_PCI_HOST_GENERIC
struct acpi_device *companion; /* ACPI companion device */+#endif
int domain;};
Using struct pci_config_window to pass along domain and companion looks bad. I think there are two possible options to do this better:
add a 'struct fwnode_handle *' or 'struct device *parent_dev' instead of the companion and domain fields above. In case of ACPI either of them can be used to get the acpi_device and both domain and companion can be set from that.
make pci_config_window fully embeddable by moving allocation out of pci_ecam_create to its callers. Then it can be embedded into acpi_pci_generic_root_info, and container_of can be used to get acpi info from ->sysdata.
The first option should be easier to implement but the second may be better on long run. I would leave it to the Bjorn or Rafael to suggest which is preferred.
Personally, I'd probably try to use fwnode_handle, but the second option makes sense too in principle.
On 13.05.2016 13:31, Rafael J. Wysocki wrote:
On Fri, May 13, 2016 at 1:25 PM, Jayachandran C jchandra@broadcom.com wrote:
On Tue, May 10, 2016 at 8:49 PM, Tomasz Nowicki tn@semihalf.com wrote:
This patch is going to implement generic PCI host controller for ACPI world, similar to what pci-host-generic.c driver does for DT world.
All such drivers, which we have seen so far, were implemented within arch/ directory since they had some arch assumptions (x86 and ia64). However, they all are doing similar thing, so it makes sense to find some common code and abstract it into the generic driver.
In order to handle PCI config space regions properly, we define new MCFG interface which does sanity checks on MCFG table and keeps its root pointer. User is able to lookup MCFG regions based on that root pointer and specified domain:bus_start:bus_end touple. We are using pci_mmcfg_late_init old prototype to avoid another function name.
The implementation of pci_acpi_scan_root() looks up the MCFG entries and sets up a new mapping (regions are not mapped until host controller ask for it). Generic PCI functions are used for accessing config space. Driver selects PCI_ECAM and uses functions from drivers/pci/ecam.h to create and access ECAM mappings.
As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice should be made on a per-architecture basis.
Signed-off-by: Tomasz Nowicki tn@semihalf.com Signed-off-by: Jayachandran C jchandra@broadcom.com
[....]
diff --git a/drivers/pci/ecam.h b/drivers/pci/ecam.h index 1ad2176..1cccf57 100644 --- a/drivers/pci/ecam.h +++ b/drivers/pci/ecam.h @@ -45,6 +45,11 @@ struct pci_config_window { void __iomem *win; /* 64-bit single mapping */ void __iomem **winp; /* 32-bit per bus mapping */ }; +#ifdef CONFIG_ACPI_PCI_HOST_GENERIC
struct acpi_device *companion; /* ACPI companion device */+#endif
int domain;- };
Using struct pci_config_window to pass along domain and companion looks bad. I think there are two possible options to do this better:
add a 'struct fwnode_handle *' or 'struct device *parent_dev' instead of the companion and domain fields above. In case of ACPI either of them can be used to get the acpi_device and both domain and companion can be set from that.
make pci_config_window fully embeddable by moving allocation out of pci_ecam_create to its callers. Then it can be embedded into acpi_pci_generic_root_info, and container_of can be used to get acpi info from ->sysdata.
The first option should be easier to implement but the second may be better on long run. I would leave it to the Bjorn or Rafael to suggest which is preferred.
Personally, I'd probably try to use fwnode_handle, but the second option makes sense too in principle.
Thanks for suggestions. I will try to use fwnode_handle
Tomasz
On Tue, May 10, 2016 at 8:49 PM, Tomasz Nowicki tn@semihalf.com wrote:
This patch is going to implement generic PCI host controller for ACPI world, similar to what pci-host-generic.c driver does for DT world.
All such drivers, which we have seen so far, were implemented within arch/ directory since they had some arch assumptions (x86 and ia64). However, they all are doing similar thing, so it makes sense to find some common code and abstract it into the generic driver.
In order to handle PCI config space regions properly, we define new MCFG interface which does sanity checks on MCFG table and keeps its root pointer. User is able to lookup MCFG regions based on that root pointer and specified domain:bus_start:bus_end touple. We are using pci_mmcfg_late_init old prototype to avoid another function name.
The implementation of pci_acpi_scan_root() looks up the MCFG entries and sets up a new mapping (regions are not mapped until host controller ask for it). Generic PCI functions are used for accessing config space. Driver selects PCI_ECAM and uses functions from drivers/pci/ecam.h to create and access ECAM mappings.
As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice should be made on a per-architecture basis.
Looking thru the new code, I see a few issues, please see below
Signed-off-by: Tomasz Nowicki tn@semihalf.com Signed-off-by: Jayachandran C jchandra@broadcom.com
drivers/acpi/Kconfig | 8 +++ drivers/acpi/Makefile | 1 + drivers/acpi/pci_mcfg.c | 97 ++++++++++++++++++++++++++ drivers/acpi/pci_root_generic.c | 149 ++++++++++++++++++++++++++++++++++++++++ drivers/pci/ecam.h | 5 ++ include/linux/pci-acpi.h | 5 ++ include/linux/pci.h | 5 +- 7 files changed, 269 insertions(+), 1 deletion(-) create mode 100644 drivers/acpi/pci_mcfg.c create mode 100644 drivers/acpi/pci_root_generic.c
diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig index 183ffa3..44afc76 100644 --- a/drivers/acpi/Kconfig +++ b/drivers/acpi/Kconfig @@ -346,6 +346,14 @@ config ACPI_PCI_SLOT i.e., segment/bus/device/function tuples, with physical slots in the system. If you are unsure, say N.
+config ACPI_PCI_HOST_GENERIC
boolselect PCI_ECAMhelpSelect this config option from the architecture Kconfig,if it is preferred to enable ACPI PCI host controller driver whichhas no arch-specific assumptions.config X86_PM_TIMER bool "Power Management Timer Support" if EXPERT depends on X86 diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile index 81e5cbc..627a2b7 100644 --- a/drivers/acpi/Makefile +++ b/drivers/acpi/Makefile @@ -40,6 +40,7 @@ acpi-$(CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC) += processor_pdc.o acpi-y += ec.o acpi-$(CONFIG_ACPI_DOCK) += dock.o acpi-y += pci_root.o pci_link.o pci_irq.o +obj-$(CONFIG_ACPI_PCI_HOST_GENERIC) += pci_root_generic.o pci_mcfg.o acpi-y += acpi_lpss.o acpi_apd.o acpi-y += acpi_platform.o acpi-y += acpi_pnp.o diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c new file mode 100644 index 0000000..373d079 --- /dev/null +++ b/drivers/acpi/pci_mcfg.c @@ -0,0 +1,97 @@ +/*
- Copyright (C) 2016 Broadcom
Author: Jayachandran C <jchandra@broadcom.com>
- Copyright (C) 2016 Semihalf
Author: Tomasz Nowicki <tn@semihalf.com>
- This program is free software; you can redistribute it and/or modify
- it under the terms of the GNU General Public License, version 2, as
- published by the Free Software Foundation (the "GPL").
- This program is distributed in the hope that it will be useful, but
- WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- General Public License version 2 (GPLv2) for more details.
- You should have received a copy of the GNU General Public License
- version 2 (GPLv2) along with this source code.
- */
+#include <linux/kernel.h> +#include <linux/pci.h> +#include <linux/pci-acpi.h>
+#define PREFIX "ACPI: "
+/* Root pointer to the mapped MCFG table */ +static struct acpi_table_mcfg *mcfg_table;
+#define MCFG_ENTRIES(mcfg_ptr) (((mcfg_ptr)->header.length - \
sizeof(struct acpi_table_mcfg)) / \sizeof(struct acpi_mcfg_allocation))
It would be better if you used static inline function here.
+static phys_addr_t pci_mcfg_lookup_static(u16 seg, u8 bus_start, u8 bus_end) +{
struct acpi_mcfg_allocation *mptr;int i;if (!mcfg_table) {pr_err(PREFIX "MCFG table not available, lookup failed\n");return -ENXIO;}mptr = (struct acpi_mcfg_allocation *) &mcfg_table[1];/** We expect exact match, unless MCFG entry end bus covers more than* specified by caller.*/for (i = 0; i < MCFG_ENTRIES(mcfg_table); i++, mptr++) {if (mptr->pci_segment == seg &&mptr->start_bus_number == bus_start &&mptr->end_bus_number >= bus_end) {return mptr->address;}}
There is an issue here, the bus range is obtained if different ways. If the _CRS has it, this should be fine. But if it is _BBN-0xff or the default 0-0xff, then I would think taking the MCFG entry end would be better. This would require updating the bus resource end.
Also (trivial), the braces are not needed.
return -ENXIO;+}
+phys_addr_t pci_mcfg_lookup(struct acpi_device *device, u16 seg,
struct resource *bus_res)+{
phys_addr_t addr;addr = acpi_pci_root_get_mcfg_addr(device->handle);if (addr)return addr;
When you have address from _CBA, you are assuming that the bus range is also set correctly (from _CRS or as _BBN-0xff). Is this assumption correct?
(this was discussed earlier) you are doing the _CBA call again, I think cleaning up the _CBA, _BBN, _CRS and MCFG based ECAM area resources and defaults should be a different patch, which would need changes to pci_root.c. We should use root->mcfg_addr in this patchset.
return pci_mcfg_lookup_static(seg, bus_res->start, bus_res->end);
There is no need to have a separate function for this, based on by above comment, you might need to change bus_res, so having it here would be better.
+}
+static __init int pci_mcfg_parse(struct acpi_table_header *header) +{
struct acpi_table_mcfg *mcfg;int n;if (!header)return -EINVAL;
This is not needed, the handler is not called if header is NULL
mcfg = (struct acpi_table_mcfg *)header;n = MCFG_ENTRIES(mcfg);if (n <= 0 || n > 255) {pr_err(PREFIX "MCFG has incorrect entries (%d).\n", n);return -EINVAL;}mcfg_table = mcfg;
Saving a reference of ACPI mapping seems dangerous, acpi_parse_table calles early_acpi_os_unmap_memory() after calling handler, which does not do anything since acpi_gbl_permanent_mmap is set.
I would suggest copying the entries.
pr_info(PREFIX "MCFG table loaded, %d entries detected\n", n);return 0;+}
+/* Interface called by ACPI - parse and save MCFG table */ +void __init pci_mmcfg_late_init(void) +{
int err = acpi_table_parse(ACPI_SIG_MCFG, pci_mcfg_parse);if (err)pr_err(PREFIX "Failed to parse MCFG (%d)\n", err);+}
JC.
On 14.05.2016 11:07, Jayachandran C wrote:
On Tue, May 10, 2016 at 8:49 PM, Tomasz Nowicki tn@semihalf.com wrote:
This patch is going to implement generic PCI host controller for ACPI world, similar to what pci-host-generic.c driver does for DT world.
All such drivers, which we have seen so far, were implemented within arch/ directory since they had some arch assumptions (x86 and ia64). However, they all are doing similar thing, so it makes sense to find some common code and abstract it into the generic driver.
In order to handle PCI config space regions properly, we define new MCFG interface which does sanity checks on MCFG table and keeps its root pointer. User is able to lookup MCFG regions based on that root pointer and specified domain:bus_start:bus_end touple. We are using pci_mmcfg_late_init old prototype to avoid another function name.
The implementation of pci_acpi_scan_root() looks up the MCFG entries and sets up a new mapping (regions are not mapped until host controller ask for it). Generic PCI functions are used for accessing config space. Driver selects PCI_ECAM and uses functions from drivers/pci/ecam.h to create and access ECAM mappings.
As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice should be made on a per-architecture basis.
Looking thru the new code, I see a few issues, please see below
Signed-off-by: Tomasz Nowicki tn@semihalf.com Signed-off-by: Jayachandran C jchandra@broadcom.com
drivers/acpi/Kconfig | 8 +++ drivers/acpi/Makefile | 1 + drivers/acpi/pci_mcfg.c | 97 ++++++++++++++++++++++++++ drivers/acpi/pci_root_generic.c | 149 ++++++++++++++++++++++++++++++++++++++++ drivers/pci/ecam.h | 5 ++ include/linux/pci-acpi.h | 5 ++ include/linux/pci.h | 5 +- 7 files changed, 269 insertions(+), 1 deletion(-) create mode 100644 drivers/acpi/pci_mcfg.c create mode 100644 drivers/acpi/pci_root_generic.c
diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig index 183ffa3..44afc76 100644 --- a/drivers/acpi/Kconfig +++ b/drivers/acpi/Kconfig @@ -346,6 +346,14 @@ config ACPI_PCI_SLOT i.e., segment/bus/device/function tuples, with physical slots in the system. If you are unsure, say N.
+config ACPI_PCI_HOST_GENERIC
boolselect PCI_ECAMhelpSelect this config option from the architecture Kconfig,if it is preferred to enable ACPI PCI host controller driver whichhas no arch-specific assumptions.- config X86_PM_TIMER bool "Power Management Timer Support" if EXPERT depends on X86
diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile index 81e5cbc..627a2b7 100644 --- a/drivers/acpi/Makefile +++ b/drivers/acpi/Makefile @@ -40,6 +40,7 @@ acpi-$(CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC) += processor_pdc.o acpi-y += ec.o acpi-$(CONFIG_ACPI_DOCK) += dock.o acpi-y += pci_root.o pci_link.o pci_irq.o +obj-$(CONFIG_ACPI_PCI_HOST_GENERIC) += pci_root_generic.o pci_mcfg.o acpi-y += acpi_lpss.o acpi_apd.o acpi-y += acpi_platform.o acpi-y += acpi_pnp.o diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c new file mode 100644 index 0000000..373d079 --- /dev/null +++ b/drivers/acpi/pci_mcfg.c @@ -0,0 +1,97 @@ +/*
- Copyright (C) 2016 Broadcom
Author: Jayachandran C <jchandra@broadcom.com>
- Copyright (C) 2016 Semihalf
Author: Tomasz Nowicki <tn@semihalf.com>
- This program is free software; you can redistribute it and/or modify
- it under the terms of the GNU General Public License, version 2, as
- published by the Free Software Foundation (the "GPL").
- This program is distributed in the hope that it will be useful, but
- WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- General Public License version 2 (GPLv2) for more details.
- You should have received a copy of the GNU General Public License
- version 2 (GPLv2) along with this source code.
- */
+#include <linux/kernel.h> +#include <linux/pci.h> +#include <linux/pci-acpi.h>
+#define PREFIX "ACPI: "
+/* Root pointer to the mapped MCFG table */ +static struct acpi_table_mcfg *mcfg_table;
+#define MCFG_ENTRIES(mcfg_ptr) (((mcfg_ptr)->header.length - \
sizeof(struct acpi_table_mcfg)) / \sizeof(struct acpi_mcfg_allocation))It would be better if you used static inline function here.
OK
+static phys_addr_t pci_mcfg_lookup_static(u16 seg, u8 bus_start, u8 bus_end) +{
struct acpi_mcfg_allocation *mptr;int i;if (!mcfg_table) {pr_err(PREFIX "MCFG table not available, lookup failed\n");return -ENXIO;}mptr = (struct acpi_mcfg_allocation *) &mcfg_table[1];/** We expect exact match, unless MCFG entry end bus covers more than* specified by caller.*/for (i = 0; i < MCFG_ENTRIES(mcfg_table); i++, mptr++) {if (mptr->pci_segment == seg &&mptr->start_bus_number == bus_start &&mptr->end_bus_number >= bus_end) {return mptr->address;}}There is an issue here, the bus range is obtained if different ways. If the _CRS has it, this should be fine. But if it is _BBN-0xff or the default 0-0xff, then I would think taking the MCFG entry end would be better. This would require updating the bus resource end.
Yeah we can implement algorithm like that but IMO it would try to guess what FW designer meant putting various combination of BUS resources into the tables. We should rather be strict about this so I agree with Bjorn: https://lkml.org/lkml/2016/4/28/807 Either we get exact match or MCFG covers more than the host bridge range.
Host bridge bus range definition -> MCFG _BBN-0xff -> we expect exact _BBN-0xff match 0-0xff -> ditto _CRS (the only way to trim host bridge bus rage) -> exact match or MCFG covers more
Also (trivial), the braces are not needed.
OK
return -ENXIO;+}
+phys_addr_t pci_mcfg_lookup(struct acpi_device *device, u16 seg,
struct resource *bus_res)+{
phys_addr_t addr;addr = acpi_pci_root_get_mcfg_addr(device->handle);if (addr)return addr;When you have address from _CBA, you are assuming that the bus range is also set correctly (from _CRS or as _BBN-0xff). Is this assumption correct?
I think so, see above. Maybe Bjorn may advise here.
(this was discussed earlier) you are doing the _CBA call again, I think cleaning up the _CBA, _BBN, _CRS and MCFG based ECAM area resources and defaults should be a different patch, which would need changes to pci_root.c. We should use root->mcfg_addr in this patchset.
OK
return pci_mcfg_lookup_static(seg, bus_res->start, bus_res->end);There is no need to have a separate function for this, based on by above comment, you might need to change bus_res, so having it here would be better.
+}
+static __init int pci_mcfg_parse(struct acpi_table_header *header) +{
struct acpi_table_mcfg *mcfg;int n;if (!header)return -EINVAL;This is not needed, the handler is not called if header is NULL
OK
mcfg = (struct acpi_table_mcfg *)header;n = MCFG_ENTRIES(mcfg);if (n <= 0 || n > 255) {pr_err(PREFIX "MCFG has incorrect entries (%d).\n", n);return -EINVAL;}mcfg_table = mcfg;Saving a reference of ACPI mapping seems dangerous, acpi_parse_table calles early_acpi_os_unmap_memory() after calling handler, which does not do anything since acpi_gbl_permanent_mmap is set.
I would suggest copying the entries.
You got the point about early_acpi_os_unmap_memory but that would be the case if we call pci_mmcfg_late_init before acpi_early_init. But pci_mmcfg_late_init is called much later. Also the code is much simpler as is now.
Thanks, Tomasz
On 10/05/16 17:19, Tomasz Nowicki wrote:
This patch is going to implement generic PCI host controller for ACPI world, similar to what pci-host-generic.c driver does for DT world.
All such drivers, which we have seen so far, were implemented within arch/ directory since they had some arch assumptions (x86 and ia64). However, they all are doing similar thing, so it makes sense to find some common code and abstract it into the generic driver.
In order to handle PCI config space regions properly, we define new MCFG interface which does sanity checks on MCFG table and keeps its root pointer. User is able to lookup MCFG regions based on that root pointer and specified domain:bus_start:bus_end touple. We are using pci_mmcfg_late_init old prototype to avoid another function name.
The implementation of pci_acpi_scan_root() looks up the MCFG entries and sets up a new mapping (regions are not mapped until host controller ask for it). Generic PCI functions are used for accessing config space. Driver selects PCI_ECAM and uses functions from drivers/pci/ecam.h to create and access ECAM mappings.
As mentioned in Kconfig help section, ACPI_PCI_HOST_GENERIC choice should be made on a per-architecture basis.
Signed-off-by: Tomasz Nowicki tn@semihalf.com Signed-off-by: Jayachandran C jchandra@broadcom.com
drivers/acpi/Kconfig | 8 +++ drivers/acpi/Makefile | 1 + drivers/acpi/pci_mcfg.c | 97 ++++++++++++++++++++++++++ drivers/acpi/pci_root_generic.c | 149 ++++++++++++++++++++++++++++++++++++++++ drivers/pci/ecam.h | 5 ++ include/linux/pci-acpi.h | 5 ++ include/linux/pci.h | 5 +- 7 files changed, 269 insertions(+), 1 deletion(-) create mode 100644 drivers/acpi/pci_mcfg.c create mode 100644 drivers/acpi/pci_root_generic.c
diff --git a/drivers/acpi/Kconfig b/drivers/acpi/Kconfig index 183ffa3..44afc76 100644 --- a/drivers/acpi/Kconfig +++ b/drivers/acpi/Kconfig @@ -346,6 +346,14 @@ config ACPI_PCI_SLOT i.e., segment/bus/device/function tuples, with physical slots in the system. If you are unsure, say N.
+config ACPI_PCI_HOST_GENERIC
- bool
- select PCI_ECAM
- help
Select this config option from the architecture Kconfig,if it is preferred to enable ACPI PCI host controller driver whichhas no arch-specific assumptions.- config X86_PM_TIMER bool "Power Management Timer Support" if EXPERT depends on X86
diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile index 81e5cbc..627a2b7 100644 --- a/drivers/acpi/Makefile +++ b/drivers/acpi/Makefile @@ -40,6 +40,7 @@ acpi-$(CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC) += processor_pdc.o acpi-y += ec.o acpi-$(CONFIG_ACPI_DOCK) += dock.o acpi-y += pci_root.o pci_link.o pci_irq.o +obj-$(CONFIG_ACPI_PCI_HOST_GENERIC) += pci_root_generic.o pci_mcfg.o acpi-y += acpi_lpss.o acpi_apd.o acpi-y += acpi_platform.o acpi-y += acpi_pnp.o diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c new file mode 100644 index 0000000..373d079 --- /dev/null +++ b/drivers/acpi/pci_mcfg.c @@ -0,0 +1,97 @@ +/*
- Copyright (C) 2016 Broadcom
- Author: Jayachandran C jchandra@broadcom.com
- Copyright (C) 2016 Semihalf
- Author: Tomasz Nowicki tn@semihalf.com
- This program is free software; you can redistribute it and/or modify
- it under the terms of the GNU General Public License, version 2, as
- published by the Free Software Foundation (the "GPL").
- This program is distributed in the hope that it will be useful, but
- WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- General Public License version 2 (GPLv2) for more details.
- You should have received a copy of the GNU General Public License
- version 2 (GPLv2) along with this source code.
- */
+#include <linux/kernel.h> +#include <linux/pci.h> +#include <linux/pci-acpi.h>
+#define PREFIX "ACPI: "
+/* Root pointer to the mapped MCFG table */ +static struct acpi_table_mcfg *mcfg_table;
+#define MCFG_ENTRIES(mcfg_ptr) (((mcfg_ptr)->header.length - \
sizeof(struct acpi_table_mcfg)) / \sizeof(struct acpi_mcfg_allocation))+static phys_addr_t pci_mcfg_lookup_static(u16 seg, u8 bus_start, u8 bus_end) +{
- struct acpi_mcfg_allocation *mptr;
- int i;
- if (!mcfg_table) {
pr_err(PREFIX "MCFG table not available, lookup failed\n");return -ENXIO;- }
- mptr = (struct acpi_mcfg_allocation *) &mcfg_table[1];
- /*
* We expect exact match, unless MCFG entry end bus covers more than* specified by caller.*/- for (i = 0; i < MCFG_ENTRIES(mcfg_table); i++, mptr++) {
if (mptr->pci_segment == seg &&mptr->start_bus_number == bus_start &&mptr->end_bus_number >= bus_end) {return mptr->address;}- }
- return -ENXIO;
+}
+phys_addr_t pci_mcfg_lookup(struct acpi_device *device, u16 seg,
struct resource *bus_res)+{
- phys_addr_t addr;
- addr = acpi_pci_root_get_mcfg_addr(device->handle);
- if (addr)
return addr;- return pci_mcfg_lookup_static(seg, bus_res->start, bus_res->end);
+}
+static __init int pci_mcfg_parse(struct acpi_table_header *header) +{
- struct acpi_table_mcfg *mcfg;
- int n;
- if (!header)
return -EINVAL;
Maybe that's bike shedding, but acpi_table_parse already checks that the table passed to the handler exists, so this is redundant.
Regards, Matthias
To enable PCI legacy IRQs on platforms booting with ACPI, arch code should include ACPI specific callbacks that parse and set-up the device IRQ number, equivalent to the DT boot path. Owing to the current ACPI core scan handlers implementation, ACPI PCI legacy IRQs bindings cannot be parsed at device add time, since that would trigger ACPI scan handlers ordering issues depending on how the ACPI tables are defined.
To solve this problem and consolidate FW PCI legacy IRQs parsing in one single pcibios callback (pending final removal), this patch moves DT PCI IRQ parsing to the pcibios_alloc_irq() callback (called by PCI core code at device probe time) and adds ACPI PCI legacy IRQs parsing to the same callback too, so that FW PCI legacy IRQs parsing is confined in one single arch callback that can be easily removed when code parsing PCI legacy IRQs is consolidated and moved to core PCI code.
Signed-off-by: Tomasz Nowicki tn@semihalf.com Suggested-by: Lorenzo Pieralisi lorenzo.pieralisi@arm.com --- arch/arm64/kernel/pci.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c index c72de66..15109c11 100644 --- a/arch/arm64/kernel/pci.c +++ b/arch/arm64/kernel/pci.c @@ -50,11 +50,16 @@ int pcibios_enable_device(struct pci_dev *dev, int mask) }
/* - * Try to assign the IRQ number from DT when adding a new device + * Try to assign the IRQ number when probing a new device */ -int pcibios_add_device(struct pci_dev *dev) +int pcibios_alloc_irq(struct pci_dev *dev) { - dev->irq = of_irq_parse_and_map_pci(dev, 0, 0); + if (acpi_disabled) + dev->irq = of_irq_parse_and_map_pci(dev, 0, 0); +#ifdef CONFIG_ACPI + else + return acpi_pci_irq_enable(dev); +#endif
return 0; }
ACPI requires to run acpi_pci_{add|remove}_bus while new PCI bus is created. This allows to do some ACPI-specific additional configuration, like PCI hotplug slot enumeration. In order to fulfill these requirements, we implement arch-specific pcibios_{add|remove}_bus calls and call acpi_pci_{add|remove}_bus from there.
Signed-off-by: Tomasz Nowicki tn@semihalf.com To: Catalin Marinas catalin.marinas@arm.com To: Lorenzo Pieralisi Lorenzo.Pieralisi@arm.com To: Will Deacon will.deacon@arm.com To: Arnd Bergmann arnd@arndb.de --- arch/arm64/kernel/pci.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c index 15109c11..eeec5f6 100644 --- a/arch/arm64/kernel/pci.c +++ b/arch/arm64/kernel/pci.c @@ -17,6 +17,7 @@ #include <linux/mm.h> #include <linux/of_pci.h> #include <linux/of_platform.h> +#include <linux/pci-acpi.h> #include <linux/slab.h>
/* @@ -64,6 +65,16 @@ int pcibios_alloc_irq(struct pci_dev *dev) return 0; }
+void pcibios_add_bus(struct pci_bus *bus) +{ + acpi_pci_add_bus(bus); +} + +void pcibios_remove_bus(struct pci_bus *bus) +{ + acpi_pci_remove_bus(bus); +} + /* * raw_pci_read/write - Platform-specific PCI config space access. */
It is perfectly fine to use ACPI_PCI_HOST_GENERIC for ARM64, so lets get rid of PCI init and RAW ACPI accessor empty stubs and go with full-blown PCI host controller driver.
Signed-off-by: Tomasz Nowicki tn@semihalf.com To: Catalin Marinas catalin.marinas@arm.com To: Lorenzo Pieralisi Lorenzo.Pieralisi@arm.com To: Will Deacon will.deacon@arm.com To: Arnd Bergmann arnd@arndb.de --- arch/arm64/Kconfig | 1 + arch/arm64/kernel/pci.c | 24 ------------------------ 2 files changed, 1 insertion(+), 24 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 4f43622..1bded87 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -2,6 +2,7 @@ config ARM64 def_bool y select ACPI_CCA_REQUIRED if ACPI select ACPI_GENERIC_GSI if ACPI + select ACPI_PCI_HOST_GENERIC if ACPI select ACPI_REDUCED_HARDWARE_ONLY if ACPI select ARCH_HAS_DEVMEM_IS_ALLOWED select ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c index eeec5f6..e484c91 100644 --- a/arch/arm64/kernel/pci.c +++ b/arch/arm64/kernel/pci.c @@ -74,27 +74,3 @@ void pcibios_remove_bus(struct pci_bus *bus) { acpi_pci_remove_bus(bus); } - -/* - * raw_pci_read/write - Platform-specific PCI config space access. - */ -int raw_pci_read(unsigned int domain, unsigned int bus, - unsigned int devfn, int reg, int len, u32 *val) -{ - return -ENXIO; -} - -int raw_pci_write(unsigned int domain, unsigned int bus, - unsigned int devfn, int reg, int len, u32 val) -{ - return -ENXIO; -} - -#ifdef CONFIG_ACPI -/* Root bridge scanning */ -struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root) -{ - /* TODO: Should be revisited when implementing PCI on ACPI */ - return NULL; -} -#endif
Hi Tomasz
-----Original Message----- From: linux-kernel-owner@vger.kernel.org [mailto:linux-kernel- owner@vger.kernel.org] On Behalf Of Tomasz Nowicki Sent: 10 May 2016 16:20 To: helgaas@kernel.org; arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com; rafael@kernel.org; hanjun.guo@linaro.org; Lorenzo.Pieralisi@arm.com; okaya@codeaurora.org; jchandra@broadcom.com Cc: robert.richter@caviumnetworks.com; mw@semihalf.com; Liviu.Dudau@arm.com; ddaney@caviumnetworks.com; Wangyijing; Suravee.Suthikulpanit@amd.com; msalter@redhat.com; linux- pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org; linux- acpi@vger.kernel.org; linux-kernel@vger.kernel.org; linaro- acpi@lists.linaro.org; jcm@redhat.com; andrea.gallo@linaro.org; dhdang@apm.com; jeremy.linton@arm.com; liudongdong (C); cov@codeaurora.org; Tomasz Nowicki Subject: [PATCH V7 00/11] Support for generic ACPI based PCI host controller
From the functionality point of view this series may be split into the following logic parts:
- New ECAM API and update for users of the pci-host-common API
- Necessary fixes as the preparation for using driver on ARM64.
- Use new MCFG interface and implement generic ACPI based PCI host
controller driver. 4. Enable above driver on ARM64
Patches has been built on top of 4.6-rc7 and can be found here: git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v7)
This has been tested on Cavium ThunderX server. Any help in reviewing and testing is very appreciated.
v6 -> v7
- drop quirks handling
Maybe I missed something in the v6 discussion thread; when was it decided to drop quirk handling?
I think it is important to have this in place to accommodate different vendors. If the intention is to keep this patchset "clean" maybe we can add it as a separate patch on top later on...
What’s your view?
Thanks
Gab
- changes for ACPI companion and domain number assignment approach
- implement arch pcibios_{add|remove}_bus and call
acpi_pci_{add|remove}_bus from there
- cleanups around nomenclature
- use resources oriented API for ECAM
- fix for based address calculation before mapping ECAM region
- remove useless lock for MCFG lookup
- move MCFG stuff to separated file pci_mcfg.c
- drop MCFG entries caching
- rebase against 4.6-rc7
v5 -> v6
- drop idea of x86 MMCONFIG code refactoring
- integrate JC's patches which introduce new ECAM API: https://lkml.org/lkml/2016/4/11/907 git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
- integrate Sinan's fix for releasing IO resources, see patch [06/13]
- added ACPI support for ThunderX ECAM and PEM drivers
- rebase against 4.6-rc2
v4 -> v5
- drop MCFG refactoring group patches 1-6 from series v4 and integrate
Jayachandran's patch https://patchwork.ozlabs.org/patch/575525/
- rewrite PCI legacy IRQs allocation
- squash two patches 11 and 12 from series v4, fixed bisection issue
- changelog improvements
- rebase against 4.5-rc3
v3 -> v4
- drop Jiang's fix
http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04318.html
- add Lorenzo's fix patch 19/24
- ACPI PCI bus domain number assigning cleanup
- change resource management, we now claim and reassign resources
- improvements for applying quirks
- drop Matthew's http://www.spinics.net/lists/linux-pci/msg45950.html
dependency
- rebase against 4.5-rc1
v2 -> v3
- fix legacy IRQ assigning and IO ports registration
- remove reference to arch specific companion device for ia64
- move ACPI PCI host controller driver to pci_root.c
- drop generic domain assignment for x86 and ia64 as I am not able to run all necessary test variants
- drop patch which cleaned legacy IRQ assignment since it belongs to Mathew's series: https://patchwork.ozlabs.org/patch/557504/
- extend MCFG quirk code
- rebase against 4.4
v1 -> v2
- move non-arch specific piece of code to dirver/acpi/ directory
- fix IO resource handling
- introduce PCI config accessors quirks matching
- moved ACPI_COMPANION_SET to generic code
v1 - https://lkml.org/lkml/2015/10/27/504 v2 - https://lkml.org/lkml/2015/12/16/246 v3 - http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04308.html v4 - https://lkml.org/lkml/2016/2/4/646 v5 - https://lkml.org/lkml/2016/2/16/426 v6 - https://lkml.org/lkml/2016/4/15/594
Jayachandran C (2): PCI: Provide common functions for ECAM mapping PCI: generic, thunder: update to use generic ECAM API
Tomasz Nowicki (9): pci, of: Move the PCI I/O space management to PCI core code. pci: Add new function to unmap IO resources. acpi, pci: Support IO resources when parsing PCI host bridge resources. pci, acpi: Provide a way to assign bus domain number. pci, acpi: Handle ACPI companion assignment. pci, acpi: Support for ACPI based generic PCI host controller arm64, pci, acpi: ACPI support for legacy IRQs parsing and consolidation with DT code. arm64, pci, acpi: Provide ACPI-specific prerequisites for PCI bus enumeration. arm64, pci, acpi: Start using ACPI based PCI host controller driver for ARM64.
arch/arm64/Kconfig | 1 + arch/arm64/kernel/pci.c | 34 +++----- drivers/acpi/Kconfig | 8 ++ drivers/acpi/Makefile | 1 + drivers/acpi/pci_mcfg.c | 97 ++++++++++++++++++++++ drivers/acpi/pci_root.c | 33 ++++++++ drivers/acpi/pci_root_generic.c | 149 +++++++++++++++++++++++++++++++++ drivers/of/address.c | 116 +------------------------- drivers/pci/Kconfig | 3 + drivers/pci/Makefile | 2 + drivers/pci/ecam.c | 161 ++++++++++++++++++++++++++++++++++++ drivers/pci/ecam.h | 72 ++++++++++++++++ drivers/pci/host/Kconfig | 1 + drivers/pci/host/pci-host-common.c | 114 +++++++++++-------------- drivers/pci/host/pci-host-common.h | 47 ----------- drivers/pci/host/pci-host-generic.c | 52 +++--------- drivers/pci/host/pci-thunder-ecam.c | 39 ++------- drivers/pci/host/pci-thunder-pem.c | 92 ++++++++++----------- drivers/pci/pci.c | 150 ++++++++++++++++++++++++++++++++- drivers/pci/probe.c | 2 + include/linux/of_address.h | 9 -- include/linux/pci-acpi.h | 14 ++++ include/linux/pci.h | 11 ++- 23 files changed, 823 insertions(+), 385 deletions(-) create mode 100644 drivers/acpi/pci_mcfg.c create mode 100644 drivers/acpi/pci_root_generic.c create mode 100644 drivers/pci/ecam.c create mode 100644 drivers/pci/ecam.h delete mode 100644 drivers/pci/host/pci-host-common.h
-- 1.9.1
Hi Gabriele,
On 11.05.2016 12:41, Gabriele Paoloni wrote:
Hi Tomasz
-----Original Message----- From: linux-kernel-owner@vger.kernel.org [mailto:linux-kernel- owner@vger.kernel.org] On Behalf Of Tomasz Nowicki Sent: 10 May 2016 16:20 To: helgaas@kernel.org; arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com; rafael@kernel.org; hanjun.guo@linaro.org; Lorenzo.Pieralisi@arm.com; okaya@codeaurora.org; jchandra@broadcom.com Cc: robert.richter@caviumnetworks.com; mw@semihalf.com; Liviu.Dudau@arm.com; ddaney@caviumnetworks.com; Wangyijing; Suravee.Suthikulpanit@amd.com; msalter@redhat.com; linux- pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org; linux- acpi@vger.kernel.org; linux-kernel@vger.kernel.org; linaro- acpi@lists.linaro.org; jcm@redhat.com; andrea.gallo@linaro.org; dhdang@apm.com; jeremy.linton@arm.com; liudongdong (C); cov@codeaurora.org; Tomasz Nowicki Subject: [PATCH V7 00/11] Support for generic ACPI based PCI host controller
From the functionality point of view this series may be split into the following logic parts:
- New ECAM API and update for users of the pci-host-common API
- Necessary fixes as the preparation for using driver on ARM64.
- Use new MCFG interface and implement generic ACPI based PCI host
controller driver. 4. Enable above driver on ARM64
Patches has been built on top of 4.6-rc7 and can be found here: git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v7)
This has been tested on Cavium ThunderX server. Any help in reviewing and testing is very appreciated.
v6 -> v7
- drop quirks handling
Maybe I missed something in the v6 discussion thread; when was it decided to drop quirk handling?
I had such requests in previous series.
I think it is important to have this in place to accommodate different vendors. If the intention is to keep this patchset "clean" maybe we can add it as a separate patch on top later on...
What’s your view?
Yes, keeping these things separated should help in review. Obviously I agree that we need quirk handling but currently there is no implementation which we all agree upon. For the test, you can use quirk handling approach from the previous series until we sort out final solution.
Thanks, Tomasz
Hi Tomasz
-----Original Message----- From: Tomasz Nowicki [mailto:tn@semihalf.com] Sent: 11 May 2016 12:08 To: Gabriele Paoloni; helgaas@kernel.org; arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com; rafael@kernel.org; hanjun.guo@linaro.org; Lorenzo.Pieralisi@arm.com; okaya@codeaurora.org; jchandra@broadcom.com Cc: robert.richter@caviumnetworks.com; mw@semihalf.com; Liviu.Dudau@arm.com; ddaney@caviumnetworks.com; Wangyijing; Suravee.Suthikulpanit@amd.com; msalter@redhat.com; linux- pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org; linux- acpi@vger.kernel.org; linux-kernel@vger.kernel.org; linaro- acpi@lists.linaro.org; jcm@redhat.com; andrea.gallo@linaro.org; dhdang@apm.com; jeremy.linton@arm.com; liudongdong (C); cov@codeaurora.org Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller
Hi Gabriele,
On 11.05.2016 12:41, Gabriele Paoloni wrote:
Hi Tomasz
-----Original Message----- From: linux-kernel-owner@vger.kernel.org [mailto:linux-kernel- owner@vger.kernel.org] On Behalf Of Tomasz Nowicki Sent: 10 May 2016 16:20 To: helgaas@kernel.org; arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com; rafael@kernel.org; hanjun.guo@linaro.org; Lorenzo.Pieralisi@arm.com; okaya@codeaurora.org;
jchandra@broadcom.com
Cc: robert.richter@caviumnetworks.com; mw@semihalf.com; Liviu.Dudau@arm.com; ddaney@caviumnetworks.com; Wangyijing; Suravee.Suthikulpanit@amd.com; msalter@redhat.com; linux- pci@vger.kernel.org; linux-arm-kernel@lists.infradead.org; linux- acpi@vger.kernel.org; linux-kernel@vger.kernel.org; linaro- acpi@lists.linaro.org; jcm@redhat.com; andrea.gallo@linaro.org; dhdang@apm.com; jeremy.linton@arm.com; liudongdong (C); cov@codeaurora.org; Tomasz Nowicki Subject: [PATCH V7 00/11] Support for generic ACPI based PCI host controller
From the functionality point of view this series may be split into
the
following logic parts:
- New ECAM API and update for users of the pci-host-common API
- Necessary fixes as the preparation for using driver on ARM64.
- Use new MCFG interface and implement generic ACPI based PCI host
controller driver. 4. Enable above driver on ARM64
Patches has been built on top of 4.6-rc7 and can be found here: git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v7)
This has been tested on Cavium ThunderX server. Any help in
reviewing
and testing is very appreciated.
v6 -> v7
- drop quirks handling
Maybe I missed something in the v6 discussion thread; when was it decided to drop quirk handling?
I had such requests in previous series.
I think it is important to have this in place to accommodate
different
vendors. If the intention is to keep this patchset "clean" maybe we can add it as a separate patch on top later on...
What's your view?
Yes, keeping these things separated should help in review. Obviously I agree that we need quirk handling but currently there is no implementation which we all agree upon. For the test, you can use quirk handling approach from the previous series until we sort out final solution.
Great
This explains perfectly
We will apply previous series quirks and try to test the patchset from Huawei side
Many Thanks
Gab
Thanks, Tomasz
Hi Tomasz, all,
On 05/11/2016 07:08 AM, Tomasz Nowicki wrote:
On 11.05.2016 12:41, Gabriele Paoloni wrote:
v6 -> v7
- drop quirks handling
Maybe I missed something in the v6 discussion thread; when was it decided to drop quirk handling?
I had such requests in previous series.
A quick note on quirk handling. This, I believe, applies post-merge of the base infrastructure, which I realize will likely not have quirks.
We've some "gen1" ARMv8 server platforms where we end up doing quirks (for things like forcing 32-bit config space accessors and the like) due to people repurposing existing embedded PCIe IP blocks or using them for the first time (especially in servers), and those being involved in the design not necessarily seeing this problem ahead of time, or not realizing that it would be an issue for servers. In the early days of ARM server designs 3-4 years ago, many of us had never really played with ECAM or realized how modern topologies are built.
Anyway. We missed this one in our SBSA requirements. They say (words to the effect of) "thou shalt do PCIe the way it is done on servers" but they aren't prescriptive, and they don't tell people how that actually is in reality. That is being fixed. A lot of things are happening behind the scenes - especially with third party IP block providers (all of whom myself and others are speaking with directly about this) - to ensure that the next wave of designs won't repeat these mistakes. We don't have a time machine, but we can contain this from becoming an ongoing mess for upstream, and we will do so. It won't be a zoo.
Various proposals have arisen for how to handle quirks in the longer term, including elaborate frameworks and tables to describe them generically. I would like to caution against such approaches, especially in the case that they deviate from practice on x86, or prior to being standardized fully with other Operating System vendors. I don't expect there to be too many more than the existing initial set of quirks we have seen posted. A number of "future" server SoCs have already been fixed prior to silicon, and new design starts are being warned not to make this a problem for us to have to clean up later.
So, I would like to suggest that the eventual framework mirror the existing approach on x86 systems (matching DMI, etc.) and not be made into some kind of generic, utopia. This is a case where we want there to be pain involved (and upstream patches required) when people screw up, so that they have a level of pain in response to ever making this mistake in the future. If we try to create too grand a generic scheme and make it too easy to handle this kind of situation beyond the small number of existing offenders, we undermine efforts to force vendors to ensure that their IP blocks are compliant going forward.
Side note: if you're a third party IP vendor and we didn't already speak about this one, drop me a line, and let's collaborate also on your test frameworks to make sure you're covered as well.
Jon.
On 20 May 2016 at 06:41, Jon Masters jcm@redhat.com wrote:
Hi Tomasz, all,
On 05/11/2016 07:08 AM, Tomasz Nowicki wrote:
On 11.05.2016 12:41, Gabriele Paoloni wrote:
v6 -> v7
- drop quirks handling
Maybe I missed something in the v6 discussion thread; when was it decided to drop quirk handling?
I had such requests in previous series.
A quick note on quirk handling. This, I believe, applies post-merge of the base infrastructure, which I realize will likely not have quirks.
We've some "gen1" ARMv8 server platforms where we end up doing quirks (for things like forcing 32-bit config space accessors and the like) due to people repurposing existing embedded PCIe IP blocks or using them for the first time (especially in servers), and those being involved in the design not necessarily seeing this problem ahead of time, or not realizing that it would be an issue for servers. In the early days of ARM server designs 3-4 years ago, many of us had never really played with ECAM or realized how modern topologies are built.
Anyway. We missed this one in our SBSA requirements. They say (words to the effect of) "thou shalt do PCIe the way it is done on servers" but they aren't prescriptive, and they don't tell people how that actually is in reality. That is being fixed. A lot of things are happening behind the scenes - especially with third party IP block providers (all of whom myself and others are speaking with directly about this) - to ensure that the next wave of designs won't repeat these mistakes. We don't have a time machine, but we can contain this from becoming an ongoing mess for upstream, and we will do so. It won't be a zoo.
Various proposals have arisen for how to handle quirks in the longer term, including elaborate frameworks and tables to describe them generically. I would like to caution against such approaches, especially in the case that they deviate from practice on x86, or prior to being standardized fully with other Operating System vendors. I don't expect there to be too many more than the existing initial set of quirks we have seen posted. A number of "future" server SoCs have already been fixed prior to silicon, and new design starts are being warned not to make this a problem for us to have to clean up later.
So, I would like to suggest that the eventual framework mirror the existing approach on x86 systems (matching DMI, etc.) and not be made into some kind of generic, utopia. This is a case where we want there to be pain involved (and upstream patches required) when people screw up, so that they have a level of pain in response to ever making this mistake in the future. If we try to create too grand a generic scheme and make it too easy to handle this kind of situation beyond the small number of existing offenders, we undermine efforts to force vendors to ensure that their IP blocks are compliant going forward.
I understand that there is a desire from the RedHat side to mimic x86 as closely as possible, but I never saw any technical justification for that. DMI contains strings that are visible to userland, and you effectively lock those down to certain values just so that the kernel can distinguish a broken PCIe root complex from a working one. Linux on x86 had no choice, since the overwhelming majority of existing hardware misrepresented itself as generic, and DMI was the only thing available to actually distinguish these broken implementations from one another. This does not mean we should allow and/or encourage this first gen hardware to misrepresent non-compliant hardware as compliant as well.
Since you are talking to all the people involved, how about you convince them to put something in the ACPI tables that allows the kernel to distinguish those non-standard PCIe implementations from hardware that is really generic? This way, we can sidestep the quirks debate entirely, since it will simply be a different device as far as the kernel is concerned. This is no worse than a quirk from a practical point of view, since an older OS will be equally unable to run on newer hardware, but it is arguably more true to the standards compliance you tend to preach about, especially since this small pool of third party IP could potentially be identified directly rather than based on some divination of the SoC we may or may not be running on. I am also convinced that adding support for an additional HID() to the ACPI ECAM driver with some special config space handling wired in is an easier sell upstream than making the same ugly mess x86 has had to make because they did not have any choice to begin with.
If we do need a quirks handling mechanism, I still don't see how the x86 situation extrapolates to ARM. ACPI offers plenty of ways for a SoC vendor to identify the make and particular revision, and quirks could be keyed off of that.
Hi Ard,
On 05/20/2016 03:37 AM, Ard Biesheuvel wrote:
On 20 May 2016 at 06:41, Jon Masters jcm@redhat.com wrote:
Hi Tomasz, all,
On 05/11/2016 07:08 AM, Tomasz Nowicki wrote:
On 11.05.2016 12:41, Gabriele Paoloni wrote:
v6 -> v7
- drop quirks handling
Maybe I missed something in the v6 discussion thread; when was it decided to drop quirk handling?
I had such requests in previous series.
A quick note on quirk handling. This, I believe, applies post-merge of the base infrastructure, which I realize will likely not have quirks.
We've some "gen1" ARMv8 server platforms where we end up doing quirks (for things like forcing 32-bit config space accessors and the like) due to people repurposing existing embedded PCIe IP blocks or using them for the first time (especially in servers), and those being involved in the design not necessarily seeing this problem ahead of time, or not realizing that it would be an issue for servers. In the early days of ARM server designs 3-4 years ago, many of us had never really played with ECAM or realized how modern topologies are built.
Anyway. We missed this one in our SBSA requirements. They say (words to the effect of) "thou shalt do PCIe the way it is done on servers" but they aren't prescriptive, and they don't tell people how that actually is in reality. That is being fixed. A lot of things are happening behind the scenes - especially with third party IP block providers (all of whom myself and others are speaking with directly about this) - to ensure that the next wave of designs won't repeat these mistakes. We don't have a time machine, but we can contain this from becoming an ongoing mess for upstream, and we will do so. It won't be a zoo.
Various proposals have arisen for how to handle quirks in the longer term, including elaborate frameworks and tables to describe them generically. I would like to caution against such approaches, especially in the case that they deviate from practice on x86, or prior to being standardized fully with other Operating System vendors. I don't expect there to be too many more than the existing initial set of quirks we have seen posted. A number of "future" server SoCs have already been fixed prior to silicon, and new design starts are being warned not to make this a problem for us to have to clean up later.
So, I would like to suggest that the eventual framework mirror the existing approach on x86 systems (matching DMI, etc.) and not be made into some kind of generic, utopia. This is a case where we want there to be pain involved (and upstream patches required) when people screw up, so that they have a level of pain in response to ever making this mistake in the future. If we try to create too grand a generic scheme and make it too easy to handle this kind of situation beyond the small number of existing offenders, we undermine efforts to force vendors to ensure that their IP blocks are compliant going forward.
I understand that there is a desire from the RedHat side to mimic x86 as closely as possible, but I never saw any technical justification for that.
Understood. My own motivation is always to make the experience as familiar as possible, both for end users, as well as for ODMs and the entire ecosystem. There are very many ODMs currently working on v8 server designs and they're already expecting this to be "just like x88". Intentionally. But as to the specifics of using DMI...
DMI contains strings that are visible to userland, and you effectively lock those down to certain values just so that the kernel can distinguish a broken PCIe root complex from a working one. Linux on x86 had no choice, since the overwhelming majority of existing hardware misrepresented itself as generic, and DMI was the only thing available to actually distinguish these broken implementations from one another. This does not mean we should allow and/or encourage this first gen hardware to misrepresent non-compliant hardware as compliant as well.
That's a very reasonable argument. I don't disagree that it would be nice to have nicer ways to distinguish the non-compliant IP than treating the whole platform with an ASCII matching sledgehammer.
Since you are talking to all the people involved, how about you convince them to put something in the ACPI tables that allows the kernel to distinguish those non-standard PCIe implementations from hardware that is really generic?
I'm open to this *BUT* it has to be something that will be adopted beyond Linux. I have reached out to some non-Linux folks about this. If there's buy-in, and if there's agreement to go standardize it through the ASWG, then we should do so. What we should not do is treat ARM as special in a way that the others aren't involved with. I'll admit DMI ended up part of the SBBR in part because I wrote that piece in with the assumption that exactly the same matches as on x86 would happen.
This way, we can sidestep the quirks debate entirely, since it will simply be a different device as far as the kernel is concerned. This is no worse than a quirk from a practical point of view, since an older OS will be equally unable to run on newer hardware, but it is arguably more true to the standards compliance you tend to preach about, especially since this small pool of third party IP could potentially be identified directly rather than based on some divination of the SoC we may or may not be running on. I am also convinced that adding support for an additional HID() to the ACPI ECAM driver with some special config space handling wired in is an easier sell upstream than making the same ugly mess x86 has had to make because they did not have any choice to begin with.
Again, open to it. I just don't want to do something that's Linux specific. So it'll take time. It would be awesome if an interim quirk solution existed that got platforms that are shipping in production (e.g. HP Moonshot) actually booting upstream kernels this year. We /really/ want F25 to be able to run on these without needing to carry an out-of-tree quirk patch or just not support them. That's much worse.
If we do need a quirks handling mechanism, I still don't see how the x86 situation extrapolates to ARM. ACPI offers plenty of ways for a SoC vendor to identify the make and particular revision, and quirks could be keyed off of that.
Fair enough. Is there any traction for an interim solution for these initial platforms do you think? If we wait to add a new table it's probably going to be the end of the year before we get this done.
Jon.
On 20 May 2016 at 10:01, Jon Masters jcm@redhat.com wrote:
Hi Ard,
On 05/20/2016 03:37 AM, Ard Biesheuvel wrote:
On 20 May 2016 at 06:41, Jon Masters jcm@redhat.com wrote:
Hi Tomasz, all,
On 05/11/2016 07:08 AM, Tomasz Nowicki wrote:
On 11.05.2016 12:41, Gabriele Paoloni wrote:
v6 -> v7
- drop quirks handling
Maybe I missed something in the v6 discussion thread; when was it decided to drop quirk handling?
I had such requests in previous series.
A quick note on quirk handling. This, I believe, applies post-merge of the base infrastructure, which I realize will likely not have quirks.
We've some "gen1" ARMv8 server platforms where we end up doing quirks (for things like forcing 32-bit config space accessors and the like) due to people repurposing existing embedded PCIe IP blocks or using them for the first time (especially in servers), and those being involved in the design not necessarily seeing this problem ahead of time, or not realizing that it would be an issue for servers. In the early days of ARM server designs 3-4 years ago, many of us had never really played with ECAM or realized how modern topologies are built.
Anyway. We missed this one in our SBSA requirements. They say (words to the effect of) "thou shalt do PCIe the way it is done on servers" but they aren't prescriptive, and they don't tell people how that actually is in reality. That is being fixed. A lot of things are happening behind the scenes - especially with third party IP block providers (all of whom myself and others are speaking with directly about this) - to ensure that the next wave of designs won't repeat these mistakes. We don't have a time machine, but we can contain this from becoming an ongoing mess for upstream, and we will do so. It won't be a zoo.
Various proposals have arisen for how to handle quirks in the longer term, including elaborate frameworks and tables to describe them generically. I would like to caution against such approaches, especially in the case that they deviate from practice on x86, or prior to being standardized fully with other Operating System vendors. I don't expect there to be too many more than the existing initial set of quirks we have seen posted. A number of "future" server SoCs have already been fixed prior to silicon, and new design starts are being warned not to make this a problem for us to have to clean up later.
So, I would like to suggest that the eventual framework mirror the existing approach on x86 systems (matching DMI, etc.) and not be made into some kind of generic, utopia. This is a case where we want there to be pain involved (and upstream patches required) when people screw up, so that they have a level of pain in response to ever making this mistake in the future. If we try to create too grand a generic scheme and make it too easy to handle this kind of situation beyond the small number of existing offenders, we undermine efforts to force vendors to ensure that their IP blocks are compliant going forward.
I understand that there is a desire from the RedHat side to mimic x86 as closely as possible, but I never saw any technical justification for that.
Understood. My own motivation is always to make the experience as familiar as possible, both for end users, as well as for ODMs and the entire ecosystem. There are very many ODMs currently working on v8 server designs and they're already expecting this to be "just like x88". Intentionally. But as to the specifics of using DMI...
DMI contains strings that are visible to userland, and you effectively lock those down to certain values just so that the kernel can distinguish a broken PCIe root complex from a working one. Linux on x86 had no choice, since the overwhelming majority of existing hardware misrepresented itself as generic, and DMI was the only thing available to actually distinguish these broken implementations from one another. This does not mean we should allow and/or encourage this first gen hardware to misrepresent non-compliant hardware as compliant as well.
That's a very reasonable argument. I don't disagree that it would be nice to have nicer ways to distinguish the non-compliant IP than treating the whole platform with an ASCII matching sledgehammer.
Since you are talking to all the people involved, how about you convince them to put something in the ACPI tables that allows the kernel to distinguish those non-standard PCIe implementations from hardware that is really generic?
I'm open to this *BUT* it has to be something that will be adopted beyond Linux. I have reached out to some non-Linux folks about this. If there's buy-in, and if there's agreement to go standardize it through the ASWG, then we should do so. What we should not do is treat ARM as special in a way that the others aren't involved with. I'll admit DMI ended up part of the SBBR in part because I wrote that piece in with the assumption that exactly the same matches as on x86 would happen.
Is the PCIe root complex so special that you cannot simply describe an implementation that is not PNP0408 compatible as something else, under its own unique HID? If everybody is onboard with using ACPI, how is this any different from describing other parts of the platform topology? Even if the SBSA mandates generic PCI, they already deviated from that when they built the hardware, so pretending that it is a PNP0408 with quirks really does not buy us anything.
This way, we can sidestep the quirks debate entirely, since it will simply be a different device as far as the kernel is concerned. This is no worse than a quirk from a practical point of view, since an older OS will be equally unable to run on newer hardware, but it is arguably more true to the standards compliance you tend to preach about, especially since this small pool of third party IP could potentially be identified directly rather than based on some divination of the SoC we may or may not be running on. I am also convinced that adding support for an additional HID() to the ACPI ECAM driver with some special config space handling wired in is an easier sell upstream than making the same ugly mess x86 has had to make because they did not have any choice to begin with.
Again, open to it. I just don't want to do something that's Linux specific. So it'll take time. It would be awesome if an interim quirk solution existed that got platforms that are shipping in production (e.g. HP Moonshot) actually booting upstream kernels this year. We /really/ want F25 to be able to run on these without needing to carry an out-of-tree quirk patch or just not support them. That's much worse.
My whole point is that we don't need quirks in the first place if non-compliant devices are not being misrepresented as compliant ones. This is fine for other platform devices, i.e., SATA, network, so again, why is PCIe so special that we *must* use a generic ID + quirks rather than a specific ID?
If we do need a quirks handling mechanism, I still don't see how the x86 situation extrapolates to ARM. ACPI offers plenty of ways for a SoC vendor to identify the make and particular revision, and quirks could be keyed off of that.
Fair enough. Is there any traction for an interim solution for these initial platforms do you think? If we wait to add a new table it's probably going to be the end of the year before we get this done.
The 'interim solution' is to come to terms with the fact that these initial platforms are not SBSA compliant, contain a PCIe root complex that is not PNP0408 but can be identified by its own HID, and we make the software work with that. This means our message from the beginning is that, yes, you can have non-compliant hardware and the burden is on you to get it supported upstream, and no, you don't get to hang out with the cool SBSA kids if you decide to go that route
Hi Ard
-----Original Message----- From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org] Sent: 20 May 2016 09:29 To: Jon Masters Cc: Tomasz Nowicki; Gabriele Paoloni; helgaas@kernel.org; arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com; rafael@kernel.org; hanjun.guo@linaro.org; Lorenzo.Pieralisi@arm.com; okaya@codeaurora.org; jchandra@broadcom.com; linaro- acpi@lists.linaro.org; linux-pci@vger.kernel.org; dhdang@apm.com; Liviu.Dudau@arm.com; ddaney@caviumnetworks.com; jeremy.linton@arm.com; linux-kernel@vger.kernel.org; linux-acpi@vger.kernel.org; robert.richter@caviumnetworks.com; Suravee.Suthikulpanit@amd.com; msalter@redhat.com; Wangyijing; mw@semihalf.com; andrea.gallo@linaro.org; linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller
On 20 May 2016 at 10:01, Jon Masters jcm@redhat.com wrote:
Hi Ard,
On 05/20/2016 03:37 AM, Ard Biesheuvel wrote:
On 20 May 2016 at 06:41, Jon Masters jcm@redhat.com wrote:
Hi Tomasz, all,
On 05/11/2016 07:08 AM, Tomasz Nowicki wrote:
On 11.05.2016 12:41, Gabriele Paoloni wrote:
> v6 -> v7 > - drop quirks handling
Maybe I missed something in the v6 discussion thread; when was it decided to drop quirk handling?
I had such requests in previous series.
A quick note on quirk handling. This, I believe, applies post-merge
of
the base infrastructure, which I realize will likely not have
quirks.
We've some "gen1" ARMv8 server platforms where we end up doing
quirks
(for things like forcing 32-bit config space accessors and the
like) due
to people repurposing existing embedded PCIe IP blocks or using
them for
the first time (especially in servers), and those being involved in
the
design not necessarily seeing this problem ahead of time, or not realizing that it would be an issue for servers. In the early days
of
ARM server designs 3-4 years ago, many of us had never really
played
with ECAM or realized how modern topologies are built.
Anyway. We missed this one in our SBSA requirements. They say
(words to
the effect of) "thou shalt do PCIe the way it is done on servers"
but
they aren't prescriptive, and they don't tell people how that
actually
is in reality. That is being fixed. A lot of things are happening
behind
the scenes - especially with third party IP block providers (all of
whom
myself and others are speaking with directly about this) - to
ensure
that the next wave of designs won't repeat these mistakes. We don't
have
a time machine, but we can contain this from becoming an ongoing
mess
for upstream, and we will do so. It won't be a zoo.
Various proposals have arisen for how to handle quirks in the
longer
term, including elaborate frameworks and tables to describe them generically. I would like to caution against such approaches,
especially
in the case that they deviate from practice on x86, or prior to
being
standardized fully with other Operating System vendors. I don't
expect
there to be too many more than the existing initial set of quirks
we
have seen posted. A number of "future" server SoCs have already
been
fixed prior to silicon, and new design starts are being warned not
to
make this a problem for us to have to clean up later.
So, I would like to suggest that the eventual framework mirror the existing approach on x86 systems (matching DMI, etc.) and not be
made
into some kind of generic, utopia. This is a case where we want
there to
be pain involved (and upstream patches required) when people screw
up,
so that they have a level of pain in response to ever making this mistake in the future. If we try to create too grand a generic
scheme
and make it too easy to handle this kind of situation beyond the
small
number of existing offenders, we undermine efforts to force vendors
to
ensure that their IP blocks are compliant going forward.
I understand that there is a desire from the RedHat side to mimic
x86
as closely as possible, but I never saw any technical justification for that.
Understood. My own motivation is always to make the experience as familiar as possible, both for end users, as well as for ODMs and the entire ecosystem. There are very many ODMs currently working on v8 server designs and they're already expecting this to be "just like
x88".
Intentionally. But as to the specifics of using DMI...
DMI contains strings that are visible to userland, and you effectively lock those down to certain values just so that the
kernel
can distinguish a broken PCIe root complex from a working one. Linux on x86 had no choice, since the overwhelming majority of existing hardware misrepresented itself as generic, and DMI was the only
thing
available to actually distinguish these broken implementations from one another. This does not mean we should allow and/or encourage
this
first gen hardware to misrepresent non-compliant hardware as
compliant
as well.
That's a very reasonable argument. I don't disagree that it would be nice to have nicer ways to distinguish the non-compliant IP than treating the whole platform with an ASCII matching sledgehammer.
Since you are talking to all the people involved, how about you convince them to put something in the ACPI tables that allows the kernel to distinguish those non-standard PCIe implementations from hardware that is really generic?
I'm open to this *BUT* it has to be something that will be adopted beyond Linux. I have reached out to some non-Linux folks about this.
If
there's buy-in, and if there's agreement to go standardize it through the ASWG, then we should do so. What we should not do is treat ARM as special in a way that the others aren't involved with. I'll admit DMI ended up part of the SBBR in part because I wrote that piece in with
the
assumption that exactly the same matches as on x86 would happen.
Is the PCIe root complex so special that you cannot simply describe an implementation that is not PNP0408 compatible as something else, under its own unique HID? If everybody is onboard with using ACPI, how is this any different from describing other parts of the platform topology? Even if the SBSA mandates generic PCI, they already deviated from that when they built the hardware, so pretending that it is a PNP0408 with quirks really does not buy us anything.
From my understanding we want to avoid this as this would allow each vendor to come up with his own code and it would be much more effort for the PCI maintainer to rework the PCI framework to accommodate X86 and "all" ARM64 Host Controllers...
I guess this approach is too risky and we want to avoid this. Through standardization we can more easily maintain the code and scale it to multiple SoCs...
So this is my understanding; maybe Jon, Tomasz or Lorenzo can give a bit more explanation...
Thanks
Gab
This way, we can sidestep the quirks debate entirely, since it will simply be a different device as far
as
the kernel is concerned. This is no worse than a quirk from a practical point of view, since an older OS will be equally unable to run on newer hardware, but it is arguably more true to the standards compliance you tend to preach about, especially since this small
pool
of third party IP could potentially be identified directly rather
than
based on some divination of the SoC we may or may not be running on.
I
am also convinced that adding support for an additional HID() to the ACPI ECAM driver with some special config space handling wired in is an easier sell upstream than making the same ugly mess x86 has had
to
make because they did not have any choice to begin with.
Again, open to it. I just don't want to do something that's Linux specific. So it'll take time. It would be awesome if an interim quirk solution existed that got platforms that are shipping in production (e.g. HP Moonshot) actually booting upstream kernels this year. We /really/ want F25 to be able to run on these without needing to carry
an
out-of-tree quirk patch or just not support them. That's much worse.
My whole point is that we don't need quirks in the first place if non-compliant devices are not being misrepresented as compliant ones. This is fine for other platform devices, i.e., SATA, network, so again, why is PCIe so special that we *must* use a generic ID + quirks rather than a specific ID?
If we do need a quirks handling mechanism, I still don't see how the x86 situation extrapolates to ARM. ACPI offers plenty of ways for a SoC vendor to identify the make and particular revision, and quirks could be keyed off of that.
Fair enough. Is there any traction for an interim solution for these initial platforms do you think? If we wait to add a new table it's probably going to be the end of the year before we get this done.
The 'interim solution' is to come to terms with the fact that these initial platforms are not SBSA compliant, contain a PCIe root complex that is not PNP0408 but can be identified by its own HID, and we make the software work with that. This means our message from the beginning is that, yes, you can have non-compliant hardware and the burden is on you to get it supported upstream, and no, you don't get to hang out with the cool SBSA kids if you decide to go that route
On 20 May 2016 at 10:40, Gabriele Paoloni gabriele.paoloni@huawei.com wrote:
Hi Ard
-----Original Message----- From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org]
[...]
Is the PCIe root complex so special that you cannot simply describe an implementation that is not PNP0408 compatible as something else, under its own unique HID? If everybody is onboard with using ACPI, how is this any different from describing other parts of the platform topology? Even if the SBSA mandates generic PCI, they already deviated from that when they built the hardware, so pretending that it is a PNP0408 with quirks really does not buy us anything.
From my understanding we want to avoid this as this would allow each vendor to come up with his own code and it would be much more effort for the PCI maintainer to rework the PCI framework to accommodate X86 and "all" ARM64 Host Controllers...
I guess this approach is too risky and we want to avoid this. Through standardization we can more easily maintain the code and scale it to multiple SoCs...
So this is my understanding; maybe Jon, Tomasz or Lorenzo can give a bit more explanation...
OK, so that boils down to recommending to vendors to represent known non-compliant hardware as compliant, just so that we don't have to change the code to support additional flavors of ECAM ? It's fine to be pragmatic, but that sucks.
We keep confusing the x86 case with the ARM case here: for x86, they needed to deal with broken hardware *after* the fact, and all they could do is find /some/ distinguishing feature in order to guess which exact hardware they might be running on. For arm64, it is the opposite case. We are currently in a position where we can demand vendors to comply with the standards they endorsed themselves, and (ab)using ACPI + DMI as a de facto platform description rather than plain ACPI makes me think the DT crowd were actually right from the beginning. It *directly* violates the standardization principle, since it requires a priori knowledge inside the OS that a certain 'generic' device must be driven in a special way.
So can anyone comment on the feasibility of adding support for devices with vendor specific HIDs (and no generic CIDs) to the current ACPI ECAM driver in Linux?
On Fri, May 20, 2016 at 11:14:03AM +0200, Ard Biesheuvel wrote:
On 20 May 2016 at 10:40, Gabriele Paoloni gabriele.paoloni@huawei.com wrote:
Hi Ard
-----Original Message----- From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org]
[...]
Is the PCIe root complex so special that you cannot simply describe an implementation that is not PNP0408 compatible as something else, under its own unique HID? If everybody is onboard with using ACPI, how is this any different from describing other parts of the platform topology? Even if the SBSA mandates generic PCI, they already deviated from that when they built the hardware, so pretending that it is a PNP0408 with quirks really does not buy us anything.
From my understanding we want to avoid this as this would allow each vendor to come up with his own code and it would be much more effort for the PCI maintainer to rework the PCI framework to accommodate X86 and "all" ARM64 Host Controllers...
I guess this approach is too risky and we want to avoid this. Through standardization we can more easily maintain the code and scale it to multiple SoCs...
So this is my understanding; maybe Jon, Tomasz or Lorenzo can give a bit more explanation...
OK, so that boils down to recommending to vendors to represent known non-compliant hardware as compliant, just so that we don't have to change the code to support additional flavors of ECAM ? It's fine to be pragmatic, but that sucks.
We keep confusing the x86 case with the ARM case here: for x86, they needed to deal with broken hardware *after* the fact, and all they could do is find /some/ distinguishing feature in order to guess which exact hardware they might be running on. For arm64, it is the opposite case. We are currently in a position where we can demand vendors to comply with the standards they endorsed themselves, and (ab)using ACPI
- DMI as a de facto platform description rather than plain ACPI makes
me think the DT crowd were actually right from the beginning. It *directly* violates the standardization principle, since it requires a priori knowledge inside the OS that a certain 'generic' device must be driven in a special way.
So can anyone comment on the feasibility of adding support for devices with vendor specific HIDs (and no generic CIDs) to the current ACPI ECAM driver in Linux?
Host bridges in ACPI are handled through PNP0A08/PNP0A03 ids, and most of the arch specific code is handled in the respective arch directories (X86 and IA64, even though IA64 does not rely on ECAM/MCFG for PCI ops), it is not a driver per-se, PNP0A08/PNP0A03 are detected through ACPI scan handlers and the respective arch code (ie pci_acpi_scan_root) sets-up resources AND config space on an arch specific basis.
X86 deals with that with code in arch/x86 that sets-up the pci_raw_ops on a platform specific basis (and it is not nice, but it works because as you all know the number of platforms in X86 world is contained).
Will this happen for ARM64 in arch/arm64 based on vendor specific HIDs ?
No.
So given the current state of play (we were requested to move the arch/arm64 specific ACPI PCI bits to arch/arm64), we would end up with arch/arm64 code requiring code in /drivers to set-up pci_ops in a platform specific way, it is horrible, if feasible at all.
The only way this can be implemented is by pretending that the ACPI/PCI arch/arm64 implementation is generic code (that's what this series does), move it to /drivers (where it is in this series), and implement _DSD vendor specific bindings (per HID) to set-up the pci operations; whether this solution should go upstream, given that it is just a short-term solution for early platforms bugs, it is another story and my personal answer is no.
Lorenzo
Hi Lorenzo
-----Original Message----- From: Lorenzo Pieralisi [mailto:lorenzo.pieralisi@arm.com] Sent: 23 May 2016 11:57 To: Ard Biesheuvel Cc: Gabriele Paoloni; Jon Masters; Tomasz Nowicki; helgaas@kernel.org; arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com; rafael@kernel.org; hanjun.guo@linaro.org; okaya@codeaurora.org; jchandra@broadcom.com; linaro-acpi@lists.linaro.org; linux- pci@vger.kernel.org; dhdang@apm.com; Liviu.Dudau@arm.com; ddaney@caviumnetworks.com; jeremy.linton@arm.com; linux- kernel@vger.kernel.org; linux-acpi@vger.kernel.org; robert.richter@caviumnetworks.com; Suravee.Suthikulpanit@amd.com; msalter@redhat.com; Wangyijing; mw@semihalf.com; andrea.gallo@linaro.org; linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller
On Fri, May 20, 2016 at 11:14:03AM +0200, Ard Biesheuvel wrote:
On 20 May 2016 at 10:40, Gabriele Paoloni
gabriele.paoloni@huawei.com wrote:
Hi Ard
-----Original Message----- From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org]
[...]
Is the PCIe root complex so special that you cannot simply
describe an
implementation that is not PNP0408 compatible as something else,
under
its own unique HID? If everybody is onboard with using ACPI, how
is
this any different from describing other parts of the platform topology? Even if the SBSA mandates generic PCI, they already
deviated
from that when they built the hardware, so pretending that it is a PNP0408 with quirks really does not buy us anything.
From my understanding we want to avoid this as this would allow
each
vendor to come up with his own code and it would be much more
effort
for the PCI maintainer to rework the PCI framework to accommodate
X86
and "all" ARM64 Host Controllers...
I guess this approach is too risky and we want to avoid this.
Through
standardization we can more easily maintain the code and scale it
to
multiple SoCs...
So this is my understanding; maybe Jon, Tomasz or Lorenzo can give a bit more explanation...
OK, so that boils down to recommending to vendors to represent known non-compliant hardware as compliant, just so that we don't have to change the code to support additional flavors of ECAM ? It's fine to be pragmatic, but that sucks.
We keep confusing the x86 case with the ARM case here: for x86, they needed to deal with broken hardware *after* the fact, and all they could do is find /some/ distinguishing feature in order to guess
which
exact hardware they might be running on. For arm64, it is the
opposite
case. We are currently in a position where we can demand vendors to comply with the standards they endorsed themselves, and (ab)using
ACPI
- DMI as a de facto platform description rather than plain ACPI makes
me think the DT crowd were actually right from the beginning. It *directly* violates the standardization principle, since it requires
a
priori knowledge inside the OS that a certain 'generic' device must
be
driven in a special way.
So can anyone comment on the feasibility of adding support for
devices
with vendor specific HIDs (and no generic CIDs) to the current ACPI ECAM driver in Linux?
Host bridges in ACPI are handled through PNP0A08/PNP0A03 ids, and most of the arch specific code is handled in the respective arch directories (X86 and IA64, even though IA64 does not rely on ECAM/MCFG for PCI ops), it is not a driver per-se, PNP0A08/PNP0A03 are detected through ACPI scan handlers and the respective arch code (ie pci_acpi_scan_root) sets-up resources AND config space on an arch specific basis.
X86 deals with that with code in arch/x86 that sets-up the pci_raw_ops on a platform specific basis (and it is not nice, but it works because as you all know the number of platforms in X86 world is contained).
Will this happen for ARM64 in arch/arm64 based on vendor specific HIDs ?
No.
So given the current state of play (we were requested to move the arch/arm64 specific ACPI PCI bits to arch/arm64), we would end up with arch/arm64 code requiring code in /drivers to set-up pci_ops in a platform specific way, it is horrible, if feasible at all.
The only way this can be implemented is by pretending that the ACPI/PCI arch/arm64 implementation is generic code (that's what this series does), move it to /drivers (where it is in this series), and implement _DSD vendor specific bindings (per HID) to set-up the pci operations; whether this solution should go upstream, given that it is just a short-term solution for early platforms bugs, it is another story and my personal answer is no.
I think it shouldn't be too bad to move quirk handling mechanism to arch/arm64. Effectively we would not move platform specific code into arch/arm64 but just the mechanism checking if there is any quirk that is defined.
i.e.:
extern struct pci_cfg_fixup __start_acpi_mcfg_fixups[]; extern struct pci_cfg_fixup __end_acpi_mcfg_fixups[];
static struct pci_ecam_ops *pci_acpi_get_ops(struct acpi_pci_root *root) { int bus_num = root->secondary.start; int domain = root->segment; struct pci_cfg_fixup *f;
/* * Match against platform specific quirks and return corresponding * CAM ops. * * First match against PCI topology domain:bus then use DMI or * custom match handler. */ for (f = __start_acpi_mcfg_fixups; f < __end_acpi_mcfg_fixups; f++) { if ((f->domain == domain || f->domain == PCI_MCFG_DOMAIN_ANY) && (f->bus_num == bus_num || f->bus_num == PCI_MCFG_BUS_ANY) && (f->system ? dmi_check_system(f->system) : 1) && (f->match ? f->match(f, root) : 1)) return f->ops; } /* No quirks, use ECAM */ return &pci_generic_ecam_ops; }
Such quirks will be defined anyway in drivers/pci/host/ in the vendor specific quirk implementations.
e.g. in HiSilicon case we would have
DECLARE_ACPI_MCFG_FIXUP(NULL, hisi_pcie_match, &hisi_pcie_ecam_ops, PCI_MCFG_DOMAIN_ANY, PCI_MCFG_BUS_ANY);
in "drivers/pci/host/pcie-hisi-acpi.c "
Thanks
Gab
Lorenzo
On Mon, May 23, 2016 at 03:16:01PM +0000, Gabriele Paoloni wrote:
Hi Lorenzo
-----Original Message----- From: Lorenzo Pieralisi [mailto:lorenzo.pieralisi@arm.com] Sent: 23 May 2016 11:57 To: Ard Biesheuvel Cc: Gabriele Paoloni; Jon Masters; Tomasz Nowicki; helgaas@kernel.org; arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com; rafael@kernel.org; hanjun.guo@linaro.org; okaya@codeaurora.org; jchandra@broadcom.com; linaro-acpi@lists.linaro.org; linux- pci@vger.kernel.org; dhdang@apm.com; Liviu.Dudau@arm.com; ddaney@caviumnetworks.com; jeremy.linton@arm.com; linux- kernel@vger.kernel.org; linux-acpi@vger.kernel.org; robert.richter@caviumnetworks.com; Suravee.Suthikulpanit@amd.com; msalter@redhat.com; Wangyijing; mw@semihalf.com; andrea.gallo@linaro.org; linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller
On Fri, May 20, 2016 at 11:14:03AM +0200, Ard Biesheuvel wrote:
On 20 May 2016 at 10:40, Gabriele Paoloni
gabriele.paoloni@huawei.com wrote:
Hi Ard
-----Original Message----- From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org]
[...]
Is the PCIe root complex so special that you cannot simply
describe an
implementation that is not PNP0408 compatible as something else,
under
its own unique HID? If everybody is onboard with using ACPI, how
is
this any different from describing other parts of the platform topology? Even if the SBSA mandates generic PCI, they already
deviated
from that when they built the hardware, so pretending that it is a PNP0408 with quirks really does not buy us anything.
From my understanding we want to avoid this as this would allow
each
vendor to come up with his own code and it would be much more
effort
for the PCI maintainer to rework the PCI framework to accommodate
X86
and "all" ARM64 Host Controllers...
I guess this approach is too risky and we want to avoid this.
Through
standardization we can more easily maintain the code and scale it
to
multiple SoCs...
So this is my understanding; maybe Jon, Tomasz or Lorenzo can give a bit more explanation...
OK, so that boils down to recommending to vendors to represent known non-compliant hardware as compliant, just so that we don't have to change the code to support additional flavors of ECAM ? It's fine to be pragmatic, but that sucks.
We keep confusing the x86 case with the ARM case here: for x86, they needed to deal with broken hardware *after* the fact, and all they could do is find /some/ distinguishing feature in order to guess
which
exact hardware they might be running on. For arm64, it is the
opposite
case. We are currently in a position where we can demand vendors to comply with the standards they endorsed themselves, and (ab)using
ACPI
- DMI as a de facto platform description rather than plain ACPI makes
me think the DT crowd were actually right from the beginning. It *directly* violates the standardization principle, since it requires
a
priori knowledge inside the OS that a certain 'generic' device must
be
driven in a special way.
So can anyone comment on the feasibility of adding support for
devices
with vendor specific HIDs (and no generic CIDs) to the current ACPI ECAM driver in Linux?
I don't think of ECAM support itself as a "driver". It's just a service available to drivers, similar to OF resource parsing.
Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe host bridge. "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that supports extended config space. It doesn't specify how we access that config space, so I think hardware with non-standard ECAM should still have PNP0A03 and PNP0A08 in _CID or _HID.
"ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware r3.2, sec 4.1) means:
(a) a memory-mapped model for config space access, and (b) a specific mapping of address bits to bus/device/function/ register
MCFG and _CBA assume both (a) and (b), so I think a device with non-standard ECAM mappings should not be described in MCFG or _CBA.
If a bridge has ECAM with non-standard mappings, I think either a vendor-specific _HID or a device-specific method, e.g., _DSM, could communicate that.
Jon, I agree that we should avoid describing non-standardized hardware in Linux-specific ways. Is there a mechanism in use already? How does Windows handle this? DMI is a poor long-term solution because it requires ongoing maintenance for new platforms, but I think it's OK for getting started with platforms already shipping.
A _DSM has the advantage that once it is defined and supported, OEMs can ship new platforms without requiring a new quirk or a new _HID to be added to a driver.
There would still be the problem of config access before the namespace is available, i.e., the MCFG use case. I don't know how important that is. Defining an MCFG extension seems like the most obvious solution.
If we only expect a few non-standard devices, maybe it's enough to have DMI quirks to statically set up ECAM and just live with the inconvenience of requiring a kernel change for every new non-standard device.
Host bridges in ACPI are handled through PNP0A08/PNP0A03 ids, and most of the arch specific code is handled in the respective arch directories (X86 and IA64, even though IA64 does not rely on ECAM/MCFG for PCI ops), it is not a driver per-se, PNP0A08/PNP0A03 are detected through ACPI scan handlers and the respective arch code (ie pci_acpi_scan_root) sets-up resources AND config space on an arch specific basis.
X86 deals with that with code in arch/x86 that sets-up the pci_raw_ops on a platform specific basis (and it is not nice, but it works because as you all know the number of platforms in X86 world is contained).
Will this happen for ARM64 in arch/arm64 based on vendor specific HIDs ?
No.
So given the current state of play (we were requested to move the arch/arm64 specific ACPI PCI bits to arch/arm64), we would end up with arch/arm64 code requiring code in /drivers to set-up pci_ops in a platform specific way, it is horrible, if feasible at all.
The only way this can be implemented is by pretending that the ACPI/PCI arch/arm64 implementation is generic code (that's what this series does), move it to /drivers (where it is in this series), and implement _DSD vendor specific bindings (per HID) to set-up the pci operations; whether this solution should go upstream, given that it is just a short-term solution for early platforms bugs, it is another story and my personal answer is no.
It seems like there should be a way to look for a _DSM before we call acpi_pci_root_get_mcfg_addr() to look for _CBA.
Currently we call acpi_pci_root_get_mcfg_addr() (to read _CBA) from the generic acpi_pci_root_add(), but the result (root->mcfg_addr) is only used in x86-specific code. I think it would be nicer if the lookup and the use were together. Then it would be easier to override it because the mapping assumptions would all be in one place.
I think it shouldn't be too bad to move quirk handling mechanism to arch/arm64. Effectively we would not move platform specific code into arch/arm64 but just the mechanism checking if there is any quirk that is defined.
i.e.:
extern struct pci_cfg_fixup __start_acpi_mcfg_fixups[]; extern struct pci_cfg_fixup __end_acpi_mcfg_fixups[];
static struct pci_ecam_ops *pci_acpi_get_ops(struct acpi_pci_root *root) { int bus_num = root->secondary.start; int domain = root->segment; struct pci_cfg_fixup *f;
/* * Match against platform specific quirks and return corresponding * CAM ops. * * First match against PCI topology <domain:bus> then use DMI or * custom match handler. */ for (f = __start_acpi_mcfg_fixups; f < __end_acpi_mcfg_fixups; f++) { if ((f->domain == domain || f->domain == PCI_MCFG_DOMAIN_ANY) && (f->bus_num == bus_num || f->bus_num == PCI_MCFG_BUS_ANY) && (f->system ? dmi_check_system(f->system) : 1) && (f->match ? f->match(f, root) : 1)) return f->ops; } /* No quirks, use ECAM */ return &pci_generic_ecam_ops;}
Such quirks will be defined anyway in drivers/pci/host/ in the vendor specific quirk implementations.
e.g. in HiSilicon case we would have
DECLARE_ACPI_MCFG_FIXUP(NULL, hisi_pcie_match, &hisi_pcie_ecam_ops, PCI_MCFG_DOMAIN_ANY, PCI_MCFG_BUS_ANY);
in "drivers/pci/host/pcie-hisi-acpi.c "
Thanks
Gab
Sorry Gab, I guess I was really responding to earlier messages :)
I don't really have much to say here, except that it doesn't seem right to have an MCFG that describes a non-standard ECAM mapping. I suppose there's already firmware in the field that does that, though?
Bjorn
Bjorn,
Out walking so sorry about top posting. Quick reply though:
1. I checked with the Windows team. They usually avoid quirks entirely but when it has happened, it has been done via the MCFG/FADT not DSDT.
2. They would be ok if we were to key off the OEM name and revision for the IP in the MCFG table.
3. I have already verified existing shipping ARMv8 systems provide enough unique data in that entry, and have asked that vendors guarantee to rev it in future IP (which I will verify on models pre tapeout and certainly in early firmware builds). One vendor has a platform that isn't public yet that uses a non-public name in the MCFG but I spoke with them on Friday and they will shortly update their firmware so that a quirk could be posted.
4. I have requested (and Linaro are investigating) that Linaro (with ARM's assistance) begin to drive a separate thread around upstreaming (independent of this core effort) quirks that use the OEM fields in the MCFG as a more scalable approach than one per platform via DMI.
5. I will drive a clarification to the SBBR that does not encourage or endorse quirks but does merely reinforce that data must be unique in such tables. I am driving a separate series of conversations with vendors to ensure that this is the case on all future platforms - though just generally, there is no more high end top shelf "Xeon class" silicon needing common quirks in the pipeline.
More later.
Jon.
Additional: I would like to thank Ard for suggesting this approach. It turns out (apparently) that Mark Salter's initial X-Gene quirks internal to RH did it this way as well. You great minds think alike. If this works for folks then I hope it leads to upstream kernel support in F25 (we have a bunch of Moonshot hardware we would like to deeply in Fedora but can't without the PCIe network...). I rant only because I care :)
Jon.
Hi Jon
-----Original Message----- From: Jon Masters [mailto:jcm@redhat.com] Sent: 24 May 2016 02:11 To: Bjorn Helgaas Cc: Gabriele Paoloni; Lorenzo Pieralisi; Ard Biesheuvel; Tomasz Nowicki; arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com; rafael@kernel.org; hanjun.guo@linaro.org; okaya@codeaurora.org; jchandra@broadcom.com; linaro-acpi@lists.linaro.org; linux- pci@vger.kernel.org; dhdang@apm.com; Liviu.Dudau@arm.com; ddaney@caviumnetworks.com; jeremy.linton@arm.com; linux- kernel@vger.kernel.org; linux-acpi@vger.kernel.org; robert.richter@caviumnetworks.com; Suravee.Suthikulpanit@amd.com; msalter@redhat.com; Wangyijing; mw@semihalf.com; andrea.gallo@linaro.org; linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller
Bjorn,
Out walking so sorry about top posting. Quick reply though:
- I checked with the Windows team. They usually avoid quirks entirely
but when it has happened, it has been done via the MCFG/FADT not DSDT.
- They would be ok if we were to key off the OEM name and revision
for the IP in the MCFG table.
I see some problems with this approach:
1) We would need to modify the ACPI specs to accommodate quirks in the MCFG, correct?
2) Just adding OEM info would not fit some other people (like us for Designware based solutions). In our case for example the addresses defined in the MCFG are not compatible with the ones used by the Designware IP, therefore we would also need specific quirk data
I think that we can use an approach where we use MCFG entries for the ECAM address spaces and motherboard reserved resources for those BUSes that are outside the MCFG table and therefore are non ECAM.
I think it is a more generic approach that would suit anybody and there is no need to redefine the ACPI specs for MCFG... ?
Thanks
Gab
- I have already verified existing shipping ARMv8 systems provide
enough unique data in that entry, and have asked that vendors guarantee to rev it in future IP (which I will verify on models pre tapeout and certainly in early firmware builds). One vendor has a platform that isn't public yet that uses a non-public name in the MCFG but I spoke with them on Friday and they will shortly update their firmware so that a quirk could be posted.
- I have requested (and Linaro are investigating) that Linaro (with
ARM's assistance) begin to drive a separate thread around upstreaming (independent of this core effort) quirks that use the OEM fields in the MCFG as a more scalable approach than one per platform via DMI.
- I will drive a clarification to the SBBR that does not encourage or
endorse quirks but does merely reinforce that data must be unique in such tables. I am driving a separate series of conversations with vendors to ensure that this is the case on all future platforms - though just generally, there is no more high end top shelf "Xeon class" silicon needing common quirks in the pipeline.
More later.
Jon.
-- Computer Architect | Sent from my 64-bit #ARM Powered phone
On May 23, 2016, at 19:39, Bjorn Helgaas helgaas@kernel.org wrote:
On Mon, May 23, 2016 at 03:16:01PM +0000, Gabriele Paoloni wrote: Hi Lorenzo
-----Original Message----- From: Lorenzo Pieralisi [mailto:lorenzo.pieralisi@arm.com] Sent: 23 May 2016 11:57 To: Ard Biesheuvel Cc: Gabriele Paoloni; Jon Masters; Tomasz Nowicki;
helgaas@kernel.org;
arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com; rafael@kernel.org; hanjun.guo@linaro.org; okaya@codeaurora.org; jchandra@broadcom.com; linaro-acpi@lists.linaro.org; linux- pci@vger.kernel.org; dhdang@apm.com; Liviu.Dudau@arm.com; ddaney@caviumnetworks.com; jeremy.linton@arm.com; linux- kernel@vger.kernel.org; linux-acpi@vger.kernel.org; robert.richter@caviumnetworks.com; Suravee.Suthikulpanit@amd.com; msalter@redhat.com; Wangyijing; mw@semihalf.com; andrea.gallo@linaro.org; linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI
host
controller
On Fri, May 20, 2016 at 11:14:03AM +0200, Ard Biesheuvel wrote: On 20 May 2016 at 10:40, Gabriele Paoloni
gabriele.paoloni@huawei.com wrote:
Hi Ard
> -----Original Message----- > From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org]
[...]
> > Is the PCIe root complex so special that you cannot simply
describe an
> implementation that is not PNP0408 compatible as something else,
under
> its own unique HID? If everybody is onboard with using ACPI, how
is
> this any different from describing other parts of the platform > topology? Even if the SBSA mandates generic PCI, they already
deviated
> from that when they built the hardware, so pretending that it is
a
> PNP0408 with quirks really does not buy us anything.
From my understanding we want to avoid this as this would allow
each
vendor to come up with his own code and it would be much more
effort
for the PCI maintainer to rework the PCI framework to accommodate
X86
and "all" ARM64 Host Controllers...
I guess this approach is too risky and we want to avoid this.
Through
standardization we can more easily maintain the code and scale it
to
multiple SoCs...
So this is my understanding; maybe Jon, Tomasz or Lorenzo can
give
a bit more explanation...
OK, so that boils down to recommending to vendors to represent
known
non-compliant hardware as compliant, just so that we don't have to change the code to support additional flavors of ECAM ? It's fine
to
be pragmatic, but that sucks.
We keep confusing the x86 case with the ARM case here: for x86,
they
needed to deal with broken hardware *after* the fact, and all they could do is find /some/ distinguishing feature in order to guess
which
exact hardware they might be running on. For arm64, it is the
opposite
case. We are currently in a position where we can demand vendors
to
comply with the standards they endorsed themselves, and (ab)using
ACPI
- DMI as a de facto platform description rather than plain ACPI
makes
me think the DT crowd were actually right from the beginning. It *directly* violates the standardization principle, since it
requires
a
priori knowledge inside the OS that a certain 'generic' device
must
be
driven in a special way.
So can anyone comment on the feasibility of adding support for
devices
with vendor specific HIDs (and no generic CIDs) to the current
ACPI
ECAM driver in Linux?
I don't think of ECAM support itself as a "driver". It's just a service available to drivers, similar to OF resource parsing.
Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe host bridge. "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that supports extended config space. It doesn't specify how we access
that
config space, so I think hardware with non-standard ECAM should still have PNP0A03 and PNP0A08 in _CID or _HID.
"ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware r3.2, sec 4.1) means:
(a) a memory-mapped model for config space access, and (b) a specific mapping of address bits to bus/device/function/ register
MCFG and _CBA assume both (a) and (b), so I think a device with non-standard ECAM mappings should not be described in MCFG or _CBA.
If a bridge has ECAM with non-standard mappings, I think either a vendor-specific _HID or a device-specific method, e.g., _DSM, could communicate that.
Jon, I agree that we should avoid describing non-standardized
hardware
in Linux-specific ways. Is there a mechanism in use already? How does Windows handle this? DMI is a poor long-term solution because
it
requires ongoing maintenance for new platforms, but I think it's OK for getting started with platforms already shipping.
A _DSM has the advantage that once it is defined and supported, OEMs can ship new platforms without requiring a new quirk or a new _HID to be added to a driver.
There would still be the problem of config access before the
namespace
is available, i.e., the MCFG use case. I don't know how important that is. Defining an MCFG extension seems like the most obvious solution.
If we only expect a few non-standard devices, maybe it's enough to have DMI quirks to statically set up ECAM and just live with the inconvenience of requiring a kernel change for every new non-standard device.
Host bridges in ACPI are handled through PNP0A08/PNP0A03 ids, and most of the arch specific code is handled in the respective arch directories (X86 and IA64, even though IA64 does not rely on
ECAM/MCFG
for PCI ops), it is not a driver per-se, PNP0A08/PNP0A03 are detected through ACPI scan handlers and the respective arch code (ie
pci_acpi_scan_root)
sets-up resources AND config space on an arch specific basis.
X86 deals with that with code in arch/x86 that sets-up the
pci_raw_ops
on a platform specific basis (and it is not nice, but it works
because
as you all know the number of platforms in X86 world is contained).
Will this happen for ARM64 in arch/arm64 based on vendor specific HIDs ?
No.
So given the current state of play (we were requested to move the arch/arm64 specific ACPI PCI bits to arch/arm64), we would end up with arch/arm64 code requiring code in /drivers to set-up pci_ops in a platform specific way, it is horrible, if feasible at all.
The only way this can be implemented is by pretending that the ACPI/PCI arch/arm64 implementation is generic code (that's what
this
series does), move it to /drivers (where it is in this series), and implement _DSD vendor specific bindings (per HID) to set-up the pci operations; whether this solution should go upstream, given that it is just a short-term solution for early platforms bugs, it is
another
story and my personal answer is no.
It seems like there should be a way to look for a _DSM before we call acpi_pci_root_get_mcfg_addr() to look for _CBA.
Currently we call acpi_pci_root_get_mcfg_addr() (to read _CBA) from the generic acpi_pci_root_add(), but the result (root->mcfg_addr) is only used in x86-specific code. I think it would be nicer if the lookup and the use were together. Then it would be easier to
override
it because the mapping assumptions would all be in one place.
I think it shouldn't be too bad to move quirk handling mechanism to arch/arm64. Effectively we would not move platform specific code
into
arch/arm64 but just the mechanism checking if there is any quirk
that
is defined.
i.e.:
extern struct pci_cfg_fixup __start_acpi_mcfg_fixups[]; extern struct pci_cfg_fixup __end_acpi_mcfg_fixups[];
static struct pci_ecam_ops *pci_acpi_get_ops(struct acpi_pci_root
*root)
{ int bus_num = root->secondary.start; int domain = root->segment; struct pci_cfg_fixup *f;
/* * Match against platform specific quirks and returncorresponding
* CAM ops. * * First match against PCI topology <domain:bus> then use DMIor
* custom match handler. */ for (f = __start_acpi_mcfg_fixups; f <__end_acpi_mcfg_fixups; f++) {
if ((f->domain == domain || f->domain ==PCI_MCFG_DOMAIN_ANY) &&
(f->bus_num == bus_num || f->bus_num ==PCI_MCFG_BUS_ANY) &&
(f->system ? dmi_check_system(f->system) : 1) && (f->match ? f->match(f, root) : 1)) return f->ops; } /* No quirks, use ECAM */ return &pci_generic_ecam_ops;}
Such quirks will be defined anyway in drivers/pci/host/ in the
vendor
specific quirk implementations.
e.g. in HiSilicon case we would have
DECLARE_ACPI_MCFG_FIXUP(NULL, hisi_pcie_match, &hisi_pcie_ecam_ops, PCI_MCFG_DOMAIN_ANY, PCI_MCFG_BUS_ANY);
in "drivers/pci/host/pcie-hisi-acpi.c "
Thanks
Gab
Sorry Gab, I guess I was really responding to earlier messages :)
I don't really have much to say here, except that it doesn't seem right to have an MCFG that describes a non-standard ECAM mapping. I suppose there's already firmware in the field that does that, though?
Bjorn
Hi Bjorn
-----Original Message----- From: Bjorn Helgaas [mailto:helgaas@kernel.org] Sent: 24 May 2016 00:39 To: Gabriele Paoloni Cc: Lorenzo Pieralisi; Ard Biesheuvel; Jon Masters; Tomasz Nowicki; arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com; rafael@kernel.org; hanjun.guo@linaro.org; okaya@codeaurora.org; jchandra@broadcom.com; linaro-acpi@lists.linaro.org; linux- pci@vger.kernel.org; dhdang@apm.com; Liviu.Dudau@arm.com; ddaney@caviumnetworks.com; jeremy.linton@arm.com; linux- kernel@vger.kernel.org; linux-acpi@vger.kernel.org; robert.richter@caviumnetworks.com; Suravee.Suthikulpanit@amd.com; msalter@redhat.com; Wangyijing; mw@semihalf.com; andrea.gallo@linaro.org; linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller
On Mon, May 23, 2016 at 03:16:01PM +0000, Gabriele Paoloni wrote:
Hi Lorenzo
-----Original Message----- From: Lorenzo Pieralisi [mailto:lorenzo.pieralisi@arm.com] Sent: 23 May 2016 11:57 To: Ard Biesheuvel Cc: Gabriele Paoloni; Jon Masters; Tomasz Nowicki;
helgaas@kernel.org;
arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com; rafael@kernel.org; hanjun.guo@linaro.org; okaya@codeaurora.org; jchandra@broadcom.com; linaro-acpi@lists.linaro.org; linux- pci@vger.kernel.org; dhdang@apm.com; Liviu.Dudau@arm.com; ddaney@caviumnetworks.com; jeremy.linton@arm.com; linux- kernel@vger.kernel.org; linux-acpi@vger.kernel.org; robert.richter@caviumnetworks.com; Suravee.Suthikulpanit@amd.com; msalter@redhat.com; Wangyijing; mw@semihalf.com; andrea.gallo@linaro.org; linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI
host
controller
On Fri, May 20, 2016 at 11:14:03AM +0200, Ard Biesheuvel wrote:
On 20 May 2016 at 10:40, Gabriele Paoloni
gabriele.paoloni@huawei.com wrote:
Hi Ard
-----Original Message----- From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org]
[...]
Is the PCIe root complex so special that you cannot simply
describe an
implementation that is not PNP0408 compatible as something
else,
under
its own unique HID? If everybody is onboard with using ACPI,
how
is
this any different from describing other parts of the platform topology? Even if the SBSA mandates generic PCI, they already
deviated
from that when they built the hardware, so pretending that it
is a
PNP0408 with quirks really does not buy us anything.
From my understanding we want to avoid this as this would allow
each
vendor to come up with his own code and it would be much more
effort
for the PCI maintainer to rework the PCI framework to
accommodate
X86
and "all" ARM64 Host Controllers...
I guess this approach is too risky and we want to avoid this.
Through
standardization we can more easily maintain the code and scale
it
to
multiple SoCs...
So this is my understanding; maybe Jon, Tomasz or Lorenzo can
give
a bit more explanation...
OK, so that boils down to recommending to vendors to represent
known
non-compliant hardware as compliant, just so that we don't have
to
change the code to support additional flavors of ECAM ? It's fine
to
be pragmatic, but that sucks.
We keep confusing the x86 case with the ARM case here: for x86,
they
needed to deal with broken hardware *after* the fact, and all
they
could do is find /some/ distinguishing feature in order to guess
which
exact hardware they might be running on. For arm64, it is the
opposite
case. We are currently in a position where we can demand vendors
to
comply with the standards they endorsed themselves, and (ab)using
ACPI
- DMI as a de facto platform description rather than plain ACPI
makes
me think the DT crowd were actually right from the beginning. It *directly* violates the standardization principle, since it
requires
a
priori knowledge inside the OS that a certain 'generic' device
must
be
driven in a special way.
So can anyone comment on the feasibility of adding support for
devices
with vendor specific HIDs (and no generic CIDs) to the current
ACPI
ECAM driver in Linux?
I don't think of ECAM support itself as a "driver". It's just a service available to drivers, similar to OF resource parsing.
Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe host bridge. "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that supports extended config space. It doesn't specify how we access that config space, so I think hardware with non-standard ECAM should still have PNP0A03 and PNP0A08 in _CID or _HID.
"ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware r3.2, sec 4.1) means:
(a) a memory-mapped model for config space access, and (b) a specific mapping of address bits to bus/device/function/ register
MCFG and _CBA assume both (a) and (b), so I think a device with non-standard ECAM mappings should not be described in MCFG or _CBA.
If a bridge has ECAM with non-standard mappings, I think either a vendor-specific _HID or a device-specific method, e.g., _DSM, could communicate that.
Jon, I agree that we should avoid describing non-standardized hardware in Linux-specific ways. Is there a mechanism in use already? How does Windows handle this? DMI is a poor long-term solution because it requires ongoing maintenance for new platforms, but I think it's OK for getting started with platforms already shipping.
A _DSM has the advantage that once it is defined and supported, OEMs can ship new platforms without requiring a new quirk or a new _HID to be added to a driver.
There would still be the problem of config access before the namespace is available, i.e., the MCFG use case. I don't know how important that is. Defining an MCFG extension seems like the most obvious solution.
If we only expect a few non-standard devices, maybe it's enough to have DMI quirks to statically set up ECAM and just live with the inconvenience of requiring a kernel change for every new non-standard device.
Host bridges in ACPI are handled through PNP0A08/PNP0A03 ids, and most of the arch specific code is handled in the respective arch directories (X86 and IA64, even though IA64 does not rely on
ECAM/MCFG
for PCI ops), it is not a driver per-se, PNP0A08/PNP0A03 are detected through ACPI scan handlers and the respective arch code (ie
pci_acpi_scan_root)
sets-up resources AND config space on an arch specific basis.
X86 deals with that with code in arch/x86 that sets-up the
pci_raw_ops
on a platform specific basis (and it is not nice, but it works
because
as you all know the number of platforms in X86 world is contained).
Will this happen for ARM64 in arch/arm64 based on vendor specific HIDs ?
No.
So given the current state of play (we were requested to move the arch/arm64 specific ACPI PCI bits to arch/arm64), we would end up with arch/arm64 code requiring code in /drivers to set-up pci_ops in a platform specific way, it is horrible, if feasible at all.
The only way this can be implemented is by pretending that the ACPI/PCI arch/arm64 implementation is generic code (that's what
this
series does), move it to /drivers (where it is in this series), and implement _DSD vendor specific bindings (per HID) to set-up the pci operations; whether this solution should go upstream, given that it is just a short-term solution for early platforms bugs, it is
another
story and my personal answer is no.
It seems like there should be a way to look for a _DSM before we call acpi_pci_root_get_mcfg_addr() to look for _CBA.
Currently we call acpi_pci_root_get_mcfg_addr() (to read _CBA) from the generic acpi_pci_root_add(), but the result (root->mcfg_addr) is only used in x86-specific code. I think it would be nicer if the lookup and the use were together. Then it would be easier to override it because the mapping assumptions would all be in one place.
I think it shouldn't be too bad to move quirk handling mechanism to arch/arm64. Effectively we would not move platform specific code into arch/arm64 but just the mechanism checking if there is any quirk that is defined.
i.e.:
extern struct pci_cfg_fixup __start_acpi_mcfg_fixups[]; extern struct pci_cfg_fixup __end_acpi_mcfg_fixups[];
static struct pci_ecam_ops *pci_acpi_get_ops(struct acpi_pci_root
*root)
{ int bus_num = root->secondary.start; int domain = root->segment; struct pci_cfg_fixup *f;
/* * Match against platform specific quirks and returncorresponding
* CAM ops. * * First match against PCI topology <domain:bus> then use DMIor
* custom match handler. */ for (f = __start_acpi_mcfg_fixups; f <__end_acpi_mcfg_fixups; f++) {
if ((f->domain == domain || f->domain ==PCI_MCFG_DOMAIN_ANY) &&
(f->bus_num == bus_num || f->bus_num ==PCI_MCFG_BUS_ANY) &&
(f->system ? dmi_check_system(f->system) : 1) && (f->match ? f->match(f, root) : 1)) return f->ops; } /* No quirks, use ECAM */ return &pci_generic_ecam_ops;}
Such quirks will be defined anyway in drivers/pci/host/ in the vendor specific quirk implementations.
e.g. in HiSilicon case we would have
DECLARE_ACPI_MCFG_FIXUP(NULL, hisi_pcie_match, &hisi_pcie_ecam_ops, PCI_MCFG_DOMAIN_ANY, PCI_MCFG_BUS_ANY);
in "drivers/pci/host/pcie-hisi-acpi.c "
Thanks
Gab
Sorry Gab, I guess I was really responding to earlier messages :)
I don't really have much to say here, except that it doesn't seem right to have an MCFG that describes a non-standard ECAM mapping.
The ACPI table that this mechanism relies upon is the one discussed in: https://lkml.org/lkml/2016/3/9/91
As you can see MCFG describes ECAM mappings, but we have a motherboard reserved resource outside the MCFG: Device (RES0) { Name (_HID, "HISI0081") // HiSi PCIe RC config base address Name (_CID, "PNP0C02") // Motherboard reserved resource Name (_CRS, ResourceTemplate (){ Memory32Fixed (ReadWrite, 0xb0080000 , 0x10000) }) }
This allows us to retrieve the address we need for accessing the config space on the RC (that is non-ECAM).
I was thinking that such mechanism could fit different vendors and allow them define their own quirks without spoiling the official and standard MCFG; also from the thread discussion you seemed quite ok with such solution...?
I suppose there's already firmware in the field that does that, though?
We have "experimental" firmware that is based on the ACPI table described above, however it is not widely distributed and, obviously, it is not supported by Linux mainline (so we have room to rework if we decide that another solution is more appropriate)
Thanks and Regards
Gab
Bjorn
Hi Gabriele, all,
On 05/24/2016 03:23 AM, Gabriele Paoloni wrote:
Sorry Gab, I guess I was really responding to earlier messages :)
I don't really have much to say here, except that it doesn't seem right to have an MCFG that describes a non-standard ECAM mapping.
The ACPI table that this mechanism relies upon is the one discussed in: https://lkml.org/lkml/2016/3/9/91
As you can see MCFG describes ECAM mappings, but we have a motherboard reserved resource outside the MCFG: Device (RES0) { Name (_HID, "HISI0081") // HiSi PCIe RC config base address Name (_CID, "PNP0C02") // Motherboard reserved resource Name (_CRS, ResourceTemplate (){ Memory32Fixed (ReadWrite, 0xb0080000 , 0x10000) }) }
This allows us to retrieve the address we need for accessing the config space on the RC (that is non-ECAM).
I was thinking that such mechanism could fit different vendors and allow them define their own quirks without spoiling the official and standard MCFG; also from the thread discussion you seemed quite ok with such solution...?
This could have been useful 2-3 years ago (when myself and others first pulled the fire alarm concerning the lack of upstreaming of the ACPI enablement for PCIe - which should have been fully upstream before the first platforms ever even shipped) but at this time we have shipping platforms that don't have tables built in this way. While we can go back around to vendors and try to get them to rebuild firmware, it would be by far preferable to adopt a solution that works with what is already being deployed in the field today. Such as OEM match in MCFG.
Jon.
Hi Bjorn,
On Mon, May 23, 2016 at 06:39:18PM -0500, Bjorn Helgaas wrote:
[...]
On Mon, May 23, 2016 at 03:16:01PM +0000, Gabriele Paoloni wrote: I don't think of ECAM support itself as a "driver". It's just a service available to drivers, similar to OF resource parsing.
Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe host bridge. "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that supports extended config space. It doesn't specify how we access that config space, so I think hardware with non-standard ECAM should still have PNP0A03 and PNP0A08 in _CID or _HID.
"ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware r3.2, sec 4.1) means:
(a) a memory-mapped model for config space access, and (b) a specific mapping of address bits to bus/device/function/ register
MCFG and _CBA assume both (a) and (b), so I think a device with non-standard ECAM mappings should not be described in MCFG or _CBA.
If a bridge has ECAM with non-standard mappings, I think either a vendor-specific _HID or a device-specific method, e.g., _DSM, could communicate that.
Jon, I agree that we should avoid describing non-standardized hardware in Linux-specific ways. Is there a mechanism in use already? How does Windows handle this? DMI is a poor long-term solution because it requires ongoing maintenance for new platforms, but I think it's OK for getting started with platforms already shipping.
A _DSM has the advantage that once it is defined and supported, OEMs can ship new platforms without requiring a new quirk or a new _HID to be added to a driver.
There would still be the problem of config access before the namespace is available, i.e., the MCFG use case. I don't know how important that is. Defining an MCFG extension seems like the most obvious solution.
Your summary above is a perfect representation of the situation.
We had an opportunity to sync-up on the current status of ACPI PCI for ARM64 (and talked about a way forward for this series, which includes quirks handling), let me summarize it here for everyone involved so that we can agree on a way forward.
1) ACPI PCI support for PNP0A03/PNP0A08 host bridges on top of MCFG ECAM for config space is basically ready (Tomasz and JC addressed Rafael's concerns in relation to ARM64 specific code, and managed to find a way to allocate domain numbers in preparation for Arnd pci_create_root_bus() clean-up, v8 to be posted shortly and should be final). This provides support for de-facto ACPI/PCI ECAM base standard for ARM64 (with a clean-split between generic code and ARM64 bits, where ARM64, like X86 and IA64, manages in arch code IO space and PCI resources, to be further consolidated in the near future). I do not think anyone can complain about the generality of what we achieved, for systems that are PCI standard (yes, PCI STANDARD) this would just be sufficient. 2) In a real world (1) is not enough. Some ARM64 platforms, not entirely ECAM compliant, already shipped with the corresponding firmware that we can't update. HW has ECAM quirks and to work around it in the kernel we put forward many solutions to the problem, it is time we found a solution (when, of course, (1) is completed and upstream). Using the MCFG table OEMID matching floated around in this thread would work fine for most of the platforms (and cross-OS) that have shipped with HW ECAM quirks, so I think that's the starting point for our solution and that's how we can sort this out, _today_.
The solution is a trivial look-up table: MCFG OEMID <-> PCI config space ops
3) (2) does not just work on some platforms (and we can't predict the future either - actually I can, it is three letters, ECAM), simply because MCFG OEMID matching does not provide a way to attach further data to the MCFG (eg if config space for, say, bus 0 domain 0, is not ECAM compliant, the config region can't be handled and must not be handled through a corresponding MCFG region. That's the problem Gabriele is facing and wants to solve through something like:
https://lkml.org/lkml/2016/3/9/91
in the respective ACPI tables-bindings. It may be an idea worth pursuing, it does not solve (2) simply because that FW has shipped, we can't patch it any longer.
Hence to finally support ACPI PCI on ARM64 I suggest we carry out the following steps, in order:
- Let's complete/merge (1), that's fundamental to this whole thread - On top of (1) we apply a quirking mechanism based on (2) that allows us to boot mainline with boxes shipping today with no FW update required. - We devise a way to handle quirks that is more generic than (2) so that can we can accomodate further platforms that can't rely on (2) but have more leeway in terms of FW updates.
I hope that's a reasonable plan, Tomasz's v8 series coming to kick it off.
Thank you, Lorenzo
A big +1 to the below :) :) :)
On Tue, May 24, 2016 at 06:24:23PM +0100, Lorenzo Pieralisi wrote:
Hi Bjorn,
On Mon, May 23, 2016 at 06:39:18PM -0500, Bjorn Helgaas wrote:
[...]
On Mon, May 23, 2016 at 03:16:01PM +0000, Gabriele Paoloni wrote: I don't think of ECAM support itself as a "driver". It's just a service available to drivers, similar to OF resource parsing.
Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe host bridge. "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that supports extended config space. It doesn't specify how we access that config space, so I think hardware with non-standard ECAM should still have PNP0A03 and PNP0A08 in _CID or _HID.
"ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware r3.2, sec 4.1) means:
(a) a memory-mapped model for config space access, and (b) a specific mapping of address bits to bus/device/function/ register
MCFG and _CBA assume both (a) and (b), so I think a device with non-standard ECAM mappings should not be described in MCFG or _CBA.
If a bridge has ECAM with non-standard mappings, I think either a vendor-specific _HID or a device-specific method, e.g., _DSM, could communicate that.
Jon, I agree that we should avoid describing non-standardized hardware in Linux-specific ways. Is there a mechanism in use already? How does Windows handle this? DMI is a poor long-term solution because it requires ongoing maintenance for new platforms, but I think it's OK for getting started with platforms already shipping.
A _DSM has the advantage that once it is defined and supported, OEMs can ship new platforms without requiring a new quirk or a new _HID to be added to a driver.
There would still be the problem of config access before the namespace is available, i.e., the MCFG use case. I don't know how important that is. Defining an MCFG extension seems like the most obvious solution.
Your summary above is a perfect representation of the situation.
We had an opportunity to sync-up on the current status of ACPI PCI for ARM64 (and talked about a way forward for this series, which includes quirks handling), let me summarize it here for everyone involved so that we can agree on a way forward.
- ACPI PCI support for PNP0A03/PNP0A08 host bridges on top of MCFG ECAM for config space is basically ready (Tomasz and JC addressed Rafael's concerns in relation to ARM64 specific code, and managed to find a way to allocate domain numbers in preparation for Arnd pci_create_root_bus() clean-up, v8 to be posted shortly and should be final). This provides support for de-facto ACPI/PCI ECAM base standard for ARM64 (with a clean-split between generic code and ARM64 bits, where ARM64, like X86 and IA64, manages in arch code IO space and PCI resources, to be further consolidated in the near future). I do not think anyone can complain about the generality of what we achieved, for systems that are PCI standard (yes, PCI STANDARD) this would just be sufficient.
Sounds good to me.
In a real world (1) is not enough. Some ARM64 platforms, not entirely ECAM compliant, already shipped with the corresponding firmware that we can't update. HW has ECAM quirks and to work around it in the kernel we put forward many solutions to the problem, it is time we found a solution (when, of course, (1) is completed and upstream). Using the MCFG table OEMID matching floated around in this thread would work fine for most of the platforms (and cross-OS) that have shipped with HW ECAM quirks, so I think that's the starting point for our solution and that's how we can sort this out, _today_.
The solution is a trivial look-up table: MCFG OEMID <-> PCI config space ops
Sounds reasonable to me.
- (2) does not just work on some platforms (and we can't predict the future either - actually I can, it is three letters, ECAM), simply because MCFG OEMID matching does not provide a way to attach further data to the MCFG (eg if config space for, say, bus 0 domain 0, is not ECAM compliant, the config region can't be handled and must not be handled through a corresponding MCFG region.
Couldn't this be handled by custom pci_ops that do something special for bus 0 domain 0, and default to some different pci_ops for the rest?
That's the problem Gabriele is facing and wants to solve through something like:
https://lkml.org/lkml/2016/3/9/91
in the respective ACPI tables-bindings. It may be an idea worth pursuing, it does not solve (2) simply because that FW has shipped, we can't patch it any longer.
(2) is for quirks to deal with MCFG. Gabriele's post is a proposal for ACPI namespace. We can't use anything in the namespace to implement MCFG quirks because MCFG is needed before the namespace is available.
I think Gabriele's post is a good proposal for the namespace, but I would propose the following modifications:
- Add PNP0A08 to the PCI1 _CID since this is a PCIe host bridge - Add a PCI1 _DSM describing the ECAM space - Remove the HISI0081 _HID
The result would be:
Device (PCI1) { Name(_HID, "HISI0080") Name(_CID, "PNP0A03,PNP0A08") Method(_CRS) { ... } Method(_DSM) { <describe ECAM base and mapping function> } } Device (RES0) { Name(_HID, "PNP0C02") Name(_CRS) { <describe ECAM base and size> } }
RES0 could also be contained within PCI1, as Gabriele suggested. I don't really care whether it's contained or not, and making it contained might make it easier for firmware, because addition/removal of PCI1 and RES0 should always happen together.
I think the _DSM is important because it is really ugly if the HISI0080 driver has to look for a separate HISI0081 device to learn about the ECAM space. There are several PCI drivers that do something similar, using for_each_pci_dev(), and I cringe every time I see them, because this totally screws up the driver model. A driver should claim a device via a .probe() method called by the core, and it shouldn't look at devices it hasn't claimed. This is required to make hotplug work correctly.
Hence to finally support ACPI PCI on ARM64 I suggest we carry out the following steps, in order:
- Let's complete/merge (1), that's fundamental to this whole thread
- On top of (1) we apply a quirking mechanism based on (2) that allows us to boot mainline with boxes shipping today with no FW update required.
- We devise a way to handle quirks that is more generic than (2) so that can we can accomodate further platforms that can't rely on (2) but have more leeway in terms of FW updates.
I hope that's a reasonable plan, Tomasz's v8 series coming to kick it off.
Sounds very good to me; I'm looking forward to v8.
Bjorn
Hi Bjorn many thanks for your suggestions
-----Original Message----- From: Bjorn Helgaas [mailto:helgaas@kernel.org] Sent: 24 May 2016 20:01 To: Lorenzo Pieralisi Cc: Gabriele Paoloni; Ard Biesheuvel; Jon Masters; Tomasz Nowicki; arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com; rafael@kernel.org; hanjun.guo@linaro.org; okaya@codeaurora.org; jchandra@broadcom.com; linaro-acpi@lists.linaro.org; linux- pci@vger.kernel.org; dhdang@apm.com; Liviu.Dudau@arm.com; ddaney@caviumnetworks.com; jeremy.linton@arm.com; linux- kernel@vger.kernel.org; linux-acpi@vger.kernel.org; robert.richter@caviumnetworks.com; Suravee.Suthikulpanit@amd.com; msalter@redhat.com; Wangyijing; mw@semihalf.com; andrea.gallo@linaro.org; linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller
On Tue, May 24, 2016 at 06:24:23PM +0100, Lorenzo Pieralisi wrote:
Hi Bjorn,
On Mon, May 23, 2016 at 06:39:18PM -0500, Bjorn Helgaas wrote:
[...]
On Mon, May 23, 2016 at 03:16:01PM +0000, Gabriele Paoloni wrote: I don't think of ECAM support itself as a "driver". It's just a service available to drivers, similar to OF resource parsing.
Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe host bridge. "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that supports extended config space. It doesn't specify how we access
that
config space, so I think hardware with non-standard ECAM should
still
have PNP0A03 and PNP0A08 in _CID or _HID.
"ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware r3.2, sec 4.1) means:
(a) a memory-mapped model for config space access, and (b) a specific mapping of address bits to bus/device/function/ register
MCFG and _CBA assume both (a) and (b), so I think a device with non-standard ECAM mappings should not be described in MCFG or _CBA.
If a bridge has ECAM with non-standard mappings, I think either a vendor-specific _HID or a device-specific method, e.g., _DSM, could communicate that.
Jon, I agree that we should avoid describing non-standardized
hardware
in Linux-specific ways. Is there a mechanism in use already? How does Windows handle this? DMI is a poor long-term solution because
it
requires ongoing maintenance for new platforms, but I think it's OK for getting started with platforms already shipping.
A _DSM has the advantage that once it is defined and supported,
OEMs
can ship new platforms without requiring a new quirk or a new _HID
to
be added to a driver.
There would still be the problem of config access before the
namespace
is available, i.e., the MCFG use case. I don't know how important that is. Defining an MCFG extension seems like the most obvious solution.
Your summary above is a perfect representation of the situation.
We had an opportunity to sync-up on the current status of ACPI PCI for ARM64 (and talked about a way forward for this series, which includes quirks handling), let me summarize it here for everyone involved so that we can agree on a way forward.
- ACPI PCI support for PNP0A03/PNP0A08 host bridges on top of MCFG ECAM for config space is basically ready (Tomasz and JC addressed Rafael's concerns in relation to ARM64 specific code, and managed to find a way to allocate domain numbers in preparation for Arnd pci_create_root_bus() clean-up, v8 to be posted shortly and should be final). This provides support for de-facto ACPI/PCI ECAM base standard for ARM64 (with a clean-split between generic code and
ARM64
bits, where ARM64, like X86 and IA64, manages in arch code IO
space and
PCI resources, to be further consolidated in the near future). I do not think anyone can complain about the generality of what we achieved, for systems that are PCI standard (yes, PCI STANDARD)
this
would just be sufficient.
Sounds good to me.
- In a real world (1) is not enough. Some ARM64 platforms, not
entirely
ECAM compliant, already shipped with the corresponding firmware
that
we can't update. HW has ECAM quirks and to work around it in the
kernel
we put forward many solutions to the problem, it is time we found
a
solution (when, of course, (1) is completed and upstream). Using the MCFG table OEMID matching floated around in this thread would work fine for most of the platforms (and cross-OS) that have shipped with HW ECAM quirks, so I think that's the starting point
for
our solution and that's how we can sort this out, _today_.
The solution is a trivial look-up table: MCFG OEMID <-> PCI config space ops
Sounds reasonable to me.
- (2) does not just work on some platforms (and we can't predict the future either - actually I can, it is three letters, ECAM), simply because MCFG OEMID matching does not provide a way to attach
further
data to the MCFG (eg if config space for, say, bus 0 domain 0, is
not
ECAM compliant, the config region can't be handled and must not be handled through a corresponding MCFG region.
Couldn't this be handled by custom pci_ops that do something special for bus 0 domain 0, and default to some different pci_ops for the rest?
My idea was to remove bus 0 from domain 0 MCFG (and also the other buses corresponding to the root complexes ports)
So the driver would use the ECAM access with addresses retrieved from MCFG for any devices except the RCs.
For the RC we could retrieve special addresses from the "PNP0C02" reserved resource...
That's the problem Gabriele is facing and wants to solve through something like:
https://lkml.org/lkml/2016/3/9/91
in the respective ACPI tables-bindings. It may be an idea worth pursuing, it does not solve (2) simply because that FW has
shipped,
we can't patch it any longer.
(2) is for quirks to deal with MCFG. Gabriele's post is a proposal for ACPI namespace. We can't use anything in the namespace to implement MCFG quirks because MCFG is needed before the namespace is available.
Honestly I have to dig better into this (and I will once V8 is out), However from my current understanding so far we have look-up where MCFG OEMID is going to tell which specific pci-ops are going to be used.
Now from my quirk idea I need to retrieve the "special address" from the ACPI namespace before cfg rd/wr take place...if this is not doable then I need to find a different solution...than can be the one you proposed below (many thanks for this).
I will look into details later on once v8 is out.
Thanks
Gab
I think Gabriele's post is a good proposal for the namespace, but I would propose the following modifications:
- Add PNP0A08 to the PCI1 _CID since this is a PCIe host bridge
- Add a PCI1 _DSM describing the ECAM space
- Remove the HISI0081 _HID
The result would be:
Device (PCI1) { Name(_HID, "HISI0080") Name(_CID, "PNP0A03,PNP0A08") Method(_CRS) { ... } Method(_DSM) { <describe ECAM base and mapping function> } } Device (RES0) { Name(_HID, "PNP0C02") Name(_CRS) { <describe ECAM base and size> } }
RES0 could also be contained within PCI1, as Gabriele suggested. I don't really care whether it's contained or not, and making it contained might make it easier for firmware, because addition/removal of PCI1 and RES0 should always happen together.
I think the _DSM is important because it is really ugly if the HISI0080 driver has to look for a separate HISI0081 device to learn about the ECAM space. There are several PCI drivers that do something similar, using for_each_pci_dev(), and I cringe every time I see them, because this totally screws up the driver model. A driver should claim a device via a .probe() method called by the core, and it shouldn't look at devices it hasn't claimed. This is required to make hotplug work correctly.
Hence to finally support ACPI PCI on ARM64 I suggest we carry out the following steps, in order:
- Let's complete/merge (1), that's fundamental to this whole thread
- On top of (1) we apply a quirking mechanism based on (2) that
allows
us to boot mainline with boxes shipping today with no FW update
required.
- We devise a way to handle quirks that is more generic than (2) so
that
can we can accomodate further platforms that can't rely on (2) but have more leeway in terms of FW updates.
I hope that's a reasonable plan, Tomasz's v8 series coming to kick it
off.
Sounds very good to me; I'm looking forward to v8.
Bjorn
Hi Lorenzo
-----Original Message----- From: Lorenzo Pieralisi [mailto:lorenzo.pieralisi@arm.com] Sent: 24 May 2016 18:24 To: Bjorn Helgaas Cc: Gabriele Paoloni; Ard Biesheuvel; Jon Masters; Tomasz Nowicki; arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com; rafael@kernel.org; hanjun.guo@linaro.org; okaya@codeaurora.org; jchandra@broadcom.com; linaro-acpi@lists.linaro.org; linux- pci@vger.kernel.org; dhdang@apm.com; Liviu.Dudau@arm.com; ddaney@caviumnetworks.com; jeremy.linton@arm.com; linux- kernel@vger.kernel.org; linux-acpi@vger.kernel.org; robert.richter@caviumnetworks.com; Suravee.Suthikulpanit@amd.com; msalter@redhat.com; Wangyijing; mw@semihalf.com; andrea.gallo@linaro.org; linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller
Hi Bjorn,
On Mon, May 23, 2016 at 06:39:18PM -0500, Bjorn Helgaas wrote:
[...]
On Mon, May 23, 2016 at 03:16:01PM +0000, Gabriele Paoloni wrote: I don't think of ECAM support itself as a "driver". It's just a service available to drivers, similar to OF resource parsing.
Per PCI Firmware r3.2, sec 4.1.5, "PNP0A03" means a PCI/PCI-X/PCIe host bridge. "PNP0A08" means a PCI-X Mode 2 or PCIe bridge that supports extended config space. It doesn't specify how we access
that
config space, so I think hardware with non-standard ECAM should still have PNP0A03 and PNP0A08 in _CID or _HID.
"ECAM" as used in the specs (PCIe r3.0, sec 7.2.2, and PCI Firmware r3.2, sec 4.1) means:
(a) a memory-mapped model for config space access, and (b) a specific mapping of address bits to bus/device/function/ register
MCFG and _CBA assume both (a) and (b), so I think a device with non-standard ECAM mappings should not be described in MCFG or _CBA.
If a bridge has ECAM with non-standard mappings, I think either a vendor-specific _HID or a device-specific method, e.g., _DSM, could communicate that.
Jon, I agree that we should avoid describing non-standardized
hardware
in Linux-specific ways. Is there a mechanism in use already? How does Windows handle this? DMI is a poor long-term solution because
it
requires ongoing maintenance for new platforms, but I think it's OK for getting started with platforms already shipping.
A _DSM has the advantage that once it is defined and supported, OEMs can ship new platforms without requiring a new quirk or a new _HID to be added to a driver.
There would still be the problem of config access before the
namespace
is available, i.e., the MCFG use case. I don't know how important that is. Defining an MCFG extension seems like the most obvious solution.
Your summary above is a perfect representation of the situation.
We had an opportunity to sync-up on the current status of ACPI PCI for ARM64 (and talked about a way forward for this series, which includes quirks handling), let me summarize it here for everyone involved so that we can agree on a way forward.
- ACPI PCI support for PNP0A03/PNP0A08 host bridges on top of MCFG ECAM for config space is basically ready (Tomasz and JC addressed Rafael's concerns in relation to ARM64 specific code, and managed to find a way to allocate domain numbers in preparation for Arnd pci_create_root_bus() clean-up, v8 to be posted shortly and should be final). This provides support for de-facto ACPI/PCI ECAM base standard for ARM64 (with a clean-split between generic code and
ARM64 bits, where ARM64, like X86 and IA64, manages in arch code IO space and PCI resources, to be further consolidated in the near future). I do not think anyone can complain about the generality of what we achieved, for systems that are PCI standard (yes, PCI STANDARD) this would just be sufficient. 2) In a real world (1) is not enough. Some ARM64 platforms, not entirely ECAM compliant, already shipped with the corresponding firmware that we can't update. HW has ECAM quirks and to work around it in the kernel we put forward many solutions to the problem, it is time we found a solution (when, of course, (1) is completed and upstream). Using the MCFG table OEMID matching floated around in this thread would work fine for most of the platforms (and cross-OS) that have shipped with HW ECAM quirks, so I think that's the starting point for our solution and that's how we can sort this out, _today_.
The solution is a trivial look-up table: MCFG OEMID <-> PCI config space ops
- (2) does not just work on some platforms (and we can't predict the future either - actually I can, it is three letters, ECAM), simply because MCFG OEMID matching does not provide a way to attach further data to the MCFG (eg if config space for, say, bus 0 domain 0, is
not ECAM compliant, the config region can't be handled and must not be handled through a corresponding MCFG region. That's the problem Gabriele is facing and wants to solve through something like:
https://lkml.org/lkml/2016/3/9/91
in the respective ACPI tables-bindings. It may be an idea worth pursuing, it does not solve (2) simply because that FW has shipped, we can't patch it any longer.
Hence to finally support ACPI PCI on ARM64 I suggest we carry out the following steps, in order:
- Let's complete/merge (1), that's fundamental to this whole thread
- On top of (1) we apply a quirking mechanism based on (2) that allows us to boot mainline with boxes shipping today with no FW update
required.
- We devise a way to handle quirks that is more generic than (2) so
that can we can accomodate further platforms that can't rely on (2) but have more leeway in terms of FW updates.
I hope that's a reasonable plan, Tomasz's v8 series coming to kick it off.
Thanks for summarizing.
100% agree on the summary and next steps.
Cheers
Gab
Thank you, Lorenzo
On 05/23/2016 06:56 AM, Lorenzo Pieralisi wrote:
The only way this can be implemented is by pretending that the ACPI/PCI arch/arm64 implementation is generic code (that's what this series does), move it to /drivers (where it is in this series), and implement _DSD vendor specific bindings (per HID) to set-up the pci operations; whether this solution should go upstream, given that it is just a short-term solution for early platforms bugs, it is another story and my personal answer is no.
Just for completeness, let me also followup to this one.
We have real, shipping, systems in the field based on ARMv8. For example, HPE Moonshot ProLiant m400. Not everyone loves the first generation of anything (Applied get a lot of stick, but the reality is that someone had to go first, and whoever that was was going to learn all of the lessons that others don't need to) but it is out there and we need an upstream kernel solution that includes support for that.
In the server world, we (speaking as a major distro vendor here) are not going to entertain a situation in which non-upstream patches are needed to even boot a platform. That simply won't do. We need to separate out the issue of getting the core in place from the quirks, but then we need quirks that include support for all early ARMv8 platforms that are out there today. If we can't get to a point where a Moonshot[0] cartridge boots out of the box with an upstream kernel, let's just give up and do something else instead :) (joke)
Jon.
[0] HPE have been *amazingly* patient with this stuff. They've reworked the firmware when someone (cough) pointed out that the early stuff they'd been fed was not built according to the standards (U-Boot). They have *really good* UEFI and ACPI enabled firmware that is running RHEL(SA) great. But that's not good enough. We don't ship a distro with hacks. We ship a distro derived from upstream code (although we might have to backport a lot of stuff later). There's wiggle room, but there is not wiggle room for core platforms. On ARM, users and developers *will* be able to take an upstream kernel and boot it on their RHEL install. And they *will* be able to reproduce problems against upstream, and help develop against upstream, and further the upstream first mentality in the ARM ecosystem. There will not be "oh well, it runs RHEL so that's good enough for the early generation...".
Hi Ard, Jon
-----Original Message----- From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org] Sent: 20 May 2016 08:38 To: Jon Masters Cc: Tomasz Nowicki; Gabriele Paoloni; helgaas@kernel.org; arnd@arndb.de; will.deacon@arm.com; catalin.marinas@arm.com; rafael@kernel.org; hanjun.guo@linaro.org; Lorenzo.Pieralisi@arm.com; okaya@codeaurora.org; jchandra@broadcom.com; linaro-acpi@lists.linaro.org; linux- pci@vger.kernel.org; dhdang@apm.com; Liviu.Dudau@arm.com; ddaney@caviumnetworks.com; jeremy.linton@arm.com; linux- kernel@vger.kernel.org; linux-acpi@vger.kernel.org; robert.richter@caviumnetworks.com; Suravee.Suthikulpanit@amd.com; msalter@redhat.com; Wangyijing; mw@semihalf.com; andrea.gallo@linaro.org; linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH V7 00/11] Support for generic ACPI based PCI host controller
On 20 May 2016 at 06:41, Jon Masters jcm@redhat.com wrote:
Hi Tomasz, all,
On 05/11/2016 07:08 AM, Tomasz Nowicki wrote:
On 11.05.2016 12:41, Gabriele Paoloni wrote:
v6 -> v7
- drop quirks handling
Maybe I missed something in the v6 discussion thread; when was it decided to drop quirk handling?
I had such requests in previous series.
A quick note on quirk handling. This, I believe, applies post-merge
of
the base infrastructure, which I realize will likely not have quirks.
We've some "gen1" ARMv8 server platforms where we end up doing quirks (for things like forcing 32-bit config space accessors and the like)
due
to people repurposing existing embedded PCIe IP blocks or using them
for
the first time (especially in servers), and those being involved in
the
design not necessarily seeing this problem ahead of time, or not realizing that it would be an issue for servers. In the early days of ARM server designs 3-4 years ago, many of us had never really played with ECAM or realized how modern topologies are built.
Anyway. We missed this one in our SBSA requirements. They say (words
to
the effect of) "thou shalt do PCIe the way it is done on servers" but they aren't prescriptive, and they don't tell people how that
actually
is in reality. That is being fixed. A lot of things are happening
behind
the scenes - especially with third party IP block providers (all of
whom
myself and others are speaking with directly about this) - to ensure that the next wave of designs won't repeat these mistakes. We don't
have
a time machine, but we can contain this from becoming an ongoing mess for upstream, and we will do so. It won't be a zoo.
Various proposals have arisen for how to handle quirks in the longer term, including elaborate frameworks and tables to describe them generically. I would like to caution against such approaches,
especially
in the case that they deviate from practice on x86, or prior to being standardized fully with other Operating System vendors. I don't
expect
there to be too many more than the existing initial set of quirks we have seen posted. A number of "future" server SoCs have already been fixed prior to silicon, and new design starts are being warned not to make this a problem for us to have to clean up later.
So, I would like to suggest that the eventual framework mirror the existing approach on x86 systems (matching DMI, etc.) and not be made into some kind of generic, utopia. This is a case where we want there
to
be pain involved (and upstream patches required) when people screw up, so that they have a level of pain in response to ever making this mistake in the future. If we try to create too grand a generic scheme and make it too easy to handle this kind of situation beyond the
small
number of existing offenders, we undermine efforts to force vendors
to
ensure that their IP blocks are compliant going forward.
I understand that there is a desire from the RedHat side to mimic x86 as closely as possible, but I never saw any technical justification for that. DMI contains strings that are visible to userland, and you effectively lock those down to certain values just so that the kernel can distinguish a broken PCIe root complex from a working one. Linux on x86 had no choice, since the overwhelming majority of existing hardware misrepresented itself as generic, and DMI was the only thing available to actually distinguish these broken implementations from one another. This does not mean we should allow and/or encourage this first gen hardware to misrepresent non-compliant hardware as compliant as well.
Since you are talking to all the people involved, how about you convince them to put something in the ACPI tables that allows the kernel to distinguish those non-standard PCIe implementations from hardware that is really generic? This way, we can sidestep the quirks debate entirely, since it will simply be a different device as far as the kernel is concerned. This is no worse than a quirk from a practical point of view, since an older OS will be equally unable to run on newer hardware, but it is arguably more true to the standards compliance you tend to preach about, especially since this small pool of third party IP could potentially be identified directly rather than based on some divination of the SoC we may or may not be running on. I am also convinced that adding support for an additional HID() to the ACPI ECAM driver with some special config space handling wired in is an easier sell upstream than making the same ugly mess x86 has had to make because they did not have any choice to begin with.
In our case (HiSilicon Hip05/Hip06) we are using the Designware IP that unfortunately is non-ECAM for the RC config space.
A possible ACPI table solution was discussed already in this thread
https://lkml.org/lkml/2016/3/14/722
where <<Name (_CID, "PNP0C02") // Motherboard reserved resource>> is used to specify an Host Controller specific resource. It looks to me that this can be an approach that can accommodate different vendors/scenarios and Bjorn seemed to be quite ok with it.
It comes without saying that for future HW releases we all should make an effort to deliver fully ECAM compliant controllers.
What's your view about this approach?
Thanks
Gab
If we do need a quirks handling mechanism, I still don't see how the x86 situation extrapolates to ARM. ACPI offers plenty of ways for a SoC vendor to identify the make and particular revision, and quirks could be keyed off of that.
-- Ard.
On 05/20/2016 04:11 AM, Gabriele Paoloni wrote:
Hi Ard, Jon
Hi Gabriele :)
In our case (HiSilicon Hip05/Hip06) we are using the Designware IP that unfortunately is non-ECAM for the RC config space.
Yea, I know, and I've pinged them already.
A possible ACPI table solution was discussed already in this thread
https://lkml.org/lkml/2016/3/14/722
where <<Name (_CID, "PNP0C02") // Motherboard reserved resource>> is used to specify an Host Controller specific resource. It looks to me that this can be an approach that can accommodate different vendors/scenarios and Bjorn seemed to be quite ok with it.
Yeah, pondering that. We'll chat with a few others about it.
It comes without saying that for future HW releases we all should make an effort to deliver fully ECAM compliant controllers.
Right. Like I said, a number of designs have been fixed already.
What's your view about this approach?
Will followup over the weekend.
Jon.
On Tue, May 10, 2016 at 8:19 AM, Tomasz Nowicki tn@semihalf.com wrote:
From the functionality point of view this series may be split into the following logic parts:
- New ECAM API and update for users of the pci-host-common API
- Necessary fixes as the preparation for using driver on ARM64.
- Use new MCFG interface and implement generic ACPI based PCI host controller driver.
- Enable above driver on ARM64
Patches has been built on top of 4.6-rc7 and can be found here: git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v7)
This has been tested on Cavium ThunderX server. Any help in reviewing and testing is very appreciated.
I tried this series on APM X-Gene platforms (with ECAM fix-up quirk added) and PCIe works fine.
Regards, Duc Dang.
v6 -> v7
- drop quirks handling
- changes for ACPI companion and domain number assignment approach
- implement arch pcibios_{add|remove}_bus and call acpi_pci_{add|remove}_bus from there
- cleanups around nomenclature
- use resources oriented API for ECAM
- fix for based address calculation before mapping ECAM region
- remove useless lock for MCFG lookup
- move MCFG stuff to separated file pci_mcfg.c
- drop MCFG entries caching
- rebase against 4.6-rc7
v5 -> v6
- drop idea of x86 MMCONFIG code refactoring
- integrate JC's patches which introduce new ECAM API: https://lkml.org/lkml/2016/4/11/907 git: https://github.com/jchandra-brcm/linux/ (arm64-acpi-pci-v3)
- integrate Sinan's fix for releasing IO resources, see patch [06/13]
- added ACPI support for ThunderX ECAM and PEM drivers
- rebase against 4.6-rc2
v4 -> v5
- drop MCFG refactoring group patches 1-6 from series v4 and integrate Jayachandran's patch https://patchwork.ozlabs.org/patch/575525/
- rewrite PCI legacy IRQs allocation
- squash two patches 11 and 12 from series v4, fixed bisection issue
- changelog improvements
- rebase against 4.5-rc3
v3 -> v4
- drop Jiang's fix http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04318.html
- add Lorenzo's fix patch 19/24
- ACPI PCI bus domain number assigning cleanup
- change resource management, we now claim and reassign resources
- improvements for applying quirks
- drop Matthew's http://www.spinics.net/lists/linux-pci/msg45950.html dependency
- rebase against 4.5-rc1
v2 -> v3
- fix legacy IRQ assigning and IO ports registration
- remove reference to arch specific companion device for ia64
- move ACPI PCI host controller driver to pci_root.c
- drop generic domain assignment for x86 and ia64 as I am not able to run all necessary test variants
- drop patch which cleaned legacy IRQ assignment since it belongs to Mathew's series: https://patchwork.ozlabs.org/patch/557504/
- extend MCFG quirk code
- rebase against 4.4
v1 -> v2
- move non-arch specific piece of code to dirver/acpi/ directory
- fix IO resource handling
- introduce PCI config accessors quirks matching
- moved ACPI_COMPANION_SET to generic code
v1 - https://lkml.org/lkml/2015/10/27/504 v2 - https://lkml.org/lkml/2015/12/16/246 v3 - http://lkml.iu.edu/hypermail/linux/kernel/1601.1/04308.html v4 - https://lkml.org/lkml/2016/2/4/646 v5 - https://lkml.org/lkml/2016/2/16/426 v6 - https://lkml.org/lkml/2016/4/15/594
Jayachandran C (2): PCI: Provide common functions for ECAM mapping PCI: generic, thunder: update to use generic ECAM API
Tomasz Nowicki (9): pci, of: Move the PCI I/O space management to PCI core code. pci: Add new function to unmap IO resources. acpi, pci: Support IO resources when parsing PCI host bridge resources. pci, acpi: Provide a way to assign bus domain number. pci, acpi: Handle ACPI companion assignment. pci, acpi: Support for ACPI based generic PCI host controller arm64, pci, acpi: ACPI support for legacy IRQs parsing and consolidation with DT code. arm64, pci, acpi: Provide ACPI-specific prerequisites for PCI bus enumeration. arm64, pci, acpi: Start using ACPI based PCI host controller driver for ARM64.
arch/arm64/Kconfig | 1 + arch/arm64/kernel/pci.c | 34 +++----- drivers/acpi/Kconfig | 8 ++ drivers/acpi/Makefile | 1 + drivers/acpi/pci_mcfg.c | 97 ++++++++++++++++++++++ drivers/acpi/pci_root.c | 33 ++++++++ drivers/acpi/pci_root_generic.c | 149 +++++++++++++++++++++++++++++++++ drivers/of/address.c | 116 +------------------------- drivers/pci/Kconfig | 3 + drivers/pci/Makefile | 2 + drivers/pci/ecam.c | 161 ++++++++++++++++++++++++++++++++++++ drivers/pci/ecam.h | 72 ++++++++++++++++ drivers/pci/host/Kconfig | 1 + drivers/pci/host/pci-host-common.c | 114 +++++++++++-------------- drivers/pci/host/pci-host-common.h | 47 ----------- drivers/pci/host/pci-host-generic.c | 52 +++--------- drivers/pci/host/pci-thunder-ecam.c | 39 ++------- drivers/pci/host/pci-thunder-pem.c | 92 ++++++++++----------- drivers/pci/pci.c | 150 ++++++++++++++++++++++++++++++++- drivers/pci/probe.c | 2 + include/linux/of_address.h | 9 -- include/linux/pci-acpi.h | 14 ++++ include/linux/pci.h | 11 ++- 23 files changed, 823 insertions(+), 385 deletions(-) create mode 100644 drivers/acpi/pci_mcfg.c create mode 100644 drivers/acpi/pci_root_generic.c create mode 100644 drivers/pci/ecam.c create mode 100644 drivers/pci/ecam.h delete mode 100644 drivers/pci/host/pci-host-common.h
-- 1.9.1
On 05/10/2016 10:19 AM, Tomasz Nowicki wrote:
From the functionality point of view this series may be split into the following logic parts:
- New ECAM API and update for users of the pci-host-common API
- Necessary fixes as the preparation for using driver on ARM64.
- Use new MCFG interface and implement generic ACPI based PCI host controller driver.
- Enable above driver on ARM64
Thomasz,
Unless i'm doing something wrong, this patch causes a build break if NUMA is enabled in a 4.6+ kernel.
Thanks,
drivers/built-in.o: In function `pci_device_add': /home/jlinton/linux-main/drivers/pci/probe.c:1744: undefined reference to `pcibus_to_node' drivers/built-in.o: In function `pci_create_root_bus': /home/jlinton/linux-main/drivers/pci/probe.c:2163: undefined reference to `pcibus_to_node' drivers/built-in.o: In function `cpulistaffinity_show': /home/jlinton/linux-main/drivers/pci/pci-sysfs.c:123: undefined reference to `pcibus_to_node' /home/jlinton/linux-main/drivers/pci/pci-sysfs.c:123: undefined reference to `pcibus_to_node' drivers/built-in.o: In function `cpuaffinity_show': /home/jlinton/linux-main/drivers/pci/pci-sysfs.c:114: undefined reference to `pcibus_to_node' drivers/built-in.o:/home/jlinton/linux-main/drivers/pci/pci-sysfs.c:114: more undefined references to `pcibus_to_node' follow Makefile:937: recipe for target 'vmlinux' failed
On 05/10/2016 11:19 AM, Tomasz Nowicki wrote:
This has been tested on Cavium ThunderX server. Any help in reviewing and testing is very appreciated.
Boot log from Intel Itanium 2 system attached.
Jon.
在 2016/5/10 23:19, Tomasz Nowicki 写道:
From the functionality point of view this series may be split into the
following logic parts:
- New ECAM API and update for users of the pci-host-common API
- Necessary fixes as the preparation for using driver on ARM64.
- Use new MCFG interface and implement generic ACPI based PCI host controller driver.
- Enable above driver on ARM64
Patches has been built on top of 4.6-rc7 and can be found here: git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v7)
This has been tested on Cavium ThunderX server. Any help in reviewing and testing is very appreciated.
Based on the patchset (with ECAM fix-up quirk added) and added the patch(Add ACPI support for HiSilicon PCIe Host Controllers). Tested on the HiSilicon ARM64 D02 board. It can work ok with Intel 82599 networking card. This is the bootup log which contains PCIe host and Intel 82599 networking card part.
Tested-by: Dongdong Liu liudongdong3@huawei.com
EFI stub: Booting Linux Kernel... EFI stub: Using DTB from configuration table EFI stub: Exiting boot services and installing virtual address map... GMAC ExitBootServicesEvent SMMU ExitBootServicesEvent [ 0.000000] Booting Linux on physical CPU 0x20000 [ 0.000000] Linux version 4.6.0-rc1+ (l00290354@linux-ioko) (gcc version 4.9.3 20150211 (prerelease) (20150316) ) #43 SMP PREEMPT Thu May 19 17:15:30 CST 2016 [ 0.000000] Boot CPU: AArch64 Processor [411fd071] [ 0.000000] earlycon: uart8250 at MMIO32 0x0000000080300000 (options '') [ 0.000000] bootconsole [uart8250] enabled [ 0.000000] efi: Getting EFI parameters from FDT: [ 0.000000] EFI v2.50 by EDK II [ 0.000000] efi: SMBIOS=0x7a650000 SMBIOS 3.0=0x7a630000 ACPI=0x7aba0000 ACPI 2.0=0x7aba0014 [ 0.000000] cma: Reserved 16 MiB at 0x000000007e800000 [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x000000007ABA0014 000024 (v02 HISI ) [ 0.000000] ACPI: XSDT 0x000000007A7000E8 000064 (v01 HISI HISI-D02 20140727 01000013) [ 0.000000] ACPI: FACP 0x000000007A5F0000 00010C (v05 HISI HISI-D02 20140727 HISI 00000099) [ 0.000000] ACPI: DSDT 0x000000007A5A0000 001656 (v01 HISI HISI-D02 20140727 INTL 20150619) [ 0.000000] ACPI: DBG2 0x000000007A610000 00005A (v00 HISI HISI-D02 20140727 HISI 00000099) [ 0.000000] ACPI: GTDT 0x000000007A5E0000 000060 (v02 HISI HISI-D02 20140727 HISI 00000099) [ 0.000000] ACPI: APIC 0x000000007A5D0000 000554 (v01 HISI HISI-D02 20140727 HISI 00000099) [ 0.000000] ACPI: MCFG 0x000000007A5C0000 00004C (v01 HISI HISI-D02 20140727 HISI 00000099) [ 0.000000] ACPI: SPCR 0x000000007A5B0000 000050 (v02 HISI HISI-D02 20140727 HISI 00000099) [ 0.000000] ACPI: IORT 0x000000007A590000 0001FC (v00 INTEL TEMPLATE 00000000 INTL 20150619) [ 0.000000] ACPI: SSDT 0x000000007A580000 00046E (v01 HISI SAS0 20140727 INTL 20150619) [ 0.000000] ACPI: SPCR: console: uart,mmio,0x80300000,115200 [ 0.000000] psci: probing for conduit method from ACPI. NOTICE: [psci_smc_handler]:[347L] PSCI_VERSION CALL NOTICE: [psci_version]:[99L] PSCI_MAJOR_VER: 10000: PSCI_MINOR_VER: 0
0808?844 [ 0.000000] psci: PSCIv1.0 detected in firmware. [ 0.000000] psci: Using standard PSCI v0.2 function IDs
0808?844 [ 0.000000] psci: MIGRATE_INFO_TYPE not supported.
0808?844
0808?844 [ 0.000000] percpu: Embedded 20 pages/cpu @ffffffd1ffe7e000 s43008 r8192 d30720 u81920 [ 0.000000] Detected PIPT I-cache on CPU0 [ 0.000000] CPU features: enabling workaround for ARM erratum 832075 [ 0.000000] CPU features: enabling workaround for ARM erratum 834220 [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 2063376 [ 0.000000] Kernel command line: console=ttyS0,115200 earlycon=uart8250,mmio32,0x80300000 initrd=filesystem.cpio.gz acpi=force pcie_aspm=off [ 0.000000] PCIe ASPM is disabled [ 0.000000] log_buf_len individual max cpu contribution: 4096 bytes [ 0.000000] log_buf_len total cpu_extra contributions: 61440 bytes [ 0.000000] log_buf_len min size: 16384 bytes [ 0.000000] log_buf_len: 131072 bytes [ 0.000000] early log buf free: 12988(79%) [ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes) [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes) [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes) [ 0.000000] software IO TLB [mem 0x764e0000-0x7a4e0000] (64MB) mapped at [ffffffc0764e0000-ffffffc07a4dffff] [ 0.000000] Memory: 8110296K/8384512K available (7240K kernel code, 632K rwdata, 3028K rodata, 840K init, 247K bss, 257832K reserved, 16384K cma-reserved) [ 0.000000] Virtual kernel memory layout: [ 0.000000] modules : 0xffffff8000000000 - 0xffffff8008000000 ( 128 MB) [ 0.000000] vmalloc : 0xffffff8008000000 - 0xffffffbdbfff0000 ( 246 GB) [ 0.000000] .text : 0xffffff8008080000 - 0xffffff8008790000 ( 7232 KB) [ 0.000000] .rodata : 0xffffff8008790000 - 0xffffff8008a89000 ( 3044 KB) [ 0.000000] .init : 0xffffff8008a89000 - 0xffffff8008b5b000 ( 840 KB) [ 0.000000] .data : 0xffffff8008b5b000 - 0xffffff8008bf9200 ( 633 KB) [ 0.000000] vmemmap : 0xffffffbdc0000000 - 0xffffffbfc0000000 ( 8 GB maximum) [ 0.000000] 0xffffffbdc0000000 - 0xffffffbe08000000 ( 1152 MB actual) [ 0.000000] fixed : 0xffffffbffe7fd000 - 0xffffffbffec00000 ( 4108 KB) [ 0.000000] PCI I/O : 0xffffffbffee00000 - 0xffffffbfffe00000 ( 16 MB) [ 0.000000] memory : 0xffffffc000000000 - 0xffffffd200000000 ( 73728 MB) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=16, Nodes=1 [ 0.000000] Preemptible hierarchical RCU implementation. [ 0.000000] Build-time adjustment of leaf fanout to 64. [ 0.000000] RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=16. [ 0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=16 [ 0.000000] NR_IRQS:64 nr_irqs:64 0 [ 0.000000] GIC: Using split EOI/Deactivate mode [ 0.000000] ITS@0x8c000000 [ 0.000000] ITS: allocated 65536 Devices @11f6c80000 (psz 4K, shr 1) [ 0.000000] ITS: allocated 512 Virtual CPUs @11f6c0f000 (psz 4K, shr 1) [ 0.000000] ITS: allocated 512 Interrupt Collections @11f6c20000 (psz 4K, shr 1) [ 0.000000] ITS@0xc6000000 [ 0.000000] ITS: allocated 65536 Devices @11f6d00000 (psz 4K, shr 1) [ 0.000000] ITS: allocated 512 Virtual CPUs @11f6c21000 (psz 4K, shr 1) [ 0.000000] ITS: allocated 512 Interrupt Collections @11f6c22000 (psz 4K, shr 1) [ 0.000000] ITS@0xa3000000 [ 0.000000] ITS: allocated 65536 Devices @11f6d80000 (psz 4K, shr 1) [ 0.000000] ITS: allocated 512 Virtual CPUs @11f6c24000 (psz 4K, shr 1) [ 0.000000] ITS: allocated 512 Interrupt Collections @11f6c25000 (psz 4K, shr 1) [ 0.000000] ITS@0xb7000000 [ 0.000000] ITS: allocated 65536 Devices @11f6e00000 (psz 4K, shr 1) [ 0.000000] ITS: allocated 512 Virtual CPUs @11f6c26000 (psz 4K, shr 1) [ 0.000000] ITS: allocated 512 Interrupt Collections @11f6c27000 (psz 4K, shr 1) [ 0.000000] GIC: using LPI property table @0x00000011f6c60000 [ 0.000000] ITS: Allocated 1792 chunks for LPIs [ 0.000000] CPU0: found redistributor 20000 region 0:0x000000008d100000 [ 0.000000] CPU0: using LPI pending table @0x00000011f6c70000 [ 0.000000] Unable to get hardware information used for virtualization [ 0.000000] GTDT: No Platform Timer structures. [ 0.000000] arch_timer: Can't find GT Block. [ 0.000000] Architected cp15 and mmio timer(s) running at 50.00MHz (phys/phys). [ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xb8812736b, max_idle_ns: 440795202655 ns [ 0.000001] sched_clock: 56 bits at 50MHz, resolution 20ns, wraps every 4398046511100ns [ 0.008025] Console: colour dummy device 80x25 [ 0.012459] Calibrating delay loop (skipped), value calculated using timer frequency.. 100.00 BogoMIPS (lpj=200000) [ 0.022850] pid_max: default: 32768 minimum: 301 [ 0.027450] ACPI: Core revision 20160108 [ 0.032284] ACPI: 2 ACPI AML tables successfully acquired and loaded [ 0.038617] [ 0.040126] Security Framework initialized [ 0.044213] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes) [ 0.051053] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes) [ 0.058747] ASID allocator initialised with 65536 entries [ 0.064393] PCI/MSI: ITS@0x8c000000 domain created [ 0.069162] PCI/MSI: ITS@0xc6000000 domain created [ 0.073927] PCI/MSI: ITS@0xa3000000 domain created [ 0.078693] PCI/MSI: ITS@0xb7000000 domain created [ 0.083463] Platform MSI: irqchip@000000008c000000 domain created [ 0.089525] Platform MSI: irqchip@00000000c6000000 domain created [ 0.095586] Platform MSI: irqchip@00000000a3000000 domain created [ 0.101648] Platform MSI: irqchip@00000000b7000000 domain created [ 0.107753] Remapping and enabling EFI services. [ 0.112370] EFI remap 0x000000007a4e0000 => 0000000020000000 [ 0.118185] EFI remap 0x000000007a530000 => 0000000020050000 [ 0.124008] EFI remap 0x000000007a620000 => 00000000200a0000 [ 0.129822] EFI remap 0x000000007a6b0000 => 0000000020130000 [ 0.135636] EFI remap 0x000000007a710000 => 0000000020180000 [ 0.141454] EFI remap 0x000000007a760000 => 00000000201d0000 [ 0.147269] EFI remap 0x000000007a7b0000 => 0000000020220000 [ 0.153083] EFI remap 0x000000007a800000 => 0000000020270000 [ 0.158897] EFI remap 0x000000007a850000 => 00000000202c0000 [ 0.164711] EFI remap 0x000000007a8a0000 => 0000000020310000 [ 0.170525] EFI remap 0x000000007a8f0000 => 0000000020360000 [ 0.176339] EFI remap 0x000000007a940000 => 00000000203b0000 [ 0.182160] EFI remap 0x000000007a990000 => 0000000020400000 [ 0.187974] EFI remap 0x000000007aa00000 => 0000000020470000 [ 0.193788] EFI remap 0x000000007aa50000 => 00000000204c0000 [ 0.199602] EFI remap 0x000000007aaa0000 => 0000000020510000 [ 0.205417] EFI remap 0x000000007aaf0000 => 0000000020560000 [ 0.211230] EFI remap 0x000000007ab40000 => 00000000205b0000 [ 0.217043] EFI remap 0x000000007fbb0000 => 0000000020600000 [ 0.222849] EFI remap 0x0000000080300000 => 0000000020630000 [ 0.228654] EFI remap 0x00000000a00f0000 => 0000000020640000 [ 0.234457] EFI remap 0x00000000a4000000 => 0000000020800000 [ 0.240266] EFI remap 0x00000000a6000000 => 0000000021800000 NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL NOTICE: [psci_smc_handler]:[409L] x1=0x20001 x2=0x82870 x3=0x0 NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x1
NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0x1
0808?8AB44 NOTICE: [psci_afflvl_power_on_finish]:[504L] NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c080 NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3d190 NOTICE: [psci_afflvl_power_on_finish]:[562L]
00082870 NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL NOTICE: [psci_smc_handler]:[409L] x1=0x20002 x2=0x82870 x3=0x0 NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x1
NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0x1
0808?8AB44 NOTICE: [psci_afflvl_power_on_finish]:[504L] NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c100 NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3d3a0 NOTICE: [psci_afflvl_power_on_finish]:[562L]
00082870 NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL NOTICE: [psci_smc_handler]:[409L] x1=0x20003 x2=0x82870 x3=0x0 NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x1
NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0x1
0808?8AB44 NOTICE: [psci_afflvl_power_on_finish]:[504L] NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c180 NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3d5b0 NOTICE: [psci_afflvl_power_on_finish]:[562L]
00082870 NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL NOTICE: [psci_smc_handler]:[409L] x1=0x20100 x2=0x82870 x3=0x0 NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x1
NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0x3
0808?8AB44 NOTICE: [psci_afflvl_power_on_finish]:[504L] NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c200 NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3d7c0 NOTICE: [psci_afflvl_power_on_finish]:[562L]
00082870 NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL NOTICE: [psci_smc_handler]:[409L] x1=0x20101 x2=0x82870 x3=0x0 NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x3
NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0x3
0808?8AB44 NOTICE: [psci_afflvl_power_on_finish]:[504L] NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c280 NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3d9d0 NOTICE: [psci_afflvl_power_on_finish]:[562L]
00082870 NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL NOTICE: [psci_smc_handler]:[409L] x1=0x20102 x2=0x82870 x3=0x0 NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x3
NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0x3
0808?8AB44 NOTICE: [psci_afflvl_power_on_finish]:[504L] NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c300 NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3dbe0 NOTICE: [psci_afflvl_power_on_finish]:[562L]
00082870 NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL NOTICE: [psci_smc_handler]:[409L] x1=0x20103 x2=0x82870 x3=0x0 NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x3
NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0x3
0808?8AB44 NOTICE: [psci_afflvl_power_on_finish]:[504L] NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c380 NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3ddf0 NOTICE: [psci_afflvl_power_on_finish]:[562L]
00082870 NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL NOTICE: [psci_smc_handler]:[409L] x1=0x20200 x2=0x82870 x3=0x0 NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x3
NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0x7
0808?8AB44 NOTICE: [psci_afflvl_power_on_finish]:[504L] NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c400 NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3e000 NOTICE: [psci_afflvl_power_on_finish]:[562L]
00082870 NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL NOTICE: [psci_smc_handler]:[409L] x1=0x20201 x2=0x82870 x3=0x0 NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x7
NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0x7
0808?8AB44 NOTICE: [psci_afflvl_power_on_finish]:[504L] NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c480 NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3e210 NOTICE: [psci_afflvl_power_on_finish]:[562L]
00082870 NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL NOTICE: [psci_smc_handler]:[409L] x1=0x20202 x2=0x82870 x3=0x0 NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x7
NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0x7
0808?8AB44 NOTICE: [psci_afflvl_power_on_finish]:[504L] NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c500 NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3e420 NOTICE: [psci_afflvl_power_on_finish]:[562L]
00082870 NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL NOTICE: [psci_smc_handler]:[409L] x1=0x20203 x2=0x82870 x3=0x0 NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x7
NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0x7
0808?8AB44 NOTICE: [psci_afflvl_power_on_finish]:[504L] NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c580 NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3e630 NOTICE: [psci_afflvl_power_on_finish]:[562L]
00082870 NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL NOTICE: [psci_smc_handler]:[409L] x1=0x20300 x2=0x82870 x3=0x0 NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0x7
NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0xf
0808?8AB44 NOTICE: [psci_afflvl_power_on_finish]:[504L] NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c600 NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3e840 NOTICE: [psci_afflvl_power_on_finish]:[562L]
00082870 NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL NOTICE: [psci_smc_handler]:[409L] x1=0x20301 x2=0x82870 x3=0x0 NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0xf
NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0xf
0808?8AB44 NOTICE: [psci_afflvl_power_on_finish]:[504L] NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c680 NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3ea50 NOTICE: [psci_afflvl_power_on_finish]:[562L]
00082870 NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL NOTICE: [psci_smc_handler]:[409L] x1=0x20302 x2=0x82870 x3=0x0 NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0xf
NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0xf
0808?8AB44 NOTICE: [psci_afflvl_power_on_finish]:[504L] NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c700 NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3ec60 NOTICE: [psci_afflvl_power_on_finish]:[562L]
00082870 NOTICE: [psci_smc_handler]:[408L] PSCI_CPU_ON_AARCH64 CALL NOTICE: [psci_smc_handler]:[409L] x1=0x20303 x2=0x82870 x3=0x0 NOTICE: [scpi_set_css_power_state]:[85L] domain_cluster=0xf
NOTICE: [scpi_set_css_power_state]:[93L] domain_cluster=0xf
0808?8AB44 NOTICE: [psci_afflvl_power_on_finish]:[504L] NOTICE: [cm_prepare_el3_exit]:[262L] read_tpidr_el3 = 7fc3c780 NOTICE: [cm_prepare_el3_exit]:[319L] ctx add = 7fc3ee70 NOTICE: [psci_afflvl_power_on_finish]:[562L]
00082870 [ 0.288031] Detected PIPT I-cache on CPU1 [ 0.288043] CPU1: found redistributor 20001 region 1:0x000000008d130000 [ 0.288064] CPU1: using LPI pending table @0x00000011f6410000 [ 0.288122] CPU1: Booted secondary processor [411fd071] [ 0.331143] Detected PIPT I-cache on CPU2 [ 0.331150] CPU2: found redistributor 20002 region 2:0x000000008d160000 [ 0.331170] CPU2: using LPI pending table @0x00000011f6440000 [ 0.331218] CPU2: Booted secondary processor [411fd071] [ 0.374257] Detected PIPT I-cache on CPU3 [ 0.374263] CPU3: found redistributor 20003 region 3:0x000000008d190000 [ 0.374283] CPU3: using LPI pending table @0x00000011f6480000 [ 0.374328] CPU3: Booted secondary processor [411fd071] [ 0.417372] Detected PIPT I-cache on CPU4 [ 0.417380] CPU4: found redistributor 20100 region 4:0x000000008d1c0000 [ 0.417400] CPU4: using LPI pending table @0x00000011f64c0000 [ 0.417447] CPU4: Booted secondary processor [411fd071] [ 0.460484] Detected PIPT I-cache on CPU5 [ 0.460491] CPU5: found redistributor 20101 region 5:0x000000008d1f0000 [ 0.460511] CPU5: using LPI pending table @0x00000011f64f0000 [ 0.460558] CPU5: Booted secondary processor [411fd071] [ 0.503598] Detected PIPT I-cache on CPU6 [ 0.503605] CPU6: found redistributor 20102 region 6:0x000000008d220000 [ 0.503625] CPU6: using LPI pending table @0x00000011f6530000 [ 0.503669] CPU6: Booted secondary processor [411fd071] [ 0.546711] Detected PIPT I-cache on CPU7 [ 0.546718] CPU7: found redistributor 20103 region 7:0x000000008d250000 [ 0.546738] CPU7: using LPI pending table @0x00000011f6560000 [ 0.546783] CPU7: Booted secondary processor [411fd071] [ 0.589826] Detected PIPT I-cache on CPU8 [ 0.589835] CPU8: found redistributor 20200 region 8:0x000000008d280000 [ 0.589857] CPU8: using LPI pending table @0x00000011f65a0000 [ 0.589910] CPU8: Booted secondary processor [411fd071] [ 0.632939] Detected PIPT I-cache on CPU9 [ 0.632946] CPU9: found redistributor 20201 region 9:0x000000008d2b0000 [ 0.632967] CPU9: using LPI pending table @0x00000011f65e0000 [ 0.633014] CPU9: Booted secondary processor [411fd071] [ 0.676052] Detected PIPT I-cache on CPU10 [ 0.676060] CPU10: found redistributor 20202 region 10:0x000000008d2e0000 [ 0.676081] CPU10: using LPI pending table @0x00000011f6610000 [ 0.676126] CPU10: Booted secondary processor [411fd071] [ 0.719166] Detected PIPT I-cache on CPU11 [ 0.719173] CPU11: found redistributor 20203 region 11:0x000000008d310000 [ 0.719195] CPU11: using LPI pending table @0x00000011f6650000 [ 0.719240] CPU11: Booted secondary processor [411fd071] [ 0.762280] Detected PIPT I-cache on CPU12 [ 0.762289] CPU12: found redistributor 20300 region 12:0x000000008d340000 [ 0.762311] CPU12: using LPI pending table @0x00000011f6680000 [ 0.762360] CPU12: Booted secondary processor [411fd071] [ 0.805392] Detected PIPT I-cache on CPU13 [ 0.805400] CPU13: found redistributor 20301 region 13:0x000000008d370000 [ 0.805421] CPU13: using LPI pending table @0x00000011f66c0000 [ 0.805466] CPU13: Booted secondary processor [411fd071] [ 0.848506] Detected PIPT I-cache on CPU14 [ 0.848513] CPU14: found redistributor 20302 region 14:0x000000008d3a0000 [ 0.848534] CPU14: using LPI pending table @0x00000011f6700000 [ 0.848579] CPU14: Booted secondary processor [411fd071] [ 0.891620] Detected PIPT I-cache on CPU15 [ 0.891628] CPU15: found redistributor 20303 region 15:0x000000008d3d0000 [ 0.891648] CPU15: using LPI pending table @0x00000011f6750000 [ 0.891693] CPU15: Booted secondary processor [411fd071] [ 0.891723] Brought up 16 CPUs [ 1.220754] SMP: Total of 16 processors activated. [ 1.225520] CPU features: detected feature: GIC system register CPU interface [ 1.232620] CPU: All CPU(s) started at EL2 [ 1.236718] alternatives: patching kernel code [ 1.243905] devtmpfs: initialized [ 1.247446] SMBIOS 3.0.0 present. [ 1.250851] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns [ 1.260790] pinctrl core: initialized pinctrl subsystem [ 1.266365] NET: Registered protocol family 16 [ 1.282800] cpuidle: using governor menu [ 1.286747] vdso: 2 pages (1 code @ ffffff8008796000, 1 data @ ffffff8008b60000) [ 1.294122] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers. [ 1.301211] DMA: preallocated 256 KiB pool for atomic allocations [ 1.307340] ACPI: bus type PCI registered [ 1.311388] Serial: AMBA PL011 UART driver [ 1.331638] HugeTLB registered 2 MB page size, pre-allocated 0 pages [ 1.338373] ACPI: Added _OSI(Module Device) [ 1.342562] ACPI: Added _OSI(Processor Device) [ 1.346984] ACPI: Added _OSI(3.0 _SCP Extensions) [ 1.351664] ACPI: Added _OSI(Processor Aggregator Device) [ 1.358469] ACPI: Interpreter enabled [ 1.362116] ACPI: Using GIC for interrupt routing [ 1.366815] ACPI: MCFG table loaded, 2 entries detected [ 1.375180] Hisilicon MBIGEN-V1 HISI0151:00: Allocated 256 MSIs [ 1.381158] Hisilicon MBIGEN-V1 HISI0151:01: Allocated 640 MSIs [ 1.387123] Hisilicon MBIGEN-V1 HISI0151:02: Allocated 256 MSIs [ 1.393091] Hisilicon MBIGEN-V1 HISI0151:03: Allocated 640 MSIs [ 1.399213] ACPI: IORT: can't find node related to (null) device [ 1.405292] ACPI: IORT: can't find node related to (null) device [ 1.411324] ACPI: IORT: can't find node related to (null) device [ 1.417348] ACPI: IORT: can't find node related to (null) device [ 1.423370] ACPI: IORT: can't find node related to (null) device [ 1.429391] ACPI: IORT: can't find node related to (null) device [ 1.436634] ACPI: IORT: can't find node related to (null) device [ 1.442681] ACPI: IORT: can't find node related to (null) device [ 1.448698] ACPI: IORT: can't find node related to (null) device [ 1.454712] ACPI: IORT: can't find node related to (null) device [ 1.460724] ACPI: IORT: can't find node related to (null) device [ 1.466764] ACPI: PCI Root Bridge [PCI1] (domain 0001 [bus 40-7f]) [ 1.472918] acpi HISI0080:00: _OSC: OS supports [ExtendedConfig Segments MSI] [ 1.480025] acpi HISI0080:00: _OSC failed (AE_NOT_FOUND); disabling ASPM [ 1.486771] acpi HISI0080:00: ECAM at [mem 0x22004000000-0x22007ffffff] for [bus 40-7f] [ 1.494762] Remapped I/O 0x000002200fff0000 to [io 0x0000-0xffff window] [ 1.501592] PCI host bridge to bus 0001:40 [ 1.505671] pci_bus 0001:40: root bus resource [mem 0x22008000000-0x2200ffeffff window] (bus address [0xb0000000-0xb7feffff]) [ 1.516919] pci_bus 0001:40: root bus resource [io 0x0000-0xffff window] [ 1.523675] pci_bus 0001:40: root bus resource [bus 40-7f] [ 1.534090] pci 0001:41:00.0: VF(n) BAR0 space: [mem 0x22008e08000-0x22008f07fff 64bit pref] (contains BAR0 for 64 VFs) [ 1.545323] pci 0001:41:00.0: VF(n) BAR3 space: [mem 0x22008f08000-0x22009007fff 64bit pref] (contains BAR3 for 64 VFs) [ 1.568223] pci 0001:41:00.1: VF(n) BAR0 space: [mem 0x22008c04000-0x22008d03fff 64bit pref] (contains BAR0 for 64 VFs) [ 1.579460] pci 0001:41:00.1: VF(n) BAR3 space: [mem 0x22008d04000-0x22008e03fff 64bit pref] (contains BAR3 for 64 VFs) [ 1.597823] pci 0001:40:00.0: BAR 15: assigned [mem 0x22008000000-0x220095fffff pref] [ 1.605616] pci 0001:40:00.0: BAR 13: assigned [io 0x1000-0x1fff] [ 1.611771] pci 0001:41:00.0: BAR 0: assigned [mem 0x22008000000-0x220083fffff 64bit pref] [ 1.620245] pci 0001:41:00.0: BAR 6: assigned [mem 0x22008400000-0x220087fffff pref] [ 1.627952] pci 0001:41:00.1: BAR 0: assigned [mem 0x22008800000-0x22008bfffff 64bit pref] [ 1.636430] pci 0001:41:00.1: BAR 6: assigned [mem 0x22008c00000-0x22008ffffff pref] [ 1.644138] pci 0001:41:00.0: BAR 4: assigned [mem 0x22009000000-0x22009003fff 64bit pref] [ 1.652610] pci 0001:41:00.0: BAR 7: assigned [mem 0x22009004000-0x22009103fff 64bit pref] [ 1.661083] pci 0001:41:00.0: BAR 10: assigned [mem 0x22009104000-0x22009203fff 64bit pref] [ 1.669640] pci 0001:41:00.1: BAR 4: assigned [mem 0x22009204000-0x22009207fff 64bit pref] [ 1.678115] pci 0001:41:00.1: BAR 7: assigned [mem 0x22009208000-0x22009307fff 64bit pref] [ 1.686591] pci 0001:41:00.1: BAR 10: assigned [mem 0x22009308000-0x22009407fff 64bit pref] [ 1.695148] pci 0001:41:00.0: BAR 2: assigned [io 0x1000-0x101f] [ 1.701285] pci 0001:41:00.1: BAR 2: assigned [io 0x1020-0x103f] [ 1.707421] pci 0001:40:00.0: PCI bridge to [bus 41-42] [ 1.712621] pci 0001:40:00.0: bridge window [io 0x1000-0x1fff] [ 1.718685] pci 0001:40:00.0: bridge window [mem 0x22008000000-0x220095fffff pref] [ 1.726434] ACPI: PCI Root Bridge [PCI2] (domain 0002 [bus 80-bf]) [ 1.732594] acpi HISI0080:01: _OSC: OS supports [ExtendedConfig Segments MSI] [ 1.739696] acpi HISI0080:01: _OSC failed (AE_NOT_FOUND); disabling ASPM [ 1.746425] acpi HISI0080:01: link status is down [ 1.751106] acpi HISI0080:01: ECAM at [mem 0x24008000000-0x2400bffffff] for [bus 80-bf] [ 1.759090] Remapped I/O 0x000002400fff0000 to [io 0x10000-0x1ffff window] [ 1.766086] PCI host bridge to bus 0002:80 [ 1.770172] pci_bus 0002:80: root bus resource [mem 0x2400c000000-0x2400ffeffff window] (bus address [0xc0000000-0xc3feffff]) [ 1.781422] pci_bus 0002:80: root bus resource [io 0x10000-0x1ffff window] (bus address [0x0000-0xffff]) [ 1.790940] pci_bus 0002:80: root bus resource [bus 80-bf] [ 1.796414] pci 0002:80:00.0: ignoring class 0x000000 (doesn't match header type 01) [ 1.804311] pci 0002:80:00.0: not setting up bridge for bus 0002:81 [ 1.810956] ACPI: IORT: can't find node related to (null) device [ 1.817228] ACPI: IORT: can't find node related to (null) device [ 1.823453] vgaarb: loaded [ 1.826246] SCSI subsystem initialized [ 1.830103] ACPI: bus type USB registered [ 1.834130] usbcore: registered new interface driver usbfs [ 1.839606] usbcore: registered new interface driver hub [ 1.844933] usbcore: registered new device driver usb [ 1.850014] pps_core: LinuxPPS API ver. 1 registered [ 1.854953] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti giometti@linux.it [ 1.864048] PTP clock support registered [ 1.868069] Advanced Linux Sound Architecture Driver Initialized. [ 1.874376] clocksource: Switched to clocksource arch_sys_counter [ 1.880489] VFS: Disk quotas dquot_6.6.0 [ 1.884414] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.891385] pnp: PnP ACPI init [ 1.894570] system 00:00: [mem 0xb0080000-0xb008ffff] has been reserved [ 1.901209] system 00:01: [mem 0xb0090000-0xb009ffff] has been reserved [ 1.907813] pnp: PnP ACPI: found 2 devices [ 1.914467] NET: Registered protocol family 2 [ 1.919030] TCP established hash table entries: 65536 (order: 7, 524288 bytes) [ 1.926352] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes) [ 1.933333] TCP: Hash tables configured (established 65536 bind 65536) [ 1.939876] UDP hash table entries: 4096 (order: 5, 131072 bytes) [ 1.945968] UDP-Lite hash table entries: 4096 (order: 5, 131072 bytes) [ 1.952576] NET: Registered protocol family 1 [ 1.957013] RPC: Registered named UNIX socket transport module. [ 1.962905] RPC: Registered udp transport module. [ 1.967586] RPC: Registered tcp transport module. [ 1.972266] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.978872] Unpacking initramfs... [ 2.340835] Freeing initrd memory: 27492K (ffffffc01e520000 - ffffffc01fff9000) [ 2.348645] kvm [1]: 8-bit VMID [ 2.351776] kvm [1]: Hyp mode initialized successfully [ 2.356892] kvm [1]: error: KVM vGIC probing failed [ 2.361790] kvm [1]: virtual timer IRQ3 [ 2.366650] ACPI: IORT: can't find node related to (null) device [ 2.373057] futex hash table entries: 4096 (order: 7, 524288 bytes) [ 2.379403] audit: initializing netlink subsys (disabled) [ 2.384813] audit: type=2000 audit(1.864:1): initialized [ 2.390302] workingset: timestamp_bits=44 max_order=21 bucket_order=0 [ 2.400222] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ 2.406342] NFS: Registering the id_resolver key type [ 2.411393] Key type id_resolver registered [ 2.415557] Key type id_legacy registered [ 2.419609] fuse init (API version 7.24) [ 2.423687] 9p: Installing v9fs 9p2000 file system support [ 2.429982] io scheduler noop registered [ 2.433946] io scheduler cfq registered (default) [ 2.439027] pcieport 0001:40:00.0: can't derive routing for PCI INT A [ 2.445438] pcieport 0001:40:00.0: PCI INT A: no GSI [ 2.451126] xenfs: not registering filesystem on non-xen platform [ 2.458262] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled [ 2.465049] console [ttyS0] disabled [ 2.468627] APMC0D08:00: ttyS0 at MMIO 0x80300000 (irq = 5, base_baud = 12500000) is a 16550A [ 2.477133] console [ttyS0] enabled [ 2.477133] console [ttyS0] enabled [ 2.484137] bootconsole [uart8250] disabled [ 2.484137] bootconsole [uart8250] disabled [ 2.492728] SuperH (H)SCI(F) driver initialized [ 2.497322] msm_serial: driver initialized [ 2.501649] Failed to find cpu0 device node [ 2.505852] Unable to detect cache hierarchy from DT for CPU 0 [ 2.514093] loop: module loaded [ 2.517694] tun: Universal TUN/TAP device driver, 1.6 [ 2.522771] tun: (C) 1999-2004 Max Krasnyansky maxk@qualcomm.com [ 2.529079] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k [ 2.534939] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. [ 2.540908] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.3.0-k [ 2.547900] igb: Copyright (c) 2007-2014 Intel Corporation. [ 2.553518] igbvf: Intel(R) Gigabit Virtual Function Network Driver - version 2.0.2-k [ 2.561382] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. [ 2.567350] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver - version 4.2.1-k [ 2.575039] ixgbe: Copyright (c) 1999-2015 Intel Corporation. [ 2.580854] pcieport 0001:40:00.0: can't derive routing for PCI INT A [ 2.587326] ixgbe 0001:41:00.0: PCI INT A: no GSI [ 2.592178] ixgbe 0001:41:00.0: enabling device (0000 -> 0002) [ 2.753716] ixgbe 0001:41:00.0: Multiqueue Enabled: Rx Queue count = 16, Tx Queue count = 16 [ 2.762393] ixgbe 0001:41:00.0: PCI Express bandwidth of 32GT/s available [ 2.769213] ixgbe 0001:41:00.0: (Speed:5.0GT/s, Width: x8, Encoding Loss:20%) [ 2.776457] ixgbe 0001:41:00.0: MAC: 2, PHY: 17, SFP+: 5, PBA No: FFFFFF-0FF [ 2.783537] ixgbe 0001:41:00.0: 68:a8:28:2e:c9:10 [ 2.792732] ixgbe 0001:41:00.0: Intel(R) 10 Gigabit Network Connection [ 2.799312] pcieport 0001:40:00.0: can't derive routing for PCI INT B [ 2.805783] ixgbe 0001:41:00.1: PCI INT B: no GSI [ 2.810598] ixgbe 0001:41:00.1: enabling device (0000 -> 0002) [ 3.949697] ixgbe 0001:41:00.1: Multiqueue Enabled: Rx Queue count = 16, Tx Queue count = 16 [ 3.958365] ixgbe 0001:41:00.1: PCI Express bandwidth of 32GT/s available [ 3.965185] ixgbe 0001:41:00.1: (Speed:5.0GT/s, Width: x8, Encoding Loss:20%) [ 3.972427] ixgbe 0001:41:00.1: MAC: 2, PHY: 1, PBA No: FFFFFF-0FF [ 3.978634] ixgbe 0001:41:00.1: 68:a8:28:2e:c9:11 [ 3.987790] ixgbe 0001:41:00.1: Intel(R) 10 Gigabit Network Connection [ 3.994388] sky2: driver version 1.30 [ 3.998201] VFIO - User Level meta-driver version: 0.3 [ 4.003889] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver [ 4.010459] ehci-pci: EHCI PCI platform driver [ 4.014944] ehci-platform: EHCI generic platform driver [ 4.020226] ehci-platform PNP0D20:00: EHCI Host Controller [ 4.025745] ehci-platform PNP0D20:00: new USB bus registered, assigned bus number 1 [ 4.033572] ehci-platform PNP0D20:00: irq 6, io mem 0xa1000000 [ 4.050387] ehci-platform PNP0D20:00: USB 2.0 started, EHCI 1.00 [ 4.056652] hub 1-0:1.0: USB hub found [ 4.060428] hub 1-0:1.0: 1 port detected [ 4.064517] ehci-msm: Qualcomm On-Chip EHCI Host Controller [ 4.070137] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver [ 4.076351] ohci-pci: OHCI PCI platform driver [ 4.080834] ohci-platform: OHCI generic platform driver [ 4.086166] usbcore: registered new interface driver usb-storage [ 4.092427] mousedev: PS/2 mouse device common for all mice [ 4.161533] rtc-efi rtc-efi: rtc core: registered rtc-efi as rtc0 [ 4.168513] i2c /dev entries driver [ 4.172251] sdhci: Secure Digital Host Controller Interface driver [ 4.178458] sdhci: Copyright(c) Pierre Ossman [ 4.182850] Synopsys Designware Multimedia Card Interface Driver [ 4.188933] sdhci-pltfm: SDHCI platform and OF driver helper [ 4.194701] ledtrig-cpu: registered to indicate activity on CPUs [ 4.200950] usbcore: registered new interface driver usbhid [ 4.206549] usbhid: USB HID core driver [ 4.210569] ACPI: IORT: can't find node related to (null) device [ 4.216837] NET: Registered protocol family 17 [ 4.221334] 9pnet: Installing 9P2000 support [ 4.225652] Key type dns_resolver registered [ 4.230166] registered taskstats version 1 [ 4.297850] rtc-efi rtc-efi: hctosys: unable to read the hardware clock [ 4.304576] ALSA device list: [ 4.307561] No soundcards found. [ 4.311063] ttyS0 - failed to request DMA [ 4.315348] Freeing unused kernel memory: 840K (ffffff8008a89000 - ffffff8008b5b000) root@(none)$ ifconfig eth0 192.168.20.188 [ 15.679957] ixgbe 0001:41:00.0: registered PHC device on eth0 root@(none)$ [ 15.851142] ixgbe 0001:41:00.0 eth0: detected SFP+: 5 [ 15.990419] ixgbe 0001:41:00.0 eth0: NIC Link is Up 10 Gbps, Flow Control: RX/TX
root@(none)$ ping 192.168.20.188 PING 192.168.20.4 (192.168.20.4): 56 data bytes 64 bytes from 192.168.20.4: seq=14 ttl=128 time=1.465 ms 64 bytes from 192.168.20.4: seq=15 ttl=128 time=0.616 ms 64 bytes from 192.168.20.4: seq=16 ttl=128 time=0.391 ms 64 bytes from 192.168.20.4: seq=17 ttl=128 time=0.698 ms 64 bytes from 192.168.20.4: seq=18 ttl=128 time=0.676 ms 64 bytes from 192.168.20.4: seq=19 ttl=128 time=0.524 ms 64 bytes from 192.168.20.4: seq=20 ttl=128 time=0.301 ms 64 bytes from 192.168.20.4: seq=21 ttl=128 time=0.248 ms 64 bytes from 192.168.20.4: seq=22 ttl=128 time=0.403 ms
On 5/10/2016 11:19 AM, Tomasz Nowicki wrote:
From the functionality point of view this series may be split into the following logic parts:
- New ECAM API and update for users of the pci-host-common API
- Necessary fixes as the preparation for using driver on ARM64.
- Use new MCFG interface and implement generic ACPI based PCI host controller driver.
- Enable above driver on ARM64
Patches has been built on top of 4.6-rc7 and can be found here: git@github.com:semihalf-nowicki-tomasz/linux.git (pci-acpi-v7)
This has been tested on Cavium ThunderX server. Any help in reviewing and testing is very appreciated.
Tested on Qualcomm QDF2XXX server using Mellanox CX3 and Intel e1000e adapters.
Tested-by: Sinan Kaya okaya@codeaurora.org
-
Ard Biesheuvel -
Arnd Bergmann -
Bjorn Helgaas -
Dongdong Liu -
Duc Dang -
Gabriele Paoloni -
Jayachandran C -
Jeremy Linton -
Jon Masters -
Jon Masters -
Lorenzo Pieralisi -
Matthias Brugger -
Rafael J. Wysocki -
Sinan Kaya -
Tomasz Nowicki