Hi,
This patch adds support for PCI to AArch64. It is based on my v5 patch that adds support for creating generic host bridge structure from device tree. With that in place, I was able to boot a platform that has PCIe host bridge support and use a PCIe network card.
Changes from v4: - Fixed the pci_domain_nr() implementation for arm64. Now we use find_pci_host_bride() to find the host bridge before we retrieve the domain number.
Changes from v3: - Added Acks accumulated so far ;) - Still carrying Catalin's patch for moving the PCI_IO_BASE until it lands in linux-next or mainline, in order to ease applying the series
Changes from v2: - Implement an arch specific version of pci_register_io_range() and pci_address_to_pio(). - Return 1 from pci_proc_domain().
Changes from v1: - Added Catalin's patch for moving the PCI_IO_BASE location and extend its size to 16MB - Integrated Arnd's version of pci_ioremap_io that uses a bitmap for keeping track of assigned IO space and returns an io_offset. At the moment the code is added in arch/arm64 but it can be moved in drivers/pci. - Added a fix for the generic ioport_map() function when !CONFIG_GENERIC_IOMAP as suggested by Arnd.
v4 thread here: https://lkml.org/lkml/2014/3/3/298 v3 thread here: https://lkml.org/lkml/2014/2/28/211 v2 thread here: https://lkml.org/lkml/2014/2/27/255 v1 thread here: https://lkml.org/lkml/2014/2/3/389
The API used is different from the one used by ARM architecture. There is no pci_common_init_dev() function and no hw_pci structure, as that is no longer needed. Once the last signature is added to the legal agreement, I will post the host bridge driver code that I am using. Meanwhile, here is an example of what the probe function looks like, posted as an example:
static int myhostbridge_probe(struct platform_device *pdev) { int err; struct device_node *dev; struct pci_host_bridge *bridge; struct myhostbridge_port *pp; resource_size_t lastbus;
dev = pdev->dev.of_node;
if (!of_device_is_available(dev)) { pr_warn("%s: disabled\n", dev->full_name); return -ENODEV; }
pp = kzalloc(sizeof(struct myhostbridge_port), GFP_KERNEL); if (!pp) return -ENOMEM;
bridge = of_create_pci_host_bridge(&pdev->dev, &myhostbridge_ops, pp); if (IS_ERR(bridge)) { err = PTR_ERR(bridge); goto bridge_create_fail; }
err = myhostbridge_setup(bridge->bus); if (err) goto bridge_setup_fail;
/* We always enable PCI domains and we keep domain 0 backward * compatible in /proc for video cards */ pci_add_flags(PCI_ENABLE_PROC_DOMAINS); pci_add_flags(PCI_REASSIGN_ALL_BUS | PCI_REASSIGN_ALL_RSRC);
lastbus = pci_scan_child_bus(bridge->bus); pci_bus_update_busn_res_end(bridge->bus, lastbus);
pci_assign_unassigned_bus_resources(bridge->bus);
pci_bus_add_devices(bridge->bus);
return 0;
bridge_setup_fail: put_device(&bridge->dev); device_unregister(&bridge->dev); bridge_create_fail: kfree(pp); return err; }
Best regards, Liviu
Catalin Marinas (1): arm64: Extend the PCI I/O space to 16MB
Liviu Dudau (2): Fix ioport_map() for !CONFIG_GENERIC_IOMAP cases. arm64: Add architecture support for PCI
Documentation/arm64/memory.txt | 16 ++-- arch/arm64/Kconfig | 19 ++++- arch/arm64/include/asm/Kbuild | 1 + arch/arm64/include/asm/io.h | 5 +- arch/arm64/include/asm/pci.h | 49 ++++++++++++ arch/arm64/kernel/Makefile | 1 + arch/arm64/kernel/pci.c | 178 +++++++++++++++++++++++++++++++++++++++++ include/asm-generic/io.h | 2 +- 8 files changed, 261 insertions(+), 10 deletions(-) create mode 100644 arch/arm64/include/asm/pci.h create mode 100644 arch/arm64/kernel/pci.c
The inline version of ioport_map() that gets used when !CONFIG_GENERIC_IOMAP is wrong. It returns a mapped (i.e. virtual) address that can start from zero and completely ignores the PCI_IOBASE and IO_SPACE_LIMIT that most architectures that use !CONFIG_GENERIC_MAP define.
Signed-off-by: Liviu Dudau Liviu.Dudau@arm.com Acked-by: Arnd Bergmann arnd@arndb.de
diff --git a/include/asm-generic/io.h b/include/asm-generic/io.h index d5afe96..df72051 100644 --- a/include/asm-generic/io.h +++ b/include/asm-generic/io.h @@ -331,7 +331,7 @@ static inline void iounmap(void __iomem *addr) #ifndef CONFIG_GENERIC_IOMAP static inline void __iomem *ioport_map(unsigned long port, unsigned int nr) { - return (void __iomem *) port; + return (void __iomem *)(PCI_IOBASE + (port & IO_SPACE_LIMIT)); }
static inline void ioport_unmap(void __iomem *p)
From: Catalin Marinas catalin.marinas@arm.com
The patch moves the PCI I/O space (currently at 64K) before the earlyprintk mapping and extends it to 16MB.
Signed-off-by: Catalin Marinas catalin.marinas@arm.com
diff --git a/Documentation/arm64/memory.txt b/Documentation/arm64/memory.txt index 5e054bf..85e24c4 100644 --- a/Documentation/arm64/memory.txt +++ b/Documentation/arm64/memory.txt @@ -35,11 +35,13 @@ ffffffbc00000000 ffffffbdffffffff 8GB vmemmap
ffffffbe00000000 ffffffbffbbfffff ~8GB [guard, future vmmemap]
-ffffffbffbc00000 ffffffbffbdfffff 2MB earlyprintk device +ffffffbffa000000 ffffffbffaffffff 16MB PCI I/O space + +ffffffbffb000000 ffffffbffbbfffff 12MB [guard]
-ffffffbffbe00000 ffffffbffbe0ffff 64KB PCI I/O space +ffffffbffbc00000 ffffffbffbdfffff 2MB earlyprintk device
-ffffffbffbe10000 ffffffbcffffffff ~2MB [guard] +ffffffbffbe00000 ffffffbffbffffff 2MB [guard]
ffffffbffc000000 ffffffbfffffffff 64MB modules
@@ -60,11 +62,13 @@ fffffdfc00000000 fffffdfdffffffff 8GB vmemmap
fffffdfe00000000 fffffdfffbbfffff ~8GB [guard, future vmmemap]
-fffffdfffbc00000 fffffdfffbdfffff 2MB earlyprintk device +fffffdfffa000000 fffffdfffaffffff 16MB PCI I/O space + +fffffdfffb000000 fffffdfffbbfffff 12MB [guard]
-fffffdfffbe00000 fffffdfffbe0ffff 64KB PCI I/O space +fffffdfffbc00000 fffffdfffbdfffff 2MB earlyprintk device
-fffffdfffbe10000 fffffdfffbffffff ~2MB [guard] +fffffdfffbe00000 fffffdfffbffffff 2MB [guard]
fffffdfffc000000 fffffdffffffffff 64MB modules
diff --git a/arch/arm64/include/asm/io.h b/arch/arm64/include/asm/io.h index 4cc813e..7846a6b 100644 --- a/arch/arm64/include/asm/io.h +++ b/arch/arm64/include/asm/io.h @@ -121,7 +121,7 @@ static inline u64 __raw_readq(const volatile void __iomem *addr) * I/O port access primitives. */ #define IO_SPACE_LIMIT 0xffff -#define PCI_IOBASE ((void __iomem *)(MODULES_VADDR - SZ_2M)) +#define PCI_IOBASE ((void __iomem *)(MODULES_VADDR - SZ_32M))
static inline u8 inb(unsigned long addr) {
Use the generic host bridge functions to provide support for PCI Express on arm64. There is no support for ISA memory.
Signed-off-by: Liviu Dudau Liviu.Dudau@arm.com
create mode 100644 arch/arm64/include/asm/pci.h create mode 100644 arch/arm64/kernel/pci.c
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 27bbcfc..d1c8568 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -62,7 +62,7 @@ config MMU def_bool y
config NO_IOPORT - def_bool y + def_bool y if !PCI
config STACKTRACE_SUPPORT def_bool y @@ -134,6 +134,23 @@ menu "Bus support" config ARM_AMBA bool
+config PCI + bool "PCI support" + help + This feature enables support for PCIe bus system. If you say Y + here, the kernel will include drivers and infrastructure code + to support PCIe bus devices. + +config PCI_DOMAINS + def_bool PCI + +config PCI_SYSCALL + def_bool PCI + +source "drivers/pci/Kconfig" +source "drivers/pci/pcie/Kconfig" +source "drivers/pci/hotplug/Kconfig" + endmenu
menu "Kernel Features" diff --git a/arch/arm64/include/asm/Kbuild b/arch/arm64/include/asm/Kbuild index 71c53ec..46924bc 100644 --- a/arch/arm64/include/asm/Kbuild +++ b/arch/arm64/include/asm/Kbuild @@ -26,6 +26,7 @@ generic-y += mman.h generic-y += msgbuf.h generic-y += mutex.h generic-y += pci.h +generic-y += pci-bridge.h generic-y += poll.h generic-y += posix_types.h generic-y += resource.h diff --git a/arch/arm64/include/asm/io.h b/arch/arm64/include/asm/io.h index 7846a6b..67463a5 100644 --- a/arch/arm64/include/asm/io.h +++ b/arch/arm64/include/asm/io.h @@ -120,7 +120,8 @@ static inline u64 __raw_readq(const volatile void __iomem *addr) /* * I/O port access primitives. */ -#define IO_SPACE_LIMIT 0xffff +#define arch_has_dev_port() (1) +#define IO_SPACE_LIMIT 0x1ffffff #define PCI_IOBASE ((void __iomem *)(MODULES_VADDR - SZ_32M))
static inline u8 inb(unsigned long addr) diff --git a/arch/arm64/include/asm/pci.h b/arch/arm64/include/asm/pci.h new file mode 100644 index 0000000..6f1fb3d --- /dev/null +++ b/arch/arm64/include/asm/pci.h @@ -0,0 +1,49 @@ +#ifndef __ASM_PCI_H +#define __ASM_PCI_H +#ifdef __KERNEL__ + +#include <linux/types.h> +#include <linux/slab.h> +#include <linux/dma-mapping.h> + +#include <asm/io.h> +#include <asm-generic/pci-bridge.h> +#include <asm-generic/pci-dma-compat.h> + +#define PCIBIOS_MIN_IO 0x1000 +#define PCIBIOS_MIN_MEM 0 + +struct pci_host_bridge *find_pci_host_bridge(struct pci_bus *bus); + +/* + * Set to 1 if the kernel should re-assign all PCI bus numbers + */ +#define pcibios_assign_all_busses() \ + (pci_has_flag(PCI_REASSIGN_ALL_BUS)) + +/* + * PCI address space differs from physical memory address space + */ +#define PCI_DMA_BUS_IS_PHYS (0) + +extern int isa_dma_bridge_buggy; + +static inline int pci_domain_nr(struct pci_bus *bus) +{ + struct pci_host_bridge *bridge = find_pci_host_bridge(bus); + + if (bridge) + return bridge->domain_nr; + + return 0; +} + +static inline int pci_proc_domain(struct pci_bus *bus) +{ + return 1; +} + +extern unsigned long pci_ioremap_io(const struct resource *res, phys_addr_t phys_addr); + +#endif /* __KERNEL__ */ +#endif /* __ASM_PCI_H */ diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile index 2d4554b..64fc479 100644 --- a/arch/arm64/kernel/Makefile +++ b/arch/arm64/kernel/Makefile @@ -20,6 +20,7 @@ arm64-obj-$(CONFIG_HAVE_HW_BREAKPOINT)+= hw_breakpoint.o arm64-obj-$(CONFIG_EARLY_PRINTK) += early_printk.o arm64-obj-$(CONFIG_ARM64_CPU_SUSPEND) += sleep.o suspend.o arm64-obj-$(CONFIG_JUMP_LABEL) += jump_label.o +arm64-obj-$(CONFIG_PCI) += pci.o
obj-y += $(arm64-obj-y) vdso/ obj-m += $(arm64-obj-m) diff --git a/arch/arm64/kernel/pci.c b/arch/arm64/kernel/pci.c new file mode 100644 index 0000000..f00471d --- /dev/null +++ b/arch/arm64/kernel/pci.c @@ -0,0 +1,178 @@ +/* + * Code borrowed from powerpc/kernel/pci-common.c + * + * Copyright (C) 2003 Anton Blanchard anton@au.ibm.com, IBM + * Copyright (C) 2014 ARM Ltd. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License + * version 2 as published by the Free Software Foundation. + * + */ + +#include <linux/init.h> +#include <linux/io.h> +#include <linux/kernel.h> +#include <linux/mm.h> +#include <linux/of_pci.h> +#include <linux/of_platform.h> +#include <linux/slab.h> + +#include <asm/pci-bridge.h> + +struct ioresource { + struct list_head list; + phys_addr_t start; + resource_size_t size; +}; + +static LIST_HEAD(io_list); + +int pci_register_io_range(phys_addr_t address, resource_size_t size) +{ + struct ioresource *res; + resource_size_t allocated_size = 0; + + /* find if the range has not been already allocated */ + list_for_each_entry(res, &io_list, list) { + if (address >= res->start && + address + size <= res->start + size) + return 0; + allocated_size += res->size; + } + + /* range not already registered, check for space */ + if (allocated_size + size > IO_SPACE_LIMIT) + return -E2BIG; + + /* add the range in the list */ + res = kzalloc(sizeof(*res), GFP_KERNEL); + if (!res) + return -ENOMEM; + res->start = address; + res->size = size; + + list_add_tail(&res->list, &io_list); + + return 0; +} +EXPORT_SYMBOL_GPL(pci_register_io_range); + +unsigned long pci_address_to_pio(phys_addr_t address) +{ + struct ioresource *res; + + list_for_each_entry(res, &io_list, list) { + if (address >= res->start && + address < res->start + res->size) { + return res->start - address; + } + } + + return (unsigned long)-1; +} +EXPORT_SYMBOL_GPL(pci_address_to_pio); + +/* + * Called after each bus is probed, but before its children are examined + */ +void pcibios_fixup_bus(struct pci_bus *bus) +{ + struct pci_dev *dev; + struct resource *res; + int i; + + if (!pci_is_root_bus(bus)) { + pci_read_bridge_bases(bus); + + pci_bus_for_each_resource(bus, res, i) { + if (!res || !res->flags || res->parent) + continue; + + /* + * If we are going to reassign everything, we can + * shrink the P2P resource to have zero size to + * save space + */ + if (pci_has_flag(PCI_REASSIGN_ALL_RSRC)) { + res->flags |= IORESOURCE_UNSET; + res->start = 0; + res->end = -1; + continue; + } + } + } + + list_for_each_entry(dev, &bus->devices, bus_list) { + /* Ignore fully discovered devices */ + if (dev->is_added) + continue; + + set_dev_node(&dev->dev, pcibus_to_node(dev->bus)); + + /* Read default IRQs and fixup if necessary */ + dev->irq = of_irq_parse_and_map_pci(dev, 0, 0); + } +} +EXPORT_SYMBOL(pcibios_fixup_bus); + +/* + * We don't have to worry about legacy ISA devices, so nothing to do here + */ +resource_size_t pcibios_align_resource(void *data, const struct resource *res, + resource_size_t size, resource_size_t align) +{ + return ALIGN(res->start, align); +} +EXPORT_SYMBOL(pcibios_align_resource); + +int pcibios_enable_device(struct pci_dev *dev, int mask) +{ + return pci_enable_resources(dev, mask); +} + +void pcibios_fixup_bridge_ranges(struct list_head *resources) +{ +} + +#define IO_SPACE_PAGES ((IO_SPACE_LIMIT + 1) / PAGE_SIZE) +static DECLARE_BITMAP(pci_iospace, IO_SPACE_PAGES); + +unsigned long pci_ioremap_io(const struct resource *res, phys_addr_t phys_addr) +{ + unsigned long start, len, virt_start; + int err; + + if (res->end > IO_SPACE_LIMIT) + return -EINVAL; + + /* + * try finding free space for the whole size first, + * fall back to 64K if not available + */ + len = resource_size(res); + start = bitmap_find_next_zero_area(pci_iospace, IO_SPACE_PAGES, + res->start / PAGE_SIZE, len / PAGE_SIZE, 0); + if (start == IO_SPACE_PAGES && len > SZ_64K) { + len = SZ_64K; + start = 0; + start = bitmap_find_next_zero_area(pci_iospace, IO_SPACE_PAGES, + start, len / PAGE_SIZE, 0); + } + + /* no 64K area found */ + if (start == IO_SPACE_PAGES) + return -ENOMEM; + + /* ioremap physical aperture to virtual aperture */ + virt_start = start * PAGE_SIZE + (unsigned long)PCI_IOBASE; + err = ioremap_page_range(virt_start, virt_start + len, + phys_addr, __pgprot(PROT_DEVICE_nGnRE)); + if (err) + return err; + + bitmap_set(pci_iospace, start, len / PAGE_SIZE); + + /* return io_offset */ + return start * PAGE_SIZE - res->start; +}
On Tuesday 04 March 2014, Liviu Dudau wrote:
+extern int isa_dma_bridge_buggy;
As commented before, I still think we can hardcode this to off
+unsigned long pci_address_to_pio(phys_addr_t address) +{
- struct ioresource *res;
- list_for_each_entry(res, &io_list, list) {
if (address >= res->start &&address < res->start + res->size) {return res->start - address;}- }
- return (unsigned long)-1;
+} +EXPORT_SYMBOL_GPL(pci_address_to_pio);
This has two problems:
- You return the address in bus space not the logical Linux port number as powerpc and microblaze do. - You imply that the window starts at bus address zero, which might not be the case.
+EXPORT_SYMBOL(pcibios_fixup_bus);
no need for this export.
+/*
- We don't have to worry about legacy ISA devices, so nothing to do here
- */
+resource_size_t pcibios_align_resource(void *data, const struct resource *res,
resource_size_t size, resource_size_t align)+{
- return ALIGN(res->start, align);
+} +EXPORT_SYMBOL(pcibios_align_resource);
or this one. Also, the ALIGN() is already done by the PCI core.
Arnd
On Tue, Mar 04, 2014 at 10:28:51PM +0000, Arnd Bergmann wrote:
On Tuesday 04 March 2014, Liviu Dudau wrote:
+extern int isa_dma_bridge_buggy;
As commented before, I still think we can hardcode this to off
Unfortunately that doesn't work when you have CONFIG_PCI=y. drivers/pci/pci.c has:
drivers/pci/pci.c:int isa_dma_bridge_buggy; drivers/pci/pci.c:EXPORT_SYMBOL(isa_dma_bridge_buggy);
and drivers/pci/quirks.c does:
drivers/pci/quirks.c: if (!isa_dma_bridge_buggy) { drivers/pci/quirks.c: isa_dma_bridge_buggy=1; drivers/pci/quirks.c- dev_info(&dev->dev, "Activating ISA DMA hang workarounds\n"); drivers/pci/quirks.c- }
+unsigned long pci_address_to_pio(phys_addr_t address) +{
- struct ioresource *res;
- list_for_each_entry(res, &io_list, list) {
if (address >= res->start &&address < res->start + res->size) {return res->start - address;}- }
- return (unsigned long)-1;
+} +EXPORT_SYMBOL_GPL(pci_address_to_pio);
This has two problems:
- You return the address in bus space not the logical Linux port number as powerpc and microblaze do.
- You imply that the window starts at bus address zero, which might not be the case.
That's definitely not the intention here. I want to return the logical port number and I want that port number to start from zero for every registered IO range.
+EXPORT_SYMBOL(pcibios_fixup_bus);
no need for this export.
OK.
+/*
- We don't have to worry about legacy ISA devices, so nothing to do here
- */
+resource_size_t pcibios_align_resource(void *data, const struct resource *res,
resource_size_t size, resource_size_t align)+{
- return ALIGN(res->start, align);
+} +EXPORT_SYMBOL(pcibios_align_resource);
or this one. Also, the ALIGN() is already done by the PCI core.
This has been fix in v6 already.
Thanks for reviewing this!
Liviu
Arnd
On Thursday 06 March 2014, Liviu Dudau wrote:
On Tue, Mar 04, 2014 at 10:28:51PM +0000, Arnd Bergmann wrote:
On Tuesday 04 March 2014, Liviu Dudau wrote:
+extern int isa_dma_bridge_buggy;
As commented before, I still think we can hardcode this to off
Unfortunately that doesn't work when you have CONFIG_PCI=y. drivers/pci/pci.c has:
drivers/pci/pci.c:int isa_dma_bridge_buggy; drivers/pci/pci.c:EXPORT_SYMBOL(isa_dma_bridge_buggy);
and drivers/pci/quirks.c does:
drivers/pci/quirks.c: if (!isa_dma_bridge_buggy) { drivers/pci/quirks.c: isa_dma_bridge_buggy=1; drivers/pci/quirks.c- dev_info(&dev->dev, "Activating ISA DMA hang workarounds\n"); drivers/pci/quirks.c- }
Ah, I see. Should we maybe move the declaration to a more generic header file then? If the architectures cannot override it anyway, it shouldn't be up to the architecture to declare it.
Arnd
On Fri, Mar 07, 2014 at 01:17:46AM +0100, Arnd Bergmann wrote:
On Thursday 06 March 2014, Liviu Dudau wrote:
On Tue, Mar 04, 2014 at 10:28:51PM +0000, Arnd Bergmann wrote:
On Tuesday 04 March 2014, Liviu Dudau wrote:
+extern int isa_dma_bridge_buggy;
As commented before, I still think we can hardcode this to off
Unfortunately that doesn't work when you have CONFIG_PCI=y. drivers/pci/pci.c has:
drivers/pci/pci.c:int isa_dma_bridge_buggy; drivers/pci/pci.c:EXPORT_SYMBOL(isa_dma_bridge_buggy);
and drivers/pci/quirks.c does:
drivers/pci/quirks.c: if (!isa_dma_bridge_buggy) { drivers/pci/quirks.c: isa_dma_bridge_buggy=1; drivers/pci/quirks.c- dev_info(&dev->dev, "Activating ISA DMA hang workarounds\n"); drivers/pci/quirks.c- }
Ah, I see. Should we maybe move the declaration to a more generic header file then? If the architectures cannot override it anyway, it shouldn't be up to the architecture to declare it.
Yes, it's one of those cleanup jobs that seems that no one wants to take on. I'm reluctant to deal with it here as it would easily double the size of the patchset, so I'm currently playing along like all other architectures.
Liviu
Arnd
To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
linaro-kernel@lists.linaro.org
-
Arnd Bergmann -
Liviu Dudau -
Liviu Dudau