Using two distinct DMA zones turned out to be problematic. Here's an attempt go back to a saner default.
Ard Biesheuvel (1): arm64: mm: Set ZONE_DMA size based on early IORT scan
Nicolas Saenz Julienne (6): arm64: mm: Move reserve_crashkernel() into mem_init() arm64: mm: Move zone_dma_bits initialization into zone_sizes_init() of/address: Introduce of_dma_get_max_cpu_address() of: unittest: Add test for of_dma_get_max_cpu_address() arm64: mm: Set ZONE_DMA size based on devicetree's dma-ranges mm: Remove examples from enum zone_type comment
arch/arm64/mm/init.c | 22 +++++++++------- drivers/acpi/arm64/iort.c | 55 +++++++++++++++++++++++++++++++++++++++ drivers/of/address.c | 42 ++++++++++++++++++++++++++++++ drivers/of/unittest.c | 18 +++++++++++++ include/linux/acpi_iort.h | 4 +++ include/linux/mmzone.h | 20 -------------- include/linux/of.h | 7 +++++ 7 files changed, 139 insertions(+), 29 deletions(-)
From: Nicolas Saenz Julienne nsaenzjulienne@suse.de
commit 0a30c53573b07d5561457e41fb0ab046cd857da5 upstream
crashkernel might reserve memory located in ZONE_DMA. We plan to delay ZONE_DMA's initialization after unflattening the devicetree and ACPI's boot table initialization, so move it later in the boot process. Specifically into bootmem_init() since request_standard_resources() depends on it.
Signed-off-by: Nicolas Saenz Julienne nsaenzjulienne@suse.de Tested-by: Jeremy Linton jeremy.linton@arm.com Link: https://lore.kernel.org/r/20201119175400.9995-2-nsaenzjulienne@suse.de Signed-off-by: Catalin Marinas catalin.marinas@arm.com Cc: stable@vger.kernel.org Signed-off-by: Jing Xiangfeng jingxiangfeng@huawei.com --- arch/arm64/mm/init.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 00576a960f11..686653e33250 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -386,8 +386,6 @@ void __init arm64_memblock_init(void) else arm64_dma32_phys_limit = PHYS_MASK + 1;
- reserve_crashkernel(); - reserve_elfcorehdr();
high_memory = __va(memblock_end_of_DRAM() - 1) + 1; @@ -427,6 +425,12 @@ void __init bootmem_init(void) sparse_init(); zone_sizes_init(min, max);
+ /* + * request_standard_resources() depends on crashkernel's memory being + * reserved, so do it here. + */ + reserve_crashkernel(); + memblock_dump_all(); }
From: Nicolas Saenz Julienne nsaenzjulienne@suse.de
commit 9804f8c69b04a39d0ba41d19e6bdc6aa91c19725 upstream
zone_dma_bits's initialization happens earlier that it's actually needed, in arm64_memblock_init(). So move it into the more suitable zone_sizes_init().
Signed-off-by: Nicolas Saenz Julienne nsaenzjulienne@suse.de Tested-by: Jeremy Linton jeremy.linton@arm.com Link: https://lore.kernel.org/r/20201119175400.9995-3-nsaenzjulienne@suse.de Signed-off-by: Catalin Marinas catalin.marinas@arm.com Cc: stable@vger.kernel.org Signed-off-by: Jing Xiangfeng jingxiangfeng@huawei.com --- arch/arm64/mm/init.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 686653e33250..7da912bf4222 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -190,6 +190,8 @@ static void __init zone_sizes_init(unsigned long min, unsigned long max) unsigned long max_zone_pfns[MAX_NR_ZONES] = {0};
#ifdef CONFIG_ZONE_DMA + zone_dma_bits = ARM64_ZONE_DMA_BITS; + arm64_dma_phys_limit = max_zone_phys(zone_dma_bits); max_zone_pfns[ZONE_DMA] = PFN_DOWN(arm64_dma_phys_limit); #endif #ifdef CONFIG_ZONE_DMA32 @@ -376,11 +378,6 @@ void __init arm64_memblock_init(void)
early_init_fdt_scan_reserved_mem();
- if (IS_ENABLED(CONFIG_ZONE_DMA)) { - zone_dma_bits = ARM64_ZONE_DMA_BITS; - arm64_dma_phys_limit = max_zone_phys(ARM64_ZONE_DMA_BITS); - } - if (IS_ENABLED(CONFIG_ZONE_DMA32)) arm64_dma32_phys_limit = max_zone_phys(32); else
From: Nicolas Saenz Julienne nsaenzjulienne@suse.de
commit 964db79d6c186cc2ecc6ae46f98eed7e0ea8cf71 upstream
Introduce of_dma_get_max_cpu_address(), which provides the highest CPU physical address addressable by all DMA masters in the system. It's specially useful for setting memory zones sizes at early boot time.
Signed-off-by: Nicolas Saenz Julienne nsaenzjulienne@suse.de Reviewed-by: Rob Herring robh@kernel.org Link: https://lore.kernel.org/r/20201119175400.9995-4-nsaenzjulienne@suse.de Signed-off-by: Catalin Marinas catalin.marinas@arm.com Cc: stable@vger.kernel.org Signed-off-by: Jing Xiangfeng jingxiangfeng@huawei.com --- drivers/of/address.c | 42 ++++++++++++++++++++++++++++++++++++++++++ include/linux/of.h | 7 +++++++ 2 files changed, 49 insertions(+)
diff --git a/drivers/of/address.c b/drivers/of/address.c index 1c3257a2d4e3..73ddf2540f3f 100644 --- a/drivers/of/address.c +++ b/drivers/of/address.c @@ -1024,6 +1024,48 @@ int of_dma_get_range(struct device_node *np, const struct bus_dma_region **map) } #endif /* CONFIG_HAS_DMA */
+/** + * of_dma_get_max_cpu_address - Gets highest CPU address suitable for DMA + * @np: The node to start searching from or NULL to start from the root + * + * Gets the highest CPU physical address that is addressable by all DMA masters + * in the sub-tree pointed by np, or the whole tree if NULL is passed. If no + * DMA constrained device is found, it returns PHYS_ADDR_MAX. + */ +phys_addr_t __init of_dma_get_max_cpu_address(struct device_node *np) +{ + phys_addr_t max_cpu_addr = PHYS_ADDR_MAX; + struct of_range_parser parser; + phys_addr_t subtree_max_addr; + struct device_node *child; + struct of_range range; + const __be32 *ranges; + u64 cpu_end = 0; + int len; + + if (!np) + np = of_root; + + ranges = of_get_property(np, "dma-ranges", &len); + if (ranges && len) { + of_dma_range_parser_init(&parser, np); + for_each_of_range(&parser, &range) + if (range.cpu_addr + range.size > cpu_end) + cpu_end = range.cpu_addr + range.size - 1; + + if (max_cpu_addr > cpu_end) + max_cpu_addr = cpu_end; + } + + for_each_available_child_of_node(np, child) { + subtree_max_addr = of_dma_get_max_cpu_address(child); + if (max_cpu_addr > subtree_max_addr) + max_cpu_addr = subtree_max_addr; + } + + return max_cpu_addr; +} + /** * of_dma_is_coherent - Check if device is coherent * @np: device node diff --git a/include/linux/of.h b/include/linux/of.h index af655d264f10..0f4e81e6fb23 100644 --- a/include/linux/of.h +++ b/include/linux/of.h @@ -558,6 +558,8 @@ int of_map_id(struct device_node *np, u32 id, const char *map_name, const char *map_mask_name, struct device_node **target, u32 *id_out);
+phys_addr_t of_dma_get_max_cpu_address(struct device_node *np); + #else /* CONFIG_OF */
static inline void of_core_init(void) @@ -995,6 +997,11 @@ static inline int of_map_id(struct device_node *np, u32 id, return -EINVAL; }
+static inline phys_addr_t of_dma_get_max_cpu_address(struct device_node *np) +{ + return PHYS_ADDR_MAX; +} + #define of_match_ptr(_ptr) NULL #define of_match_node(_matches, _node) NULL #endif /* CONFIG_OF */
From: Nicolas Saenz Julienne nsaenzjulienne@suse.de
commit 07d13a1d6120d453c3c1f020578693d072deded5 upstream
Introduce a test for of_dma_get_max_cup_address(), it uses the same DT data as the rest of dma-ranges unit tests.
Signed-off-by: Nicolas Saenz Julienne nsaenzjulienne@suse.de Reviewed-by: Rob Herring robh@kernel.org Link: https://lore.kernel.org/r/20201119175400.9995-5-nsaenzjulienne@suse.de Signed-off-by: Catalin Marinas catalin.marinas@arm.com Cc: stable@vger.kernel.org Signed-off-by: Jing Xiangfeng jingxiangfeng@huawei.com --- drivers/of/unittest.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
diff --git a/drivers/of/unittest.c b/drivers/of/unittest.c index 06cc988faf78..98cc0163301b 100644 --- a/drivers/of/unittest.c +++ b/drivers/of/unittest.c @@ -869,6 +869,23 @@ static void __init of_unittest_changeset(void) #endif }
+static void __init of_unittest_dma_get_max_cpu_address(void) +{ + struct device_node *np; + phys_addr_t cpu_addr; + + np = of_find_node_by_path("/testcase-data/address-tests"); + if (!np) { + pr_err("missing testcase data\n"); + return; + } + + cpu_addr = of_dma_get_max_cpu_address(np); + unittest(cpu_addr == 0x4fffffff, + "of_dma_get_max_cpu_address: wrong CPU addr %pad (expecting %x)\n", + &cpu_addr, 0x4fffffff); +} + static void __init of_unittest_dma_ranges_one(const char *path, u64 expect_dma_addr, u64 expect_paddr) { @@ -3266,6 +3283,7 @@ static int __init of_unittest(void) of_unittest_changeset(); of_unittest_parse_interrupts(); of_unittest_parse_interrupts_extended(); + of_unittest_dma_get_max_cpu_address(); of_unittest_parse_dma_ranges(); of_unittest_pci_dma_ranges(); of_unittest_match_node();
From: Nicolas Saenz Julienne nsaenzjulienne@suse.de
commit 8424ecdde7df99d5426e1a1fd9f0fb36f4183032 upstream
We recently introduced a 1 GB sized ZONE_DMA to cater for platforms incorporating masters that can address less than 32 bits of DMA, in particular the Raspberry Pi 4, which has 4 or 8 GB of DRAM, but has peripherals that can only address up to 1 GB (and its PCIe host bridge can only access the bottom 3 GB)
The DMA layer also needs to be able to allocate memory that is guaranteed to meet those DMA constraints, for bounce buffering as well as allocating the backing for consistent mappings. This is why the 1 GB ZONE_DMA was introduced recently. Unfortunately, it turns out the having a 1 GB ZONE_DMA as well as a ZONE_DMA32 causes problems with kdump, and potentially in other places where allocations cannot cross zone boundaries. Therefore, we should avoid having two separate DMA zones when possible.
So, with the help of of_dma_get_max_cpu_address() get the topmost physical address accessible to all DMA masters in system and use that information to fine-tune ZONE_DMA's size. In the absence of addressing limited masters ZONE_DMA will span the whole 32-bit address space, otherwise, in the case of the Raspberry Pi 4 it'll only span the 30-bit address space, and have ZONE_DMA32 cover the rest of the 32-bit address space.
Signed-off-by: Nicolas Saenz Julienne nsaenzjulienne@suse.de Link: https://lore.kernel.org/r/20201119175400.9995-6-nsaenzjulienne@suse.de Signed-off-by: Catalin Marinas catalin.marinas@arm.com Cc: stable@vger.kernel.org Signed-off-by: Jing Xiangfeng jingxiangfeng@huawei.com --- arch/arm64/mm/init.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 7da912bf4222..05a1c2773629 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -42,8 +42,6 @@ #include <asm/tlb.h> #include <asm/alternative.h>
-#define ARM64_ZONE_DMA_BITS 30 - /* * We need to be able to catch inadvertent references to memstart_addr * that occur (potentially in generic code) before arm64_memblock_init() @@ -188,9 +186,11 @@ static phys_addr_t __init max_zone_phys(unsigned int zone_bits) static void __init zone_sizes_init(unsigned long min, unsigned long max) { unsigned long max_zone_pfns[MAX_NR_ZONES] = {0}; + unsigned int __maybe_unused dt_zone_dma_bits;
#ifdef CONFIG_ZONE_DMA - zone_dma_bits = ARM64_ZONE_DMA_BITS; + dt_zone_dma_bits = fls64(of_dma_get_max_cpu_address(NULL)); + zone_dma_bits = min(32U, dt_zone_dma_bits); arm64_dma_phys_limit = max_zone_phys(zone_dma_bits); max_zone_pfns[ZONE_DMA] = PFN_DOWN(arm64_dma_phys_limit); #endif
From: Ard Biesheuvel ardb@kernel.org
commit 2b8652936f0ca9ca2e6c984ae76c7bfcda1b3f22 upstream
We recently introduced a 1 GB sized ZONE_DMA to cater for platforms incorporating masters that can address less than 32 bits of DMA, in particular the Raspberry Pi 4, which has 4 or 8 GB of DRAM, but has peripherals that can only address up to 1 GB (and its PCIe host bridge can only access the bottom 3 GB)
Instructing the DMA layer about these limitations is straight-forward, even though we had to fix some issues regarding memory limits set in the IORT for named components, and regarding the handling of ACPI _DMA methods. However, the DMA layer also needs to be able to allocate memory that is guaranteed to meet those DMA constraints, for bounce buffering as well as allocating the backing for consistent mappings.
This is why the 1 GB ZONE_DMA was introduced recently. Unfortunately, it turns out the having a 1 GB ZONE_DMA as well as a ZONE_DMA32 causes problems with kdump, and potentially in other places where allocations cannot cross zone boundaries. Therefore, we should avoid having two separate DMA zones when possible.
So let's do an early scan of the IORT, and only create the ZONE_DMA if we encounter any devices that need it. This puts the burden on the firmware to describe such limitations in the IORT, which may be redundant (and less precise) if _DMA methods are also being provided. However, it should be noted that this situation is highly unusual for arm64 ACPI machines. Also, the DMA subsystem still gives precedence to the _DMA method if implemented, and so we will not lose the ability to perform streaming DMA outside the ZONE_DMA if the _DMA method permits it.
[nsaenz: unified implementation with DT's counterpart]
Signed-off-by: Ard Biesheuvel ardb@kernel.org Signed-off-by: Nicolas Saenz Julienne nsaenzjulienne@suse.de Tested-by: Jeremy Linton jeremy.linton@arm.com Acked-by: Lorenzo Pieralisi lorenzo.pieralisi@arm.com Acked-by: Hanjun Guo guohanjun@huawei.com Cc: Jeremy Linton jeremy.linton@arm.com Cc: Lorenzo Pieralisi lorenzo.pieralisi@arm.com Cc: Nicolas Saenz Julienne nsaenzjulienne@suse.de Cc: Rob Herring robh+dt@kernel.org Cc: Christoph Hellwig hch@lst.de Cc: Robin Murphy robin.murphy@arm.com Cc: Hanjun Guo guohanjun@huawei.com Cc: Sudeep Holla sudeep.holla@arm.com Cc: Anshuman Khandual anshuman.khandual@arm.com Link: https://lore.kernel.org/r/20201119175400.9995-7-nsaenzjulienne@suse.de Signed-off-by: Catalin Marinas catalin.marinas@arm.com Cc: stable@vger.kernel.org Signed-off-by: Jing Xiangfeng jingxiangfeng@huawei.com --- arch/arm64/mm/init.c | 5 +++- drivers/acpi/arm64/iort.c | 55 +++++++++++++++++++++++++++++++++++++++ include/linux/acpi_iort.h | 4 +++ 3 files changed, 63 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 05a1c2773629..b913844ab740 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -29,6 +29,7 @@ #include <linux/kexec.h> #include <linux/crash_dump.h> #include <linux/hugetlb.h> +#include <linux/acpi_iort.h>
#include <asm/boot.h> #include <asm/fixmap.h> @@ -186,11 +187,13 @@ static phys_addr_t __init max_zone_phys(unsigned int zone_bits) static void __init zone_sizes_init(unsigned long min, unsigned long max) { unsigned long max_zone_pfns[MAX_NR_ZONES] = {0}; + unsigned int __maybe_unused acpi_zone_dma_bits; unsigned int __maybe_unused dt_zone_dma_bits;
#ifdef CONFIG_ZONE_DMA + acpi_zone_dma_bits = fls64(acpi_iort_dma_get_max_cpu_address()); dt_zone_dma_bits = fls64(of_dma_get_max_cpu_address(NULL)); - zone_dma_bits = min(32U, dt_zone_dma_bits); + zone_dma_bits = min3(32U, dt_zone_dma_bits, acpi_zone_dma_bits); arm64_dma_phys_limit = max_zone_phys(zone_dma_bits); max_zone_pfns[ZONE_DMA] = PFN_DOWN(arm64_dma_phys_limit); #endif diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c index 94f34109695c..2494138a6905 100644 --- a/drivers/acpi/arm64/iort.c +++ b/drivers/acpi/arm64/iort.c @@ -1730,3 +1730,58 @@ void __init acpi_iort_init(void)
iort_init_platform_devices(); } + +#ifdef CONFIG_ZONE_DMA +/* + * Extract the highest CPU physical address accessible to all DMA masters in + * the system. PHYS_ADDR_MAX is returned when no constrained device is found. + */ +phys_addr_t __init acpi_iort_dma_get_max_cpu_address(void) +{ + phys_addr_t limit = PHYS_ADDR_MAX; + struct acpi_iort_node *node, *end; + struct acpi_table_iort *iort; + acpi_status status; + int i; + + if (acpi_disabled) + return limit; + + status = acpi_get_table(ACPI_SIG_IORT, 0, + (struct acpi_table_header **)&iort); + if (ACPI_FAILURE(status)) + return limit; + + node = ACPI_ADD_PTR(struct acpi_iort_node, iort, iort->node_offset); + end = ACPI_ADD_PTR(struct acpi_iort_node, iort, iort->header.length); + + for (i = 0; i < iort->node_count; i++) { + if (node >= end) + break; + + switch (node->type) { + struct acpi_iort_named_component *ncomp; + struct acpi_iort_root_complex *rc; + phys_addr_t local_limit; + + case ACPI_IORT_NODE_NAMED_COMPONENT: + ncomp = (struct acpi_iort_named_component *)node->node_data; + local_limit = DMA_BIT_MASK(ncomp->memory_address_limit); + limit = min_not_zero(limit, local_limit); + break; + + case ACPI_IORT_NODE_PCI_ROOT_COMPLEX: + if (node->revision < 1) + break; + + rc = (struct acpi_iort_root_complex *)node->node_data; + local_limit = DMA_BIT_MASK(rc->memory_address_limit); + limit = min_not_zero(limit, local_limit); + break; + } + node = ACPI_ADD_PTR(struct acpi_iort_node, node, node->length); + } + acpi_put_table(&iort->header); + return limit; +} +#endif diff --git a/include/linux/acpi_iort.h b/include/linux/acpi_iort.h index 20a32120bb88..1a12baa58e40 100644 --- a/include/linux/acpi_iort.h +++ b/include/linux/acpi_iort.h @@ -38,6 +38,7 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *size); const struct iommu_ops *iort_iommu_configure_id(struct device *dev, const u32 *id_in); int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head); +phys_addr_t acpi_iort_dma_get_max_cpu_address(void); #else static inline void acpi_iort_init(void) { } static inline u32 iort_msi_map_id(struct device *dev, u32 id) @@ -55,6 +56,9 @@ static inline const struct iommu_ops *iort_iommu_configure_id( static inline int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head) { return 0; } + +static inline phys_addr_t acpi_iort_dma_get_max_cpu_address(void) +{ return PHYS_ADDR_MAX; } #endif
#endif /* __ACPI_IORT_H__ */
From: Nicolas Saenz Julienne nsaenzjulienne@suse.de
commit 04435217f96869ac3a8f055ff68c5237a60bcd7e upstream
We can't really list every setup in common code. On top of that they are unlikely to stay true for long as things change in the arch trees independently of this comment.
Suggested-by: Christoph Hellwig hch@lst.de Signed-off-by: Nicolas Saenz Julienne nsaenzjulienne@suse.de Reviewed-by: Christoph Hellwig hch@lst.de Link: https://lore.kernel.org/r/20201119175400.9995-8-nsaenzjulienne@suse.de Signed-off-by: Catalin Marinas catalin.marinas@arm.com Cc: stable@vger.kernel.org Signed-off-by: Jing Xiangfeng jingxiangfeng@huawei.com --- include/linux/mmzone.h | 20 -------------------- 1 file changed, 20 deletions(-)
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index fb3bf696c05e..9d0c454d23cd 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -354,26 +354,6 @@ enum zone_type { * DMA mask is assumed when ZONE_DMA32 is defined. Some 64-bit * platforms may need both zones as they support peripherals with * different DMA addressing limitations. - * - * Some examples: - * - * - i386 and x86_64 have a fixed 16M ZONE_DMA and ZONE_DMA32 for the - * rest of the lower 4G. - * - * - arm only uses ZONE_DMA, the size, up to 4G, may vary depending on - * the specific device. - * - * - arm64 has a fixed 1G ZONE_DMA and ZONE_DMA32 for the rest of the - * lower 4G. - * - * - powerpc only uses ZONE_DMA, the size, up to 2G, may vary - * depending on the specific device. - * - * - s390 uses ZONE_DMA fixed to the lower 2G. - * - * - ia64 and riscv only use ZONE_DMA32. - * - * - parisc uses neither. */ #ifdef CONFIG_ZONE_DMA ZONE_DMA,
Hi Greg.
On Thu, 2021-03-04 at 14:46 +0100, Greg KH wrote:
On Wed, Mar 03, 2021 at 03:33:12PM +0800, Jing Xiangfeng wrote:
Using two distinct DMA zones turned out to be problematic. Here's an attempt go back to a saner default.
What problem does this solve? How does this fit into the stable kernel rules?
We changed the way we setup memory zones in arm64 in order to cater for Raspberry Pi 4's weird DMA constraints: ZONE_DMA spans the lower 1GB of memory and ZONE_DMA32 the rest of the 32bit address space. Since you can't allocate memory that crosses zone boundaries, this broke crashkernel allocations on big machines. This series fixes all this by parsing the HW description and checking for DMA constrained buses. When not found, the unnecessary zone creation is skipped.
That said, I have no clue whether this falls or not into the stable kernel rules.
Regards, Nicolas
On Thu, Mar 04, 2021 at 03:05:32PM +0100, Nicolas Saenz Julienne wrote:
Hi Greg.
On Thu, 2021-03-04 at 14:46 +0100, Greg KH wrote:
On Wed, Mar 03, 2021 at 03:33:12PM +0800, Jing Xiangfeng wrote:
Using two distinct DMA zones turned out to be problematic. Here's an attempt go back to a saner default.
What problem does this solve? How does this fit into the stable kernel rules?
We changed the way we setup memory zones in arm64 in order to cater for Raspberry Pi 4's weird DMA constraints: ZONE_DMA spans the lower 1GB of memory and ZONE_DMA32 the rest of the 32bit address space. Since you can't allocate memory that crosses zone boundaries, this broke crashkernel allocations on big machines. This series fixes all this by parsing the HW description and checking for DMA constrained buses. When not found, the unnecessary zone creation is skipped.
What kernel/commit caused this "breakage"?
thanks,
greg k-h
On Thu, 2021-03-04 at 15:17 +0100, Greg KH wrote:
On Thu, Mar 04, 2021 at 03:05:32PM +0100, Nicolas Saenz Julienne wrote:
Hi Greg.
On Thu, 2021-03-04 at 14:46 +0100, Greg KH wrote:
On Wed, Mar 03, 2021 at 03:33:12PM +0800, Jing Xiangfeng wrote:
Using two distinct DMA zones turned out to be problematic. Here's an attempt go back to a saner default.
What problem does this solve? How does this fit into the stable kernel rules?
We changed the way we setup memory zones in arm64 in order to cater for Raspberry Pi 4's weird DMA constraints: ZONE_DMA spans the lower 1GB of memory and ZONE_DMA32 the rest of the 32bit address space. Since you can't allocate memory that crosses zone boundaries, this broke crashkernel allocations on big machines. This series fixes all this by parsing the HW description and checking for DMA constrained buses. When not found, the unnecessary zone creation is skipped.
What kernel/commit caused this "breakage"?
1a8e1cef7603 arm64: use both ZONE_DMA and ZONE_DMA32
Regards, Nicolas
On Thu, Mar 04, 2021 at 04:09:28PM +0100, Nicolas Saenz Julienne wrote:
On Thu, 2021-03-04 at 15:17 +0100, Greg KH wrote:
On Thu, Mar 04, 2021 at 03:05:32PM +0100, Nicolas Saenz Julienne wrote:
Hi Greg.
On Thu, 2021-03-04 at 14:46 +0100, Greg KH wrote:
On Wed, Mar 03, 2021 at 03:33:12PM +0800, Jing Xiangfeng wrote:
Using two distinct DMA zones turned out to be problematic. Here's an attempt go back to a saner default.
What problem does this solve? How does this fit into the stable kernel rules?
We changed the way we setup memory zones in arm64 in order to cater for Raspberry Pi 4's weird DMA constraints: ZONE_DMA spans the lower 1GB of memory and ZONE_DMA32 the rest of the 32bit address space. Since you can't allocate memory that crosses zone boundaries, this broke crashkernel allocations on big machines. This series fixes all this by parsing the HW description and checking for DMA constrained buses. When not found, the unnecessary zone creation is skipped.
What kernel/commit caused this "breakage"?
1a8e1cef7603 arm64: use both ZONE_DMA and ZONE_DMA32
Thanks for the info, all now queued up.
greg k-h
On 2021/3/7 23:24, Greg KH wrote:
On Thu, Mar 04, 2021 at 04:09:28PM +0100, Nicolas Saenz Julienne wrote:
On Thu, 2021-03-04 at 15:17 +0100, Greg KH wrote:
On Thu, Mar 04, 2021 at 03:05:32PM +0100, Nicolas Saenz Julienne wrote:
Hi Greg.
On Thu, 2021-03-04 at 14:46 +0100, Greg KH wrote:
On Wed, Mar 03, 2021 at 03:33:12PM +0800, Jing Xiangfeng wrote:
Using two distinct DMA zones turned out to be problematic. Here's an attempt go back to a saner default.
What problem does this solve? How does this fit into the stable kernel rules?
We changed the way we setup memory zones in arm64 in order to cater for Raspberry Pi 4's weird DMA constraints: ZONE_DMA spans the lower 1GB of memory and ZONE_DMA32 the rest of the 32bit address space. Since you can't allocate memory that crosses zone boundaries, this broke crashkernel allocations on big machines. This series fixes all this by parsing the HW description and checking for DMA constrained buses. When not found, the unnecessary zone creation is skipped.
What kernel/commit caused this "breakage"?
1a8e1cef7603 arm64: use both ZONE_DMA and ZONE_DMA32
Thanks for the info, all now queued up.
There is a fix in 5.11. Please consider applying the following commit to 5.10.y:
aed5041ef9a3 of: unittest: Fix build on architectures without CONFIG_OF_ADDRES
Thanks
greg k-h .
On Mon, Mar 08, 2021 at 11:20:03AM +0800, Jing Xiangfeng wrote:
On 2021/3/7 23:24, Greg KH wrote:
On Thu, Mar 04, 2021 at 04:09:28PM +0100, Nicolas Saenz Julienne wrote:
On Thu, 2021-03-04 at 15:17 +0100, Greg KH wrote:
On Thu, Mar 04, 2021 at 03:05:32PM +0100, Nicolas Saenz Julienne wrote:
Hi Greg.
On Thu, 2021-03-04 at 14:46 +0100, Greg KH wrote:
On Wed, Mar 03, 2021 at 03:33:12PM +0800, Jing Xiangfeng wrote: > Using two distinct DMA zones turned out to be problematic. Here's an > attempt go back to a saner default. What problem does this solve? How does this fit into the stable kernel rules?
We changed the way we setup memory zones in arm64 in order to cater for Raspberry Pi 4's weird DMA constraints: ZONE_DMA spans the lower 1GB of memory and ZONE_DMA32 the rest of the 32bit address space. Since you can't allocate memory that crosses zone boundaries, this broke crashkernel allocations on big machines. This series fixes all this by parsing the HW description and checking for DMA constrained buses. When not found, the unnecessary zone creation is skipped.
What kernel/commit caused this "breakage"?
1a8e1cef7603 arm64: use both ZONE_DMA and ZONE_DMA32
Thanks for the info, all now queued up.
There is a fix in 5.11. Please consider applying the following commit to 5.10.y:
aed5041ef9a3 of: unittest: Fix build on architectures without CONFIG_OF_ADDRES
Thanks, now queued up.
greg k-h
On 2021/3/8 17:58, Greg KH wrote:
On Mon, Mar 08, 2021 at 11:20:03AM +0800, Jing Xiangfeng wrote:
On 2021/3/7 23:24, Greg KH wrote:
On Thu, Mar 04, 2021 at 04:09:28PM +0100, Nicolas Saenz Julienne wrote:
On Thu, 2021-03-04 at 15:17 +0100, Greg KH wrote:
On Thu, Mar 04, 2021 at 03:05:32PM +0100, Nicolas Saenz Julienne wrote:
Hi Greg.
On Thu, 2021-03-04 at 14:46 +0100, Greg KH wrote: > On Wed, Mar 03, 2021 at 03:33:12PM +0800, Jing Xiangfeng wrote: >> Using two distinct DMA zones turned out to be problematic. Here's an >> attempt go back to a saner default. > What problem does this solve? How does this fit into the stable kernel > rules? We changed the way we setup memory zones in arm64 in order to cater for Raspberry Pi 4's weird DMA constraints: ZONE_DMA spans the lower 1GB of memory and ZONE_DMA32 the rest of the 32bit address space. Since you can't allocate memory that crosses zone boundaries, this broke crashkernel allocations on big machines. This series fixes all this by parsing the HW description and checking for DMA constrained buses. When not found, the unnecessary zone creation is skipped.
What kernel/commit caused this "breakage"?
1a8e1cef7603 arm64: use both ZONE_DMA and ZONE_DMA32
Thanks for the info, all now queued up.
There is a fix in 5.11. Please consider applying the following commit to 5.10.y:
aed5041ef9a3 of: unittest: Fix build on architectures without CONFIG_OF_ADDRES
Thanks, now queued up.
Hi Grep, another commit d78050ee3544 "arm64: Remove arm64_dma32_phys_limit and its uses" should be involved, thanks.
"Prior to this patch, disabling CONFIG_ZONE_DMA32 leads to CMA allocation from the whole RAM as arm64_dma32_phys_limit becomes PHYS_MASK+1." from Catalin, see more from the link https://www.spinics.net/lists/arm-kernel/msg867356.html
greg k-h .
On Tue, May 11, 2021 at 08:35:47PM +0800, Kefeng Wang wrote:
On 2021/3/8 17:58, Greg KH wrote:
On Mon, Mar 08, 2021 at 11:20:03AM +0800, Jing Xiangfeng wrote:
On 2021/3/7 23:24, Greg KH wrote:
On Thu, Mar 04, 2021 at 04:09:28PM +0100, Nicolas Saenz Julienne wrote:
On Thu, 2021-03-04 at 15:17 +0100, Greg KH wrote:
On Thu, Mar 04, 2021 at 03:05:32PM +0100, Nicolas Saenz Julienne wrote: > Hi Greg. > > On Thu, 2021-03-04 at 14:46 +0100, Greg KH wrote: > > On Wed, Mar 03, 2021 at 03:33:12PM +0800, Jing Xiangfeng wrote: > > > Using two distinct DMA zones turned out to be problematic. Here's an > > > attempt go back to a saner default. > > What problem does this solve? How does this fit into the stable kernel > > rules? > We changed the way we setup memory zones in arm64 in order to cater for > Raspberry Pi 4's weird DMA constraints: ZONE_DMA spans the lower 1GB of memory > and ZONE_DMA32 the rest of the 32bit address space. Since you can't allocate > memory that crosses zone boundaries, this broke crashkernel allocations on big > machines. This series fixes all this by parsing the HW description and checking > for DMA constrained buses. When not found, the unnecessary zone creation is > skipped. What kernel/commit caused this "breakage"?
1a8e1cef7603 arm64: use both ZONE_DMA and ZONE_DMA32
Thanks for the info, all now queued up.
There is a fix in 5.11. Please consider applying the following commit to 5.10.y:
aed5041ef9a3 of: unittest: Fix build on architectures without CONFIG_OF_ADDRES
Thanks, now queued up.
Hi Grep, another commit d78050ee3544 "arm64: Remove arm64_dma32_phys_limit and its uses" should be involved, thanks.
"Prior to this patch, disabling CONFIG_ZONE_DMA32 leads to CMA allocation from the whole RAM as arm64_dma32_phys_limit becomes PHYS_MASK+1." from Catalin, see more from the link https://www.spinics.net/lists/arm-kernel/msg867356.html
Ok, now queued up.
greg k-h
Hi Greg,
There are two more patches about the ZONE_DMA[32] changes, especially the second one, both them need be backported, thanks.
791ab8b2e3db - arm64: Ignore any DMA offsets in the max_zone_phys() calculation 2687275a5843 - arm64: Force NO_BLOCK_MAPPINGS if crashkernel reservation is required
On 2021/5/11 20:35, Kefeng Wang wrote:
On 2021/3/8 17:58, Greg KH wrote:
On Mon, Mar 08, 2021 at 11:20:03AM +0800, Jing Xiangfeng wrote:
On 2021/3/7 23:24, Greg KH wrote:
On Thu, Mar 04, 2021 at 04:09:28PM +0100, Nicolas Saenz Julienne wrote:
On Thu, 2021-03-04 at 15:17 +0100, Greg KH wrote:
On Thu, Mar 04, 2021 at 03:05:32PM +0100, Nicolas Saenz Julienne wrote: > Hi Greg. > > On Thu, 2021-03-04 at 14:46 +0100, Greg KH wrote: >> On Wed, Mar 03, 2021 at 03:33:12PM +0800, Jing Xiangfeng wrote: >>> Using two distinct DMA zones turned out to be problematic. >>> Here's an >>> attempt go back to a saner default. >> What problem does this solve? How does this fit into the stable >> kernel >> rules? > We changed the way we setup memory zones in arm64 in order to > cater for > Raspberry Pi 4's weird DMA constraints: ZONE_DMA spans the lower > 1GB of memory > and ZONE_DMA32 the rest of the 32bit address space. Since you > can't allocate > memory that crosses zone boundaries, this broke crashkernel > allocations on big > machines. This series fixes all this by parsing the HW > description and checking > for DMA constrained buses. When not found, the unnecessary zone > creation is > skipped. What kernel/commit caused this "breakage"?
1a8e1cef7603 arm64: use both ZONE_DMA and ZONE_DMA32
Thanks for the info, all now queued up.
There is a fix in 5.11. Please consider applying the following commit to 5.10.y:
aed5041ef9a3 of: unittest: Fix build on architectures without CONFIG_OF_ADDRES
Thanks, now queued up.
Hi Grep, another commit d78050ee3544 "arm64: Remove arm64_dma32_phys_limit and its uses" should be involved, thanks.
"Prior to this patch, disabling CONFIG_ZONE_DMA32 leads to CMA allocation from the whole RAM as arm64_dma32_phys_limit becomes PHYS_MASK+1." from Catalin, see more from the link https://www.spinics.net/lists/arm-kernel/msg867356.html
greg k-h .
On Wed, Jun 23, 2021 at 02:59:59PM +0800, Kefeng Wang wrote:
Hi Greg,
There are two more patches about the ZONE_DMA[32] changes,
What ZONE_DMA changes?
especially the second one, both them need be backported, thanks.
Backported to where?
791ab8b2e3db - arm64: Ignore any DMA offsets in the max_zone_phys() calculation 2687275a5843 - arm64: Force NO_BLOCK_MAPPINGS if crashkernel reservation is required
Have you tried these patches? Where do they need to be applied to?
confused,
greg k-h
On 2021/6/23 15:12, Greg KH wrote:
On Wed, Jun 23, 2021 at 02:59:59PM +0800, Kefeng Wang wrote:
Hi Greg,
There are two more patches about the ZONE_DMA[32] changes,
What ZONE_DMA changes?
See the subject, [PATCH stable v5.10 0/7] arm64: Default to 32-bit wide ZONE_DMA, We asked the ARM64 ZONE_DMA change backport before, link[1]
especially the second one, both them need be backported, thanks.
Backported to where?
stable 5.10
791ab8b2e3db - arm64: Ignore any DMA offsets in the max_zone_phys() calculation 2687275a5843 - arm64: Force NO_BLOCK_MAPPINGS if crashkernel reservation is required
Have you tried these patches? Where do they need to be applied to?
Yes, we tested it, without them, especially the second one, we will meet crash when using kexec boot, also there is discussion in [2] and [3] from Catalin.
confused,
Sorry about this, should add more information, thanks.
[1] https://lore.kernel.org/linux-riscv/20210303073319.2215839-1-jingxiangfeng@h... [2] https://lore.kernel.org/linux-devicetree/e60d643e-4879-3fc3-737d-2c145332a6d... [3] https://lore.kernel.org/linux-arm-kernel/20201119175556.18681-1-catalin.mari...
greg k-h .
On Wed, Jun 23, 2021 at 03:25:10PM +0800, Kefeng Wang wrote:
On 2021/6/23 15:12, Greg KH wrote:
On Wed, Jun 23, 2021 at 02:59:59PM +0800, Kefeng Wang wrote:
Hi Greg,
There are two more patches about the ZONE_DMA[32] changes,
What ZONE_DMA changes?
See the subject, [PATCH stable v5.10 0/7] arm64: Default to 32-bit wide ZONE_DMA, We asked the ARM64 ZONE_DMA change backport before, link[1]
The subject doesn't help much, sorry, what commit does this refer to? What happened to it? Was it accepted or rejected?
especially the second one, both them need be backported, thanks.
Backported to where?
stable 5.10
Why?
791ab8b2e3db - arm64: Ignore any DMA offsets in the max_zone_phys() calculation 2687275a5843 - arm64: Force NO_BLOCK_MAPPINGS if crashkernel reservation is required
Have you tried these patches? Where do they need to be applied to?
Yes, we tested it, without them, especially the second one, we will meet crash when using kexec boot, also there is discussion in [2] and [3] from Catalin.
These [] do not seem to be links :(
thanks,
greg k-h
On 2021/6/23 15:34, Greg KH wrote:
On Wed, Jun 23, 2021 at 03:25:10PM +0800, Kefeng Wang wrote:
On 2021/6/23 15:12, Greg KH wrote:
On Wed, Jun 23, 2021 at 02:59:59PM +0800, Kefeng Wang wrote:
Hi Greg,
There are two more patches about the ZONE_DMA[32] changes,
What ZONE_DMA changes?
See the subject, [PATCH stable v5.10 0/7] arm64: Default to 32-bit wide ZONE_DMA, We asked the ARM64 ZONE_DMA change backport before, link[1]
Let's inline the link: https://lore.kernel.org/lkml/20210303073319.2215839-1-jingxiangfeng@huawei.c...
The following 7 patches(we asked from link) has merged into lts5.10(tag: v5.10.22)
4d7ed9a49b0c mm: Remove examples from enum zone_type comment 8eaef922e938 arm64: mm: Set ZONE_DMA size based on early IORT scan 35ec3d09ff6a arm64: mm: Set ZONE_DMA size based on devicetree's dma-ranges a9861e7fa4f8 of: unittest: Add test for of_dma_get_max_cpu_address() 18bf6e998d08 of/address: Introduce of_dma_get_max_cpu_address() 3fbe62ffbb54 arm64: mm: Move zone_dma_bits initialization into zone_sizes_init() 407b173adfac arm64: mm: Move reserve_crashkernel() into mem_init()
but the patch "arm64: mm: Move reserve_crashkernel() into mem_init()" has some issue, see the following discussion from Catalin,
https://lore.kernel.org/linux-devicetree/e60d643e-4879-3fc3-737d-2c145332a6d... https://lore.kernel.org/linux-arm-kernel/20201119175556.18681-1-catalin.mari...
and yes, we met crash in lts5.10 when kexec boot due to "arm64: mm: Move reserve_crashkernel() into mem_init()" too, which could be fixed by commit 2687275a5843 "arm64: Force NO_BLOCK_MAPPINGS if crashkernel reservation is required", and the commit 791ab8b2e3db "arm64: Ignore any DMA offsets in the max_zone_phys() calculation" also about DMA set, So I only asked the two patches(both in v5.11) related ARM64 ZONE_DMA changes backported into lts5.10.
The subject doesn't help much, sorry, what commit does this refer to? What happened to it? Was it accepted or rejected?
especially the second one, both them need be backported, thanks.
Backported to where?
stable 5.10
Why?
791ab8b2e3db - arm64: Ignore any DMA offsets in the max_zone_phys() calculation 2687275a5843 - arm64: Force NO_BLOCK_MAPPINGS if crashkernel reservation is required
Have you tried these patches? Where do they need to be applied to?
Yes, we tested it, without them, especially the second one, we will meet crash when using kexec boot, also there is discussion in [2] and [3] from Catalin.
These [] do not seem to be links :(
I could see the links is in the end, see https://lore.kernel.org/lkml/e47df0fd-0ddd-408b-2972-1b6d0a786f00@huawei.com...
thanks,
greg k-h .
On Wed, Jun 23, 2021 at 04:01:10PM +0800, Kefeng Wang wrote:
On 2021/6/23 15:34, Greg KH wrote:
On Wed, Jun 23, 2021 at 03:25:10PM +0800, Kefeng Wang wrote:
On 2021/6/23 15:12, Greg KH wrote:
On Wed, Jun 23, 2021 at 02:59:59PM +0800, Kefeng Wang wrote:
Hi Greg,
There are two more patches about the ZONE_DMA[32] changes,
What ZONE_DMA changes?
See the subject, [PATCH stable v5.10 0/7] arm64: Default to 32-bit wide ZONE_DMA, We asked the ARM64 ZONE_DMA change backport before, link[1]
Let's inline the link: https://lore.kernel.org/lkml/20210303073319.2215839-1-jingxiangfeng@huawei.c...
The following 7 patches(we asked from link) has merged into lts5.10(tag: v5.10.22)
4d7ed9a49b0c mm: Remove examples from enum zone_type comment 8eaef922e938 arm64: mm: Set ZONE_DMA size based on early IORT scan 35ec3d09ff6a arm64: mm: Set ZONE_DMA size based on devicetree's dma-ranges a9861e7fa4f8 of: unittest: Add test for of_dma_get_max_cpu_address() 18bf6e998d08 of/address: Introduce of_dma_get_max_cpu_address() 3fbe62ffbb54 arm64: mm: Move zone_dma_bits initialization into zone_sizes_init() 407b173adfac arm64: mm: Move reserve_crashkernel() into mem_init()
but the patch "arm64: mm: Move reserve_crashkernel() into mem_init()" has some issue, see the following discussion from Catalin,
https://lore.kernel.org/linux-devicetree/e60d643e-4879-3fc3-737d-2c145332a6d... https://lore.kernel.org/linux-arm-kernel/20201119175556.18681-1-catalin.mari...
and yes, we met crash in lts5.10 when kexec boot due to "arm64: mm: Move reserve_crashkernel() into mem_init()" too, which could be fixed by commit 2687275a5843 "arm64: Force NO_BLOCK_MAPPINGS if crashkernel reservation is required", and the commit 791ab8b2e3db "arm64: Ignore any DMA offsets in the max_zone_phys() calculation" also about DMA set, So I only asked the two patches(both in v5.11) related ARM64 ZONE_DMA changes backported into lts5.10.
Thanks, all now queued up.
greg k-h
On 2021/6/25 18:19, Greg KH wrote:
On Wed, Jun 23, 2021 at 04:01:10PM +0800, Kefeng Wang wrote:
Let's inline the link: https://lore.kernel.org/lkml/20210303073319.2215839-1-jingxiangfeng@huawei.c...
The following 7 patches(we asked from link) has merged into lts5.10(tag: v5.10.22)
4d7ed9a49b0c mm: Remove examples from enum zone_type comment 8eaef922e938 arm64: mm: Set ZONE_DMA size based on early IORT scan 35ec3d09ff6a arm64: mm: Set ZONE_DMA size based on devicetree's dma-ranges a9861e7fa4f8 of: unittest: Add test for of_dma_get_max_cpu_address() 18bf6e998d08 of/address: Introduce of_dma_get_max_cpu_address() 3fbe62ffbb54 arm64: mm: Move zone_dma_bits initialization into zone_sizes_init() 407b173adfac arm64: mm: Move reserve_crashkernel() into mem_init()
but the patch "arm64: mm: Move reserve_crashkernel() into mem_init()" has some issue, see the following discussion from Catalin,
https://lore.kernel.org/linux-devicetree/e60d643e-4879-3fc3-737d-2c145332a6d... https://lore.kernel.org/linux-arm-kernel/20201119175556.18681-1-catalin.mari...
and yes, we met crash in lts5.10 when kexec boot due to "arm64: mm: Move reserve_crashkernel() into mem_init()" too, which could be fixed by commit 2687275a5843 "arm64: Force NO_BLOCK_MAPPINGS if crashkernel reservation is required", and the commit 791ab8b2e3db "arm64: Ignore any DMA offsets in the max_zone_phys() calculation" also about DMA set, So I only asked the two patches(both in v5.11) related ARM64 ZONE_DMA changes backported into lts5.10.
Thanks, all now queued up.
Thank you again ;)
greg k-h .
linux-stable-mirror@lists.linaro.org
-
Greg KH -
Jing Xiangfeng -
Kefeng Wang -
Nicolas Saenz Julienne