Backport of the two riscv mmap patches from master. In effect, these two patches removes arch_get_mmap_{base,end} for riscv.
Guo Ren: Please take a look. Patch 1 has a slightly non-trivial conflict with your commit 97b7ac69be2e ("riscv: mm: Fixup compat arch_get_mmap_end"), which changed STACK_TOP_MAX from TASK_SIZE_64 to TASK_SIZE when CONFIG_64BIT=y. This shouldn't be a problem, but, well, just to be safe.
--- Charlie Jenkins (2): riscv: mm: Use hint address in mmap if available riscv: mm: Do not restrict mmap address based on hint
arch/riscv/include/asm/processor.h | 33 +++++---------------------------- 1 file changed, 5 insertions(+), 28 deletions(-) --- base-commit: 60a9e718726fa7019ae00916e4b1c52498da5b60 change-id: 20250917-riscv-mmap-addr-space-6-6-15e7db6b5db6
Best regards,
From: Charlie Jenkins charlie@rivosinc.com
[ Upstream commit b5b4287accd702f562a49a60b10dbfaf7d40270f ]
On riscv it is guaranteed that the address returned by mmap is less than the hint address. Allow mmap to return an address all the way up to addr, if provided, rather than just up to the lower address space.
This provides a performance benefit as well, allowing mmap to exit after checking that the address is in range rather than searching for a valid address.
It is possible to provide an address that uses at most the same number of bits, however it is significantly more computationally expensive to provide that number rather than setting the max to be the hint address. There is the instruction clz/clzw in Zbb that returns the highest set bit which could be used to performantly implement this, but it would still be slower than the current implementation. At worst case, half of the address would not be able to be allocated when a hint address is provided.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com Link: https://lore.kernel.org/r/20240130-use_mmap_hint_address-v3-1-8a655cfa8bcb@r... Signed-off-by: Palmer Dabbelt palmer@rivosinc.com [ Adjust TASK_SIZE64 -> TASK_SIZE in moved lines ] Signed-off-by: Vivian Wang wangruikang@iscas.ac.cn Tested-by: Han Gao rabenda.cn@gmail.com --- arch/riscv/include/asm/processor.h | 27 +++++++++++---------------- 1 file changed, 11 insertions(+), 16 deletions(-)
diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h index 4f6af8c6cfa060380594c6d0e727af6b02d08d70..938aef30dfb42ee477b7c59b5d2afc3871d8004d 100644 --- a/arch/riscv/include/asm/processor.h +++ b/arch/riscv/include/asm/processor.h @@ -13,22 +13,16 @@
#include <asm/ptrace.h>
-#ifdef CONFIG_64BIT -#define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1)) -#define STACK_TOP_MAX TASK_SIZE - #define arch_get_mmap_end(addr, len, flags) \ ({ \ unsigned long mmap_end; \ typeof(addr) _addr = (addr); \ - if ((_addr) == 0 || (IS_ENABLED(CONFIG_COMPAT) && is_compat_task())) \ + if ((_addr) == 0 || \ + (IS_ENABLED(CONFIG_COMPAT) && is_compat_task()) || \ + ((_addr + len) > BIT(VA_BITS - 1))) \ mmap_end = STACK_TOP_MAX; \ - else if ((_addr) >= VA_USER_SV57) \ - mmap_end = STACK_TOP_MAX; \ - else if ((((_addr) >= VA_USER_SV48)) && (VA_BITS >= VA_BITS_SV48)) \ - mmap_end = VA_USER_SV48; \ else \ - mmap_end = VA_USER_SV39; \ + mmap_end = (_addr + len); \ mmap_end; \ })
@@ -38,17 +32,18 @@ typeof(addr) _addr = (addr); \ typeof(base) _base = (base); \ unsigned long rnd_gap = DEFAULT_MAP_WINDOW - (_base); \ - if ((_addr) == 0 || (IS_ENABLED(CONFIG_COMPAT) && is_compat_task())) \ + if ((_addr) == 0 || \ + (IS_ENABLED(CONFIG_COMPAT) && is_compat_task()) || \ + ((_addr + len) > BIT(VA_BITS - 1))) \ mmap_base = (_base); \ - else if (((_addr) >= VA_USER_SV57) && (VA_BITS >= VA_BITS_SV57)) \ - mmap_base = VA_USER_SV57 - rnd_gap; \ - else if ((((_addr) >= VA_USER_SV48)) && (VA_BITS >= VA_BITS_SV48)) \ - mmap_base = VA_USER_SV48 - rnd_gap; \ else \ - mmap_base = VA_USER_SV39 - rnd_gap; \ + mmap_base = (_addr + len) - rnd_gap; \ mmap_base; \ })
+#ifdef CONFIG_64BIT +#define DEFAULT_MAP_WINDOW (UL(1) << (MMAP_VA_BITS - 1)) +#define STACK_TOP_MAX TASK_SIZE #else #define DEFAULT_MAP_WINDOW TASK_SIZE #define STACK_TOP_MAX TASK_SIZE
From: Charlie Jenkins charlie@rivosinc.com
[ Upstream commit 2116988d5372aec51f8c4fb85bf8e305ecda47a0 ]
The hint address should not forcefully restrict the addresses returned by mmap as this causes mmap to report ENOMEM when there is memory still available.
Signed-off-by: Charlie Jenkins charlie@rivosinc.com Fixes: b5b4287accd7 ("riscv: mm: Use hint address in mmap if available") Fixes: add2cc6b6515 ("RISC-V: mm: Restrict address space for sv39,sv48,sv57") Closes: https://lore.kernel.org/linux-kernel/ZbxTNjQPFKBatMq+@ghost/T/#mccb1890466bf... Link: https://lore.kernel.org/r/20240826-riscv_mmap-v1-3-cd8962afe47f@rivosinc.com Signed-off-by: Palmer Dabbelt palmer@rivosinc.com [ Adjust removed lines ] Signed-off-by: Vivian Wang wangruikang@iscas.ac.cn Tested-by: Han Gao rabenda.cn@gmail.com --- arch/riscv/include/asm/processor.h | 22 ++-------------------- 1 file changed, 2 insertions(+), 20 deletions(-)
diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h index 938aef30dfb42ee477b7c59b5d2afc3871d8004d..4747277983ad1a2a9666c03a4e69758f56d22dbc 100644 --- a/arch/riscv/include/asm/processor.h +++ b/arch/riscv/include/asm/processor.h @@ -15,30 +15,12 @@
#define arch_get_mmap_end(addr, len, flags) \ ({ \ - unsigned long mmap_end; \ - typeof(addr) _addr = (addr); \ - if ((_addr) == 0 || \ - (IS_ENABLED(CONFIG_COMPAT) && is_compat_task()) || \ - ((_addr + len) > BIT(VA_BITS - 1))) \ - mmap_end = STACK_TOP_MAX; \ - else \ - mmap_end = (_addr + len); \ - mmap_end; \ + STACK_TOP_MAX; \ })
#define arch_get_mmap_base(addr, base) \ ({ \ - unsigned long mmap_base; \ - typeof(addr) _addr = (addr); \ - typeof(base) _base = (base); \ - unsigned long rnd_gap = DEFAULT_MAP_WINDOW - (_base); \ - if ((_addr) == 0 || \ - (IS_ENABLED(CONFIG_COMPAT) && is_compat_task()) || \ - ((_addr + len) > BIT(VA_BITS - 1))) \ - mmap_base = (_base); \ - else \ - mmap_base = (_addr + len) - rnd_gap; \ - mmap_base; \ + base; \ })
#ifdef CONFIG_64BIT
linux-stable-mirror@lists.linaro.org
-
Greg KH -
Vivian Wang