Commit a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservations") introduced early_reserve_memory() to do all needed initial memblock_reserve() calls in one function. Unfortunately the call of early_reserve_memory() is done too late for Xen dom0, as in some cases a Xen hook called by e820__memory_setup() will need those memory reservations to have happened already.
Move the call of early_reserve_memory() before the call of e820__memory_setup() in order to avoid such problems.
Cc: stable@vger.kernel.org Fixes: a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservations") Reported-by: Marek Marczykowski-Górecki marmarek@invisiblethingslab.com Signed-off-by: Juergen Gross jgross@suse.com --- V2: - update comment (Jan Beulich, Boris Petkov) - move call down in setup_arch() (Mike Galbraith) --- arch/x86/kernel/setup.c | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-)
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 79f164141116..40ed44ead063 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -830,6 +830,20 @@ void __init setup_arch(char **cmdline_p)
x86_init.oem.arch_setup();
+ /* + * Do some memory reservations *before* memory is added to memblock, so + * memblock allocations won't overwrite it. + * + * After this point, everything still needed from the boot loader or + * firmware or kernel text should be early reserved or marked not RAM in + * e820. All other memory is free game. + * + * This call needs to happen before e820__memory_setup() which calls the + * xen_memory_setup() on Xen dom0 which relies on the fact that those + * early reservations have happened already. + */ + early_reserve_memory(); + iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1; e820__memory_setup(); parse_setup_data(); @@ -876,18 +890,6 @@ void __init setup_arch(char **cmdline_p)
parse_early_param();
- /* - * Do some memory reservations *before* memory is added to - * memblock, so memblock allocations won't overwrite it. - * Do it after early param, so we could get (unlikely) panic from - * serial. - * - * After this point everything still needed from the boot loader or - * firmware or kernel text should be early reserved or marked not - * RAM in e820. All other memory is free game. - */ - early_reserve_memory(); - #ifdef CONFIG_MEMORY_HOTPLUG /* * Memory used by the kernel cannot be hot-removed because Linux
On Mon, Sep 20, 2021 at 02:04:21PM +0200, Juergen Gross wrote:
Commit a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservations") introduced early_reserve_memory() to do all needed initial memblock_reserve() calls in one function. Unfortunately the call of early_reserve_memory() is done too late for Xen dom0, as in some cases a Xen hook called by e820__memory_setup() will need those memory reservations to have happened already.
Move the call of early_reserve_memory() before the call of e820__memory_setup() in order to avoid such problems.
Cc: stable@vger.kernel.org Fixes: a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservations") Reported-by: Marek Marczykowski-Górecki marmarek@invisiblethingslab.com Signed-off-by: Juergen Gross jgross@suse.com
I confirm this fixes my boot issue too.
Tested-by: Marek Marczykowski-Górecki marmarek@invisiblethingslab.com
V2:
- update comment (Jan Beulich, Boris Petkov)
- move call down in setup_arch() (Mike Galbraith)
arch/x86/kernel/setup.c | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-)
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 79f164141116..40ed44ead063 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -830,6 +830,20 @@ void __init setup_arch(char **cmdline_p) x86_init.oem.arch_setup();
- /*
* Do some memory reservations *before* memory is added to memblock, so* memblock allocations won't overwrite it.** After this point, everything still needed from the boot loader or* firmware or kernel text should be early reserved or marked not RAM in* e820. All other memory is free game.** This call needs to happen before e820__memory_setup() which calls the* xen_memory_setup() on Xen dom0 which relies on the fact that those* early reservations have happened already.*/- early_reserve_memory();
- iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1; e820__memory_setup(); parse_setup_data();
@@ -876,18 +890,6 @@ void __init setup_arch(char **cmdline_p) parse_early_param();
- /*
* Do some memory reservations *before* memory is added to* memblock, so memblock allocations won't overwrite it.* Do it after early param, so we could get (unlikely) panic from* serial.** After this point everything still needed from the boot loader or* firmware or kernel text should be early reserved or marked not* RAM in e820. All other memory is free game.*/- early_reserve_memory();
#ifdef CONFIG_MEMORY_HOTPLUG /* * Memory used by the kernel cannot be hot-removed because Linux -- 2.26.2
On Mon, Sep 20, 2021 at 02:04:21PM +0200, Juergen Gross wrote:
Commit a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservations") introduced early_reserve_memory() to do all needed initial memblock_reserve() calls in one function. Unfortunately the call of early_reserve_memory() is done too late for Xen dom0, as in some cases a Xen hook called by e820__memory_setup() will need those memory reservations to have happened already.
Move the call of early_reserve_memory() before the call of e820__memory_setup() in order to avoid such problems.
Cc: stable@vger.kernel.org Fixes: a799c2bd29d19c565 ("x86/setup: Consolidate early memory reservations") Reported-by: Marek Marczykowski-Górecki marmarek@invisiblethingslab.com Signed-off-by: Juergen Gross jgross@suse.com
I had issues on an AMD Ryzen 3 4300G based system with v1. v2 does not trigger any boot issues on that same machine or an Intel i5-4210U based system that I also test with.
Tested-by: Nathan Chancellor nathan@kernel.org
V2:
- update comment (Jan Beulich, Boris Petkov)
- move call down in setup_arch() (Mike Galbraith)
arch/x86/kernel/setup.c | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-)
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 79f164141116..40ed44ead063 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -830,6 +830,20 @@ void __init setup_arch(char **cmdline_p) x86_init.oem.arch_setup();
- /*
* Do some memory reservations *before* memory is added to memblock, so* memblock allocations won't overwrite it.** After this point, everything still needed from the boot loader or* firmware or kernel text should be early reserved or marked not RAM in* e820. All other memory is free game.** This call needs to happen before e820__memory_setup() which calls the* xen_memory_setup() on Xen dom0 which relies on the fact that those* early reservations have happened already.*/- early_reserve_memory();
- iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1; e820__memory_setup(); parse_setup_data();
@@ -876,18 +890,6 @@ void __init setup_arch(char **cmdline_p) parse_early_param();
- /*
* Do some memory reservations *before* memory is added to* memblock, so memblock allocations won't overwrite it.* Do it after early param, so we could get (unlikely) panic from* serial.** After this point everything still needed from the boot loader or* firmware or kernel text should be early reserved or marked not* RAM in e820. All other memory is free game.*/- early_reserve_memory();
#ifdef CONFIG_MEMORY_HOTPLUG /* * Memory used by the kernel cannot be hot-removed because Linux -- 2.26.2
The following commit has been merged into the x86/urgent branch of tip:
Commit-ID: 8aa83e6395ce047a506f0b16edca45f36c1ae7f8 Gitweb: https://git.kernel.org/tip/8aa83e6395ce047a506f0b16edca45f36c1ae7f8 Author: Juergen Gross jgross@suse.com AuthorDate: Mon, 20 Sep 2021 14:04:21 +02:00 Committer: Borislav Petkov bp@suse.de CommitterDate: Tue, 21 Sep 2021 09:52:08 +02:00
x86/setup: Call early_reserve_memory() earlier
Commit in Fixes introduced early_reserve_memory() to do all needed initial memblock_reserve() calls in one function. Unfortunately, the call of early_reserve_memory() is done too late for Xen dom0, as in some cases a Xen hook called by e820__memory_setup() will need those memory reservations to have happened already.
Move the call of early_reserve_memory() before the call of e820__memory_setup() in order to avoid such problems.
Fixes: a799c2bd29d1 ("x86/setup: Consolidate early memory reservations") Reported-by: Marek Marczykowski-Górecki marmarek@invisiblethingslab.com Signed-off-by: Juergen Gross jgross@suse.com Signed-off-by: Borislav Petkov bp@suse.de Tested-by: Marek Marczykowski-Górecki marmarek@invisiblethingslab.com Tested-by: Nathan Chancellor nathan@kernel.org Cc: stable@vger.kernel.org Link: https://lkml.kernel.org/r/20210920120421.29276-1-jgross@suse.com --- arch/x86/kernel/setup.c | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-)
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 79f1641..40ed44e 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -830,6 +830,20 @@ void __init setup_arch(char **cmdline_p)
x86_init.oem.arch_setup();
+ /* + * Do some memory reservations *before* memory is added to memblock, so + * memblock allocations won't overwrite it. + * + * After this point, everything still needed from the boot loader or + * firmware or kernel text should be early reserved or marked not RAM in + * e820. All other memory is free game. + * + * This call needs to happen before e820__memory_setup() which calls the + * xen_memory_setup() on Xen dom0 which relies on the fact that those + * early reservations have happened already. + */ + early_reserve_memory(); + iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1; e820__memory_setup(); parse_setup_data(); @@ -876,18 +890,6 @@ void __init setup_arch(char **cmdline_p)
parse_early_param();
- /* - * Do some memory reservations *before* memory is added to - * memblock, so memblock allocations won't overwrite it. - * Do it after early param, so we could get (unlikely) panic from - * serial. - * - * After this point everything still needed from the boot loader or - * firmware or kernel text should be early reserved or marked not - * RAM in e820. All other memory is free game. - */ - early_reserve_memory(); - #ifdef CONFIG_MEMORY_HOTPLUG /* * Memory used by the kernel cannot be hot-removed because Linux
On Wed, 2021-09-22 at 17:18 +0000, tip-bot2 for Juergen Gross wrote:
The following commit has been merged into the x86/urgent branch of tip:
Commit-ID: 8aa83e6395ce047a506f0b16edca45f36c1ae7f8 Gitweb: https://git.kernel.org/tip/8aa83e6395ce047a506f0b16edca45f36c1ae7f8 Author: Juergen Gross jgross@suse.com AuthorDate: Mon, 20 Sep 2021 14:04:21 +02:00 Committer: Borislav Petkov bp@suse.de CommitterDate: Tue, 21 Sep 2021 09:52:08 +02:00
x86/setup: Call early_reserve_memory() earlier
Hi,
I got a report from Anjaneya that starting with the v5.15 kernel he can no longer use the "efi=nosoftreserve" kernel command line parameter to suppress "soft reservation" behavior. Recall that "soft reserved" is the Linux designation for memory that is marked with the EFI "Special Purpose" attribute.
By inspection, this commit looks like the source of the problem because early_reserve_memory() now runs before parse_early_param(). I suspect that this also affects memmap= since it is also an early_param(). I'll verify that tomorrow when I'm more awake, but wanted to give a heads up in the meantime.
Commit in Fixes introduced early_reserve_memory() to do all needed initial memblock_reserve() calls in one function. Unfortunately, the call of early_reserve_memory() is done too late for Xen dom0, as in some cases a Xen hook called by e820__memory_setup() will need those memory reservations to have happened already.
Move the call of early_reserve_memory() before the call of e820__memory_setup() in order to avoid such problems.
Fixes: a799c2bd29d1 ("x86/setup: Consolidate early memory reservations") Reported-by: Marek Marczykowski-Górecki marmarek@invisiblethingslab.com Signed-off-by: Juergen Gross jgross@suse.com Signed-off-by: Borislav Petkov bp@suse.de Tested-by: Marek Marczykowski-Górecki marmarek@invisiblethingslab.com Tested-by: Nathan Chancellor nathan@kernel.org Cc: stable@vger.kernel.org Link: https://lkml.kernel.org/r/20210920120421.29276-1-jgross@suse.com
arch/x86/kernel/setup.c | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-)
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 79f1641..40ed44e 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -830,6 +830,20 @@ void __init setup_arch(char **cmdline_p) x86_init.oem.arch_setup(); + /* + * Do some memory reservations *before* memory is added to memblock, so + * memblock allocations won't overwrite it. + * + * After this point, everything still needed from the boot loader or + * firmware or kernel text should be early reserved or marked not RAM in + * e820. All other memory is free game. + * + * This call needs to happen before e820__memory_setup() which calls the + * xen_memory_setup() on Xen dom0 which relies on the fact that those + * early reservations have happened already. + */ + early_reserve_memory();
iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1; e820__memory_setup(); parse_setup_data(); @@ -876,18 +890,6 @@ void __init setup_arch(char **cmdline_p) parse_early_param(); - /* - * Do some memory reservations *before* memory is added to - * memblock, so memblock allocations won't overwrite it. - * Do it after early param, so we could get (unlikely) panic from - * serial. - * - * After this point everything still needed from the boot loader or - * firmware or kernel text should be early reserved or marked not - * RAM in e820. All other memory is free game. - */ - early_reserve_memory();
#ifdef CONFIG_MEMORY_HOTPLUG /* * Memory used by the kernel cannot be hot-removed because Linux
+ Mike.
On Thu, Nov 04, 2021 at 05:38:54AM +0000, Williams, Dan J wrote:
By inspection, this commit looks like the source of the problem because early_reserve_memory() now runs before parse_early_param().
Yah, I see it too:
early_reserve_memory |-> efi_memblock_x86_reserve_range |-> efi_fake_memmap_early
which does
if (!efi_soft_reserve_enabled()) return;
and that would have set EFI_MEM_NO_SOFT_RESERVE after having parsed "nosoftreserve".
And parse_early_param() happens later.
Uuuf, that early memory reservation dance is not going to be over, ever...
Well, I guess we can do something like this below. The intent being to carve out the command line preparation into a separate function which does the early param parsing too. So that it all goes together.
And then call that function before early_reserve_memory() so that the params have been parsed by then.
Looking at the changed flow, I think we should be ok but I've said that a bunch of times already regarding this memory reservation early and something in our house of cards called early boot order always broke.
Damn.
--- diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 40ed44ead063..05f69e7d84dc 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -742,6 +742,28 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p) return 0; }
+static char *prepare_command_line(void) +{ +#ifdef CONFIG_CMDLINE_BOOL +#ifdef CONFIG_CMDLINE_OVERRIDE + strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE); +#else + if (builtin_cmdline[0]) { + /* append boot loader cmdline to builtin */ + strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE); + strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE); + strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE); + } +#endif +#endif + + strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE); + + parse_early_param(); + + return command_line; +} + /* * Determine if we were loaded by an EFI loader. If so, then we have also been * passed the efi memmap, systab, etc., so we should use these data structures @@ -830,6 +852,23 @@ void __init setup_arch(char **cmdline_p)
x86_init.oem.arch_setup();
+ /* + * x86_configure_nx() is called before parse_early_param() (called by + * prepare_command_line()) to detect whether hardware doesn't support + * NX (so that the early EHCI debug console setup can safely call + * set_fixmap()). It may then be called again from within noexec_setup() + * during parsing early parameters to honor the respective command line + * option. + */ + x86_configure_nx(); + + /* + * This parses early params and it needs to run before + * early_reserve_memory() because latter relies on such settings + * supplies as early params. + */ + *cmdline_p = prepare_command_line(); + /* * Do some memory reservations *before* memory is added to memblock, so * memblock allocations won't overwrite it. @@ -863,33 +902,6 @@ void __init setup_arch(char **cmdline_p) bss_resource.start = __pa_symbol(__bss_start); bss_resource.end = __pa_symbol(__bss_stop)-1;
-#ifdef CONFIG_CMDLINE_BOOL -#ifdef CONFIG_CMDLINE_OVERRIDE - strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE); -#else - if (builtin_cmdline[0]) { - /* append boot loader cmdline to builtin */ - strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE); - strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE); - strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE); - } -#endif -#endif - - strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE); - *cmdline_p = command_line; - - /* - * x86_configure_nx() is called before parse_early_param() to detect - * whether hardware doesn't support NX (so that the early EHCI debug - * console setup can safely call set_fixmap()). It may then be called - * again from within noexec_setup() during parsing early parameters - * to honor the respective command line option. - */ - x86_configure_nx(); - - parse_early_param(); - #ifdef CONFIG_MEMORY_HOTPLUG /* * Memory used by the kernel cannot be hot-removed because Linux
On Thu, Nov 4, 2021 at 4:40 AM Borislav Petkov bp@alien8.de wrote:
- Mike.
On Thu, Nov 04, 2021 at 05:38:54AM +0000, Williams, Dan J wrote:
By inspection, this commit looks like the source of the problem because early_reserve_memory() now runs before parse_early_param().
Yah, I see it too:
early_reserve_memory |-> efi_memblock_x86_reserve_range |-> efi_fake_memmap_early
which does
if (!efi_soft_reserve_enabled()) return;and that would have set EFI_MEM_NO_SOFT_RESERVE after having parsed "nosoftreserve".
And parse_early_param() happens later.
Uuuf, that early memory reservation dance is not going to be over, ever...
Well, I guess we can do something like this below. The intent being to carve out the command line preparation into a separate function which does the early param parsing too. So that it all goes together.
And then call that function before early_reserve_memory() so that the params have been parsed by then.
Looking at the changed flow, I think we should be ok but I've said that a bunch of times already regarding this memory reservation early and something in our house of cards called early boot order always broke.
Damn.
Thanks Boris!
You can add:
Tested-by: Anjaneya Chagam anjaneya.chagam@intel.com Reviewed-by: Dan Williams dan.j.williams@intel.com
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 40ed44ead063..05f69e7d84dc 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -742,6 +742,28 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p) return 0; }
+static char *prepare_command_line(void) +{ +#ifdef CONFIG_CMDLINE_BOOL +#ifdef CONFIG_CMDLINE_OVERRIDE
strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);+#else
if (builtin_cmdline[0]) {/* append boot loader cmdline to builtin */strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);}+#endif +#endif
strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);parse_early_param();return command_line;+}
/*
- Determine if we were loaded by an EFI loader. If so, then we have also been
- passed the efi memmap, systab, etc., so we should use these data structures
@@ -830,6 +852,23 @@ void __init setup_arch(char **cmdline_p)
x86_init.oem.arch_setup();
/** x86_configure_nx() is called before parse_early_param() (called by* prepare_command_line()) to detect whether hardware doesn't support* NX (so that the early EHCI debug console setup can safely call* set_fixmap()). It may then be called again from within noexec_setup()* during parsing early parameters to honor the respective command line* option.*/x86_configure_nx();/** This parses early params and it needs to run before* early_reserve_memory() because latter relies on such settings* supplies as early params.*/*cmdline_p = prepare_command_line();/* * Do some memory reservations *before* memory is added to memblock, so * memblock allocations won't overwrite it.@@ -863,33 +902,6 @@ void __init setup_arch(char **cmdline_p) bss_resource.start = __pa_symbol(__bss_start); bss_resource.end = __pa_symbol(__bss_stop)-1;
-#ifdef CONFIG_CMDLINE_BOOL -#ifdef CONFIG_CMDLINE_OVERRIDE
strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);-#else
if (builtin_cmdline[0]) {/* append boot loader cmdline to builtin */strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE);strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE);strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE);}-#endif -#endif
strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE);*cmdline_p = command_line;/** x86_configure_nx() is called before parse_early_param() to detect* whether hardware doesn't support NX (so that the early EHCI debug* console setup can safely call set_fixmap()). It may then be called* again from within noexec_setup() during parsing early parameters* to honor the respective command line option.*/x86_configure_nx();parse_early_param();#ifdef CONFIG_MEMORY_HOTPLUG /* * Memory used by the kernel cannot be hot-removed because Linux
-- Regards/Gruss, Boris.
The following commit has been merged into the x86/urgent branch of tip:
Commit-ID: 8d48bf8206f77aa8687f0e241e901e5197e52423 Gitweb: https://git.kernel.org/tip/8d48bf8206f77aa8687f0e241e901e5197e52423 Author: Borislav Petkov bp@suse.de AuthorDate: Fri, 05 Nov 2021 10:41:51 +01:00 Committer: Borislav Petkov bp@suse.de CommitterDate: Mon, 15 Nov 2021 12:27:40 +01:00
x86/boot: Pull up cmdline preparation and early param parsing
Dan reports that Anjaneya Chagam can no longer use the efi=nosoftreserve kernel command line parameter to suppress "soft reservation" behavior.
This is due to the fact that the following call-chain happens at boot:
early_reserve_memory |-> efi_memblock_x86_reserve_range |-> efi_fake_memmap_early
which does
if (!efi_soft_reserve_enabled()) return;
and that would have set EFI_MEM_NO_SOFT_RESERVE after having parsed "nosoftreserve".
However, parse_early_param() gets called *after* it, leading to the boot cmdline not being taken into account.
Therefore, carve out the command line preparation into a separate function which does the early param parsing too. So that it all goes together.
And then call that function before early_reserve_memory() so that the params would have been parsed by then.
Fixes: 8aa83e6395ce ("x86/setup: Call early_reserve_memory() earlier") Reported-by: Dan Williams dan.j.williams@intel.com Reviewed-by: Dan Williams dan.j.williams@intel.com Signed-off-by: Borislav Petkov bp@suse.de Tested-by: Anjaneya Chagam anjaneya.chagam@intel.com Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/e8dd8993c38702ee6dd73b3c11f158617e665607.camel@int... --- arch/x86/kernel/setup.c | 66 +++++++++++++++++++++++----------------- 1 file changed, 39 insertions(+), 27 deletions(-)
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 49b596d..c410be7 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -742,6 +742,28 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p) return 0; }
+static char *prepare_command_line(void) +{ +#ifdef CONFIG_CMDLINE_BOOL +#ifdef CONFIG_CMDLINE_OVERRIDE + strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE); +#else + if (builtin_cmdline[0]) { + /* append boot loader cmdline to builtin */ + strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE); + strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE); + strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE); + } +#endif +#endif + + strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE); + + parse_early_param(); + + return command_line; +} + /* * Determine if we were loaded by an EFI loader. If so, then we have also been * passed the efi memmap, systab, etc., so we should use these data structures @@ -831,6 +853,23 @@ void __init setup_arch(char **cmdline_p) x86_init.oem.arch_setup();
/* + * x86_configure_nx() is called before parse_early_param() (called by + * prepare_command_line()) to detect whether hardware doesn't support + * NX (so that the early EHCI debug console setup can safely call + * set_fixmap()). It may then be called again from within noexec_setup() + * during parsing early parameters to honor the respective command line + * option. + */ + x86_configure_nx(); + + /* + * This parses early params and it needs to run before + * early_reserve_memory() because latter relies on such settings + * supplied as early params. + */ + *cmdline_p = prepare_command_line(); + + /* * Do some memory reservations *before* memory is added to memblock, so * memblock allocations won't overwrite it. * @@ -863,33 +902,6 @@ void __init setup_arch(char **cmdline_p) bss_resource.start = __pa_symbol(__bss_start); bss_resource.end = __pa_symbol(__bss_stop)-1;
-#ifdef CONFIG_CMDLINE_BOOL -#ifdef CONFIG_CMDLINE_OVERRIDE - strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE); -#else - if (builtin_cmdline[0]) { - /* append boot loader cmdline to builtin */ - strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE); - strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE); - strlcpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE); - } -#endif -#endif - - strlcpy(command_line, boot_command_line, COMMAND_LINE_SIZE); - *cmdline_p = command_line; - - /* - * x86_configure_nx() is called before parse_early_param() to detect - * whether hardware doesn't support NX (so that the early EHCI debug - * console setup can safely call set_fixmap()). It may then be called - * again from within noexec_setup() during parsing early parameters - * to honor the respective command line option. - */ - x86_configure_nx(); - - parse_early_param(); - #ifdef CONFIG_MEMORY_HOTPLUG /* * Memory used by the kernel cannot be hot-removed because Linux
Greetings;
It seems that this patch causes a mem= parameter to the kernel to have no effect, unfortunately...
As far as I understand, the x86 mem parameter handler parse_memopt() (called by parse_early_param()) relies on being called after e820__memory_setup(): it simply removes any memory above the specified limit at that moment, allowing memory to later be hotplugged without regard for the initial limit. However, the initial non-hotplugged memory must already have been set up, in e820__memory_setup(), so that it can be removed in parse_memopt(); if parse_early_param() is called before e820__memory_setup(), as this change does, the parameter ends up having no effect.
I apologize that I don't know how to fix this, but I'm happy to test patches.
Typical dmesg output showing the lack of effect, built from the prior change and this change:
With a git tree synced to 8d48bf8206f77aa8687f0e241e901e5197e52423^ (working): [ 0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.16.0-rc1 root=UUID=a4f7bd84-4f29-40bc-8c98-f4a72d0856c4 ro net.ifnames=0 crashkernel=128M mem=4G ... [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009abff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009ac00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007dd3afff] usable [ 0.000000] BIOS-e820: [mem 0x000000007dd3b000-0x000000007deeffff] reserved [ 0.000000] BIOS-e820: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS [ 0.000000] BIOS-e820: [mem 0x000000007e0d4000-0x000000007f367fff] reserved [ 0.000000] BIOS-e820: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS [ 0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000207fffffff] usable [ 0.000000] e820: remove [mem 0x100000000-0xfffffffffffffffe] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] user-defined physical RAM map: [ 0.000000] user: [mem 0x0000000000000000-0x000000000009abff] usable [ 0.000000] user: [mem 0x000000000009ac00-0x000000000009ffff] reserved [ 0.000000] user: [mem 0x00000000000e0000-0x00000000000fffff] reserved [ 0.000000] user: [mem 0x0000000000100000-0x000000007dd3afff] usable [ 0.000000] user: [mem 0x000000007dd3b000-0x000000007deeffff] reserved [ 0.000000] user: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS [ 0.000000] user: [mem 0x000000007e0d4000-0x000000007f367fff] reserved [ 0.000000] user: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS [ 0.000000] user: [mem 0x0000000080000000-0x000000008fffffff] reserved [ 0.000000] user: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved [ 0.000000] user: [mem 0x00000000ff000000-0x00000000ffffffff] reserved ... [ 0.025617] Memory: 1762876K/2061136K available (16394K kernel code, 3568K rwdata, 10324K rodata, 2676K init, 4924K bss, 298000K reserved, 0K cma-reserved)
Synced 8d48bf8206f77aa8687f0e241e901e5197e52423 (not working):
[ 0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.16.0-rc4+ root=UUID=0e750e61-b92e-4708-a974-c50a3fb7e969 ro net.ifnames=0 crashkernel=128M mem=4G [ 0.000000] e820: remove [mem 0x100000000-0xfffffffffffffffe] usable [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009abff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009ac00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007dd3afff] usable [ 0.000000] BIOS-e820: [mem 0x000000007dd3b000-0x000000007deeffff] reserved [ 0.000000] BIOS-e820: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS [ 0.000000] BIOS-e820: [mem 0x000000007e0d4000-0x000000007f367fff] reserved [ 0.000000] BIOS-e820: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS [ 0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000207fffffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] user-defined physical RAM map: [ 0.000000] user: [mem 0x0000000000000000-0x000000000009abff] usable [ 0.000000] user: [mem 0x000000000009ac00-0x000000000009ffff] reserved [ 0.000000] user: [mem 0x00000000000e0000-0x00000000000fffff] reserved [ 0.000000] user: [mem 0x0000000000100000-0x000000007dd3afff] usable [ 0.000000] user: [mem 0x000000007dd3b000-0x000000007deeffff] reserved [ 0.000000] user: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS [ 0.000000] user: [mem 0x000000007e0d4000-0x000000007f367fff] reserved [ 0.000000] user: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS [ 0.000000] user: [mem 0x0000000080000000-0x000000008fffffff] reserved [ 0.000000] user: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved [ 0.000000] user: [mem 0x00000000ff000000-0x00000000ffffffff] reserved [ 0.000000] user: [mem 0x0000000100000000-0x000000207fffffff] usable ... [ 0.695267] Memory: 131657608K/134181712K available (16394K kernel code, 3568K rwdata, 10328K rodata, 2676K init, 4924K bss, 2523844K reserved, 0K cma-reserved)
+ Hugh and Patrick.
On Thu, Dec 09, 2021 at 09:38:10AM -0500, John Dorminy wrote:
Greetings;
It seems that this patch causes a mem= parameter to the kernel to have no effect, unfortunately...
As far as I understand, the x86 mem parameter handler parse_memopt() (called by parse_early_param()) relies on being called after e820__memory_setup(): it simply removes any memory above the specified limit at that moment, allowing memory to later be hotplugged without regard for the initial limit. However, the initial non-hotplugged memory must already have been set up, in e820__memory_setup(), so that it can be removed in parse_memopt(); if parse_early_param() is called before e820__memory_setup(), as this change does, the parameter ends up having no effect.
I apologize that I don't know how to fix this, but I'm happy to test patches.
Yeah, people have been reporting boot failures with mem= on the cmdline.
I think I see why, can you try this one:
--- diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 6a190c7f4d71..6db971e61e4b 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -862,6 +862,8 @@ void __init setup_arch(char **cmdline_p) */ x86_configure_nx();
+ e820__memory_setup(); + /* * This parses early params and it needs to run before * early_reserve_memory() because latter relies on such settings @@ -884,7 +886,6 @@ void __init setup_arch(char **cmdline_p) early_reserve_memory();
iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1; - e820__memory_setup(); parse_setup_data();
copy_edd(); ---
Leaving in the rest for the newly added folks.
Typical dmesg output showing the lack of effect, built from the prior change and this change:
With a git tree synced to 8d48bf8206f77aa8687f0e241e901e5197e52423^ (working): [ 0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.16.0-rc1 root=UUID=a4f7bd84-4f29-40bc-8c98-f4a72d0856c4 ro net.ifnames=0 crashkernel=128M mem=4G ... [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009abff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009ac00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007dd3afff] usable [ 0.000000] BIOS-e820: [mem 0x000000007dd3b000-0x000000007deeffff] reserved [ 0.000000] BIOS-e820: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS [ 0.000000] BIOS-e820: [mem 0x000000007e0d4000-0x000000007f367fff] reserved [ 0.000000] BIOS-e820: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS [ 0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000207fffffff] usable [ 0.000000] e820: remove [mem 0x100000000-0xfffffffffffffffe] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] user-defined physical RAM map: [ 0.000000] user: [mem 0x0000000000000000-0x000000000009abff] usable [ 0.000000] user: [mem 0x000000000009ac00-0x000000000009ffff] reserved [ 0.000000] user: [mem 0x00000000000e0000-0x00000000000fffff] reserved [ 0.000000] user: [mem 0x0000000000100000-0x000000007dd3afff] usable [ 0.000000] user: [mem 0x000000007dd3b000-0x000000007deeffff] reserved [ 0.000000] user: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS [ 0.000000] user: [mem 0x000000007e0d4000-0x000000007f367fff] reserved [ 0.000000] user: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS [ 0.000000] user: [mem 0x0000000080000000-0x000000008fffffff] reserved [ 0.000000] user: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved [ 0.000000] user: [mem 0x00000000ff000000-0x00000000ffffffff] reserved ... [ 0.025617] Memory: 1762876K/2061136K available (16394K kernel code, 3568K rwdata, 10324K rodata, 2676K init, 4924K bss, 298000K reserved, 0K cma-reserved)
Synced 8d48bf8206f77aa8687f0e241e901e5197e52423 (not working):
[ 0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.16.0-rc4+ root=UUID=0e750e61-b92e-4708-a974-c50a3fb7e969 ro net.ifnames=0 crashkernel=128M mem=4G [ 0.000000] e820: remove [mem 0x100000000-0xfffffffffffffffe] usable [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009abff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009ac00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007dd3afff] usable [ 0.000000] BIOS-e820: [mem 0x000000007dd3b000-0x000000007deeffff] reserved [ 0.000000] BIOS-e820: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS [ 0.000000] BIOS-e820: [mem 0x000000007e0d4000-0x000000007f367fff] reserved [ 0.000000] BIOS-e820: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS [ 0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000207fffffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] user-defined physical RAM map: [ 0.000000] user: [mem 0x0000000000000000-0x000000000009abff] usable [ 0.000000] user: [mem 0x000000000009ac00-0x000000000009ffff] reserved [ 0.000000] user: [mem 0x00000000000e0000-0x00000000000fffff] reserved [ 0.000000] user: [mem 0x0000000000100000-0x000000007dd3afff] usable [ 0.000000] user: [mem 0x000000007dd3b000-0x000000007deeffff] reserved [ 0.000000] user: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS [ 0.000000] user: [mem 0x000000007e0d4000-0x000000007f367fff] reserved [ 0.000000] user: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS [ 0.000000] user: [mem 0x0000000080000000-0x000000008fffffff] reserved [ 0.000000] user: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved [ 0.000000] user: [mem 0x00000000ff000000-0x00000000ffffffff] reserved [ 0.000000] user: [mem 0x0000000100000000-0x000000207fffffff] usable ... [ 0.695267] Memory: 131657608K/134181712K available (16394K kernel code, 3568K rwdata, 10328K rodata, 2676K init, 4924K bss, 2523844K reserved, 0K cma-reserved)
On 09.12.21 16:19, Borislav Petkov wrote:
- Hugh and Patrick.
On Thu, Dec 09, 2021 at 09:38:10AM -0500, John Dorminy wrote:
Greetings;
It seems that this patch causes a mem= parameter to the kernel to have no effect, unfortunately...
As far as I understand, the x86 mem parameter handler parse_memopt() (called by parse_early_param()) relies on being called after e820__memory_setup(): it simply removes any memory above the specified limit at that moment, allowing memory to later be hotplugged without regard for the initial limit. However, the initial non-hotplugged memory must already have been set up, in e820__memory_setup(), so that it can be removed in parse_memopt(); if parse_early_param() is called before e820__memory_setup(), as this change does, the parameter ends up having no effect.
I apologize that I don't know how to fix this, but I'm happy to test patches.
Yeah, people have been reporting boot failures with mem= on the cmdline.
I think I see why, can you try this one:
Sigh. This will break Xen PV. Again. The comment above the call of early_reserve_memory() tells you why.
Juergen
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 6a190c7f4d71..6db971e61e4b 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -862,6 +862,8 @@ void __init setup_arch(char **cmdline_p) */ x86_configure_nx();
- e820__memory_setup();
- /*
- This parses early params and it needs to run before
- early_reserve_memory() because latter relies on such settings
@@ -884,7 +886,6 @@ void __init setup_arch(char **cmdline_p) early_reserve_memory(); iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1;
- e820__memory_setup(); parse_setup_data();
copy_edd();
Leaving in the rest for the newly added folks.
Typical dmesg output showing the lack of effect, built from the prior change and this change:
With a git tree synced to 8d48bf8206f77aa8687f0e241e901e5197e52423^ (working): [ 0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.16.0-rc1 root=UUID=a4f7bd84-4f29-40bc-8c98-f4a72d0856c4 ro net.ifnames=0 crashkernel=128M mem=4G ... [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009abff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009ac00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007dd3afff] usable [ 0.000000] BIOS-e820: [mem 0x000000007dd3b000-0x000000007deeffff] reserved [ 0.000000] BIOS-e820: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS [ 0.000000] BIOS-e820: [mem 0x000000007e0d4000-0x000000007f367fff] reserved [ 0.000000] BIOS-e820: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS [ 0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000207fffffff] usable [ 0.000000] e820: remove [mem 0x100000000-0xfffffffffffffffe] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] user-defined physical RAM map: [ 0.000000] user: [mem 0x0000000000000000-0x000000000009abff] usable [ 0.000000] user: [mem 0x000000000009ac00-0x000000000009ffff] reserved [ 0.000000] user: [mem 0x00000000000e0000-0x00000000000fffff] reserved [ 0.000000] user: [mem 0x0000000000100000-0x000000007dd3afff] usable [ 0.000000] user: [mem 0x000000007dd3b000-0x000000007deeffff] reserved [ 0.000000] user: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS [ 0.000000] user: [mem 0x000000007e0d4000-0x000000007f367fff] reserved [ 0.000000] user: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS [ 0.000000] user: [mem 0x0000000080000000-0x000000008fffffff] reserved [ 0.000000] user: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved [ 0.000000] user: [mem 0x00000000ff000000-0x00000000ffffffff] reserved ... [ 0.025617] Memory: 1762876K/2061136K available (16394K kernel code, 3568K rwdata, 10324K rodata, 2676K init, 4924K bss, 298000K reserved, 0K cma-reserved)
Synced 8d48bf8206f77aa8687f0e241e901e5197e52423 (not working):
[ 0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.16.0-rc4+ root=UUID=0e750e61-b92e-4708-a974-c50a3fb7e969 ro net.ifnames=0 crashkernel=128M mem=4G [ 0.000000] e820: remove [mem 0x100000000-0xfffffffffffffffe] usable [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009abff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009ac00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007dd3afff] usable [ 0.000000] BIOS-e820: [mem 0x000000007dd3b000-0x000000007deeffff] reserved [ 0.000000] BIOS-e820: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS [ 0.000000] BIOS-e820: [mem 0x000000007e0d4000-0x000000007f367fff] reserved [ 0.000000] BIOS-e820: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS [ 0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000207fffffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] user-defined physical RAM map: [ 0.000000] user: [mem 0x0000000000000000-0x000000000009abff] usable [ 0.000000] user: [mem 0x000000000009ac00-0x000000000009ffff] reserved [ 0.000000] user: [mem 0x00000000000e0000-0x00000000000fffff] reserved [ 0.000000] user: [mem 0x0000000000100000-0x000000007dd3afff] usable [ 0.000000] user: [mem 0x000000007dd3b000-0x000000007deeffff] reserved [ 0.000000] user: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS [ 0.000000] user: [mem 0x000000007e0d4000-0x000000007f367fff] reserved [ 0.000000] user: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS [ 0.000000] user: [mem 0x0000000080000000-0x000000008fffffff] reserved [ 0.000000] user: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved [ 0.000000] user: [mem 0x00000000ff000000-0x00000000ffffffff] reserved [ 0.000000] user: [mem 0x0000000100000000-0x000000207fffffff] usable ... [ 0.695267] Memory: 131657608K/134181712K available (16394K kernel code, 3568K rwdata, 10328K rodata, 2676K init, 4924K bss, 2523844K reserved, 0K cma-reserved)
On Thu, Dec 09, 2021 at 04:26:55PM +0100, Juergen Gross wrote:
Sigh. This will break Xen PV. Again. The comment above the call of early_reserve_memory() tells you why.
I know. I was just looking at how to fix that particular thing and was going to find you on IRC to talk to you about it...
On Thu, Dec 09, 2021 at 04:28:48PM +0100, Borislav Petkov wrote:
On Thu, Dec 09, 2021 at 04:26:55PM +0100, Juergen Gross wrote:
Sigh. This will break Xen PV. Again. The comment above the call of early_reserve_memory() tells you why.
I know. I was just looking at how to fix that particular thing and was going to find you on IRC to talk to you about it...
The memory reservation in arch/x86/platform/efi/efi.c depends on at least two command line parameters, I think it's better put it back later in the boot process and move efi_memblock_x86_reserve_range() out of early_memory_reserve().
I.e. revert c0f2077baa41 ("x86/boot: Mark prepare_command_line() __init") and 8d48bf8206f7 ("x86/boot: Pull up cmdline preparation and early param parsing") and add the patch below on top.
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 49b596db5631..da36b8f8430b 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -713,9 +713,6 @@ static void __init early_reserve_memory(void)
early_reserve_initrd();
- if (efi_enabled(EFI_BOOT)) - efi_memblock_x86_reserve_range(); - memblock_x86_reserve_range_setup_data();
reserve_ibft_region(); @@ -890,6 +887,9 @@ void __init setup_arch(char **cmdline_p)
parse_early_param();
+ if (efi_enabled(EFI_BOOT)) { + efi_memblock_x86_reserve_range(); + #ifdef CONFIG_MEMORY_HOTPLUG /* * Memory used by the kernel cannot be hot-removed because Linux
On Thu, Dec 09, 2021 at 06:29:27PM +0200, Mike Rapoport wrote:
On Thu, Dec 09, 2021 at 04:28:48PM +0100, Borislav Petkov wrote:
On Thu, Dec 09, 2021 at 04:26:55PM +0100, Juergen Gross wrote:
Sigh. This will break Xen PV. Again. The comment above the call of early_reserve_memory() tells you why.
I know. I was just looking at how to fix that particular thing and was going to find you on IRC to talk to you about it...
The memory reservation in arch/x86/platform/efi/efi.c depends on at least two command line parameters, I think it's better put it back later in the boot process and move efi_memblock_x86_reserve_range() out of early_memory_reserve().
I.e. revert c0f2077baa41 ("x86/boot: Mark prepare_command_line() __init") and 8d48bf8206f7 ("x86/boot: Pull up cmdline preparation and early param parsing") and add the patch below on top.
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 49b596db5631..da36b8f8430b 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -713,9 +713,6 @@ static void __init early_reserve_memory(void) early_reserve_initrd();
- if (efi_enabled(EFI_BOOT))
efi_memblock_x86_reserve_range();- memblock_x86_reserve_range_setup_data();
reserve_ibft_region(); @@ -890,6 +887,9 @@ void __init setup_arch(char **cmdline_p) parse_early_param();
- if (efi_enabled(EFI_BOOT)) {
efi_memblock_x86_reserve_range();#ifdef CONFIG_MEMORY_HOTPLUG /* * Memory used by the kernel cannot be hot-removed because Linux
--
Jürgen and I were thinking about a different fix but that's probably ok too. But I've said that already about this mess and there's always something we haven't thought about.
Whatever we do, it needs to be tested by all folks on Cc who already reported regressions, i.e., Anjaneya, Hugh, John and Patrick.
Thx.
On Thu, Dec 09, 2021 at 05:37:42PM +0100, Borislav Petkov wrote:
Whatever we do, it needs to be tested by all folks on Cc who already reported regressions, i.e., Anjaneya, Hugh, John and Patrick.
Ok, Mike is busy so here are some patches for testing:
https://git.kernel.org/pub/scm/linux/kernel/git/bp/bp.git/log/?h=rc4-boot
I'd appreciate it if folks who reported an issue, verify those.
The first two are reverts which should address the issues with mem= folks have reported. And the last one should address Anjaneya's issue.
I guess doing it the way as Mike suggested is cleaner/better.
Thx!
On Fri, 10 Dec 2021, Borislav Petkov wrote:
On Thu, Dec 09, 2021 at 05:37:42PM +0100, Borislav Petkov wrote:
Whatever we do, it needs to be tested by all folks on Cc who already reported regressions, i.e., Anjaneya, Hugh, John and Patrick.
Ok, Mike is busy so here are some patches for testing:
https://git.kernel.org/pub/scm/linux/kernel/git/bp/bp.git/log/?h=rc4-boot
I'd appreciate it if folks who reported an issue, verify those.
The first two are reverts which should address the issues with mem= folks have reported. And the last one should address Anjaneya's issue.
I guess doing it the way as Mike suggested is cleaner/better.
Yes, mem= works fine for me, on both machines, 64-bit and 32-bit, thanks; but I'm not exercising the troublesome EFI case at all.
Hugh
On Fri, Dec 10, 2021 at 12:11:02PM -0800, Hugh Dickins wrote:
Yes, mem= works fine for me, on both machines, 64-bit and 32-bit, thanks;
Thanks!
but I'm not exercising the troublesome EFI case at all.
Yeah, I added some debug printks in a VM yesterday to confirm the ordering. But will give Anjaneya some more time to verify, before I queue them next week.
On Fri, Dec 10, 2021 at 5:28 AM Borislav Petkov bp@alien8.de wrote:
On Thu, Dec 09, 2021 at 05:37:42PM +0100, Borislav Petkov wrote:
Whatever we do, it needs to be tested by all folks on Cc who already reported regressions, i.e., Anjaneya, Hugh, John and Patrick.
Ok, Mike is busy so here are some patches for testing:
https://git.kernel.org/pub/scm/linux/kernel/git/bp/bp.git/log/?h=rc4-boot
I'd appreciate it if folks who reported an issue, verify those.
The first two are reverts which should address the issues with mem= folks have reported. And the last one should address Anjaneya's issue.
I applied the two revert patches to 5.15.7 (the last one won't apply so I skipped it) and the resulting x86 32-bit kernel boots fine here on the Thinkpad X1E that was having issues previously.
Then I tested an unpatched 5.16-rc4, which (as expected) got the boot hang on the affected machine. Applied the three patches, and the resulting kernel boots fine.
Take care,
Pat
On Fri, Dec 10, 2021 at 02:32:38PM -0600, Patrick J. Volkerding wrote:
I applied the two revert patches to 5.15.7 (the last one won't apply so I skipped it)
Looks like it is only a contextual conflict.
and the resulting x86 32-bit kernel boots fine here on the Thinkpad X1E that was having issues previously.
Good.
Then I tested an unpatched 5.16-rc4, which (as expected) got the boot hang on the affected machine. Applied the three patches, and the resulting kernel boots fine.
Thanks for testing!
Apologies for delay; my dev machine was broken much of today. But I have tested this patch under the same conditions as previously, and agree with Hugh that the patches make mem= work correctly.
Thanks!
John Dorminy
On Sat, Dec 11, 2021 at 12:24:25AM -0500, John Dorminy wrote:
Apologies for delay; my dev machine was broken much of today. But I have tested this patch under the same conditions as previously, and agree with Hugh that the patches make mem= work correctly.
Thanks!
Thanks for testing!
On Fri, Dec 10, 2021 at 12:28:09PM +0100, Borislav Petkov wrote:
On Thu, Dec 09, 2021 at 05:37:42PM +0100, Borislav Petkov wrote:
Whatever we do, it needs to be tested by all folks on Cc who already reported regressions, i.e., Anjaneya, Hugh, John and Patrick.
Ok, Mike is busy so here are some patches for testing:
https://git.kernel.org/pub/scm/linux/kernel/git/bp/bp.git/log/?h=rc4-boot
Thanks for taking care of this!
On Mon, Dec 13, 2021 at 10:20:03AM +0200, Mike Rapoport wrote:
Thanks for taking care of this!
Sure, no probs.
Lemme send them out officially so they're on the list. Will queue them this week.
Thx.
On Thu, Dec 9, 2021 at 10:19 AM Borislav Petkov bp@alien8.de wrote:
- Hugh and Patrick.
On Thu, Dec 09, 2021 at 09:38:10AM -0500, John Dorminy wrote:
Greetings;
It seems that this patch causes a mem= parameter to the kernel to have no effect, unfortunately...
As far as I understand, the x86 mem parameter handler parse_memopt() (called by parse_early_param()) relies on being called after e820__memory_setup(): it simply removes any memory above the specified limit at that moment, allowing memory to later be hotplugged without regard for the initial limit. However, the initial non-hotplugged memory must already have been set up, in e820__memory_setup(), so that it can be removed in parse_memopt(); if parse_early_param() is called before e820__memory_setup(), as this change does, the parameter ends up having no effect.
I apologize that I don't know how to fix this, but I'm happy to test patches.
Yeah, people have been reporting boot failures with mem= on the cmdline.
I think I see why, can you try this one:
diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 6a190c7f4d71..6db971e61e4b 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -862,6 +862,8 @@ void __init setup_arch(char **cmdline_p) */ x86_configure_nx();
e820__memory_setup();/* * This parses early params and it needs to run before * early_reserve_memory() because latter relies on such settings@@ -884,7 +886,6 @@ void __init setup_arch(char **cmdline_p) early_reserve_memory();
iomem_resource.end = (1ULL << boot_cpu_data.x86_phys_bits) - 1;
e820__memory_setup(); parse_setup_data(); copy_edd();
Confirmed that that patch makes mem= work again:
[ 0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.16.0-rc4+ root=UUID=0e 750e61-b92e-4708-a974-c50a3fb7e969 ro net.ifnames=0 crashkernel=128M mem=4G ... [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009abff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009ac00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007dd3afff] usable [ 0.000000] BIOS-e820: [mem 0x000000007dd3b000-0x000000007deeffff] reserved [ 0.000000] BIOS-e820: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS [ 0.000000] BIOS-e820: [mem 0x000000007e0d4000-0x000000007f367fff] reserved [ 0.000000] BIOS-e820: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS [ 0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000207fffffff] usable [ 0.000000] e820: remove [mem 0x100000000-0xfffffffffffffffe] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] user-defined physical RAM map: [ 0.000000] user: [mem 0x0000000000000000-0x000000000009abff] usable [ 0.000000] user: [mem 0x000000000009ac00-0x000000000009ffff] reserved [ 0.000000] user: [mem 0x00000000000e0000-0x00000000000fffff] reserved [ 0.000000] user: [mem 0x0000000000100000-0x000000007dd3afff] usable [ 0.000000] user: [mem 0x000000007dd3b000-0x000000007deeffff] reserved [ 0.000000] user: [mem 0x000000007def0000-0x000000007e0d3fff] ACPI NVS [ 0.000000] user: [mem 0x000000007e0d4000-0x000000007f367fff] reserved [ 0.000000] user: [mem 0x000000007f368000-0x000000007f7fffff] ACPI NVS [ 0.000000] user: [mem 0x0000000080000000-0x000000008fffffff] reserved [ 0.000000] user: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved [ 0.000000] user: [mem 0x00000000ff000000-0x00000000ffffffff] reserved ... [ 0.025520] Memory: 1762976K/2061136K available (16394K kernel code, 3568K rwdata, 10328K rodata, 2676K init, 4924K bss, 297900K reserved, 0K cma-reserved)
On Thu, Dec 09, 2021 at 10:49:06AM -0500, John Dorminy wrote:
Confirmed that that patch makes mem= work again:
Thanks!
linux-stable-mirror@lists.linaro.org
-
Borislav Petkov -
Dan Williams -
Hugh Dickins -
John Dorminy -
Juergen Gross -
Marek Marczykowski-Górecki -
Mike Rapoport -
Nathan Chancellor -
Patrick J. Volkerding -
tip-bot2 for Borislav Petkov -
tip-bot2 for Juergen Gross
-
Williams, Dan J