[PATCH] x86/speculation/l1tf: fix off-by-one error when warning that system has too much RAM
Two users have reported [1] that they have an "extremely unlikely" system with more than MAX_PA/2 memory and L1TF mitigation is not effective. In fact it's a CPU with 36bits phys limit (64GB) and 32GB memory, but due to holes in the e820 map, the main region is almost 500MB over the 32GB limit:
[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000081effffff] usable
Suggestions to use 'mem=32G' to prefer L1TF mitigation while losing the 500MB revealed, that there's an off-by-one error in the check in l1tf_select_mitigation(). l1tf_pfn_limit() returns the last usable pfn (inclusive), but it's more common and hopefully less error-prone to return the first pfn that's over limit, so this patch changes that and updates the other callers.
[1] https://bugzilla.suse.com/show_bug.cgi?id=1105536
Reported-by: George Anchev studio@anchev.net Reported-by: Christopher Snowhill kode54@gmail.com Fixes: 17dbca119312 ("x86/speculation/l1tf: Add sysfs reporting for l1tf") Cc: stable@vger.kernel.org Signed-off-by: Vlastimil Babka vbabka@suse.cz --- arch/x86/include/asm/processor.h | 2 +- arch/x86/mm/init.c | 2 +- arch/x86/mm/mmap.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h index a0a52274cb4a..c24297268ebc 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -183,7 +183,7 @@ extern void cpu_detect(struct cpuinfo_x86 *c);
static inline unsigned long long l1tf_pfn_limit(void) { - return BIT_ULL(boot_cpu_data.x86_phys_bits - 1 - PAGE_SHIFT) - 1; + return BIT_ULL(boot_cpu_data.x86_phys_bits - 1 - PAGE_SHIFT); }
extern void early_cpu_init(void); diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c index 02de3d6065c4..63a6f9fcaf20 100644 --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -923,7 +923,7 @@ unsigned long max_swapfile_size(void)
if (boot_cpu_has_bug(X86_BUG_L1TF)) { /* Limit the swap file size to MAX_PA/2 for L1TF workaround */ - unsigned long long l1tf_limit = l1tf_pfn_limit() + 1; + unsigned long long l1tf_limit = l1tf_pfn_limit(); /* * We encode swap offsets also with 3 bits below those for pfn * which makes the usable limit higher. diff --git a/arch/x86/mm/mmap.c b/arch/x86/mm/mmap.c index f40ab8185d94..1e95d57760cf 100644 --- a/arch/x86/mm/mmap.c +++ b/arch/x86/mm/mmap.c @@ -257,7 +257,7 @@ bool pfn_modify_allowed(unsigned long pfn, pgprot_t prot) /* If it's real memory always allow */ if (pfn_valid(pfn)) return true; - if (pfn > l1tf_pfn_limit() && !capable(CAP_SYS_ADMIN)) + if (pfn >= l1tf_pfn_limit() && !capable(CAP_SYS_ADMIN)) return false; return true; }
On Thu 23-08-18 15:44:18, Vlastimil Babka wrote:
Two users have reported [1] that they have an "extremely unlikely" system with more than MAX_PA/2 memory and L1TF mitigation is not effective. In fact it's a CPU with 36bits phys limit (64GB) and 32GB memory, but due to holes in the e820 map, the main region is almost 500MB over the 32GB limit:
[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000081effffff] usable
Suggestions to use 'mem=32G' to prefer L1TF mitigation while losing the 500MB revealed, that there's an off-by-one error in the check in l1tf_select_mitigation(). l1tf_pfn_limit() returns the last usable pfn (inclusive), but it's more common and hopefully less error-prone to return the first pfn that's over limit, so this patch changes that and updates the other callers.
[1] https://bugzilla.suse.com/show_bug.cgi?id=1105536
Reported-by: George Anchev studio@anchev.net Reported-by: Christopher Snowhill kode54@gmail.com Fixes: 17dbca119312 ("x86/speculation/l1tf: Add sysfs reporting for l1tf") Cc: stable@vger.kernel.org Signed-off-by: Vlastimil Babka vbabka@suse.cz
Yes this should be less error prone. Acked-by: Michal Hocko mhocko@suse.com
arch/x86/include/asm/processor.h | 2 +- arch/x86/mm/init.c | 2 +- arch/x86/mm/mmap.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h index a0a52274cb4a..c24297268ebc 100644 --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -183,7 +183,7 @@ extern void cpu_detect(struct cpuinfo_x86 *c); static inline unsigned long long l1tf_pfn_limit(void) {
- return BIT_ULL(boot_cpu_data.x86_phys_bits - 1 - PAGE_SHIFT) - 1;
- return BIT_ULL(boot_cpu_data.x86_phys_bits - 1 - PAGE_SHIFT);
} extern void early_cpu_init(void); diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c index 02de3d6065c4..63a6f9fcaf20 100644 --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -923,7 +923,7 @@ unsigned long max_swapfile_size(void) if (boot_cpu_has_bug(X86_BUG_L1TF)) { /* Limit the swap file size to MAX_PA/2 for L1TF workaround */
unsigned long long l1tf_limit = l1tf_pfn_limit() + 1;
unsigned long long l1tf_limit = l1tf_pfn_limit();
- We encode swap offsets also with 3 bits below those for pfn
- which makes the usable limit higher.
diff --git a/arch/x86/mm/mmap.c b/arch/x86/mm/mmap.c index f40ab8185d94..1e95d57760cf 100644 --- a/arch/x86/mm/mmap.c +++ b/arch/x86/mm/mmap.c @@ -257,7 +257,7 @@ bool pfn_modify_allowed(unsigned long pfn, pgprot_t prot) /* If it's real memory always allow */ if (pfn_valid(pfn)) return true;
- if (pfn > l1tf_pfn_limit() && !capable(CAP_SYS_ADMIN))
- if (pfn >= l1tf_pfn_limit() && !capable(CAP_SYS_ADMIN)) return false; return true;
}
2.18.0
Two users have reported [1] that they have an "extremely unlikely" system with more than MAX_PA/2 memory and L1TF mitigation is not effective. Let's make the warning more helpful by suggesting the proper mem=X kernel boot param, a rough calculation of how much RAM can be lost (not precise if there's holes between MAX_PA/2 and max_pfn in the e820 map) and a link to the L1TF document to help decide if the mitigation is worth the unusable RAM.
[1] https://bugzilla.suse.com/show_bug.cgi?id=1105536
Suggested-by: Michal Hocko mhocko@suse.com Cc: stable@vger.kernel.org Signed-off-by: Vlastimil Babka vbabka@suse.cz --- arch/x86/kernel/cpu/bugs.c | 5 +++++ 1 file changed, 5 insertions(+)
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c index cb4a16292aa7..4b820bb6c504 100644 --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -702,6 +702,11 @@ static void __init l1tf_select_mitigation(void) half_pa = (u64)l1tf_pfn_limit() << PAGE_SHIFT; if (e820__mapped_any(half_pa, ULLONG_MAX - half_pa, E820_TYPE_RAM)) { pr_warn("System has more than MAX_PA/2 memory. L1TF mitigation not effective.\n"); + pr_info("You may make it effective by booting the kernel with mem=%llu parameter.\n", + half_pa); + pr_info("However, doing so will make up to %llu bytes of RAM unusable.\n", + ((u64) max_pfn << PAGE_SHIFT) - half_pa); + pr_info("Reading Documentation/admin-guide/l1tf.rst might help you decide."); return; }
On Thu, Aug 23, 2018 at 04:28:12PM +0200, Vlastimil Babka wrote:
Two users have reported [1] that they have an "extremely unlikely" system with more than MAX_PA/2 memory and L1TF mitigation is not effective. Let's make the warning more helpful by suggesting the proper mem=X kernel boot param, a rough calculation of how much RAM can be lost (not precise if there's holes between MAX_PA/2 and max_pfn in the e820 map) and a link to the L1TF document to help decide if the mitigation is worth the unusable RAM.
I'm not sure anyone would really do that. After all you probably prefer your memory. And if it's really a non ECC client part they are are already used to to live very dangerously because their undetected RAM bit error rate will be significant. L1TF is probably one of your smaller problems in this case...
-Andi
On Thu 23-08-18 08:46:48, Andi Kleen wrote:
On Thu, Aug 23, 2018 at 04:28:12PM +0200, Vlastimil Babka wrote:
Two users have reported [1] that they have an "extremely unlikely" system with more than MAX_PA/2 memory and L1TF mitigation is not effective. Let's make the warning more helpful by suggesting the proper mem=X kernel boot param, a rough calculation of how much RAM can be lost (not precise if there's holes between MAX_PA/2 and max_pfn in the e820 map) and a link to the L1TF document to help decide if the mitigation is worth the unusable RAM.
I'm not sure anyone would really do that. After all you probably prefer your memory. And if it's really a non ECC client part they are are already used to to live very dangerously because their undetected RAM bit error rate will be significant. L1TF is probably one of your smaller problems in this case...
There are people who care about L1TF mitigations. I am not going to question their motivation. In any case a hint how to make the mitigation active again sounds more useful than something that sounds as scary as "you are vulnerable".
There are people who care about L1TF mitigations. I am not going to question their motivation. In any case a hint how to make the mitigation active again sounds more useful than something that sounds as scary as "you are vulnerable".
FWIW an early version of these patches automatically limited the available memory, but Linus pointed out that people likely prefer their memory. I still think his reasoning was right, and likely applies to your message too.
-Andi
On Thu 23-08-18 12:38:33, Andi Kleen wrote:
There are people who care about L1TF mitigations. I am not going to question their motivation. In any case a hint how to make the mitigation active again sounds more useful than something that sounds as scary as "you are vulnerable".
FWIW an early version of these patches automatically limited the available memory, but Linus pointed out that people likely prefer their memory.
Nobody is questioning that. The point is to give them a hint on how to make the mitigation active again without going to call for help. The message does tell them how to _enable_ it and point them to the documentation on how to _decide_.
On Thu, Aug 23, 2018 at 10:05:20PM +0200, Michal Hocko wrote:
On Thu 23-08-18 12:38:33, Andi Kleen wrote:
There are people who care about L1TF mitigations. I am not going to question their motivation. In any case a hint how to make the mitigation active again sounds more useful than something that sounds as scary as "you are vulnerable".
FWIW an early version of these patches automatically limited the available memory, but Linus pointed out that people likely prefer their memory.
Nobody is questioning that. The point is to give them a hint on how to make the mitigation active again without going to call for help. The message does tell them how to _enable_ it and point them to the documentation on how to _decide_.
On the message I guess there are two cases:
- either it's very little memory that is lost like in the 32GB + memory hole case. In this case maybe it's better if we just limit automatically if the overlap is small enough (<2GB perhaps?)
- Or it's a lot of memory then people are unlikely to want to lose their memory and I don't think we really need the message either.
Also I checked the bug again and it looks like the reporter has an IvyBridge. There is actually a better solution for those (anything Nehalem and newer) because they internally have at least 44 bits in the cache, which is good enough for the mitigation. Just need a quirk to override the bit width in this case (will submit a patch)
-Andi
Hi Vlastimil,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on tip/auto-latest] [also build test ERROR on next-20180822] [cannot apply to v4.18] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Vlastimil-Babka/x86-speculation-l1t... config: i386-randconfig-x077-201833 (attached as .config) compiler: gcc-7 (Debian 7.3.0-16) 7.3.0 reproduce: # save the attached .config to linux build tree make ARCH=i386
All errors (new ones prefixed by >>):
In file included from include/linux/kernel.h:14:0, from arch/x86/include/asm/percpu.h:45, from arch/x86/include/asm/current.h:6, from include/linux/sched.h:12, from include/linux/utsname.h:6, from arch/x86/kernel/cpu/bugs.c:12: arch/x86/kernel/cpu/bugs.c: In function 'l1tf_select_mitigation':
arch/x86/kernel/cpu/bugs.c:709:12: error: 'max_pfn' undeclared (first use in this function); did you mean 'pgd_pfn'?
((u64) max_pfn << PAGE_SHIFT) - half_pa); ^ include/linux/printk.h:311:34: note: in definition of macro 'pr_info' printk(KERN_INFO pr_fmt(fmt), ##__VA_ARGS__) ^~~~~~~~~~~ arch/x86/kernel/cpu/bugs.c:709:12: note: each undeclared identifier is reported only once for each function it appears in ((u64) max_pfn << PAGE_SHIFT) - half_pa); ^ include/linux/printk.h:311:34: note: in definition of macro 'pr_info' printk(KERN_INFO pr_fmt(fmt), ##__VA_ARGS__) ^~~~~~~~~~~
vim +709 arch/x86/kernel/cpu/bugs.c
697 698 /* 699 * This is extremely unlikely to happen because almost all 700 * systems have far more MAX_PA/2 than RAM can be fit into 701 * DIMM slots. 702 */ 703 half_pa = (u64)l1tf_pfn_limit() << PAGE_SHIFT; 704 if (e820__mapped_any(half_pa, ULLONG_MAX - half_pa, E820_TYPE_RAM)) { 705 pr_warn("System has more than MAX_PA/2 memory. L1TF mitigation not effective.\n"); 706 pr_info("You may make it effective by booting the kernel with mem=%llu parameter.\n", 707 half_pa); 708 pr_info("However, doing so will make up to %llu bytes of RAM unusable.\n",
709 ((u64) max_pfn << PAGE_SHIFT) - half_pa);
710 pr_info("Reading Documentation/admin-guide/l1tf.rst might help you decide."); 711 return; 712 } 713 714 setup_force_cpu_cap(X86_FEATURE_L1TF_PTEINV); 715 } 716
--- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation
Hi Vlastimil,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on tip/auto-latest] [also build test ERROR on next-20180822] [cannot apply to v4.18] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Vlastimil-Babka/x86-speculation-l1t... config: i386-randconfig-a1-201833 (attached as .config) compiler: gcc-4.9 (Debian 4.9.4-2) 4.9.4 reproduce: # save the attached .config to linux build tree make ARCH=i386
All errors (new ones prefixed by >>):
In file included from include/linux/kernel.h:14:0, from arch/x86/include/asm/percpu.h:45, from arch/x86/include/asm/current.h:6, from include/linux/sched.h:12, from include/linux/utsname.h:6, from arch/x86//kernel/cpu/bugs.c:12: arch/x86//kernel/cpu/bugs.c: In function 'l1tf_select_mitigation':
arch/x86//kernel/cpu/bugs.c:709:12: error: 'max_pfn' undeclared (first use in this function)
((u64) max_pfn << PAGE_SHIFT) - half_pa); ^ include/linux/printk.h:311:34: note: in definition of macro 'pr_info' printk(KERN_INFO pr_fmt(fmt), ##__VA_ARGS__) ^ arch/x86//kernel/cpu/bugs.c:709:12: note: each undeclared identifier is reported only once for each function it appears in ((u64) max_pfn << PAGE_SHIFT) - half_pa); ^ include/linux/printk.h:311:34: note: in definition of macro 'pr_info' printk(KERN_INFO pr_fmt(fmt), ##__VA_ARGS__) ^
vim +/max_pfn +709 arch/x86//kernel/cpu/bugs.c
697 698 /* 699 * This is extremely unlikely to happen because almost all 700 * systems have far more MAX_PA/2 than RAM can be fit into 701 * DIMM slots. 702 */ 703 half_pa = (u64)l1tf_pfn_limit() << PAGE_SHIFT; 704 if (e820__mapped_any(half_pa, ULLONG_MAX - half_pa, E820_TYPE_RAM)) { 705 pr_warn("System has more than MAX_PA/2 memory. L1TF mitigation not effective.\n"); 706 pr_info("You may make it effective by booting the kernel with mem=%llu parameter.\n", 707 half_pa); 708 pr_info("However, doing so will make up to %llu bytes of RAM unusable.\n",
709 ((u64) max_pfn << PAGE_SHIFT) - half_pa);
710 pr_info("Reading Documentation/admin-guide/l1tf.rst might help you decide."); 711 return; 712 } 713 714 setup_force_cpu_cap(X86_FEATURE_L1TF_PTEINV); 715 } 716
--- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation
On Thu 23-08-18 16:28:12, Vlastimil Babka wrote:
Two users have reported [1] that they have an "extremely unlikely" system with more than MAX_PA/2 memory and L1TF mitigation is not effective. Let's make the warning more helpful by suggesting the proper mem=X kernel boot param, a rough calculation of how much RAM can be lost (not precise if there's holes between MAX_PA/2 and max_pfn in the e820 map) and a link to the L1TF document to help decide if the mitigation is worth the unusable RAM.
[1] https://bugzilla.suse.com/show_bug.cgi?id=1105536
Suggested-by: Michal Hocko mhocko@suse.com Cc: stable@vger.kernel.org Signed-off-by: Vlastimil Babka vbabka@suse.cz
I wouldn't bother with max_pfn-half_pa part but other than that this is much more useful than the original message.
Acked-by: Michal Hocko mhocko@suse.com
arch/x86/kernel/cpu/bugs.c | 5 +++++ 1 file changed, 5 insertions(+)
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c index cb4a16292aa7..4b820bb6c504 100644 --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -702,6 +702,11 @@ static void __init l1tf_select_mitigation(void) half_pa = (u64)l1tf_pfn_limit() << PAGE_SHIFT; if (e820__mapped_any(half_pa, ULLONG_MAX - half_pa, E820_TYPE_RAM)) { pr_warn("System has more than MAX_PA/2 memory. L1TF mitigation not effective.\n");
pr_info("You may make it effective by booting the kernel with mem=%llu parameter.\n",half_pa);pr_info("However, doing so will make up to %llu bytes of RAM unusable.\n",((u64) max_pfn << PAGE_SHIFT) - half_pa);pr_info("Reading Documentation/admin-guide/l1tf.rst might help you decide.");2.18.0
On 08/23/2018 09:27 PM, Michal Hocko wrote:
On Thu 23-08-18 16:28:12, Vlastimil Babka wrote:
Two users have reported [1] that they have an "extremely unlikely" system with more than MAX_PA/2 memory and L1TF mitigation is not effective. Let's make the warning more helpful by suggesting the proper mem=X kernel boot param, a rough calculation of how much RAM can be lost (not precise if there's holes between MAX_PA/2 and max_pfn in the e820 map) and a link to the L1TF document to help decide if the mitigation is worth the unusable RAM.
[1] https://bugzilla.suse.com/show_bug.cgi?id=1105536
Suggested-by: Michal Hocko mhocko@suse.com Cc: stable@vger.kernel.org Signed-off-by: Vlastimil Babka vbabka@suse.cz
I wouldn't bother with max_pfn-half_pa part but other than that this is much more useful than the original message.
Right, and it causes build failures on some configs.
Acked-by: Michal Hocko mhocko@suse.com
Thanks! Here's a v2:
----8<----
From 977c5db27fe35a84807850b947bc5678c4d467b3 Mon Sep 17 00:00:00 2001
From: Vlastimil Babka vbabka@suse.cz Date: Thu, 23 Aug 2018 16:21:29 +0200 Subject: [PATCH] x86/speculation/l1tf: suggest what to do on systems with too much RAM
Two users have reported [1] that they have an "extremely unlikely" system with more than MAX_PA/2 memory and L1TF mitigation is not effective. Let's make the warning more helpful by suggesting the proper mem=X kernel boot param to make it effective and a link to the L1TF document to help decide if the mitigation is worth the unusable RAM.
[1] https://bugzilla.suse.com/show_bug.cgi?id=1105536
Suggested-by: Michal Hocko mhocko@suse.com Acked-by: Michal Hocko mhocko@suse.com Signed-off-by: Vlastimil Babka vbabka@suse.cz --- arch/x86/kernel/cpu/bugs.c | 4 ++++ 1 file changed, 4 insertions(+)
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c index cb4a16292aa7..5c32b5006738 100644 --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -702,6 +702,10 @@ static void __init l1tf_select_mitigation(void) half_pa = (u64)l1tf_pfn_limit() << PAGE_SHIFT; if (e820__mapped_any(half_pa, ULLONG_MAX - half_pa, E820_TYPE_RAM)) { pr_warn("System has more than MAX_PA/2 memory. L1TF mitigation not effective.\n"); + pr_info("You may make it effective by booting the kernel with mem=%llu parameter.\n", + half_pa); + pr_info("However, doing so will make a part of your RAM unusable.\n"); + pr_info("Reading Documentation/admin-guide/l1tf.rst might help you decide.\n"); return; }
On 08/24/2018 09:32 AM, Vlastimil Babka wrote:
On 08/23/2018 09:27 PM, Michal Hocko wrote:
On Thu 23-08-18 16:28:12, Vlastimil Babka wrote:
Two users have reported [1] that they have an "extremely unlikely" system with more than MAX_PA/2 memory and L1TF mitigation is not effective. Let's make the warning more helpful by suggesting the proper mem=X kernel boot param, a rough calculation of how much RAM can be lost (not precise if there's holes between MAX_PA/2 and max_pfn in the e820 map) and a link to the L1TF document to help decide if the mitigation is worth the unusable RAM.
[1] https://bugzilla.suse.com/show_bug.cgi?id=1105536
Suggested-by: Michal Hocko mhocko@suse.com Cc: stable@vger.kernel.org Signed-off-by: Vlastimil Babka vbabka@suse.cz
I wouldn't bother with max_pfn-half_pa part but other than that this is much more useful than the original message.
Right, and it causes build failures on some configs.
Acked-by: Michal Hocko mhocko@suse.com
Thanks! Here's a v2:
Just realized that kvm printk's refer to the online version at https://www.kernel.org/doc/html/latest/admin-guide/l1tf.html which should be easier for the users of distro kernels, should I change that?
On 08/24/2018 12:36 PM, Vlastimil Babka wrote:
On 08/24/2018 09:32 AM, Vlastimil Babka wrote:
On 08/23/2018 09:27 PM, Michal Hocko wrote:
On Thu 23-08-18 16:28:12, Vlastimil Babka wrote:
Two users have reported [1] that they have an "extremely unlikely" system with more than MAX_PA/2 memory and L1TF mitigation is not effective. Let's make the warning more helpful by suggesting the proper mem=X kernel boot param, a rough calculation of how much RAM can be lost (not precise if there's holes between MAX_PA/2 and max_pfn in the e820 map) and a link to the L1TF document to help decide if the mitigation is worth the unusable RAM.
[1] https://bugzilla.suse.com/show_bug.cgi?id=1105536
Suggested-by: Michal Hocko mhocko@suse.com Cc: stable@vger.kernel.org Signed-off-by: Vlastimil Babka vbabka@suse.cz
I wouldn't bother with max_pfn-half_pa part but other than that this is much more useful than the original message.
Right, and it causes build failures on some configs.
Acked-by: Michal Hocko mhocko@suse.com
Thanks! Here's a v2:
Just realized that kvm printk's refer to the online version at https://www.kernel.org/doc/html/latest/admin-guide/l1tf.html which should be easier for the users of distro kernels, should I change that?
FWIW, if it's not possible to amend anymore... ----8<----
From a650e6a4b989c6e029ac1ab4e0a68553e074ba7a Mon Sep 17 00:00:00 2001
From: Vlastimil Babka vbabka@suse.cz Date: Fri, 24 Aug 2018 14:05:45 +0200 Subject: [PATCH] x86/speculation/l1tf: replace Documentation path with kernel.org URL
The URL might be easier to reach for users of distro kernels without unpacked source, and be more up-to-date. It's also used in KVM warnings related to L1TF.
Signed-off-by: Vlastimil Babka vbabka@suse.cz --- arch/x86/kernel/cpu/bugs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c index 5c32b5006738..4c2313d0b9ca 100644 --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -705,7 +705,7 @@ static void __init l1tf_select_mitigation(void) pr_info("You may make it effective by booting the kernel with mem=%llu parameter.\n", half_pa); pr_info("However, doing so will make a part of your RAM unusable.\n"); - pr_info("Reading Documentation/admin-guide/l1tf.rst might help you decide.\n"); + pr_info("Reading https://www.kernel.org/doc/html/latest/admin-guide/l1tf.html might help you decide.\n"); return; }
On Fri 24-08-18 14:10:54, Vlastimil Babka wrote:
On 08/24/2018 12:36 PM, Vlastimil Babka wrote:
On 08/24/2018 09:32 AM, Vlastimil Babka wrote:
On 08/23/2018 09:27 PM, Michal Hocko wrote:
On Thu 23-08-18 16:28:12, Vlastimil Babka wrote:
Two users have reported [1] that they have an "extremely unlikely" system with more than MAX_PA/2 memory and L1TF mitigation is not effective. Let's make the warning more helpful by suggesting the proper mem=X kernel boot param, a rough calculation of how much RAM can be lost (not precise if there's holes between MAX_PA/2 and max_pfn in the e820 map) and a link to the L1TF document to help decide if the mitigation is worth the unusable RAM.
[1] https://bugzilla.suse.com/show_bug.cgi?id=1105536
Suggested-by: Michal Hocko mhocko@suse.com Cc: stable@vger.kernel.org Signed-off-by: Vlastimil Babka vbabka@suse.cz
I wouldn't bother with max_pfn-half_pa part but other than that this is much more useful than the original message.
Right, and it causes build failures on some configs.
Acked-by: Michal Hocko mhocko@suse.com
Thanks! Here's a v2:
Just realized that kvm printk's refer to the online version at https://www.kernel.org/doc/html/latest/admin-guide/l1tf.html which should be easier for the users of distro kernels, should I change that?
FWIW, if it's not possible to amend anymore... ----8<----
From a650e6a4b989c6e029ac1ab4e0a68553e074ba7a Mon Sep 17 00:00:00 2001
From: Vlastimil Babka vbabka@suse.cz Date: Fri, 24 Aug 2018 14:05:45 +0200 Subject: [PATCH] x86/speculation/l1tf: replace Documentation path with kernel.org URL
The URL might be easier to reach for users of distro kernels without unpacked source, and be more up-to-date. It's also used in KVM warnings related to L1TF.
Yes, URL is much more convenient than a reference to the kernel source documentation. If the link below is meant to be long lived then sure.
Signed-off-by: Vlastimil Babka vbabka@suse.cz
Acked-by: Michal Hocko mhocko@suse.com
arch/x86/kernel/cpu/bugs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c index 5c32b5006738..4c2313d0b9ca 100644 --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -705,7 +705,7 @@ static void __init l1tf_select_mitigation(void) pr_info("You may make it effective by booting the kernel with mem=%llu parameter.\n", half_pa); pr_info("However, doing so will make a part of your RAM unusable.\n");
pr_info("Reading Documentation/admin-guide/l1tf.rst might help you decide.\n");
pr_info("Reading https://www.kernel.org/doc/html/latest/admin-guide/l1tf.html might help you decide.\n");2.18.0
On Thu, Aug 23, 2018 at 03:44:18PM +0200, Vlastimil Babka wrote:
Two users have reported [1] that they have an "extremely unlikely" system with more than MAX_PA/2 memory and L1TF mitigation is not effective. In fact it's a CPU with 36bits phys limit (64GB) and 32GB memory, but due to holes in the e820 map, the main region is almost 500MB over the 32GB limit:
Ah I see it's a client part with very large DIMMs and someone being very brave and using that much memory without ECC.
[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000081effffff] usable
Suggestions to use 'mem=32G' to prefer L1TF mitigation while losing the 500MB revealed, that there's an off-by-one error in the check in l1tf_select_mitigation(). l1tf_pfn_limit() returns the last usable pfn (inclusive), but it's more common and hopefully less error-prone to return the first pfn that's over limit, so this patch changes that and updates the other callers.
I can see the off by one, but does it really cause the user's problem?
They will be still over the limit in any case, with or without off-by-one.
So the description has nothing to do with the fix. Or do I miss something?
-Andi
On 8/23/18 5:44 PM, Andi Kleen wrote:
On Thu, Aug 23, 2018 at 03:44:18PM +0200, Vlastimil Babka wrote:
Two users have reported [1] that they have an "extremely unlikely" system with more than MAX_PA/2 memory and L1TF mitigation is not effective. In fact it's a CPU with 36bits phys limit (64GB) and 32GB memory, but due to holes in the e820 map, the main region is almost 500MB over the 32GB limit:
Ah I see it's a client part with very large DIMMs and someone being very brave and using that much memory without ECC.
Are you trying to mock the users and diminsh their report because of that?
[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000081effffff] usable
Suggestions to use 'mem=32G' to prefer L1TF mitigation while losing the 500MB revealed, that there's an off-by-one error in the check in l1tf_select_mitigation(). l1tf_pfn_limit() returns the last usable pfn (inclusive), but it's more common and hopefully less error-prone to return the first pfn that's over limit, so this patch changes that and updates the other callers.
I can see the off by one, but does it really cause the user's problem?
They will be still over the limit in any case, with or without off-by-one.
So the description has nothing to do with the fix. Or do I miss something?
The off-by-one happens when 'mem=32G' is used to limit the amount memory. It's easier to do "mem=32G" with fixed code than "mem=33554428k" to workaround the error.
-Andi
On 23. aug. 2018 17:44, Andi Kleen wrote:
On Thu, Aug 23, 2018 at 03:44:18PM +0200, Vlastimil Babka wrote:
Two users have reported [1] that they have an "extremely unlikely" system with more than MAX_PA/2 memory and L1TF mitigation is not effective. In fact it's a CPU with 36bits phys limit (64GB) and 32GB memory, but due to holes in the e820 map, the main region is almost 500MB over the 32GB limit:
Ah I see it's a client part with very large DIMMs and someone being very brave and using that much memory without ECC.
FYI; It is also happening on Xeon E3v2 (Ivy Bridge generation) w/ 32GB of ECC RAM here, a low-end server part that officially supports up to 32GB.
[ 0.000000] microcode: microcode updated early to revision 0x20, date = 2018-04-10 [ 0.029728] L1TF: System has more than MAX_PA/2 memory. L1TF mitigation not effective. [ 1.063155] microcode: sig=0x306a9, pf=0x2, revision=0x20
processor : 7 vendor_id : GenuineIntel cpu family : 6 model : 58 model name : Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz stepping : 9 microcode : 0x20 cpu MHz : 3500.044 cache size : 8192 KB physical id : 0 siblings : 8 core id : 3 cpu cores : 4 apicid : 7 initial apicid : 7 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts flush_l1d bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf bogomips : 6602.15 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management:
On Fri, Aug 24, 2018 at 04:22:57AM +0200, Andre Tomt wrote:
On 23. aug. 2018 17:44, Andi Kleen wrote:
On Thu, Aug 23, 2018 at 03:44:18PM +0200, Vlastimil Babka wrote:
Two users have reported [1] that they have an "extremely unlikely" system with more than MAX_PA/2 memory and L1TF mitigation is not effective. In fact it's a CPU with 36bits phys limit (64GB) and 32GB memory, but due to holes in the e820 map, the main region is almost 500MB over the 32GB limit:
Ah I see it's a client part with very large DIMMs and someone being very brave and using that much memory without ECC.
FYI; It is also happening on Xeon E3v2 (Ivy Bridge generation) w/ 32GB of ECC RAM here, a low-end server part that officially supports up to 32GB.
Good point.
-andi
On 08/23/2018 07:22 PM, Andre Tomt wrote:
On 23. aug. 2018 17:44, Andi Kleen wrote:
On Thu, Aug 23, 2018 at 03:44:18PM +0200, Vlastimil Babka wrote:
Two users have reported [1] that they have an "extremely unlikely" system with more than MAX_PA/2 memory and L1TF mitigation is not effective. In fact it's a CPU with 36bits phys limit (64GB) and 32GB memory, but due to holes in the e820 map, the main region is almost 500MB over the 32GB limit:
Ah I see it's a client part with very large DIMMs and someone being very brave and using that much memory without ECC.
FYI; It is also happening on Xeon E3v2 (Ivy Bridge generation) w/ 32GB of ECC RAM here, a low-end server part that officially supports up to 32GB.
Indeed, I must be "very brave" to not have chucked this CPU and motherboard and RAM in the bin, and bought a new board, Xeon CPU, and ECC RAM. Maybe I'll consider that in the future, when I again have $1000+ to buy new kit. Which will probably be never, at this rate.
[ 0.000000] microcode: microcode updated early to revision 0x20, date = 2018-04-10 [ 0.029728] L1TF: System has more than MAX_PA/2 memory. L1TF mitigation not effective. [ 1.063155] microcode: sig=0x306a9, pf=0x2, revision=0x20
processor : 7 vendor_id : GenuineIntel cpu family : 6 model : 58 model name : Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz stepping : 9 microcode : 0x20 cpu MHz : 3500.044 cache size : 8192 KB physical id : 0 siblings : 8 core id : 3 cpu cores : 4 apicid : 7 initial apicid : 7 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts flush_l1d bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf bogomips : 6602.15 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management:
On Thu, 23 Aug 2018 08:44:37 -0700 Andi Kleen wrote:
Ah I see it's a client part with very large DIMMs and someone being very brave and using that much memory without ECC.
It is not about being "brave" but about being informed. As of 2018 you can probably call "brave" everyone who uses any modern computer. However this machine was purchased in 2012. Consider what was known then and what is known now. The motherboard ASUS P8Z77-V does not support ECC memory. Still we needed and paid for 32GB of RAM, not for 31.5.
-- George
linux-stable-mirror@lists.linaro.org
-
Andi Kleen -
Andre Tomt -
Christopher Snowhill -
George Anchev -
kbuild test robot -
Michal Hocko -
Vlastimil Babka