- Linux-stable-mirror - lists.linaro.org

[Linux-stable-mirror] Patch "kaiser: asm/tlbflush.h handle noPGE at lower level" has been added to the 4.9-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled kaiser: asm/tlbflush.h handle noPGE at lower level to the 4.9-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: kaiser-asm-tlbflush.h-handle-nopge-at-lower-level.patch and it can be found in the queue-4.9 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From foo@baz Wed Jan 3 20:37:21 CET 2018 From: Hugh Dickins <hughd(a)google.com> Date: Sat, 4 Nov 2017 18:23:24 -0700 Subject: kaiser: asm/tlbflush.h handle noPGE at lower level From: Hugh Dickins <hughd(a)google.com> I found asm/tlbflush.h too twisty, and think it safer not to avoid __native_flush_tlb_global_irq_disabled() in the kaiser_enabled case, but instead let it handle kaiser_enabled along with cr3: it can just use __native_flush_tlb() for that, no harm in re-disabling preemption. (This is not the same change as Kirill and Dave have suggested for upstream, flipping PGE in cr4: that's neat, but needs a cpu_has_pge check; cr3 is enough for kaiser, and thought to be cheaper than cr4.) Also delete the X86_FEATURE_INVPCID invpcid_flush_all_nonglobals() preference from __native_flush_tlb(): unlike the invpcid_flush_all() preference in __native_flush_tlb_global(), it's not seen in upstream 4.14, and was recently reported to be surprisingly slow. Signed-off-by: Hugh Dickins <hughd(a)google.com> Acked-by: Jiri Kosina <jkosina(a)suse.cz> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- arch/x86/include/asm/tlbflush.h | 27 +++------------------------ 1 file changed, 3 insertions(+), 24 deletions(-) --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -152,14 +152,6 @@ static inline void kaiser_flush_tlb_on_r static inline void __native_flush_tlb(void) { - if (this_cpu_has(X86_FEATURE_INVPCID)) { - /* - * Note, this works with CR4.PCIDE=0 or 1. - */ - invpcid_flush_all_nonglobals(); - return; - } - /* * If current->mm == NULL then we borrow a mm which may change during a * task switch and therefore we must not be preempted while we write CR3 @@ -183,11 +175,8 @@ static inline void __native_flush_tlb_gl /* restore PGE as it was before */ native_write_cr4(cr4); } else { - /* - * x86_64 microcode update comes this way when CR4.PGE is not - * enabled, and it's safer for all callers to allow this case. - */ - native_write_cr3(native_read_cr3()); + /* do it with cr3, letting kaiser flush user PCID */ + __native_flush_tlb(); } } @@ -195,12 +184,6 @@ static inline void __native_flush_tlb_gl { unsigned long flags; - if (kaiser_enabled) { - /* Globals are not used at all */ - __native_flush_tlb(); - return; - } - if (this_cpu_has(X86_FEATURE_INVPCID)) { /* * Using INVPCID is considerably faster than a pair of writes @@ -256,11 +239,7 @@ static inline void __native_flush_tlb_si static inline void __flush_tlb_all(void) { - if (boot_cpu_has(X86_FEATURE_PGE)) - __flush_tlb_global(); - else - __flush_tlb(); - + __flush_tlb_global(); /* * Note: if we somehow had PCID but not PGE, then this wouldn't work -- * we'd end up flushing kernel translations for the current ASID but Patches currently in stable-queue which might be from hughd(a)google.com are queue-4.9/kaiser-vmstat-show-nr_kaisertable-as-nr_overhead.patch queue-4.9/kaiser-add-nokaiser-boot-option-using-alternative.patch queue-4.9/kaiser-fix-unlikely-error-in-alloc_ldt_struct.patch queue-4.9/kaiser-kaiser_flush_tlb_on_return_to_user-check-pcid.patch queue-4.9/x86-paravirt-dont-patch-flush_tlb_single.patch queue-4.9/kaiser-merged-update.patch queue-4.9/kaiser-delete-kaiser_real_switch-option.patch queue-4.9/kaiser-kaiser_remove_mapping-move-along-the-pgd.patch queue-4.9/kaiser-fix-perf-crashes.patch queue-4.9/kaiser-drop-is_atomic-arg-to-kaiser_pagetable_walk.patch queue-4.9/kaiser-load_new_mm_cr3-let-switch_user_cr3-flush-user.patch queue-4.9/kaiser-enhanced-by-kernel-and-user-pcids.patch queue-4.9/kaiser-x86_cr3_pcid_noflush-and-x86_cr3_pcid_user.patch queue-4.9/kaiser-align-addition-to-x86-mm-makefile.patch queue-4.9/kaiser-use-alternative-instead-of-x86_cr3_pcid_noflush.patch queue-4.9/kaiser-stack-map-page_size-at-thread_size-page_size.patch queue-4.9/kaiser-name-that-0x1000-kaiser_shadow_pgd_offset.patch queue-4.9/kaiser-fix-regs-to-do_nmi-ifndef-config_kaiser.patch queue-4.9/kaiser-do-not-set-_page_nx-on-pgd_none.patch queue-4.9/kaiser-tidied-up-asm-kaiser.h-somewhat.patch queue-4.9/kaiser-cleanups-while-trying-for-gold-link.patch queue-4.9/kaiser-tidied-up-kaiser_add-remove_mapping-slightly.patch queue-4.9/kaiser-fix-build-and-fixme-in-alloc_ldt_struct.patch queue-4.9/kaiser-kernel-address-isolation.patch queue-4.9/kaiser-enomem-if-kaiser_pagetable_walk-null.patch queue-4.9/kaiser-asm-tlbflush.h-handle-nopge-at-lower-level.patch queue-4.9/kaiser-paranoid_entry-pass-cr3-need-to-paranoid_exit.patch queue-4.9/kaiser-kaiser-depends-on-smp.patch queue-4.9/kaiser-pcid-0-for-kernel-and-128-for-user.patch

7 years, 8 months

1
0
0 0

[Linux-stable-mirror] Patch "kaiser: delete KAISER_REAL_SWITCH option" has been added to the 4.9-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled kaiser: delete KAISER_REAL_SWITCH option to the 4.9-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: kaiser-delete-kaiser_real_switch-option.patch and it can be found in the queue-4.9 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From foo@baz Wed Jan 3 20:37:21 CET 2018 From: Hugh Dickins <hughd(a)google.com> Date: Sun, 3 Sep 2017 18:30:43 -0700 Subject: kaiser: delete KAISER_REAL_SWITCH option From: Hugh Dickins <hughd(a)google.com> We fail to see what CONFIG_KAISER_REAL_SWITCH is for: it seems to be left over from early development, and now just obscures tricky parts of the code. Delete it before adding PCIDs, or nokaiser boot option. (Or if there is some good reason to keep the option, then it needs a help text - and a "depends on KAISER", so that all those without KAISER are not asked the question. But we'd much rather delete it.) Signed-off-by: Hugh Dickins <hughd(a)google.com> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- arch/x86/entry/entry_64.S | 4 ---- arch/x86/include/asm/kaiser.h | 4 ---- security/Kconfig | 4 ---- 3 files changed, 12 deletions(-) --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -1317,9 +1317,7 @@ ENTRY(nmi) /* %rax is saved above, so OK to clobber here */ movq %cr3, %rax pushq %rax -#ifdef CONFIG_KAISER_REAL_SWITCH andq $(~KAISER_SHADOW_PGD_OFFSET), %rax -#endif movq %rax, %cr3 #endif call do_nmi @@ -1560,9 +1558,7 @@ end_repeat_nmi: /* %rax is saved above, so OK to clobber here */ movq %cr3, %rax pushq %rax -#ifdef CONFIG_KAISER_REAL_SWITCH andq $(~KAISER_SHADOW_PGD_OFFSET), %rax -#endif movq %rax, %cr3 #endif --- a/arch/x86/include/asm/kaiser.h +++ b/arch/x86/include/asm/kaiser.h @@ -21,17 +21,13 @@ .macro _SWITCH_TO_KERNEL_CR3 reg movq %cr3, \reg -#ifdef CONFIG_KAISER_REAL_SWITCH andq $(~KAISER_SHADOW_PGD_OFFSET), \reg -#endif movq \reg, %cr3 .endm .macro _SWITCH_TO_USER_CR3 reg movq %cr3, \reg -#ifdef CONFIG_KAISER_REAL_SWITCH orq $(KAISER_SHADOW_PGD_OFFSET), \reg -#endif movq \reg, %cr3 .endm --- a/security/Kconfig +++ b/security/Kconfig @@ -41,10 +41,6 @@ config KAISER If you are unsure how to answer this question, answer Y. -config KAISER_REAL_SWITCH - bool "KAISER: actually switch page tables" - default y - config SECURITYFS bool "Enable the securityfs filesystem" help Patches currently in stable-queue which might be from hughd(a)google.com are queue-4.9/kaiser-vmstat-show-nr_kaisertable-as-nr_overhead.patch queue-4.9/kaiser-add-nokaiser-boot-option-using-alternative.patch queue-4.9/kaiser-fix-unlikely-error-in-alloc_ldt_struct.patch queue-4.9/kaiser-kaiser_flush_tlb_on_return_to_user-check-pcid.patch queue-4.9/x86-paravirt-dont-patch-flush_tlb_single.patch queue-4.9/kaiser-merged-update.patch queue-4.9/kaiser-delete-kaiser_real_switch-option.patch queue-4.9/kaiser-kaiser_remove_mapping-move-along-the-pgd.patch queue-4.9/kaiser-fix-perf-crashes.patch queue-4.9/kaiser-drop-is_atomic-arg-to-kaiser_pagetable_walk.patch queue-4.9/kaiser-load_new_mm_cr3-let-switch_user_cr3-flush-user.patch queue-4.9/kaiser-enhanced-by-kernel-and-user-pcids.patch queue-4.9/kaiser-x86_cr3_pcid_noflush-and-x86_cr3_pcid_user.patch queue-4.9/kaiser-align-addition-to-x86-mm-makefile.patch queue-4.9/kaiser-use-alternative-instead-of-x86_cr3_pcid_noflush.patch queue-4.9/kaiser-stack-map-page_size-at-thread_size-page_size.patch queue-4.9/kaiser-name-that-0x1000-kaiser_shadow_pgd_offset.patch queue-4.9/kaiser-fix-regs-to-do_nmi-ifndef-config_kaiser.patch queue-4.9/kaiser-do-not-set-_page_nx-on-pgd_none.patch queue-4.9/kaiser-tidied-up-asm-kaiser.h-somewhat.patch queue-4.9/kaiser-cleanups-while-trying-for-gold-link.patch queue-4.9/kaiser-tidied-up-kaiser_add-remove_mapping-slightly.patch queue-4.9/kaiser-fix-build-and-fixme-in-alloc_ldt_struct.patch queue-4.9/kaiser-kernel-address-isolation.patch queue-4.9/kaiser-enomem-if-kaiser_pagetable_walk-null.patch queue-4.9/kaiser-asm-tlbflush.h-handle-nopge-at-lower-level.patch queue-4.9/kaiser-paranoid_entry-pass-cr3-need-to-paranoid_exit.patch queue-4.9/kaiser-kaiser-depends-on-smp.patch queue-4.9/kaiser-pcid-0-for-kernel-and-128-for-user.patch

7 years, 8 months

1
0
0 0

[Linux-stable-mirror] Patch "kaiser: cleanups while trying for gold link" has been added to the 4.9-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled kaiser: cleanups while trying for gold link to the 4.9-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: kaiser-cleanups-while-trying-for-gold-link.patch and it can be found in the queue-4.9 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From foo@baz Wed Jan 3 20:37:21 CET 2018 From: Hugh Dickins <hughd(a)google.com> Date: Mon, 21 Aug 2017 20:11:43 -0700 Subject: kaiser: cleanups while trying for gold link From: Hugh Dickins <hughd(a)google.com> While trying to get our gold link to work, four cleanups: matched the gdt_page declaration to its definition; in fiddling unsuccessfully with PERCPU_INPUT(), lined up backslashes; lined up the backslashes according to convention in percpu-defs.h; deleted the unused irq_stack_pointer addition to irq_stack_union. Sad to report that aligning backslashes does not appear to help gold align to 8192: but while these did not help, they are worth keeping. Signed-off-by: Hugh Dickins <hughd(a)google.com> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- arch/x86/include/asm/desc.h | 2 +- arch/x86/include/asm/processor.h | 5 ----- include/asm-generic/vmlinux.lds.h | 18 ++++++++---------- include/linux/percpu-defs.h | 24 ++++++++++++------------ 4 files changed, 21 insertions(+), 28 deletions(-) --- a/arch/x86/include/asm/desc.h +++ b/arch/x86/include/asm/desc.h @@ -43,7 +43,7 @@ struct gdt_page { struct desc_struct gdt[GDT_ENTRIES]; } __attribute__((aligned(PAGE_SIZE))); -DECLARE_PER_CPU_PAGE_ALIGNED(struct gdt_page, gdt_page); +DECLARE_PER_CPU_PAGE_ALIGNED_USER_MAPPED(struct gdt_page, gdt_page); static inline struct desc_struct *get_cpu_gdt_table(unsigned int cpu) { --- a/arch/x86/include/asm/processor.h +++ b/arch/x86/include/asm/processor.h @@ -335,11 +335,6 @@ union irq_stack_union { char gs_base[40]; unsigned long stack_canary; }; - - struct { - char irq_stack_pointer[64]; - char unused[IRQ_STACK_SIZE - 64]; - }; }; DECLARE_PER_CPU_FIRST(union irq_stack_union, irq_stack_union) __visible; --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -778,16 +778,14 @@ */ #define PERCPU_INPUT(cacheline) \ VMLINUX_SYMBOL(__per_cpu_start) = .; \ - \ - VMLINUX_SYMBOL(__per_cpu_user_mapped_start) = .; \ - *(.data..percpu..first) \ - . = ALIGN(cacheline); \ - *(.data..percpu..user_mapped) \ - *(.data..percpu..user_mapped..shared_aligned) \ - . = ALIGN(PAGE_SIZE); \ - *(.data..percpu..user_mapped..page_aligned) \ - VMLINUX_SYMBOL(__per_cpu_user_mapped_end) = .; \ - \ + VMLINUX_SYMBOL(__per_cpu_user_mapped_start) = .; \ + *(.data..percpu..first) \ + . = ALIGN(cacheline); \ + *(.data..percpu..user_mapped) \ + *(.data..percpu..user_mapped..shared_aligned) \ + . = ALIGN(PAGE_SIZE); \ + *(.data..percpu..user_mapped..page_aligned) \ + VMLINUX_SYMBOL(__per_cpu_user_mapped_end) = .; \ . = ALIGN(PAGE_SIZE); \ *(.data..percpu..page_aligned) \ . = ALIGN(cacheline); \ --- a/include/linux/percpu-defs.h +++ b/include/linux/percpu-defs.h @@ -121,10 +121,10 @@ #define DEFINE_PER_CPU(type, name) \ DEFINE_PER_CPU_SECTION(type, name, "") -#define DECLARE_PER_CPU_USER_MAPPED(type, name) \ +#define DECLARE_PER_CPU_USER_MAPPED(type, name) \ DECLARE_PER_CPU_SECTION(type, name, USER_MAPPED_SECTION) -#define DEFINE_PER_CPU_USER_MAPPED(type, name) \ +#define DEFINE_PER_CPU_USER_MAPPED(type, name) \ DEFINE_PER_CPU_SECTION(type, name, USER_MAPPED_SECTION) /* @@ -156,11 +156,11 @@ DEFINE_PER_CPU_SECTION(type, name, PER_CPU_SHARED_ALIGNED_SECTION) \ ____cacheline_aligned_in_smp -#define DECLARE_PER_CPU_SHARED_ALIGNED_USER_MAPPED(type, name) \ +#define DECLARE_PER_CPU_SHARED_ALIGNED_USER_MAPPED(type, name) \ DECLARE_PER_CPU_SECTION(type, name, USER_MAPPED_SECTION PER_CPU_SHARED_ALIGNED_SECTION) \ ____cacheline_aligned_in_smp -#define DEFINE_PER_CPU_SHARED_ALIGNED_USER_MAPPED(type, name) \ +#define DEFINE_PER_CPU_SHARED_ALIGNED_USER_MAPPED(type, name) \ DEFINE_PER_CPU_SECTION(type, name, USER_MAPPED_SECTION PER_CPU_SHARED_ALIGNED_SECTION) \ ____cacheline_aligned_in_smp @@ -185,18 +185,18 @@ /* * Declaration/definition used for per-CPU variables that must be page aligned and need to be mapped in user mode. */ -#define DECLARE_PER_CPU_PAGE_ALIGNED_USER_MAPPED(type, name) \ - DECLARE_PER_CPU_SECTION(type, name, USER_MAPPED_SECTION"..page_aligned") \ - __aligned(PAGE_SIZE) - -#define DEFINE_PER_CPU_PAGE_ALIGNED_USER_MAPPED(type, name) \ - DEFINE_PER_CPU_SECTION(type, name, USER_MAPPED_SECTION"..page_aligned") \ - __aligned(PAGE_SIZE) +#define DECLARE_PER_CPU_PAGE_ALIGNED_USER_MAPPED(type, name) \ + DECLARE_PER_CPU_SECTION(type, name, USER_MAPPED_SECTION"..page_aligned") \ + __aligned(PAGE_SIZE) + +#define DEFINE_PER_CPU_PAGE_ALIGNED_USER_MAPPED(type, name) \ + DEFINE_PER_CPU_SECTION(type, name, USER_MAPPED_SECTION"..page_aligned") \ + __aligned(PAGE_SIZE) /* * Declaration/definition used for per-CPU variables that must be read mostly. */ -#define DECLARE_PER_CPU_READ_MOSTLY(type, name) \ +#define DECLARE_PER_CPU_READ_MOSTLY(type, name) \ DECLARE_PER_CPU_SECTION(type, name, "..read_mostly") #define DEFINE_PER_CPU_READ_MOSTLY(type, name) \ Patches currently in stable-queue which might be from hughd(a)google.com are queue-4.9/kaiser-vmstat-show-nr_kaisertable-as-nr_overhead.patch queue-4.9/kaiser-add-nokaiser-boot-option-using-alternative.patch queue-4.9/kaiser-fix-unlikely-error-in-alloc_ldt_struct.patch queue-4.9/kaiser-kaiser_flush_tlb_on_return_to_user-check-pcid.patch queue-4.9/x86-paravirt-dont-patch-flush_tlb_single.patch queue-4.9/kaiser-merged-update.patch queue-4.9/kaiser-delete-kaiser_real_switch-option.patch queue-4.9/kaiser-kaiser_remove_mapping-move-along-the-pgd.patch queue-4.9/kaiser-fix-perf-crashes.patch queue-4.9/kaiser-drop-is_atomic-arg-to-kaiser_pagetable_walk.patch queue-4.9/kaiser-load_new_mm_cr3-let-switch_user_cr3-flush-user.patch queue-4.9/kaiser-enhanced-by-kernel-and-user-pcids.patch queue-4.9/kaiser-x86_cr3_pcid_noflush-and-x86_cr3_pcid_user.patch queue-4.9/kaiser-align-addition-to-x86-mm-makefile.patch queue-4.9/kaiser-use-alternative-instead-of-x86_cr3_pcid_noflush.patch queue-4.9/kaiser-stack-map-page_size-at-thread_size-page_size.patch queue-4.9/kaiser-name-that-0x1000-kaiser_shadow_pgd_offset.patch queue-4.9/kaiser-fix-regs-to-do_nmi-ifndef-config_kaiser.patch queue-4.9/kaiser-do-not-set-_page_nx-on-pgd_none.patch queue-4.9/kaiser-tidied-up-asm-kaiser.h-somewhat.patch queue-4.9/kaiser-cleanups-while-trying-for-gold-link.patch queue-4.9/kaiser-tidied-up-kaiser_add-remove_mapping-slightly.patch queue-4.9/kaiser-fix-build-and-fixme-in-alloc_ldt_struct.patch queue-4.9/kaiser-kernel-address-isolation.patch queue-4.9/kaiser-enomem-if-kaiser_pagetable_walk-null.patch queue-4.9/kaiser-asm-tlbflush.h-handle-nopge-at-lower-level.patch queue-4.9/kaiser-paranoid_entry-pass-cr3-need-to-paranoid_exit.patch queue-4.9/kaiser-kaiser-depends-on-smp.patch queue-4.9/kaiser-pcid-0-for-kernel-and-128-for-user.patch

7 years, 8 months

1
0
0 0

[Linux-stable-mirror] Patch "kaiser: align addition to x86/mm/Makefile" has been added to the 4.9-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled kaiser: align addition to x86/mm/Makefile to the 4.9-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: kaiser-align-addition-to-x86-mm-makefile.patch and it can be found in the queue-4.9 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From foo@baz Wed Jan 3 20:37:21 CET 2018 From: Hugh Dickins <hughd(a)google.com> Date: Sun, 3 Sep 2017 19:51:10 -0700 Subject: kaiser: align addition to x86/mm/Makefile From: Hugh Dickins <hughd(a)google.com> Use tab not space so they line up properly, kaslr.o also. Signed-off-by: Hugh Dickins <hughd(a)google.com> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- arch/x86/mm/Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/arch/x86/mm/Makefile +++ b/arch/x86/mm/Makefile @@ -37,5 +37,5 @@ obj-$(CONFIG_NUMA_EMU) += numa_emulatio obj-$(CONFIG_X86_INTEL_MPX) += mpx.o obj-$(CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS) += pkeys.o -obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o -obj-$(CONFIG_KAISER) += kaiser.o +obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o +obj-$(CONFIG_KAISER) += kaiser.o Patches currently in stable-queue which might be from hughd(a)google.com are queue-4.9/kaiser-vmstat-show-nr_kaisertable-as-nr_overhead.patch queue-4.9/kaiser-add-nokaiser-boot-option-using-alternative.patch queue-4.9/kaiser-fix-unlikely-error-in-alloc_ldt_struct.patch queue-4.9/kaiser-kaiser_flush_tlb_on_return_to_user-check-pcid.patch queue-4.9/x86-paravirt-dont-patch-flush_tlb_single.patch queue-4.9/kaiser-merged-update.patch queue-4.9/kaiser-delete-kaiser_real_switch-option.patch queue-4.9/kaiser-kaiser_remove_mapping-move-along-the-pgd.patch queue-4.9/kaiser-fix-perf-crashes.patch queue-4.9/kaiser-drop-is_atomic-arg-to-kaiser_pagetable_walk.patch queue-4.9/kaiser-load_new_mm_cr3-let-switch_user_cr3-flush-user.patch queue-4.9/kaiser-enhanced-by-kernel-and-user-pcids.patch queue-4.9/kaiser-x86_cr3_pcid_noflush-and-x86_cr3_pcid_user.patch queue-4.9/kaiser-align-addition-to-x86-mm-makefile.patch queue-4.9/kaiser-use-alternative-instead-of-x86_cr3_pcid_noflush.patch queue-4.9/kaiser-stack-map-page_size-at-thread_size-page_size.patch queue-4.9/kaiser-name-that-0x1000-kaiser_shadow_pgd_offset.patch queue-4.9/kaiser-fix-regs-to-do_nmi-ifndef-config_kaiser.patch queue-4.9/kaiser-do-not-set-_page_nx-on-pgd_none.patch queue-4.9/kaiser-tidied-up-asm-kaiser.h-somewhat.patch queue-4.9/kaiser-cleanups-while-trying-for-gold-link.patch queue-4.9/kaiser-tidied-up-kaiser_add-remove_mapping-slightly.patch queue-4.9/kaiser-fix-build-and-fixme-in-alloc_ldt_struct.patch queue-4.9/kaiser-kernel-address-isolation.patch queue-4.9/kaiser-enomem-if-kaiser_pagetable_walk-null.patch queue-4.9/kaiser-asm-tlbflush.h-handle-nopge-at-lower-level.patch queue-4.9/kaiser-paranoid_entry-pass-cr3-need-to-paranoid_exit.patch queue-4.9/kaiser-kaiser-depends-on-smp.patch queue-4.9/kaiser-pcid-0-for-kernel-and-128-for-user.patch

7 years, 8 months

1
0
0 0

[Linux-stable-mirror] Patch "kaiser: add "nokaiser" boot option, using ALTERNATIVE" has been added to the 4.9-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled kaiser: add "nokaiser" boot option, using ALTERNATIVE to the 4.9-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: kaiser-add-nokaiser-boot-option-using-alternative.patch and it can be found in the queue-4.9 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From foo@baz Wed Jan 3 20:37:21 CET 2018 From: Hugh Dickins <hughd(a)google.com> Date: Sun, 24 Sep 2017 16:59:49 -0700 Subject: kaiser: add "nokaiser" boot option, using ALTERNATIVE From: Hugh Dickins <hughd(a)google.com> Added "nokaiser" boot option: an early param like "noinvpcid". Most places now check int kaiser_enabled (#defined 0 when not CONFIG_KAISER) instead of #ifdef CONFIG_KAISER; but entry_64.S and entry_64_compat.S are using the ALTERNATIVE technique, which patches in the preferred instructions at runtime. That technique is tied to x86 cpu features, so X86_FEATURE_KAISER is fabricated. Prior to "nokaiser", Kaiser #defined _PAGE_GLOBAL 0: revert that, but be careful with both _PAGE_GLOBAL and CR4.PGE: setting them when nokaiser like when !CONFIG_KAISER, but not setting either when kaiser - neither matters on its own, but it's hard to be sure that _PAGE_GLOBAL won't get set in some obscure corner, or something add PGE into CR4. By omitting _PAGE_GLOBAL from __supported_pte_mask when kaiser_enabled, all page table setup which uses pte_pfn() masks it out of the ptes. It's slightly shameful that the same declaration versus definition of kaiser_enabled appears in not one, not two, but in three header files (asm/kaiser.h, asm/pgtable.h, asm/tlbflush.h). I felt safer that way, than with #including any of those in any of the others; and did not feel it worth an asm/kaiser_enabled.h - kernel/cpu/common.c includes them all, so we shall hear about it if they get out of synch. Cleanups while in the area: removed the silly #ifdef CONFIG_KAISER from kaiser.c; removed the unused native_get_normal_pgd(); removed the spurious reg clutter from SWITCH_*_CR3 macro stubs; corrected some comments. But more interestingly, set CR4.PSE in secondary_startup_64: the manual is clear that it does not matter whether it's 0 or 1 when 4-level-pts are enabled, but I was distracted to find cr4 different on BSP and auxiliaries - BSP alone was adding PSE, in probe_page_size_mask(). Signed-off-by: Hugh Dickins <hughd(a)google.com> Acked-by: Jiri Kosina <jkosina(a)suse.cz> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- Documentation/kernel-parameters.txt | 2 + arch/x86/entry/entry_64.S | 15 ++++++----- arch/x86/include/asm/cpufeatures.h | 3 ++ arch/x86/include/asm/kaiser.h | 27 +++++++++++++++------ arch/x86/include/asm/pgtable.h | 20 +++++++++++---- arch/x86/include/asm/pgtable_64.h | 13 +++------- arch/x86/include/asm/pgtable_types.h | 4 --- arch/x86/include/asm/tlbflush.h | 39 +++++++++++++++++++------------ arch/x86/kernel/cpu/common.c | 28 +++++++++++++++++++++- arch/x86/kernel/espfix_64.c | 3 +- arch/x86/kernel/head_64.S | 4 +-- arch/x86/mm/init.c | 2 - arch/x86/mm/init_64.c | 10 +++++++ arch/x86/mm/kaiser.c | 26 +++++++++++++++++--- arch/x86/mm/pgtable.c | 8 +----- arch/x86/mm/tlb.c | 4 --- tools/arch/x86/include/asm/cpufeatures.h | 3 ++ 17 files changed, 146 insertions(+), 65 deletions(-) --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -2763,6 +2763,8 @@ bytes respectively. Such letter suffixes nojitter [IA-64] Disables jitter checking for ITC timers. + nokaiser [X86-64] Disable KAISER isolation of kernel from user. + no-kvmclock [X86,KVM] Disable paravirtualized KVM clock driver no-kvmapf [X86,KVM] Disable paravirtualized asynchronous page --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -1079,7 +1079,7 @@ ENTRY(paranoid_entry) * unconditionally, but we need to find out whether the reverse * should be done on return (conveyed to paranoid_exit in %ebx). */ - movq %cr3, %rax + ALTERNATIVE "jmp 2f", "movq %cr3, %rax", X86_FEATURE_KAISER testl $KAISER_SHADOW_PGD_OFFSET, %eax jz 2f orl $2, %ebx @@ -1111,6 +1111,7 @@ ENTRY(paranoid_exit) TRACE_IRQS_OFF_DEBUG TRACE_IRQS_IRETQ_DEBUG #ifdef CONFIG_KAISER + /* No ALTERNATIVE for X86_FEATURE_KAISER: paranoid_entry sets %ebx */ testl $2, %ebx /* SWITCH_USER_CR3 needed? */ jz paranoid_exit_no_switch SWITCH_USER_CR3 @@ -1341,13 +1342,14 @@ ENTRY(nmi) #ifdef CONFIG_KAISER /* Unconditionally use kernel CR3 for do_nmi() */ /* %rax is saved above, so OK to clobber here */ - movq %cr3, %rax + ALTERNATIVE "jmp 2f", "movq %cr3, %rax", X86_FEATURE_KAISER /* If PCID enabled, NOFLUSH now and NOFLUSH on return */ orq x86_cr3_pcid_noflush, %rax pushq %rax /* mask off "user" bit of pgd address and 12 PCID bits: */ andq $(~(X86_CR3_PCID_ASID_MASK | KAISER_SHADOW_PGD_OFFSET)), %rax movq %rax, %cr3 +2: #endif call do_nmi @@ -1357,8 +1359,7 @@ ENTRY(nmi) * kernel code that needs user CR3, but do we ever return * to "user mode" where we need the kernel CR3? */ - popq %rax - mov %rax, %cr3 + ALTERNATIVE "", "popq %rax; movq %rax, %cr3", X86_FEATURE_KAISER #endif /* @@ -1585,13 +1586,14 @@ end_repeat_nmi: #ifdef CONFIG_KAISER /* Unconditionally use kernel CR3 for do_nmi() */ /* %rax is saved above, so OK to clobber here */ - movq %cr3, %rax + ALTERNATIVE "jmp 2f", "movq %cr3, %rax", X86_FEATURE_KAISER /* If PCID enabled, NOFLUSH now and NOFLUSH on return */ orq x86_cr3_pcid_noflush, %rax pushq %rax /* mask off "user" bit of pgd address and 12 PCID bits: */ andq $(~(X86_CR3_PCID_ASID_MASK | KAISER_SHADOW_PGD_OFFSET)), %rax movq %rax, %cr3 +2: #endif /* paranoidentry do_nmi, 0; without TRACE_IRQS_OFF */ @@ -1603,8 +1605,7 @@ end_repeat_nmi: * kernel code that needs user CR3, like just just before * a sysret. */ - popq %rax - mov %rax, %cr3 + ALTERNATIVE "", "popq %rax; movq %rax, %cr3", X86_FEATURE_KAISER #endif testl %ebx, %ebx /* swapgs needed? */ --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -198,6 +198,9 @@ #define X86_FEATURE_AVX512_4VNNIW (7*32+16) /* AVX-512 Neural Network Instructions */ #define X86_FEATURE_AVX512_4FMAPS (7*32+17) /* AVX-512 Multiply Accumulation Single precision */ +/* Because the ALTERNATIVE scheme is for members of the X86_FEATURE club... */ +#define X86_FEATURE_KAISER ( 7*32+31) /* CONFIG_KAISER w/o nokaiser */ + /* Virtualization flags: Linux defined, word 8 */ #define X86_FEATURE_TPR_SHADOW ( 8*32+ 0) /* Intel TPR Shadow */ #define X86_FEATURE_VNMI ( 8*32+ 1) /* Intel Virtual NMI */ --- a/arch/x86/include/asm/kaiser.h +++ b/arch/x86/include/asm/kaiser.h @@ -46,28 +46,33 @@ movq \reg, %cr3 .endm .macro SWITCH_KERNEL_CR3 -pushq %rax +ALTERNATIVE "jmp 8f", "pushq %rax", X86_FEATURE_KAISER _SWITCH_TO_KERNEL_CR3 %rax popq %rax +8: .endm .macro SWITCH_USER_CR3 -pushq %rax +ALTERNATIVE "jmp 8f", "pushq %rax", X86_FEATURE_KAISER _SWITCH_TO_USER_CR3 %rax %al popq %rax +8: .endm .macro SWITCH_KERNEL_CR3_NO_STACK -movq %rax, PER_CPU_VAR(unsafe_stack_register_backup) +ALTERNATIVE "jmp 8f", \ + __stringify(movq %rax, PER_CPU_VAR(unsafe_stack_register_backup)), \ + X86_FEATURE_KAISER _SWITCH_TO_KERNEL_CR3 %rax movq PER_CPU_VAR(unsafe_stack_register_backup), %rax +8: .endm #else /* CONFIG_KAISER */ -.macro SWITCH_KERNEL_CR3 reg +.macro SWITCH_KERNEL_CR3 .endm -.macro SWITCH_USER_CR3 reg regb +.macro SWITCH_USER_CR3 .endm .macro SWITCH_KERNEL_CR3_NO_STACK .endm @@ -90,6 +95,16 @@ DECLARE_PER_CPU(unsigned long, x86_cr3_p extern char __per_cpu_user_mapped_start[], __per_cpu_user_mapped_end[]; +extern int kaiser_enabled; +#else +#define kaiser_enabled 0 +#endif /* CONFIG_KAISER */ + +/* + * Kaiser function prototypes are needed even when CONFIG_KAISER is not set, + * so as to build with tests on kaiser_enabled instead of #ifdefs. + */ + /** * kaiser_add_mapping - map a virtual memory part to the shadow (user) mapping * @addr: the start address of the range @@ -119,8 +134,6 @@ extern void kaiser_remove_mapping(unsign */ extern void kaiser_init(void); -#endif /* CONFIG_KAISER */ - #endif /* __ASSEMBLY */ #endif /* _ASM_X86_KAISER_H */ --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -18,6 +18,12 @@ #ifndef __ASSEMBLY__ #include <asm/x86_init.h> +#ifdef CONFIG_KAISER +extern int kaiser_enabled; +#else +#define kaiser_enabled 0 +#endif + void ptdump_walk_pgd_level(struct seq_file *m, pgd_t *pgd); void ptdump_walk_pgd_level_checkwx(void); @@ -697,7 +703,7 @@ static inline int pgd_bad(pgd_t pgd) * page table by accident; it will fault on the first * instruction it tries to run. See native_set_pgd(). */ - if (IS_ENABLED(CONFIG_KAISER)) + if (kaiser_enabled) ignore_flags |= _PAGE_NX; return (pgd_flags(pgd) & ~ignore_flags) != _KERNPG_TABLE; @@ -913,12 +919,14 @@ static inline void pmdp_set_wrprotect(st */ static inline void clone_pgd_range(pgd_t *dst, pgd_t *src, int count) { - memcpy(dst, src, count * sizeof(pgd_t)); + memcpy(dst, src, count * sizeof(pgd_t)); #ifdef CONFIG_KAISER - /* Clone the shadow pgd part as well */ - memcpy(native_get_shadow_pgd(dst), - native_get_shadow_pgd(src), - count * sizeof(pgd_t)); + if (kaiser_enabled) { + /* Clone the shadow pgd part as well */ + memcpy(native_get_shadow_pgd(dst), + native_get_shadow_pgd(src), + count * sizeof(pgd_t)); + } #endif } --- a/arch/x86/include/asm/pgtable_64.h +++ b/arch/x86/include/asm/pgtable_64.h @@ -111,13 +111,12 @@ extern pgd_t kaiser_set_shadow_pgd(pgd_t static inline pgd_t *native_get_shadow_pgd(pgd_t *pgdp) { +#ifdef CONFIG_DEBUG_VM + /* linux/mmdebug.h may not have been included at this point */ + BUG_ON(!kaiser_enabled); +#endif return (pgd_t *)((unsigned long)pgdp | (unsigned long)PAGE_SIZE); } - -static inline pgd_t *native_get_normal_pgd(pgd_t *pgdp) -{ - return (pgd_t *)((unsigned long)pgdp & ~(unsigned long)PAGE_SIZE); -} #else static inline pgd_t kaiser_set_shadow_pgd(pgd_t *pgdp, pgd_t pgd) { @@ -128,10 +127,6 @@ static inline pgd_t *native_get_shadow_p BUILD_BUG_ON(1); return NULL; } -static inline pgd_t *native_get_normal_pgd(pgd_t *pgdp) -{ - return pgdp; -} #endif /* CONFIG_KAISER */ static inline void native_set_pgd(pgd_t *pgdp, pgd_t pgd) --- a/arch/x86/include/asm/pgtable_types.h +++ b/arch/x86/include/asm/pgtable_types.h @@ -45,11 +45,7 @@ #define _PAGE_ACCESSED (_AT(pteval_t, 1) << _PAGE_BIT_ACCESSED) #define _PAGE_DIRTY (_AT(pteval_t, 1) << _PAGE_BIT_DIRTY) #define _PAGE_PSE (_AT(pteval_t, 1) << _PAGE_BIT_PSE) -#ifdef CONFIG_KAISER -#define _PAGE_GLOBAL (_AT(pteval_t, 0)) -#else #define _PAGE_GLOBAL (_AT(pteval_t, 1) << _PAGE_BIT_GLOBAL) -#endif #define _PAGE_SOFTW1 (_AT(pteval_t, 1) << _PAGE_BIT_SOFTW1) #define _PAGE_SOFTW2 (_AT(pteval_t, 1) << _PAGE_BIT_SOFTW2) #define _PAGE_PAT (_AT(pteval_t, 1) << _PAGE_BIT_PAT) --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -137,9 +137,11 @@ static inline void cr4_set_bits_and_upda * to avoid the need for asm/kaiser.h in unexpected places. */ #ifdef CONFIG_KAISER +extern int kaiser_enabled; extern void kaiser_setup_pcid(void); extern void kaiser_flush_tlb_on_return_to_user(void); #else +#define kaiser_enabled 0 static inline void kaiser_setup_pcid(void) { } @@ -164,7 +166,7 @@ static inline void __native_flush_tlb(vo * back: */ preempt_disable(); - if (this_cpu_has(X86_FEATURE_PCID)) + if (kaiser_enabled && this_cpu_has(X86_FEATURE_PCID)) kaiser_flush_tlb_on_return_to_user(); native_write_cr3(native_read_cr3()); preempt_enable(); @@ -175,20 +177,30 @@ static inline void __native_flush_tlb_gl unsigned long cr4; cr4 = this_cpu_read(cpu_tlbstate.cr4); - /* clear PGE */ - native_write_cr4(cr4 & ~X86_CR4_PGE); - /* write old PGE again and flush TLBs */ - native_write_cr4(cr4); + if (cr4 & X86_CR4_PGE) { + /* clear PGE and flush TLB of all entries */ + native_write_cr4(cr4 & ~X86_CR4_PGE); + /* restore PGE as it was before */ + native_write_cr4(cr4); + } else { + /* + * x86_64 microcode update comes this way when CR4.PGE is not + * enabled, and it's safer for all callers to allow this case. + */ + native_write_cr3(native_read_cr3()); + } } static inline void __native_flush_tlb_global(void) { -#ifdef CONFIG_KAISER - /* Globals are not used at all */ - __native_flush_tlb(); -#else unsigned long flags; + if (kaiser_enabled) { + /* Globals are not used at all */ + __native_flush_tlb(); + return; + } + if (this_cpu_has(X86_FEATURE_INVPCID)) { /* * Using INVPCID is considerably faster than a pair of writes @@ -208,7 +220,6 @@ static inline void __native_flush_tlb_gl raw_local_irq_save(flags); __native_flush_tlb_global_irq_disabled(); raw_local_irq_restore(flags); -#endif } static inline void __native_flush_tlb_single(unsigned long addr) @@ -223,7 +234,7 @@ static inline void __native_flush_tlb_si */ if (!this_cpu_has(X86_FEATURE_INVPCID_SINGLE)) { - if (this_cpu_has(X86_FEATURE_PCID)) + if (kaiser_enabled && this_cpu_has(X86_FEATURE_PCID)) kaiser_flush_tlb_on_return_to_user(); asm volatile("invlpg (%0)" ::"r" (addr) : "memory"); return; @@ -238,9 +249,9 @@ static inline void __native_flush_tlb_si * Make sure to do only a single invpcid when KAISER is * disabled and we have only a single ASID. */ - if (X86_CR3_PCID_ASID_KERN != X86_CR3_PCID_ASID_USER) - invpcid_flush_one(X86_CR3_PCID_ASID_KERN, addr); - invpcid_flush_one(X86_CR3_PCID_ASID_USER, addr); + if (kaiser_enabled) + invpcid_flush_one(X86_CR3_PCID_ASID_USER, addr); + invpcid_flush_one(X86_CR3_PCID_ASID_KERN, addr); } static inline void __flush_tlb_all(void) --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -179,6 +179,20 @@ static int __init x86_pcid_setup(char *s return 1; } __setup("nopcid", x86_pcid_setup); + +static int __init x86_nokaiser_setup(char *s) +{ + /* nokaiser doesn't accept parameters */ + if (s) + return -EINVAL; +#ifdef CONFIG_KAISER + kaiser_enabled = 0; + setup_clear_cpu_cap(X86_FEATURE_KAISER); + pr_info("nokaiser: KAISER feature disabled\n"); +#endif + return 0; +} +early_param("nokaiser", x86_nokaiser_setup); #endif static int __init x86_noinvpcid_setup(char *s) @@ -327,7 +341,7 @@ static __always_inline void setup_smap(s static void setup_pcid(struct cpuinfo_x86 *c) { if (cpu_has(c, X86_FEATURE_PCID)) { - if (cpu_has(c, X86_FEATURE_PGE)) { + if (cpu_has(c, X86_FEATURE_PGE) || kaiser_enabled) { cr4_set_bits(X86_CR4_PCIDE); /* * INVPCID has two "groups" of types: @@ -799,6 +813,10 @@ void get_cpu_cap(struct cpuinfo_x86 *c) c->x86_capability[CPUID_8000_000A_EDX] = cpuid_edx(0x8000000a); init_scattered_cpuid_features(c); +#ifdef CONFIG_KAISER + if (kaiser_enabled) + set_cpu_cap(c, X86_FEATURE_KAISER); +#endif } static void identify_cpu_without_cpuid(struct cpuinfo_x86 *c) @@ -1537,6 +1555,14 @@ void cpu_init(void) * try to read it. */ cr4_init_shadow(); + if (!kaiser_enabled) { + /* + * secondary_startup_64() deferred setting PGE in cr4: + * probe_page_size_mask() sets it on the boot cpu, + * but it needs to be set on each secondary cpu. + */ + cr4_set_bits(X86_CR4_PGE); + } /* * Load microcode on this cpu if a valid microcode is available. --- a/arch/x86/kernel/espfix_64.c +++ b/arch/x86/kernel/espfix_64.c @@ -132,9 +132,10 @@ void __init init_espfix_bsp(void) * area to ensure it is mapped into the shadow user page * tables. */ - if (IS_ENABLED(CONFIG_KAISER)) + if (kaiser_enabled) { set_pgd(native_get_shadow_pgd(pgd_p), __pgd(_KERNPG_TABLE | __pa((pud_t *)espfix_pud_page))); + } /* Randomize the locations */ init_espfix_random(); --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -190,8 +190,8 @@ ENTRY(secondary_startup_64) movq $(init_level4_pgt - __START_KERNEL_map), %rax 1: - /* Enable PAE mode and PGE */ - movl $(X86_CR4_PAE | X86_CR4_PGE), %ecx + /* Enable PAE and PSE, but defer PGE until kaiser_enabled is decided */ + movl $(X86_CR4_PAE | X86_CR4_PSE), %ecx movq %rcx, %cr4 /* Setup early boot stage 4 level pagetables. */ --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -177,7 +177,7 @@ static void __init probe_page_size_mask( cr4_set_bits_and_update_boot(X86_CR4_PSE); /* Enable PGE if available */ - if (boot_cpu_has(X86_FEATURE_PGE)) { + if (boot_cpu_has(X86_FEATURE_PGE) && !kaiser_enabled) { cr4_set_bits_and_update_boot(X86_CR4_PGE); __supported_pte_mask |= _PAGE_GLOBAL; } else --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -324,6 +324,16 @@ void __init cleanup_highmap(void) continue; if (vaddr < (unsigned long) _text || vaddr > end) set_pmd(pmd, __pmd(0)); + else if (kaiser_enabled) { + /* + * level2_kernel_pgt is initialized with _PAGE_GLOBAL: + * clear that now. This is not important, so long as + * CR4.PGE remains clear, but it removes an anomaly. + * Physical mapping setup below avoids _PAGE_GLOBAL + * by use of massage_pgprot() inside pfn_pte() etc. + */ + set_pmd(pmd, pmd_clear_flags(*pmd, _PAGE_GLOBAL)); + } } } --- a/arch/x86/mm/kaiser.c +++ b/arch/x86/mm/kaiser.c @@ -16,7 +16,9 @@ #include <asm/pgalloc.h> #include <asm/desc.h> -#ifdef CONFIG_KAISER +int kaiser_enabled __read_mostly = 1; +EXPORT_SYMBOL(kaiser_enabled); /* for inlined TLB flush functions */ + __visible DEFINE_PER_CPU_USER_MAPPED(unsigned long, unsafe_stack_register_backup); @@ -167,8 +169,8 @@ static pte_t *kaiser_pagetable_walk(unsi return pte_offset_kernel(pmd, address); } -int kaiser_add_user_map(const void *__start_addr, unsigned long size, - unsigned long flags) +static int kaiser_add_user_map(const void *__start_addr, unsigned long size, + unsigned long flags) { int ret = 0; pte_t *pte; @@ -177,6 +179,15 @@ int kaiser_add_user_map(const void *__st unsigned long end_addr = PAGE_ALIGN(start_addr + size); unsigned long target_address; + /* + * It is convenient for callers to pass in __PAGE_KERNEL etc, + * and there is no actual harm from setting _PAGE_GLOBAL, so + * long as CR4.PGE is not set. But it is nonetheless troubling + * to see Kaiser itself setting _PAGE_GLOBAL (now that "nokaiser" + * requires that not to be #defined to 0): so mask it off here. + */ + flags &= ~_PAGE_GLOBAL; + for (; address < end_addr; address += PAGE_SIZE) { target_address = get_pa_from_mapping(address); if (target_address == -1) { @@ -263,6 +274,8 @@ void __init kaiser_init(void) { int cpu; + if (!kaiser_enabled) + return; kaiser_init_all_pgds(); for_each_possible_cpu(cpu) { @@ -311,6 +324,8 @@ void __init kaiser_init(void) /* Add a mapping to the shadow mapping, and synchronize the mappings */ int kaiser_add_mapping(unsigned long addr, unsigned long size, unsigned long flags) { + if (!kaiser_enabled) + return 0; return kaiser_add_user_map((const void *)addr, size, flags); } @@ -322,6 +337,8 @@ void kaiser_remove_mapping(unsigned long unsigned long addr, next; pgd_t *pgd; + if (!kaiser_enabled) + return; pgd = native_get_shadow_pgd(pgd_offset_k(start)); for (addr = start; addr < end; pgd++, addr = next) { next = pgd_addr_end(addr, end); @@ -343,6 +360,8 @@ static inline bool is_userspace_pgd(pgd_ pgd_t kaiser_set_shadow_pgd(pgd_t *pgdp, pgd_t pgd) { + if (!kaiser_enabled) + return pgd; /* * Do we need to also populate the shadow pgd? Check _PAGE_USER to * skip cases like kexec and EFI which make temporary low mappings. @@ -399,4 +418,3 @@ void kaiser_flush_tlb_on_return_to_user( X86_CR3_PCID_USER_FLUSH | KAISER_SHADOW_PGD_OFFSET); } EXPORT_SYMBOL(kaiser_flush_tlb_on_return_to_user); -#endif /* CONFIG_KAISER */ --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -345,16 +345,12 @@ static inline void _pgd_free(pgd_t *pgd) } #else -#ifdef CONFIG_KAISER /* - * Instead of one pmd, we aquire two pmds. Being order-1, it is + * Instead of one pgd, Kaiser acquires two pgds. Being order-1, it is * both 8k in size and 8k-aligned. That lets us just flip bit 12 * in a pointer to swap between the two 4k halves. */ -#define PGD_ALLOCATION_ORDER 1 -#else -#define PGD_ALLOCATION_ORDER 0 -#endif +#define PGD_ALLOCATION_ORDER kaiser_enabled static inline pgd_t *_pgd_alloc(void) { --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -39,8 +39,7 @@ static void load_new_mm_cr3(pgd_t *pgdir { unsigned long new_mm_cr3 = __pa(pgdir); -#ifdef CONFIG_KAISER - if (this_cpu_has(X86_FEATURE_PCID)) { + if (kaiser_enabled && this_cpu_has(X86_FEATURE_PCID)) { /* * We reuse the same PCID for different tasks, so we must * flush all the entries for the PCID out when we change tasks. @@ -57,7 +56,6 @@ static void load_new_mm_cr3(pgd_t *pgdir new_mm_cr3 |= X86_CR3_PCID_KERN_FLUSH; kaiser_flush_tlb_on_return_to_user(); } -#endif /* CONFIG_KAISER */ /* * Caution: many callers of this function expect --- a/tools/arch/x86/include/asm/cpufeatures.h +++ b/tools/arch/x86/include/asm/cpufeatures.h @@ -197,6 +197,9 @@ #define X86_FEATURE_AVX512_4VNNIW (7*32+16) /* AVX-512 Neural Network Instructions */ #define X86_FEATURE_AVX512_4FMAPS (7*32+17) /* AVX-512 Multiply Accumulation Single precision */ +/* Because the ALTERNATIVE scheme is for members of the X86_FEATURE club... */ +#define X86_FEATURE_KAISER ( 7*32+31) /* CONFIG_KAISER w/o nokaiser */ + /* Virtualization flags: Linux defined, word 8 */ #define X86_FEATURE_TPR_SHADOW ( 8*32+ 0) /* Intel TPR Shadow */ #define X86_FEATURE_VNMI ( 8*32+ 1) /* Intel Virtual NMI */ Patches currently in stable-queue which might be from hughd(a)google.com are queue-4.9/kaiser-vmstat-show-nr_kaisertable-as-nr_overhead.patch queue-4.9/kaiser-add-nokaiser-boot-option-using-alternative.patch queue-4.9/kaiser-fix-unlikely-error-in-alloc_ldt_struct.patch queue-4.9/kaiser-kaiser_flush_tlb_on_return_to_user-check-pcid.patch queue-4.9/x86-paravirt-dont-patch-flush_tlb_single.patch queue-4.9/kaiser-merged-update.patch queue-4.9/kaiser-delete-kaiser_real_switch-option.patch queue-4.9/kaiser-kaiser_remove_mapping-move-along-the-pgd.patch queue-4.9/kaiser-fix-perf-crashes.patch queue-4.9/kaiser-drop-is_atomic-arg-to-kaiser_pagetable_walk.patch queue-4.9/kaiser-load_new_mm_cr3-let-switch_user_cr3-flush-user.patch queue-4.9/kaiser-enhanced-by-kernel-and-user-pcids.patch queue-4.9/kaiser-x86_cr3_pcid_noflush-and-x86_cr3_pcid_user.patch queue-4.9/kaiser-align-addition-to-x86-mm-makefile.patch queue-4.9/kaiser-use-alternative-instead-of-x86_cr3_pcid_noflush.patch queue-4.9/kaiser-stack-map-page_size-at-thread_size-page_size.patch queue-4.9/kaiser-name-that-0x1000-kaiser_shadow_pgd_offset.patch queue-4.9/kaiser-fix-regs-to-do_nmi-ifndef-config_kaiser.patch queue-4.9/kaiser-do-not-set-_page_nx-on-pgd_none.patch queue-4.9/kaiser-tidied-up-asm-kaiser.h-somewhat.patch queue-4.9/kaiser-cleanups-while-trying-for-gold-link.patch queue-4.9/kaiser-tidied-up-kaiser_add-remove_mapping-slightly.patch queue-4.9/kaiser-fix-build-and-fixme-in-alloc_ldt_struct.patch queue-4.9/kaiser-kernel-address-isolation.patch queue-4.9/kaiser-enomem-if-kaiser_pagetable_walk-null.patch queue-4.9/kaiser-asm-tlbflush.h-handle-nopge-at-lower-level.patch queue-4.9/kaiser-paranoid_entry-pass-cr3-need-to-paranoid_exit.patch queue-4.9/kaiser-kaiser-depends-on-smp.patch queue-4.9/kaiser-pcid-0-for-kernel-and-128-for-user.patch

7 years, 8 months

1
0
0 0

[Linux-stable-mirror] Patch "x86/paravirt: Dont patch flush_tlb_single" has been added to the 4.4-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled x86/paravirt: Dont patch flush_tlb_single to the 4.4-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: x86-paravirt-dont-patch-flush_tlb_single.patch and it can be found in the queue-4.4 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From foo@baz Wed Jan 3 18:58:12 CET 2018 From: Thomas Gleixner <tglx(a)linutronix.de> Date: Mon, 4 Dec 2017 15:07:30 +0100 Subject: x86/paravirt: Dont patch flush_tlb_single From: Thomas Gleixner <tglx(a)linutronix.de> commit a035795499ca1c2bd1928808d1a156eda1420383 upstream native_flush_tlb_single() will be changed with the upcoming PAGE_TABLE_ISOLATION feature. This requires to have more code in there than INVLPG. Remove the paravirt patching for it. Signed-off-by: Thomas Gleixner <tglx(a)linutronix.de> Reviewed-by: Josh Poimboeuf <jpoimboe(a)redhat.com> Reviewed-by: Juergen Gross <jgross(a)suse.com> Acked-by: Peter Zijlstra <peterz(a)infradead.org> Cc: Andy Lutomirski <luto(a)kernel.org> Cc: Boris Ostrovsky <boris.ostrovsky(a)oracle.com> Cc: Borislav Petkov <bp(a)alien8.de> Cc: Borislav Petkov <bpetkov(a)suse.de> Cc: Brian Gerst <brgerst(a)gmail.com> Cc: Dave Hansen <dave.hansen(a)intel.com> Cc: Dave Hansen <dave.hansen(a)linux.intel.com> Cc: David Laight <David.Laight(a)aculab.com> Cc: Denys Vlasenko <dvlasenk(a)redhat.com> Cc: Eduardo Valentin <eduval(a)amazon.com> Cc: Greg KH <gregkh(a)linuxfoundation.org> Cc: H. Peter Anvin <hpa(a)zytor.com> Cc: Linus Torvalds <torvalds(a)linux-foundation.org> Cc: Rik van Riel <riel(a)redhat.com> Cc: Will Deacon <will.deacon(a)arm.com> Cc: aliguori(a)amazon.com Cc: daniel.gruss(a)iaik.tugraz.at Cc: hughd(a)google.com Cc: keescook(a)google.com Cc: linux-mm(a)kvack.org Cc: michael.schwarz(a)iaik.tugraz.at Cc: moritz.lipp(a)iaik.tugraz.at Cc: richard.fellner(a)student.tugraz.at Link: https://lkml.kernel.org/r/20171204150606.828111617@linutronix.de Signed-off-by: Ingo Molnar <mingo(a)kernel.org> Acked-by: Borislav Petkov <bp(a)suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- arch/x86/kernel/paravirt_patch_64.c | 2 -- 1 file changed, 2 deletions(-) --- a/arch/x86/kernel/paravirt_patch_64.c +++ b/arch/x86/kernel/paravirt_patch_64.c @@ -9,7 +9,6 @@ DEF_NATIVE(pv_irq_ops, save_fl, "pushfq; DEF_NATIVE(pv_mmu_ops, read_cr2, "movq %cr2, %rax"); DEF_NATIVE(pv_mmu_ops, read_cr3, "movq %cr3, %rax"); DEF_NATIVE(pv_mmu_ops, write_cr3, "movq %rdi, %cr3"); -DEF_NATIVE(pv_mmu_ops, flush_tlb_single, "invlpg (%rdi)"); DEF_NATIVE(pv_cpu_ops, clts, "clts"); DEF_NATIVE(pv_cpu_ops, wbinvd, "wbinvd"); @@ -62,7 +61,6 @@ unsigned native_patch(u8 type, u16 clobb PATCH_SITE(pv_mmu_ops, read_cr3); PATCH_SITE(pv_mmu_ops, write_cr3); PATCH_SITE(pv_cpu_ops, clts); - PATCH_SITE(pv_mmu_ops, flush_tlb_single); PATCH_SITE(pv_cpu_ops, wbinvd); #if defined(CONFIG_PARAVIRT_SPINLOCKS) && defined(CONFIG_QUEUED_SPINLOCKS) case PARAVIRT_PATCH(pv_lock_ops.queued_spin_unlock): Patches currently in stable-queue which might be from tglx(a)linutronix.de are queue-4.4/x86-paravirt-dont-patch-flush_tlb_single.patch queue-4.4/x86-boot-add-early-cmdline-parsing-for-options-with-arguments.patch

7 years, 8 months

1
0
0 0

[Linux-stable-mirror] Patch "x86/kaiser: Move feature detection up" has been added to the 4.4-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled x86/kaiser: Move feature detection up to the 4.4-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: x86-kaiser-move-feature-detection-up.patch and it can be found in the queue-4.4 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From foo@baz Wed Jan 3 18:58:12 CET 2018 From: Borislav Petkov <bp(a)suse.de> Date: Mon, 25 Dec 2017 13:57:16 +0100 Subject: x86/kaiser: Move feature detection up From: Borislav Petkov <bp(a)suse.de> ... before the first use of kaiser_enabled as otherwise funky things happen: about to get started... (XEN) d0v0 Unhandled page fault fault/trap [#14, ec=0000] (XEN) Pagetable walk from ffff88022a449090: (XEN) L4[0x110] = 0000000229e0e067 0000000000001e0e (XEN) L3[0x008] = 0000000000000000 ffffffffffffffff (XEN) domain_crash_sync called from entry.S: fault at ffff82d08033fd08 entry.o#create_bounce_frame+0x135/0x14d (XEN) Domain 0 (vcpu#0) crashed on cpu#0: (XEN) ----[ Xen-4.9.1_02-3.21 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e033:[<ffffffff81007460>] (XEN) RFLAGS: 0000000000000286 EM: 1 CONTEXT: pv guest (d0v0) Signed-off-by: Borislav Petkov <bp(a)suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- arch/x86/include/asm/kaiser.h | 2 ++ arch/x86/kernel/setup.c | 7 +++++++ arch/x86/mm/kaiser.c | 2 -- 3 files changed, 9 insertions(+), 2 deletions(-) --- a/arch/x86/include/asm/kaiser.h +++ b/arch/x86/include/asm/kaiser.h @@ -96,8 +96,10 @@ DECLARE_PER_CPU(unsigned long, x86_cr3_p extern char __per_cpu_user_mapped_start[], __per_cpu_user_mapped_end[]; extern int kaiser_enabled; +extern void __init kaiser_check_boottime_disable(void); #else #define kaiser_enabled 0 +static inline void __init kaiser_check_boottime_disable(void) {} #endif /* CONFIG_KAISER */ /* --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -112,6 +112,7 @@ #include <asm/alternative.h> #include <asm/prom.h> #include <asm/microcode.h> +#include <asm/kaiser.h> /* * max_low_pfn_mapped: highest direct mapped pfn under 4GB @@ -1016,6 +1017,12 @@ void __init setup_arch(char **cmdline_p) */ init_hypervisor_platform(); + /* + * This needs to happen right after XENPV is set on xen and + * kaiser_enabled is checked below in cleanup_highmap(). + */ + kaiser_check_boottime_disable(); + x86_init.resources.probe_roms(); /* after parse_early_param, so could debug it */ --- a/arch/x86/mm/kaiser.c +++ b/arch/x86/mm/kaiser.c @@ -311,8 +311,6 @@ void __init kaiser_init(void) { int cpu; - kaiser_check_boottime_disable(); - if (!kaiser_enabled) return; Patches currently in stable-queue which might be from bp(a)suse.de are queue-4.4/x86-paravirt-dont-patch-flush_tlb_single.patch queue-4.4/x86-kaiser-reenable-paravirt.patch queue-4.4/x86-kaiser-rename-and-simplify-x86_feature_kaiser-handling.patch queue-4.4/x86-kaiser-check-boottime-cmdline-params.patch queue-4.4/x86-kaiser-move-feature-detection-up.patch

7 years, 8 months

1
0
0 0

[Linux-stable-mirror] Patch "x86/kaiser: Reenable PARAVIRT" has been added to the 4.4-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled x86/kaiser: Reenable PARAVIRT to the 4.4-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: x86-kaiser-reenable-paravirt.patch and it can be found in the queue-4.4 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From foo@baz Wed Jan 3 18:58:12 CET 2018 From: Borislav Petkov <bp(a)suse.de> Date: Tue, 2 Jan 2018 14:19:49 +0100 Subject: x86/kaiser: Reenable PARAVIRT From: Borislav Petkov <bp(a)suse.de> Now that the required bits have been addressed, reenable PARAVIRT. Signed-off-by: Borislav Petkov <bp(a)suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- security/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/security/Kconfig +++ b/security/Kconfig @@ -34,7 +34,7 @@ config SECURITY config KAISER bool "Remove the kernel mapping in user mode" default y - depends on X86_64 && SMP && !PARAVIRT + depends on X86_64 && SMP help This enforces a strict kernel and user space isolation, in order to close hardware side channels on kernel address information. Patches currently in stable-queue which might be from bp(a)suse.de are queue-4.4/x86-paravirt-dont-patch-flush_tlb_single.patch queue-4.4/x86-kaiser-reenable-paravirt.patch queue-4.4/x86-kaiser-rename-and-simplify-x86_feature_kaiser-handling.patch queue-4.4/x86-kaiser-check-boottime-cmdline-params.patch queue-4.4/x86-kaiser-move-feature-detection-up.patch

7 years, 8 months

1
0
0 0

[Linux-stable-mirror] Patch "x86/kaiser: Check boottime cmdline params" has been added to the 4.4-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled x86/kaiser: Check boottime cmdline params to the 4.4-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: x86-kaiser-check-boottime-cmdline-params.patch and it can be found in the queue-4.4 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From foo@baz Wed Jan 3 18:58:12 CET 2018 From: Borislav Petkov <bp(a)suse.de> Date: Tue, 2 Jan 2018 14:19:48 +0100 Subject: x86/kaiser: Check boottime cmdline params From: Borislav Petkov <bp(a)suse.de> AMD (and possibly other vendors) are not affected by the leak KAISER is protecting against. Keep the "nopti" for traditional reasons and add pti=<on|off|auto> like upstream. Signed-off-by: Borislav Petkov <bp(a)suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- Documentation/kernel-parameters.txt | 6 +++ arch/x86/mm/kaiser.c | 59 +++++++++++++++++++++++++----------- 2 files changed, 47 insertions(+), 18 deletions(-) --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -3056,6 +3056,12 @@ bytes respectively. Such letter suffixes pt. [PARIDE] See Documentation/blockdev/paride.txt. + pti= [X86_64] + Control KAISER user/kernel address space isolation: + on - enable + off - disable + auto - default setting + pty.legacy_count= [KNL] Number of legacy pty's. Overwrites compiled-in default number. --- a/arch/x86/mm/kaiser.c +++ b/arch/x86/mm/kaiser.c @@ -16,6 +16,7 @@ #include <asm/pgtable.h> #include <asm/pgalloc.h> #include <asm/desc.h> +#include <asm/cmdline.h> int kaiser_enabled __read_mostly = 1; EXPORT_SYMBOL(kaiser_enabled); /* for inlined TLB flush functions */ @@ -264,6 +265,43 @@ static void __init kaiser_init_all_pgds( WARN_ON(__ret); \ } while (0) +void __init kaiser_check_boottime_disable(void) +{ + bool enable = true; + char arg[5]; + int ret; + + ret = cmdline_find_option(boot_command_line, "pti", arg, sizeof(arg)); + if (ret > 0) { + if (!strncmp(arg, "on", 2)) + goto enable; + + if (!strncmp(arg, "off", 3)) + goto disable; + + if (!strncmp(arg, "auto", 4)) + goto skip; + } + + if (cmdline_find_option_bool(boot_command_line, "nopti")) + goto disable; + +skip: + if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD) + goto disable; + +enable: + if (enable) + setup_force_cpu_cap(X86_FEATURE_KAISER); + + return; + +disable: + pr_info("Kernel/User page tables isolation: disabled\n"); + kaiser_enabled = 0; + setup_clear_cpu_cap(X86_FEATURE_KAISER); +} + /* * If anything in here fails, we will likely die on one of the * first kernel->user transitions and init will die. But, we @@ -275,12 +313,10 @@ void __init kaiser_init(void) { int cpu; - if (!kaiser_enabled) { - setup_clear_cpu_cap(X86_FEATURE_KAISER); - return; - } + kaiser_check_boottime_disable(); - setup_force_cpu_cap(X86_FEATURE_KAISER); + if (!kaiser_enabled) + return; kaiser_init_all_pgds(); @@ -424,16 +460,3 @@ void kaiser_flush_tlb_on_return_to_user( X86_CR3_PCID_USER_FLUSH | KAISER_SHADOW_PGD_OFFSET); } EXPORT_SYMBOL(kaiser_flush_tlb_on_return_to_user); - -static int __init x86_nokaiser_setup(char *s) -{ - /* nopti doesn't accept parameters */ - if (s) - return -EINVAL; - - kaiser_enabled = 0; - pr_info("Kernel/User page tables isolation: disabled\n"); - - return 0; -} -early_param("nopti", x86_nokaiser_setup); Patches currently in stable-queue which might be from bp(a)suse.de are queue-4.4/x86-paravirt-dont-patch-flush_tlb_single.patch queue-4.4/x86-kaiser-reenable-paravirt.patch queue-4.4/x86-kaiser-rename-and-simplify-x86_feature_kaiser-handling.patch queue-4.4/x86-kaiser-check-boottime-cmdline-params.patch queue-4.4/x86-kaiser-move-feature-detection-up.patch

7 years, 8 months

1
0
0 0

[Linux-stable-mirror] Patch "kaiser: x86_cr3_pcid_noflush and x86_cr3_pcid_user" has been added to the 4.4-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled kaiser: x86_cr3_pcid_noflush and x86_cr3_pcid_user to the 4.4-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: kaiser-x86_cr3_pcid_noflush-and-x86_cr3_pcid_user.patch and it can be found in the queue-4.4 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From foo@baz Wed Jan 3 18:58:12 CET 2018 From: Hugh Dickins <hughd(a)google.com> Date: Sun, 27 Aug 2017 16:24:27 -0700 Subject: kaiser: x86_cr3_pcid_noflush and x86_cr3_pcid_user From: Hugh Dickins <hughd(a)google.com> Mostly this commit is just unshouting X86_CR3_PCID_KERN_VAR and X86_CR3_PCID_USER_VAR: we usually name variables in lower-case. But why does x86_cr3_pcid_noflush need to be __aligned(PAGE_SIZE)? Ah, it's a leftover from when kaiser_add_user_map() once complained about mapping the same page twice. Make it __read_mostly instead. (I'm a little uneasy about all the unrelated data which shares its page getting user-mapped too, but that was so before, and not a big deal: though we call it user-mapped, it's not mapped with _PAGE_USER.) And there is a little change around the two calls to do_nmi(). Previously they set the NOFLUSH bit (if PCID supported) when forcing to kernel context before do_nmi(); now they also have the NOFLUSH bit set (if PCID supported) when restoring context after: nothing done in do_nmi() should require a TLB to be flushed here. Signed-off-by: Hugh Dickins <hughd(a)google.com> Acked-by: Jiri Kosina <jkosina(a)suse.cz> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- arch/x86/entry/entry_64.S | 8 ++++---- arch/x86/include/asm/kaiser.h | 11 +++++------ arch/x86/mm/kaiser.c | 13 +++++++------ 3 files changed, 16 insertions(+), 16 deletions(-) --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -1290,11 +1290,11 @@ ENTRY(nmi) /* Unconditionally use kernel CR3 for do_nmi() */ /* %rax is saved above, so OK to clobber here */ movq %cr3, %rax + /* If PCID enabled, NOFLUSH now and NOFLUSH on return */ + orq x86_cr3_pcid_noflush, %rax pushq %rax /* mask off "user" bit of pgd address and 12 PCID bits: */ andq $(~(X86_CR3_PCID_ASID_MASK | KAISER_SHADOW_PGD_OFFSET)), %rax - /* Add back kernel PCID and "no flush" bit */ - orq X86_CR3_PCID_KERN_VAR, %rax movq %rax, %cr3 #endif call do_nmi @@ -1534,11 +1534,11 @@ end_repeat_nmi: /* Unconditionally use kernel CR3 for do_nmi() */ /* %rax is saved above, so OK to clobber here */ movq %cr3, %rax + /* If PCID enabled, NOFLUSH now and NOFLUSH on return */ + orq x86_cr3_pcid_noflush, %rax pushq %rax /* mask off "user" bit of pgd address and 12 PCID bits: */ andq $(~(X86_CR3_PCID_ASID_MASK | KAISER_SHADOW_PGD_OFFSET)), %rax - /* Add back kernel PCID and "no flush" bit */ - orq X86_CR3_PCID_KERN_VAR, %rax movq %rax, %cr3 #endif --- a/arch/x86/include/asm/kaiser.h +++ b/arch/x86/include/asm/kaiser.h @@ -25,7 +25,7 @@ .macro _SWITCH_TO_KERNEL_CR3 reg movq %cr3, \reg andq $(~(X86_CR3_PCID_ASID_MASK | KAISER_SHADOW_PGD_OFFSET)), \reg -orq X86_CR3_PCID_KERN_VAR, \reg +orq x86_cr3_pcid_noflush, \reg movq \reg, %cr3 .endm @@ -37,11 +37,10 @@ movq \reg, %cr3 * not enabled): so that the one register can update both memory and cr3. */ movq %cr3, \reg -andq $(~(X86_CR3_PCID_ASID_MASK | KAISER_SHADOW_PGD_OFFSET)), \reg -orq PER_CPU_VAR(X86_CR3_PCID_USER_VAR), \reg +orq PER_CPU_VAR(x86_cr3_pcid_user), \reg js 9f /* FLUSH this time, reset to NOFLUSH for next time (if PCID enabled) */ -movb \regb, PER_CPU_VAR(X86_CR3_PCID_USER_VAR+7) +movb \regb, PER_CPU_VAR(x86_cr3_pcid_user+7) 9: movq \reg, %cr3 .endm @@ -94,8 +93,8 @@ movq PER_CPU_VAR(unsafe_stack_register_b */ DECLARE_PER_CPU_USER_MAPPED(unsigned long, unsafe_stack_register_backup); -extern unsigned long X86_CR3_PCID_KERN_VAR; -DECLARE_PER_CPU(unsigned long, X86_CR3_PCID_USER_VAR); +extern unsigned long x86_cr3_pcid_noflush; +DECLARE_PER_CPU(unsigned long, x86_cr3_pcid_user); extern char __per_cpu_user_mapped_start[], __per_cpu_user_mapped_end[]; --- a/arch/x86/mm/kaiser.c +++ b/arch/x86/mm/kaiser.c @@ -29,8 +29,8 @@ DEFINE_PER_CPU_USER_MAPPED(unsigned long * This is also handy because systems that do not support PCIDs * just end up or'ing a 0 into their CR3, which does no harm. */ -__aligned(PAGE_SIZE) unsigned long X86_CR3_PCID_KERN_VAR; -DEFINE_PER_CPU(unsigned long, X86_CR3_PCID_USER_VAR); +unsigned long x86_cr3_pcid_noflush __read_mostly; +DEFINE_PER_CPU(unsigned long, x86_cr3_pcid_user); /* * At runtime, the only things we map are some things for CPU @@ -304,7 +304,8 @@ void __init kaiser_init(void) sizeof(gate_desc) * NR_VECTORS, __PAGE_KERNEL); - kaiser_add_user_map_early(&X86_CR3_PCID_KERN_VAR, PAGE_SIZE, + kaiser_add_user_map_early(&x86_cr3_pcid_noflush, + sizeof(x86_cr3_pcid_noflush), __PAGE_KERNEL); } @@ -384,8 +385,8 @@ void kaiser_setup_pcid(void) * These variables are used by the entry/exit * code to change PCID and pgd and TLB flushing. */ - X86_CR3_PCID_KERN_VAR = kern_cr3; - this_cpu_write(X86_CR3_PCID_USER_VAR, user_cr3); + x86_cr3_pcid_noflush = kern_cr3; + this_cpu_write(x86_cr3_pcid_user, user_cr3); } /* @@ -395,7 +396,7 @@ void kaiser_setup_pcid(void) */ void kaiser_flush_tlb_on_return_to_user(void) { - this_cpu_write(X86_CR3_PCID_USER_VAR, + this_cpu_write(x86_cr3_pcid_user, X86_CR3_PCID_USER_FLUSH | KAISER_SHADOW_PGD_OFFSET); } EXPORT_SYMBOL(kaiser_flush_tlb_on_return_to_user); Patches currently in stable-queue which might be from hughd(a)google.com are queue-4.4/kaiser-vmstat-show-nr_kaisertable-as-nr_overhead.patch queue-4.4/kaiser-add-nokaiser-boot-option-using-alternative.patch queue-4.4/kaiser-fix-unlikely-error-in-alloc_ldt_struct.patch queue-4.4/kaiser-_pgd_alloc-without-__gfp_repeat-to-avoid-stalls.patch queue-4.4/kaiser-kaiser_flush_tlb_on_return_to_user-check-pcid.patch queue-4.4/x86-paravirt-dont-patch-flush_tlb_single.patch queue-4.4/kaiser-merged-update.patch queue-4.4/kaiser-delete-kaiser_real_switch-option.patch queue-4.4/kaiser-kaiser_remove_mapping-move-along-the-pgd.patch queue-4.4/kaiser-fix-perf-crashes.patch queue-4.4/kaiser-drop-is_atomic-arg-to-kaiser_pagetable_walk.patch queue-4.4/kaiser-load_new_mm_cr3-let-switch_user_cr3-flush-user.patch queue-4.4/kaiser-enhanced-by-kernel-and-user-pcids.patch queue-4.4/kaiser-x86_cr3_pcid_noflush-and-x86_cr3_pcid_user.patch queue-4.4/kaiser-use-alternative-instead-of-x86_cr3_pcid_noflush.patch queue-4.4/kaiser-stack-map-page_size-at-thread_size-page_size.patch queue-4.4/kaiser-name-that-0x1000-kaiser_shadow_pgd_offset.patch queue-4.4/kaiser-fix-regs-to-do_nmi-ifndef-config_kaiser.patch queue-4.4/kaiser-do-not-set-_page_nx-on-pgd_none.patch queue-4.4/kaiser-tidied-up-asm-kaiser.h-somewhat.patch queue-4.4/kaiser-cleanups-while-trying-for-gold-link.patch queue-4.4/kaiser-tidied-up-kaiser_add-remove_mapping-slightly.patch queue-4.4/kaiser-fix-build-and-fixme-in-alloc_ldt_struct.patch queue-4.4/kaiser-kernel-address-isolation.patch queue-4.4/kaiser-enomem-if-kaiser_pagetable_walk-null.patch queue-4.4/kaiser-asm-tlbflush.h-handle-nopge-at-lower-level.patch queue-4.4/kaiser-paranoid_entry-pass-cr3-need-to-paranoid_exit.patch queue-4.4/kaiser-kaiser-depends-on-smp.patch queue-4.4/kaiser-pcid-0-for-kernel-and-128-for-user.patch

7 years, 8 months

1
0
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-stable-mirror