January 2018 - Linux-stable-mirror

[Linux-stable-mirror] Patch "kaiser: align addition to x86/mm/Makefile" has been added to the 4.9-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled kaiser: align addition to x86/mm/Makefile to the 4.9-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: kaiser-align-addition-to-x86-mm-makefile.patch and it can be found in the queue-4.9 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From foo@baz Wed Jan 3 20:37:21 CET 2018 From: Hugh Dickins <hughd(a)google.com> Date: Sun, 3 Sep 2017 19:51:10 -0700 Subject: kaiser: align addition to x86/mm/Makefile From: Hugh Dickins <hughd(a)google.com> Use tab not space so they line up properly, kaslr.o also. Signed-off-by: Hugh Dickins <hughd(a)google.com> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- arch/x86/mm/Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/arch/x86/mm/Makefile +++ b/arch/x86/mm/Makefile @@ -37,5 +37,5 @@ obj-$(CONFIG_NUMA_EMU) += numa_emulatio obj-$(CONFIG_X86_INTEL_MPX) += mpx.o obj-$(CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS) += pkeys.o -obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o -obj-$(CONFIG_KAISER) += kaiser.o +obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o +obj-$(CONFIG_KAISER) += kaiser.o Patches currently in stable-queue which might be from hughd(a)google.com are queue-4.9/kaiser-vmstat-show-nr_kaisertable-as-nr_overhead.patch queue-4.9/kaiser-add-nokaiser-boot-option-using-alternative.patch queue-4.9/kaiser-fix-unlikely-error-in-alloc_ldt_struct.patch queue-4.9/kaiser-kaiser_flush_tlb_on_return_to_user-check-pcid.patch queue-4.9/x86-paravirt-dont-patch-flush_tlb_single.patch queue-4.9/kaiser-merged-update.patch queue-4.9/kaiser-delete-kaiser_real_switch-option.patch queue-4.9/kaiser-kaiser_remove_mapping-move-along-the-pgd.patch queue-4.9/kaiser-fix-perf-crashes.patch queue-4.9/kaiser-drop-is_atomic-arg-to-kaiser_pagetable_walk.patch queue-4.9/kaiser-load_new_mm_cr3-let-switch_user_cr3-flush-user.patch queue-4.9/kaiser-enhanced-by-kernel-and-user-pcids.patch queue-4.9/kaiser-x86_cr3_pcid_noflush-and-x86_cr3_pcid_user.patch queue-4.9/kaiser-align-addition-to-x86-mm-makefile.patch queue-4.9/kaiser-use-alternative-instead-of-x86_cr3_pcid_noflush.patch queue-4.9/kaiser-stack-map-page_size-at-thread_size-page_size.patch queue-4.9/kaiser-name-that-0x1000-kaiser_shadow_pgd_offset.patch queue-4.9/kaiser-fix-regs-to-do_nmi-ifndef-config_kaiser.patch queue-4.9/kaiser-do-not-set-_page_nx-on-pgd_none.patch queue-4.9/kaiser-tidied-up-asm-kaiser.h-somewhat.patch queue-4.9/kaiser-cleanups-while-trying-for-gold-link.patch queue-4.9/kaiser-tidied-up-kaiser_add-remove_mapping-slightly.patch queue-4.9/kaiser-fix-build-and-fixme-in-alloc_ldt_struct.patch queue-4.9/kaiser-kernel-address-isolation.patch queue-4.9/kaiser-enomem-if-kaiser_pagetable_walk-null.patch queue-4.9/kaiser-asm-tlbflush.h-handle-nopge-at-lower-level.patch queue-4.9/kaiser-paranoid_entry-pass-cr3-need-to-paranoid_exit.patch queue-4.9/kaiser-kaiser-depends-on-smp.patch queue-4.9/kaiser-pcid-0-for-kernel-and-128-for-user.patch

6 years, 11 months

1
0
0 0

[Linux-stable-mirror] Patch "kaiser: add "nokaiser" boot option, using ALTERNATIVE" has been added to the 4.9-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled kaiser: add "nokaiser" boot option, using ALTERNATIVE to the 4.9-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: kaiser-add-nokaiser-boot-option-using-alternative.patch and it can be found in the queue-4.9 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From foo@baz Wed Jan 3 20:37:21 CET 2018 From: Hugh Dickins <hughd(a)google.com> Date: Sun, 24 Sep 2017 16:59:49 -0700 Subject: kaiser: add "nokaiser" boot option, using ALTERNATIVE From: Hugh Dickins <hughd(a)google.com> Added "nokaiser" boot option: an early param like "noinvpcid". Most places now check int kaiser_enabled (#defined 0 when not CONFIG_KAISER) instead of #ifdef CONFIG_KAISER; but entry_64.S and entry_64_compat.S are using the ALTERNATIVE technique, which patches in the preferred instructions at runtime. That technique is tied to x86 cpu features, so X86_FEATURE_KAISER is fabricated. Prior to "nokaiser", Kaiser #defined _PAGE_GLOBAL 0: revert that, but be careful with both _PAGE_GLOBAL and CR4.PGE: setting them when nokaiser like when !CONFIG_KAISER, but not setting either when kaiser - neither matters on its own, but it's hard to be sure that _PAGE_GLOBAL won't get set in some obscure corner, or something add PGE into CR4. By omitting _PAGE_GLOBAL from __supported_pte_mask when kaiser_enabled, all page table setup which uses pte_pfn() masks it out of the ptes. It's slightly shameful that the same declaration versus definition of kaiser_enabled appears in not one, not two, but in three header files (asm/kaiser.h, asm/pgtable.h, asm/tlbflush.h). I felt safer that way, than with #including any of those in any of the others; and did not feel it worth an asm/kaiser_enabled.h - kernel/cpu/common.c includes them all, so we shall hear about it if they get out of synch. Cleanups while in the area: removed the silly #ifdef CONFIG_KAISER from kaiser.c; removed the unused native_get_normal_pgd(); removed the spurious reg clutter from SWITCH_*_CR3 macro stubs; corrected some comments. But more interestingly, set CR4.PSE in secondary_startup_64: the manual is clear that it does not matter whether it's 0 or 1 when 4-level-pts are enabled, but I was distracted to find cr4 different on BSP and auxiliaries - BSP alone was adding PSE, in probe_page_size_mask(). Signed-off-by: Hugh Dickins <hughd(a)google.com> Acked-by: Jiri Kosina <jkosina(a)suse.cz> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- Documentation/kernel-parameters.txt | 2 + arch/x86/entry/entry_64.S | 15 ++++++----- arch/x86/include/asm/cpufeatures.h | 3 ++ arch/x86/include/asm/kaiser.h | 27 +++++++++++++++------ arch/x86/include/asm/pgtable.h | 20 +++++++++++---- arch/x86/include/asm/pgtable_64.h | 13 +++------- arch/x86/include/asm/pgtable_types.h | 4 --- arch/x86/include/asm/tlbflush.h | 39 +++++++++++++++++++------------ arch/x86/kernel/cpu/common.c | 28 +++++++++++++++++++++- arch/x86/kernel/espfix_64.c | 3 +- arch/x86/kernel/head_64.S | 4 +-- arch/x86/mm/init.c | 2 - arch/x86/mm/init_64.c | 10 +++++++ arch/x86/mm/kaiser.c | 26 +++++++++++++++++--- arch/x86/mm/pgtable.c | 8 +----- arch/x86/mm/tlb.c | 4 --- tools/arch/x86/include/asm/cpufeatures.h | 3 ++ 17 files changed, 146 insertions(+), 65 deletions(-) --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -2763,6 +2763,8 @@ bytes respectively. Such letter suffixes nojitter [IA-64] Disables jitter checking for ITC timers. + nokaiser [X86-64] Disable KAISER isolation of kernel from user. + no-kvmclock [X86,KVM] Disable paravirtualized KVM clock driver no-kvmapf [X86,KVM] Disable paravirtualized asynchronous page --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -1079,7 +1079,7 @@ ENTRY(paranoid_entry) * unconditionally, but we need to find out whether the reverse * should be done on return (conveyed to paranoid_exit in %ebx). */ - movq %cr3, %rax + ALTERNATIVE "jmp 2f", "movq %cr3, %rax", X86_FEATURE_KAISER testl $KAISER_SHADOW_PGD_OFFSET, %eax jz 2f orl $2, %ebx @@ -1111,6 +1111,7 @@ ENTRY(paranoid_exit) TRACE_IRQS_OFF_DEBUG TRACE_IRQS_IRETQ_DEBUG #ifdef CONFIG_KAISER + /* No ALTERNATIVE for X86_FEATURE_KAISER: paranoid_entry sets %ebx */ testl $2, %ebx /* SWITCH_USER_CR3 needed? */ jz paranoid_exit_no_switch SWITCH_USER_CR3 @@ -1341,13 +1342,14 @@ ENTRY(nmi) #ifdef CONFIG_KAISER /* Unconditionally use kernel CR3 for do_nmi() */ /* %rax is saved above, so OK to clobber here */ - movq %cr3, %rax + ALTERNATIVE "jmp 2f", "movq %cr3, %rax", X86_FEATURE_KAISER /* If PCID enabled, NOFLUSH now and NOFLUSH on return */ orq x86_cr3_pcid_noflush, %rax pushq %rax /* mask off "user" bit of pgd address and 12 PCID bits: */ andq $(~(X86_CR3_PCID_ASID_MASK | KAISER_SHADOW_PGD_OFFSET)), %rax movq %rax, %cr3 +2: #endif call do_nmi @@ -1357,8 +1359,7 @@ ENTRY(nmi) * kernel code that needs user CR3, but do we ever return * to "user mode" where we need the kernel CR3? */ - popq %rax - mov %rax, %cr3 + ALTERNATIVE "", "popq %rax; movq %rax, %cr3", X86_FEATURE_KAISER #endif /* @@ -1585,13 +1586,14 @@ end_repeat_nmi: #ifdef CONFIG_KAISER /* Unconditionally use kernel CR3 for do_nmi() */ /* %rax is saved above, so OK to clobber here */ - movq %cr3, %rax + ALTERNATIVE "jmp 2f", "movq %cr3, %rax", X86_FEATURE_KAISER /* If PCID enabled, NOFLUSH now and NOFLUSH on return */ orq x86_cr3_pcid_noflush, %rax pushq %rax /* mask off "user" bit of pgd address and 12 PCID bits: */ andq $(~(X86_CR3_PCID_ASID_MASK | KAISER_SHADOW_PGD_OFFSET)), %rax movq %rax, %cr3 +2: #endif /* paranoidentry do_nmi, 0; without TRACE_IRQS_OFF */ @@ -1603,8 +1605,7 @@ end_repeat_nmi: * kernel code that needs user CR3, like just just before * a sysret. */ - popq %rax - mov %rax, %cr3 + ALTERNATIVE "", "popq %rax; movq %rax, %cr3", X86_FEATURE_KAISER #endif testl %ebx, %ebx /* swapgs needed? */ --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -198,6 +198,9 @@ #define X86_FEATURE_AVX512_4VNNIW (7*32+16) /* AVX-512 Neural Network Instructions */ #define X86_FEATURE_AVX512_4FMAPS (7*32+17) /* AVX-512 Multiply Accumulation Single precision */ +/* Because the ALTERNATIVE scheme is for members of the X86_FEATURE club... */ +#define X86_FEATURE_KAISER ( 7*32+31) /* CONFIG_KAISER w/o nokaiser */ + /* Virtualization flags: Linux defined, word 8 */ #define X86_FEATURE_TPR_SHADOW ( 8*32+ 0) /* Intel TPR Shadow */ #define X86_FEATURE_VNMI ( 8*32+ 1) /* Intel Virtual NMI */ --- a/arch/x86/include/asm/kaiser.h +++ b/arch/x86/include/asm/kaiser.h @@ -46,28 +46,33 @@ movq \reg, %cr3 .endm .macro SWITCH_KERNEL_CR3 -pushq %rax +ALTERNATIVE "jmp 8f", "pushq %rax", X86_FEATURE_KAISER _SWITCH_TO_KERNEL_CR3 %rax popq %rax +8: .endm .macro SWITCH_USER_CR3 -pushq %rax +ALTERNATIVE "jmp 8f", "pushq %rax", X86_FEATURE_KAISER _SWITCH_TO_USER_CR3 %rax %al popq %rax +8: .endm .macro SWITCH_KERNEL_CR3_NO_STACK -movq %rax, PER_CPU_VAR(unsafe_stack_register_backup) +ALTERNATIVE "jmp 8f", \ + __stringify(movq %rax, PER_CPU_VAR(unsafe_stack_register_backup)), \ + X86_FEATURE_KAISER _SWITCH_TO_KERNEL_CR3 %rax movq PER_CPU_VAR(unsafe_stack_register_backup), %rax +8: .endm #else /* CONFIG_KAISER */ -.macro SWITCH_KERNEL_CR3 reg +.macro SWITCH_KERNEL_CR3 .endm -.macro SWITCH_USER_CR3 reg regb +.macro SWITCH_USER_CR3 .endm .macro SWITCH_KERNEL_CR3_NO_STACK .endm @@ -90,6 +95,16 @@ DECLARE_PER_CPU(unsigned long, x86_cr3_p extern char __per_cpu_user_mapped_start[], __per_cpu_user_mapped_end[]; +extern int kaiser_enabled; +#else +#define kaiser_enabled 0 +#endif /* CONFIG_KAISER */ + +/* + * Kaiser function prototypes are needed even when CONFIG_KAISER is not set, + * so as to build with tests on kaiser_enabled instead of #ifdefs. + */ + /** * kaiser_add_mapping - map a virtual memory part to the shadow (user) mapping * @addr: the start address of the range @@ -119,8 +134,6 @@ extern void kaiser_remove_mapping(unsign */ extern void kaiser_init(void); -#endif /* CONFIG_KAISER */ - #endif /* __ASSEMBLY */ #endif /* _ASM_X86_KAISER_H */ --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -18,6 +18,12 @@ #ifndef __ASSEMBLY__ #include <asm/x86_init.h> +#ifdef CONFIG_KAISER +extern int kaiser_enabled; +#else +#define kaiser_enabled 0 +#endif + void ptdump_walk_pgd_level(struct seq_file *m, pgd_t *pgd); void ptdump_walk_pgd_level_checkwx(void); @@ -697,7 +703,7 @@ static inline int pgd_bad(pgd_t pgd) * page table by accident; it will fault on the first * instruction it tries to run. See native_set_pgd(). */ - if (IS_ENABLED(CONFIG_KAISER)) + if (kaiser_enabled) ignore_flags |= _PAGE_NX; return (pgd_flags(pgd) & ~ignore_flags) != _KERNPG_TABLE; @@ -913,12 +919,14 @@ static inline void pmdp_set_wrprotect(st */ static inline void clone_pgd_range(pgd_t *dst, pgd_t *src, int count) { - memcpy(dst, src, count * sizeof(pgd_t)); + memcpy(dst, src, count * sizeof(pgd_t)); #ifdef CONFIG_KAISER - /* Clone the shadow pgd part as well */ - memcpy(native_get_shadow_pgd(dst), - native_get_shadow_pgd(src), - count * sizeof(pgd_t)); + if (kaiser_enabled) { + /* Clone the shadow pgd part as well */ + memcpy(native_get_shadow_pgd(dst), + native_get_shadow_pgd(src), + count * sizeof(pgd_t)); + } #endif } --- a/arch/x86/include/asm/pgtable_64.h +++ b/arch/x86/include/asm/pgtable_64.h @@ -111,13 +111,12 @@ extern pgd_t kaiser_set_shadow_pgd(pgd_t static inline pgd_t *native_get_shadow_pgd(pgd_t *pgdp) { +#ifdef CONFIG_DEBUG_VM + /* linux/mmdebug.h may not have been included at this point */ + BUG_ON(!kaiser_enabled); +#endif return (pgd_t *)((unsigned long)pgdp | (unsigned long)PAGE_SIZE); } - -static inline pgd_t *native_get_normal_pgd(pgd_t *pgdp) -{ - return (pgd_t *)((unsigned long)pgdp & ~(unsigned long)PAGE_SIZE); -} #else static inline pgd_t kaiser_set_shadow_pgd(pgd_t *pgdp, pgd_t pgd) { @@ -128,10 +127,6 @@ static inline pgd_t *native_get_shadow_p BUILD_BUG_ON(1); return NULL; } -static inline pgd_t *native_get_normal_pgd(pgd_t *pgdp) -{ - return pgdp; -} #endif /* CONFIG_KAISER */ static inline void native_set_pgd(pgd_t *pgdp, pgd_t pgd) --- a/arch/x86/include/asm/pgtable_types.h +++ b/arch/x86/include/asm/pgtable_types.h @@ -45,11 +45,7 @@ #define _PAGE_ACCESSED (_AT(pteval_t, 1) << _PAGE_BIT_ACCESSED) #define _PAGE_DIRTY (_AT(pteval_t, 1) << _PAGE_BIT_DIRTY) #define _PAGE_PSE (_AT(pteval_t, 1) << _PAGE_BIT_PSE) -#ifdef CONFIG_KAISER -#define _PAGE_GLOBAL (_AT(pteval_t, 0)) -#else #define _PAGE_GLOBAL (_AT(pteval_t, 1) << _PAGE_BIT_GLOBAL) -#endif #define _PAGE_SOFTW1 (_AT(pteval_t, 1) << _PAGE_BIT_SOFTW1) #define _PAGE_SOFTW2 (_AT(pteval_t, 1) << _PAGE_BIT_SOFTW2) #define _PAGE_PAT (_AT(pteval_t, 1) << _PAGE_BIT_PAT) --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -137,9 +137,11 @@ static inline void cr4_set_bits_and_upda * to avoid the need for asm/kaiser.h in unexpected places. */ #ifdef CONFIG_KAISER +extern int kaiser_enabled; extern void kaiser_setup_pcid(void); extern void kaiser_flush_tlb_on_return_to_user(void); #else +#define kaiser_enabled 0 static inline void kaiser_setup_pcid(void) { } @@ -164,7 +166,7 @@ static inline void __native_flush_tlb(vo * back: */ preempt_disable(); - if (this_cpu_has(X86_FEATURE_PCID)) + if (kaiser_enabled && this_cpu_has(X86_FEATURE_PCID)) kaiser_flush_tlb_on_return_to_user(); native_write_cr3(native_read_cr3()); preempt_enable(); @@ -175,20 +177,30 @@ static inline void __native_flush_tlb_gl unsigned long cr4; cr4 = this_cpu_read(cpu_tlbstate.cr4); - /* clear PGE */ - native_write_cr4(cr4 & ~X86_CR4_PGE); - /* write old PGE again and flush TLBs */ - native_write_cr4(cr4); + if (cr4 & X86_CR4_PGE) { + /* clear PGE and flush TLB of all entries */ + native_write_cr4(cr4 & ~X86_CR4_PGE); + /* restore PGE as it was before */ + native_write_cr4(cr4); + } else { + /* + * x86_64 microcode update comes this way when CR4.PGE is not + * enabled, and it's safer for all callers to allow this case. + */ + native_write_cr3(native_read_cr3()); + } } static inline void __native_flush_tlb_global(void) { -#ifdef CONFIG_KAISER - /* Globals are not used at all */ - __native_flush_tlb(); -#else unsigned long flags; + if (kaiser_enabled) { + /* Globals are not used at all */ + __native_flush_tlb(); + return; + } + if (this_cpu_has(X86_FEATURE_INVPCID)) { /* * Using INVPCID is considerably faster than a pair of writes @@ -208,7 +220,6 @@ static inline void __native_flush_tlb_gl raw_local_irq_save(flags); __native_flush_tlb_global_irq_disabled(); raw_local_irq_restore(flags); -#endif } static inline void __native_flush_tlb_single(unsigned long addr) @@ -223,7 +234,7 @@ static inline void __native_flush_tlb_si */ if (!this_cpu_has(X86_FEATURE_INVPCID_SINGLE)) { - if (this_cpu_has(X86_FEATURE_PCID)) + if (kaiser_enabled && this_cpu_has(X86_FEATURE_PCID)) kaiser_flush_tlb_on_return_to_user(); asm volatile("invlpg (%0)" ::"r" (addr) : "memory"); return; @@ -238,9 +249,9 @@ static inline void __native_flush_tlb_si * Make sure to do only a single invpcid when KAISER is * disabled and we have only a single ASID. */ - if (X86_CR3_PCID_ASID_KERN != X86_CR3_PCID_ASID_USER) - invpcid_flush_one(X86_CR3_PCID_ASID_KERN, addr); - invpcid_flush_one(X86_CR3_PCID_ASID_USER, addr); + if (kaiser_enabled) + invpcid_flush_one(X86_CR3_PCID_ASID_USER, addr); + invpcid_flush_one(X86_CR3_PCID_ASID_KERN, addr); } static inline void __flush_tlb_all(void) --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -179,6 +179,20 @@ static int __init x86_pcid_setup(char *s return 1; } __setup("nopcid", x86_pcid_setup); + +static int __init x86_nokaiser_setup(char *s) +{ + /* nokaiser doesn't accept parameters */ + if (s) + return -EINVAL; +#ifdef CONFIG_KAISER + kaiser_enabled = 0; + setup_clear_cpu_cap(X86_FEATURE_KAISER); + pr_info("nokaiser: KAISER feature disabled\n"); +#endif + return 0; +} +early_param("nokaiser", x86_nokaiser_setup); #endif static int __init x86_noinvpcid_setup(char *s) @@ -327,7 +341,7 @@ static __always_inline void setup_smap(s static void setup_pcid(struct cpuinfo_x86 *c) { if (cpu_has(c, X86_FEATURE_PCID)) { - if (cpu_has(c, X86_FEATURE_PGE)) { + if (cpu_has(c, X86_FEATURE_PGE) || kaiser_enabled) { cr4_set_bits(X86_CR4_PCIDE); /* * INVPCID has two "groups" of types: @@ -799,6 +813,10 @@ void get_cpu_cap(struct cpuinfo_x86 *c) c->x86_capability[CPUID_8000_000A_EDX] = cpuid_edx(0x8000000a); init_scattered_cpuid_features(c); +#ifdef CONFIG_KAISER + if (kaiser_enabled) + set_cpu_cap(c, X86_FEATURE_KAISER); +#endif } static void identify_cpu_without_cpuid(struct cpuinfo_x86 *c) @@ -1537,6 +1555,14 @@ void cpu_init(void) * try to read it. */ cr4_init_shadow(); + if (!kaiser_enabled) { + /* + * secondary_startup_64() deferred setting PGE in cr4: + * probe_page_size_mask() sets it on the boot cpu, + * but it needs to be set on each secondary cpu. + */ + cr4_set_bits(X86_CR4_PGE); + } /* * Load microcode on this cpu if a valid microcode is available. --- a/arch/x86/kernel/espfix_64.c +++ b/arch/x86/kernel/espfix_64.c @@ -132,9 +132,10 @@ void __init init_espfix_bsp(void) * area to ensure it is mapped into the shadow user page * tables. */ - if (IS_ENABLED(CONFIG_KAISER)) + if (kaiser_enabled) { set_pgd(native_get_shadow_pgd(pgd_p), __pgd(_KERNPG_TABLE | __pa((pud_t *)espfix_pud_page))); + } /* Randomize the locations */ init_espfix_random(); --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -190,8 +190,8 @@ ENTRY(secondary_startup_64) movq $(init_level4_pgt - __START_KERNEL_map), %rax 1: - /* Enable PAE mode and PGE */ - movl $(X86_CR4_PAE | X86_CR4_PGE), %ecx + /* Enable PAE and PSE, but defer PGE until kaiser_enabled is decided */ + movl $(X86_CR4_PAE | X86_CR4_PSE), %ecx movq %rcx, %cr4 /* Setup early boot stage 4 level pagetables. */ --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -177,7 +177,7 @@ static void __init probe_page_size_mask( cr4_set_bits_and_update_boot(X86_CR4_PSE); /* Enable PGE if available */ - if (boot_cpu_has(X86_FEATURE_PGE)) { + if (boot_cpu_has(X86_FEATURE_PGE) && !kaiser_enabled) { cr4_set_bits_and_update_boot(X86_CR4_PGE); __supported_pte_mask |= _PAGE_GLOBAL; } else --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -324,6 +324,16 @@ void __init cleanup_highmap(void) continue; if (vaddr < (unsigned long) _text || vaddr > end) set_pmd(pmd, __pmd(0)); + else if (kaiser_enabled) { + /* + * level2_kernel_pgt is initialized with _PAGE_GLOBAL: + * clear that now. This is not important, so long as + * CR4.PGE remains clear, but it removes an anomaly. + * Physical mapping setup below avoids _PAGE_GLOBAL + * by use of massage_pgprot() inside pfn_pte() etc. + */ + set_pmd(pmd, pmd_clear_flags(*pmd, _PAGE_GLOBAL)); + } } } --- a/arch/x86/mm/kaiser.c +++ b/arch/x86/mm/kaiser.c @@ -16,7 +16,9 @@ #include <asm/pgalloc.h> #include <asm/desc.h> -#ifdef CONFIG_KAISER +int kaiser_enabled __read_mostly = 1; +EXPORT_SYMBOL(kaiser_enabled); /* for inlined TLB flush functions */ + __visible DEFINE_PER_CPU_USER_MAPPED(unsigned long, unsafe_stack_register_backup); @@ -167,8 +169,8 @@ static pte_t *kaiser_pagetable_walk(unsi return pte_offset_kernel(pmd, address); } -int kaiser_add_user_map(const void *__start_addr, unsigned long size, - unsigned long flags) +static int kaiser_add_user_map(const void *__start_addr, unsigned long size, + unsigned long flags) { int ret = 0; pte_t *pte; @@ -177,6 +179,15 @@ int kaiser_add_user_map(const void *__st unsigned long end_addr = PAGE_ALIGN(start_addr + size); unsigned long target_address; + /* + * It is convenient for callers to pass in __PAGE_KERNEL etc, + * and there is no actual harm from setting _PAGE_GLOBAL, so + * long as CR4.PGE is not set. But it is nonetheless troubling + * to see Kaiser itself setting _PAGE_GLOBAL (now that "nokaiser" + * requires that not to be #defined to 0): so mask it off here. + */ + flags &= ~_PAGE_GLOBAL; + for (; address < end_addr; address += PAGE_SIZE) { target_address = get_pa_from_mapping(address); if (target_address == -1) { @@ -263,6 +274,8 @@ void __init kaiser_init(void) { int cpu; + if (!kaiser_enabled) + return; kaiser_init_all_pgds(); for_each_possible_cpu(cpu) { @@ -311,6 +324,8 @@ void __init kaiser_init(void) /* Add a mapping to the shadow mapping, and synchronize the mappings */ int kaiser_add_mapping(unsigned long addr, unsigned long size, unsigned long flags) { + if (!kaiser_enabled) + return 0; return kaiser_add_user_map((const void *)addr, size, flags); } @@ -322,6 +337,8 @@ void kaiser_remove_mapping(unsigned long unsigned long addr, next; pgd_t *pgd; + if (!kaiser_enabled) + return; pgd = native_get_shadow_pgd(pgd_offset_k(start)); for (addr = start; addr < end; pgd++, addr = next) { next = pgd_addr_end(addr, end); @@ -343,6 +360,8 @@ static inline bool is_userspace_pgd(pgd_ pgd_t kaiser_set_shadow_pgd(pgd_t *pgdp, pgd_t pgd) { + if (!kaiser_enabled) + return pgd; /* * Do we need to also populate the shadow pgd? Check _PAGE_USER to * skip cases like kexec and EFI which make temporary low mappings. @@ -399,4 +418,3 @@ void kaiser_flush_tlb_on_return_to_user( X86_CR3_PCID_USER_FLUSH | KAISER_SHADOW_PGD_OFFSET); } EXPORT_SYMBOL(kaiser_flush_tlb_on_return_to_user); -#endif /* CONFIG_KAISER */ --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -345,16 +345,12 @@ static inline void _pgd_free(pgd_t *pgd) } #else -#ifdef CONFIG_KAISER /* - * Instead of one pmd, we aquire two pmds. Being order-1, it is + * Instead of one pgd, Kaiser acquires two pgds. Being order-1, it is * both 8k in size and 8k-aligned. That lets us just flip bit 12 * in a pointer to swap between the two 4k halves. */ -#define PGD_ALLOCATION_ORDER 1 -#else -#define PGD_ALLOCATION_ORDER 0 -#endif +#define PGD_ALLOCATION_ORDER kaiser_enabled static inline pgd_t *_pgd_alloc(void) { --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -39,8 +39,7 @@ static void load_new_mm_cr3(pgd_t *pgdir { unsigned long new_mm_cr3 = __pa(pgdir); -#ifdef CONFIG_KAISER - if (this_cpu_has(X86_FEATURE_PCID)) { + if (kaiser_enabled && this_cpu_has(X86_FEATURE_PCID)) { /* * We reuse the same PCID for different tasks, so we must * flush all the entries for the PCID out when we change tasks. @@ -57,7 +56,6 @@ static void load_new_mm_cr3(pgd_t *pgdir new_mm_cr3 |= X86_CR3_PCID_KERN_FLUSH; kaiser_flush_tlb_on_return_to_user(); } -#endif /* CONFIG_KAISER */ /* * Caution: many callers of this function expect --- a/tools/arch/x86/include/asm/cpufeatures.h +++ b/tools/arch/x86/include/asm/cpufeatures.h @@ -197,6 +197,9 @@ #define X86_FEATURE_AVX512_4VNNIW (7*32+16) /* AVX-512 Neural Network Instructions */ #define X86_FEATURE_AVX512_4FMAPS (7*32+17) /* AVX-512 Multiply Accumulation Single precision */ +/* Because the ALTERNATIVE scheme is for members of the X86_FEATURE club... */ +#define X86_FEATURE_KAISER ( 7*32+31) /* CONFIG_KAISER w/o nokaiser */ + /* Virtualization flags: Linux defined, word 8 */ #define X86_FEATURE_TPR_SHADOW ( 8*32+ 0) /* Intel TPR Shadow */ #define X86_FEATURE_VNMI ( 8*32+ 1) /* Intel Virtual NMI */ Patches currently in stable-queue which might be from hughd(a)google.com are queue-4.9/kaiser-vmstat-show-nr_kaisertable-as-nr_overhead.patch queue-4.9/kaiser-add-nokaiser-boot-option-using-alternative.patch queue-4.9/kaiser-fix-unlikely-error-in-alloc_ldt_struct.patch queue-4.9/kaiser-kaiser_flush_tlb_on_return_to_user-check-pcid.patch queue-4.9/x86-paravirt-dont-patch-flush_tlb_single.patch queue-4.9/kaiser-merged-update.patch queue-4.9/kaiser-delete-kaiser_real_switch-option.patch queue-4.9/kaiser-kaiser_remove_mapping-move-along-the-pgd.patch queue-4.9/kaiser-fix-perf-crashes.patch queue-4.9/kaiser-drop-is_atomic-arg-to-kaiser_pagetable_walk.patch queue-4.9/kaiser-load_new_mm_cr3-let-switch_user_cr3-flush-user.patch queue-4.9/kaiser-enhanced-by-kernel-and-user-pcids.patch queue-4.9/kaiser-x86_cr3_pcid_noflush-and-x86_cr3_pcid_user.patch queue-4.9/kaiser-align-addition-to-x86-mm-makefile.patch queue-4.9/kaiser-use-alternative-instead-of-x86_cr3_pcid_noflush.patch queue-4.9/kaiser-stack-map-page_size-at-thread_size-page_size.patch queue-4.9/kaiser-name-that-0x1000-kaiser_shadow_pgd_offset.patch queue-4.9/kaiser-fix-regs-to-do_nmi-ifndef-config_kaiser.patch queue-4.9/kaiser-do-not-set-_page_nx-on-pgd_none.patch queue-4.9/kaiser-tidied-up-asm-kaiser.h-somewhat.patch queue-4.9/kaiser-cleanups-while-trying-for-gold-link.patch queue-4.9/kaiser-tidied-up-kaiser_add-remove_mapping-slightly.patch queue-4.9/kaiser-fix-build-and-fixme-in-alloc_ldt_struct.patch queue-4.9/kaiser-kernel-address-isolation.patch queue-4.9/kaiser-enomem-if-kaiser_pagetable_walk-null.patch queue-4.9/kaiser-asm-tlbflush.h-handle-nopge-at-lower-level.patch queue-4.9/kaiser-paranoid_entry-pass-cr3-need-to-paranoid_exit.patch queue-4.9/kaiser-kaiser-depends-on-smp.patch queue-4.9/kaiser-pcid-0-for-kernel-and-128-for-user.patch

6 years, 11 months

1
0
0 0

[Linux-stable-mirror] Patch "x86/paravirt: Dont patch flush_tlb_single" has been added to the 4.4-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled x86/paravirt: Dont patch flush_tlb_single to the 4.4-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: x86-paravirt-dont-patch-flush_tlb_single.patch and it can be found in the queue-4.4 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From foo@baz Wed Jan 3 18:58:12 CET 2018 From: Thomas Gleixner <tglx(a)linutronix.de> Date: Mon, 4 Dec 2017 15:07:30 +0100 Subject: x86/paravirt: Dont patch flush_tlb_single From: Thomas Gleixner <tglx(a)linutronix.de> commit a035795499ca1c2bd1928808d1a156eda1420383 upstream native_flush_tlb_single() will be changed with the upcoming PAGE_TABLE_ISOLATION feature. This requires to have more code in there than INVLPG. Remove the paravirt patching for it. Signed-off-by: Thomas Gleixner <tglx(a)linutronix.de> Reviewed-by: Josh Poimboeuf <jpoimboe(a)redhat.com> Reviewed-by: Juergen Gross <jgross(a)suse.com> Acked-by: Peter Zijlstra <peterz(a)infradead.org> Cc: Andy Lutomirski <luto(a)kernel.org> Cc: Boris Ostrovsky <boris.ostrovsky(a)oracle.com> Cc: Borislav Petkov <bp(a)alien8.de> Cc: Borislav Petkov <bpetkov(a)suse.de> Cc: Brian Gerst <brgerst(a)gmail.com> Cc: Dave Hansen <dave.hansen(a)intel.com> Cc: Dave Hansen <dave.hansen(a)linux.intel.com> Cc: David Laight <David.Laight(a)aculab.com> Cc: Denys Vlasenko <dvlasenk(a)redhat.com> Cc: Eduardo Valentin <eduval(a)amazon.com> Cc: Greg KH <gregkh(a)linuxfoundation.org> Cc: H. Peter Anvin <hpa(a)zytor.com> Cc: Linus Torvalds <torvalds(a)linux-foundation.org> Cc: Rik van Riel <riel(a)redhat.com> Cc: Will Deacon <will.deacon(a)arm.com> Cc: aliguori(a)amazon.com Cc: daniel.gruss(a)iaik.tugraz.at Cc: hughd(a)google.com Cc: keescook(a)google.com Cc: linux-mm(a)kvack.org Cc: michael.schwarz(a)iaik.tugraz.at Cc: moritz.lipp(a)iaik.tugraz.at Cc: richard.fellner(a)student.tugraz.at Link: https://lkml.kernel.org/r/20171204150606.828111617@linutronix.de Signed-off-by: Ingo Molnar <mingo(a)kernel.org> Acked-by: Borislav Petkov <bp(a)suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- arch/x86/kernel/paravirt_patch_64.c | 2 -- 1 file changed, 2 deletions(-) --- a/arch/x86/kernel/paravirt_patch_64.c +++ b/arch/x86/kernel/paravirt_patch_64.c @@ -9,7 +9,6 @@ DEF_NATIVE(pv_irq_ops, save_fl, "pushfq; DEF_NATIVE(pv_mmu_ops, read_cr2, "movq %cr2, %rax"); DEF_NATIVE(pv_mmu_ops, read_cr3, "movq %cr3, %rax"); DEF_NATIVE(pv_mmu_ops, write_cr3, "movq %rdi, %cr3"); -DEF_NATIVE(pv_mmu_ops, flush_tlb_single, "invlpg (%rdi)"); DEF_NATIVE(pv_cpu_ops, clts, "clts"); DEF_NATIVE(pv_cpu_ops, wbinvd, "wbinvd"); @@ -62,7 +61,6 @@ unsigned native_patch(u8 type, u16 clobb PATCH_SITE(pv_mmu_ops, read_cr3); PATCH_SITE(pv_mmu_ops, write_cr3); PATCH_SITE(pv_cpu_ops, clts); - PATCH_SITE(pv_mmu_ops, flush_tlb_single); PATCH_SITE(pv_cpu_ops, wbinvd); #if defined(CONFIG_PARAVIRT_SPINLOCKS) && defined(CONFIG_QUEUED_SPINLOCKS) case PARAVIRT_PATCH(pv_lock_ops.queued_spin_unlock): Patches currently in stable-queue which might be from tglx(a)linutronix.de are queue-4.4/x86-paravirt-dont-patch-flush_tlb_single.patch queue-4.4/x86-boot-add-early-cmdline-parsing-for-options-with-arguments.patch

6 years, 11 months

1
0
0 0

[Linux-stable-mirror] Patch "x86/kaiser: Move feature detection up" has been added to the 4.4-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled x86/kaiser: Move feature detection up to the 4.4-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: x86-kaiser-move-feature-detection-up.patch and it can be found in the queue-4.4 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From foo@baz Wed Jan 3 18:58:12 CET 2018 From: Borislav Petkov <bp(a)suse.de> Date: Mon, 25 Dec 2017 13:57:16 +0100 Subject: x86/kaiser: Move feature detection up From: Borislav Petkov <bp(a)suse.de> ... before the first use of kaiser_enabled as otherwise funky things happen: about to get started... (XEN) d0v0 Unhandled page fault fault/trap [#14, ec=0000] (XEN) Pagetable walk from ffff88022a449090: (XEN) L4[0x110] = 0000000229e0e067 0000000000001e0e (XEN) L3[0x008] = 0000000000000000 ffffffffffffffff (XEN) domain_crash_sync called from entry.S: fault at ffff82d08033fd08 entry.o#create_bounce_frame+0x135/0x14d (XEN) Domain 0 (vcpu#0) crashed on cpu#0: (XEN) ----[ Xen-4.9.1_02-3.21 x86_64 debug=n Not tainted ]---- (XEN) CPU: 0 (XEN) RIP: e033:[<ffffffff81007460>] (XEN) RFLAGS: 0000000000000286 EM: 1 CONTEXT: pv guest (d0v0) Signed-off-by: Borislav Petkov <bp(a)suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- arch/x86/include/asm/kaiser.h | 2 ++ arch/x86/kernel/setup.c | 7 +++++++ arch/x86/mm/kaiser.c | 2 -- 3 files changed, 9 insertions(+), 2 deletions(-) --- a/arch/x86/include/asm/kaiser.h +++ b/arch/x86/include/asm/kaiser.h @@ -96,8 +96,10 @@ DECLARE_PER_CPU(unsigned long, x86_cr3_p extern char __per_cpu_user_mapped_start[], __per_cpu_user_mapped_end[]; extern int kaiser_enabled; +extern void __init kaiser_check_boottime_disable(void); #else #define kaiser_enabled 0 +static inline void __init kaiser_check_boottime_disable(void) {} #endif /* CONFIG_KAISER */ /* --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -112,6 +112,7 @@ #include <asm/alternative.h> #include <asm/prom.h> #include <asm/microcode.h> +#include <asm/kaiser.h> /* * max_low_pfn_mapped: highest direct mapped pfn under 4GB @@ -1016,6 +1017,12 @@ void __init setup_arch(char **cmdline_p) */ init_hypervisor_platform(); + /* + * This needs to happen right after XENPV is set on xen and + * kaiser_enabled is checked below in cleanup_highmap(). + */ + kaiser_check_boottime_disable(); + x86_init.resources.probe_roms(); /* after parse_early_param, so could debug it */ --- a/arch/x86/mm/kaiser.c +++ b/arch/x86/mm/kaiser.c @@ -311,8 +311,6 @@ void __init kaiser_init(void) { int cpu; - kaiser_check_boottime_disable(); - if (!kaiser_enabled) return; Patches currently in stable-queue which might be from bp(a)suse.de are queue-4.4/x86-paravirt-dont-patch-flush_tlb_single.patch queue-4.4/x86-kaiser-reenable-paravirt.patch queue-4.4/x86-kaiser-rename-and-simplify-x86_feature_kaiser-handling.patch queue-4.4/x86-kaiser-check-boottime-cmdline-params.patch queue-4.4/x86-kaiser-move-feature-detection-up.patch

6 years, 11 months

1
0
0 0

[Linux-stable-mirror] Patch "x86/kaiser: Reenable PARAVIRT" has been added to the 4.4-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled x86/kaiser: Reenable PARAVIRT to the 4.4-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: x86-kaiser-reenable-paravirt.patch and it can be found in the queue-4.4 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From foo@baz Wed Jan 3 18:58:12 CET 2018 From: Borislav Petkov <bp(a)suse.de> Date: Tue, 2 Jan 2018 14:19:49 +0100 Subject: x86/kaiser: Reenable PARAVIRT From: Borislav Petkov <bp(a)suse.de> Now that the required bits have been addressed, reenable PARAVIRT. Signed-off-by: Borislav Petkov <bp(a)suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- security/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/security/Kconfig +++ b/security/Kconfig @@ -34,7 +34,7 @@ config SECURITY config KAISER bool "Remove the kernel mapping in user mode" default y - depends on X86_64 && SMP && !PARAVIRT + depends on X86_64 && SMP help This enforces a strict kernel and user space isolation, in order to close hardware side channels on kernel address information. Patches currently in stable-queue which might be from bp(a)suse.de are queue-4.4/x86-paravirt-dont-patch-flush_tlb_single.patch queue-4.4/x86-kaiser-reenable-paravirt.patch queue-4.4/x86-kaiser-rename-and-simplify-x86_feature_kaiser-handling.patch queue-4.4/x86-kaiser-check-boottime-cmdline-params.patch queue-4.4/x86-kaiser-move-feature-detection-up.patch

6 years, 11 months

1
0
0 0

[Linux-stable-mirror] Patch "x86/kaiser: Check boottime cmdline params" has been added to the 4.4-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled x86/kaiser: Check boottime cmdline params to the 4.4-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: x86-kaiser-check-boottime-cmdline-params.patch and it can be found in the queue-4.4 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From foo@baz Wed Jan 3 18:58:12 CET 2018 From: Borislav Petkov <bp(a)suse.de> Date: Tue, 2 Jan 2018 14:19:48 +0100 Subject: x86/kaiser: Check boottime cmdline params From: Borislav Petkov <bp(a)suse.de> AMD (and possibly other vendors) are not affected by the leak KAISER is protecting against. Keep the "nopti" for traditional reasons and add pti=<on|off|auto> like upstream. Signed-off-by: Borislav Petkov <bp(a)suse.de> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- Documentation/kernel-parameters.txt | 6 +++ arch/x86/mm/kaiser.c | 59 +++++++++++++++++++++++++----------- 2 files changed, 47 insertions(+), 18 deletions(-) --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -3056,6 +3056,12 @@ bytes respectively. Such letter suffixes pt. [PARIDE] See Documentation/blockdev/paride.txt. + pti= [X86_64] + Control KAISER user/kernel address space isolation: + on - enable + off - disable + auto - default setting + pty.legacy_count= [KNL] Number of legacy pty's. Overwrites compiled-in default number. --- a/arch/x86/mm/kaiser.c +++ b/arch/x86/mm/kaiser.c @@ -16,6 +16,7 @@ #include <asm/pgtable.h> #include <asm/pgalloc.h> #include <asm/desc.h> +#include <asm/cmdline.h> int kaiser_enabled __read_mostly = 1; EXPORT_SYMBOL(kaiser_enabled); /* for inlined TLB flush functions */ @@ -264,6 +265,43 @@ static void __init kaiser_init_all_pgds( WARN_ON(__ret); \ } while (0) +void __init kaiser_check_boottime_disable(void) +{ + bool enable = true; + char arg[5]; + int ret; + + ret = cmdline_find_option(boot_command_line, "pti", arg, sizeof(arg)); + if (ret > 0) { + if (!strncmp(arg, "on", 2)) + goto enable; + + if (!strncmp(arg, "off", 3)) + goto disable; + + if (!strncmp(arg, "auto", 4)) + goto skip; + } + + if (cmdline_find_option_bool(boot_command_line, "nopti")) + goto disable; + +skip: + if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD) + goto disable; + +enable: + if (enable) + setup_force_cpu_cap(X86_FEATURE_KAISER); + + return; + +disable: + pr_info("Kernel/User page tables isolation: disabled\n"); + kaiser_enabled = 0; + setup_clear_cpu_cap(X86_FEATURE_KAISER); +} + /* * If anything in here fails, we will likely die on one of the * first kernel->user transitions and init will die. But, we @@ -275,12 +313,10 @@ void __init kaiser_init(void) { int cpu; - if (!kaiser_enabled) { - setup_clear_cpu_cap(X86_FEATURE_KAISER); - return; - } + kaiser_check_boottime_disable(); - setup_force_cpu_cap(X86_FEATURE_KAISER); + if (!kaiser_enabled) + return; kaiser_init_all_pgds(); @@ -424,16 +460,3 @@ void kaiser_flush_tlb_on_return_to_user( X86_CR3_PCID_USER_FLUSH | KAISER_SHADOW_PGD_OFFSET); } EXPORT_SYMBOL(kaiser_flush_tlb_on_return_to_user); - -static int __init x86_nokaiser_setup(char *s) -{ - /* nopti doesn't accept parameters */ - if (s) - return -EINVAL; - - kaiser_enabled = 0; - pr_info("Kernel/User page tables isolation: disabled\n"); - - return 0; -} -early_param("nopti", x86_nokaiser_setup); Patches currently in stable-queue which might be from bp(a)suse.de are queue-4.4/x86-paravirt-dont-patch-flush_tlb_single.patch queue-4.4/x86-kaiser-reenable-paravirt.patch queue-4.4/x86-kaiser-rename-and-simplify-x86_feature_kaiser-handling.patch queue-4.4/x86-kaiser-check-boottime-cmdline-params.patch queue-4.4/x86-kaiser-move-feature-detection-up.patch

6 years, 11 months

1
0
0 0

[Linux-stable-mirror] Patch "kaiser: x86_cr3_pcid_noflush and x86_cr3_pcid_user" has been added to the 4.4-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled kaiser: x86_cr3_pcid_noflush and x86_cr3_pcid_user to the 4.4-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: kaiser-x86_cr3_pcid_noflush-and-x86_cr3_pcid_user.patch and it can be found in the queue-4.4 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From foo@baz Wed Jan 3 18:58:12 CET 2018 From: Hugh Dickins <hughd(a)google.com> Date: Sun, 27 Aug 2017 16:24:27 -0700 Subject: kaiser: x86_cr3_pcid_noflush and x86_cr3_pcid_user From: Hugh Dickins <hughd(a)google.com> Mostly this commit is just unshouting X86_CR3_PCID_KERN_VAR and X86_CR3_PCID_USER_VAR: we usually name variables in lower-case. But why does x86_cr3_pcid_noflush need to be __aligned(PAGE_SIZE)? Ah, it's a leftover from when kaiser_add_user_map() once complained about mapping the same page twice. Make it __read_mostly instead. (I'm a little uneasy about all the unrelated data which shares its page getting user-mapped too, but that was so before, and not a big deal: though we call it user-mapped, it's not mapped with _PAGE_USER.) And there is a little change around the two calls to do_nmi(). Previously they set the NOFLUSH bit (if PCID supported) when forcing to kernel context before do_nmi(); now they also have the NOFLUSH bit set (if PCID supported) when restoring context after: nothing done in do_nmi() should require a TLB to be flushed here. Signed-off-by: Hugh Dickins <hughd(a)google.com> Acked-by: Jiri Kosina <jkosina(a)suse.cz> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- arch/x86/entry/entry_64.S | 8 ++++---- arch/x86/include/asm/kaiser.h | 11 +++++------ arch/x86/mm/kaiser.c | 13 +++++++------ 3 files changed, 16 insertions(+), 16 deletions(-) --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -1290,11 +1290,11 @@ ENTRY(nmi) /* Unconditionally use kernel CR3 for do_nmi() */ /* %rax is saved above, so OK to clobber here */ movq %cr3, %rax + /* If PCID enabled, NOFLUSH now and NOFLUSH on return */ + orq x86_cr3_pcid_noflush, %rax pushq %rax /* mask off "user" bit of pgd address and 12 PCID bits: */ andq $(~(X86_CR3_PCID_ASID_MASK | KAISER_SHADOW_PGD_OFFSET)), %rax - /* Add back kernel PCID and "no flush" bit */ - orq X86_CR3_PCID_KERN_VAR, %rax movq %rax, %cr3 #endif call do_nmi @@ -1534,11 +1534,11 @@ end_repeat_nmi: /* Unconditionally use kernel CR3 for do_nmi() */ /* %rax is saved above, so OK to clobber here */ movq %cr3, %rax + /* If PCID enabled, NOFLUSH now and NOFLUSH on return */ + orq x86_cr3_pcid_noflush, %rax pushq %rax /* mask off "user" bit of pgd address and 12 PCID bits: */ andq $(~(X86_CR3_PCID_ASID_MASK | KAISER_SHADOW_PGD_OFFSET)), %rax - /* Add back kernel PCID and "no flush" bit */ - orq X86_CR3_PCID_KERN_VAR, %rax movq %rax, %cr3 #endif --- a/arch/x86/include/asm/kaiser.h +++ b/arch/x86/include/asm/kaiser.h @@ -25,7 +25,7 @@ .macro _SWITCH_TO_KERNEL_CR3 reg movq %cr3, \reg andq $(~(X86_CR3_PCID_ASID_MASK | KAISER_SHADOW_PGD_OFFSET)), \reg -orq X86_CR3_PCID_KERN_VAR, \reg +orq x86_cr3_pcid_noflush, \reg movq \reg, %cr3 .endm @@ -37,11 +37,10 @@ movq \reg, %cr3 * not enabled): so that the one register can update both memory and cr3. */ movq %cr3, \reg -andq $(~(X86_CR3_PCID_ASID_MASK | KAISER_SHADOW_PGD_OFFSET)), \reg -orq PER_CPU_VAR(X86_CR3_PCID_USER_VAR), \reg +orq PER_CPU_VAR(x86_cr3_pcid_user), \reg js 9f /* FLUSH this time, reset to NOFLUSH for next time (if PCID enabled) */ -movb \regb, PER_CPU_VAR(X86_CR3_PCID_USER_VAR+7) +movb \regb, PER_CPU_VAR(x86_cr3_pcid_user+7) 9: movq \reg, %cr3 .endm @@ -94,8 +93,8 @@ movq PER_CPU_VAR(unsafe_stack_register_b */ DECLARE_PER_CPU_USER_MAPPED(unsigned long, unsafe_stack_register_backup); -extern unsigned long X86_CR3_PCID_KERN_VAR; -DECLARE_PER_CPU(unsigned long, X86_CR3_PCID_USER_VAR); +extern unsigned long x86_cr3_pcid_noflush; +DECLARE_PER_CPU(unsigned long, x86_cr3_pcid_user); extern char __per_cpu_user_mapped_start[], __per_cpu_user_mapped_end[]; --- a/arch/x86/mm/kaiser.c +++ b/arch/x86/mm/kaiser.c @@ -29,8 +29,8 @@ DEFINE_PER_CPU_USER_MAPPED(unsigned long * This is also handy because systems that do not support PCIDs * just end up or'ing a 0 into their CR3, which does no harm. */ -__aligned(PAGE_SIZE) unsigned long X86_CR3_PCID_KERN_VAR; -DEFINE_PER_CPU(unsigned long, X86_CR3_PCID_USER_VAR); +unsigned long x86_cr3_pcid_noflush __read_mostly; +DEFINE_PER_CPU(unsigned long, x86_cr3_pcid_user); /* * At runtime, the only things we map are some things for CPU @@ -304,7 +304,8 @@ void __init kaiser_init(void) sizeof(gate_desc) * NR_VECTORS, __PAGE_KERNEL); - kaiser_add_user_map_early(&X86_CR3_PCID_KERN_VAR, PAGE_SIZE, + kaiser_add_user_map_early(&x86_cr3_pcid_noflush, + sizeof(x86_cr3_pcid_noflush), __PAGE_KERNEL); } @@ -384,8 +385,8 @@ void kaiser_setup_pcid(void) * These variables are used by the entry/exit * code to change PCID and pgd and TLB flushing. */ - X86_CR3_PCID_KERN_VAR = kern_cr3; - this_cpu_write(X86_CR3_PCID_USER_VAR, user_cr3); + x86_cr3_pcid_noflush = kern_cr3; + this_cpu_write(x86_cr3_pcid_user, user_cr3); } /* @@ -395,7 +396,7 @@ void kaiser_setup_pcid(void) */ void kaiser_flush_tlb_on_return_to_user(void) { - this_cpu_write(X86_CR3_PCID_USER_VAR, + this_cpu_write(x86_cr3_pcid_user, X86_CR3_PCID_USER_FLUSH | KAISER_SHADOW_PGD_OFFSET); } EXPORT_SYMBOL(kaiser_flush_tlb_on_return_to_user); Patches currently in stable-queue which might be from hughd(a)google.com are queue-4.4/kaiser-vmstat-show-nr_kaisertable-as-nr_overhead.patch queue-4.4/kaiser-add-nokaiser-boot-option-using-alternative.patch queue-4.4/kaiser-fix-unlikely-error-in-alloc_ldt_struct.patch queue-4.4/kaiser-_pgd_alloc-without-__gfp_repeat-to-avoid-stalls.patch queue-4.4/kaiser-kaiser_flush_tlb_on_return_to_user-check-pcid.patch queue-4.4/x86-paravirt-dont-patch-flush_tlb_single.patch queue-4.4/kaiser-merged-update.patch queue-4.4/kaiser-delete-kaiser_real_switch-option.patch queue-4.4/kaiser-kaiser_remove_mapping-move-along-the-pgd.patch queue-4.4/kaiser-fix-perf-crashes.patch queue-4.4/kaiser-drop-is_atomic-arg-to-kaiser_pagetable_walk.patch queue-4.4/kaiser-load_new_mm_cr3-let-switch_user_cr3-flush-user.patch queue-4.4/kaiser-enhanced-by-kernel-and-user-pcids.patch queue-4.4/kaiser-x86_cr3_pcid_noflush-and-x86_cr3_pcid_user.patch queue-4.4/kaiser-use-alternative-instead-of-x86_cr3_pcid_noflush.patch queue-4.4/kaiser-stack-map-page_size-at-thread_size-page_size.patch queue-4.4/kaiser-name-that-0x1000-kaiser_shadow_pgd_offset.patch queue-4.4/kaiser-fix-regs-to-do_nmi-ifndef-config_kaiser.patch queue-4.4/kaiser-do-not-set-_page_nx-on-pgd_none.patch queue-4.4/kaiser-tidied-up-asm-kaiser.h-somewhat.patch queue-4.4/kaiser-cleanups-while-trying-for-gold-link.patch queue-4.4/kaiser-tidied-up-kaiser_add-remove_mapping-slightly.patch queue-4.4/kaiser-fix-build-and-fixme-in-alloc_ldt_struct.patch queue-4.4/kaiser-kernel-address-isolation.patch queue-4.4/kaiser-enomem-if-kaiser_pagetable_walk-null.patch queue-4.4/kaiser-asm-tlbflush.h-handle-nopge-at-lower-level.patch queue-4.4/kaiser-paranoid_entry-pass-cr3-need-to-paranoid_exit.patch queue-4.4/kaiser-kaiser-depends-on-smp.patch queue-4.4/kaiser-pcid-0-for-kernel-and-128-for-user.patch

6 years, 11 months

1
0
0 0

[Linux-stable-mirror] Patch "kaiser: vmstat show NR_KAISERTABLE as nr_overhead" has been added to the 4.4-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled kaiser: vmstat show NR_KAISERTABLE as nr_overhead to the 4.4-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: kaiser-vmstat-show-nr_kaisertable-as-nr_overhead.patch and it can be found in the queue-4.4 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From foo@baz Wed Jan 3 18:58:12 CET 2018 From: Hugh Dickins <hughd(a)google.com> Date: Sat, 9 Sep 2017 21:27:32 -0700 Subject: kaiser: vmstat show NR_KAISERTABLE as nr_overhead From: Hugh Dickins <hughd(a)google.com> The kaiser update made an interesting choice, never to free any shadow page tables. Contention on global spinlock was worrying, particularly with it held across page table scans when freeing. Something had to be done: I was going to add refcounting; but simply never to free them is an appealing choice, minimizing contention without complicating the code (the more a page table is found already, the less the spinlock is used). But leaking pages in this way is also a worry: can we get away with it? At the very least, we need a count to show how bad it actually gets: in principle, one might end up wasting about 1/256 of memory that way (1/512 for when direct-mapped pages have to be user-mapped, plus 1/512 for when they are user-mapped from the vmalloc area on another occasion (but we don't have vmalloc'ed stacks, so only large ldts are vmalloc'ed). Add per-cpu stat NR_KAISERTABLE: including 256 at startup for the shared pgd entries, and 1 for each intermediate page table added thereafter for user-mapping - but leave out the 1 per mm, for its shadow pgd, because that distracts from the monotonic increase. Shown in /proc/vmstat as nr_overhead (0 if kaiser not enabled). In practice, it doesn't look so bad so far: more like 1/12000 after nine hours of gtests below; and movable pageblock segregation should tend to cluster the kaiser tables into a subset of the address space (if not, they will be bad for compaction too). But production may tell a different story: keep an eye on this number, and bring back lighter freeing if it gets out of control (maybe a shrinker). Signed-off-by: Hugh Dickins <hughd(a)google.com> Acked-by: Jiri Kosina <jkosina(a)suse.cz> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- arch/x86/mm/kaiser.c | 16 +++++++++++----- include/linux/mmzone.h | 3 ++- mm/vmstat.c | 1 + 3 files changed, 14 insertions(+), 6 deletions(-) --- a/arch/x86/mm/kaiser.c +++ b/arch/x86/mm/kaiser.c @@ -122,9 +122,11 @@ static pte_t *kaiser_pagetable_walk(unsi if (!new_pmd_page) return NULL; spin_lock(&shadow_table_allocation_lock); - if (pud_none(*pud)) + if (pud_none(*pud)) { set_pud(pud, __pud(_KERNPG_TABLE | __pa(new_pmd_page))); - else + __inc_zone_page_state(virt_to_page((void *) + new_pmd_page), NR_KAISERTABLE); + } else free_page(new_pmd_page); spin_unlock(&shadow_table_allocation_lock); } @@ -140,9 +142,11 @@ static pte_t *kaiser_pagetable_walk(unsi if (!new_pte_page) return NULL; spin_lock(&shadow_table_allocation_lock); - if (pmd_none(*pmd)) + if (pmd_none(*pmd)) { set_pmd(pmd, __pmd(_KERNPG_TABLE | __pa(new_pte_page))); - else + __inc_zone_page_state(virt_to_page((void *) + new_pte_page), NR_KAISERTABLE); + } else free_page(new_pte_page); spin_unlock(&shadow_table_allocation_lock); } @@ -206,11 +210,13 @@ static void __init kaiser_init_all_pgds( pgd = native_get_shadow_pgd(pgd_offset_k((unsigned long )0)); for (i = PTRS_PER_PGD / 2; i < PTRS_PER_PGD; i++) { pgd_t new_pgd; - pud_t *pud = pud_alloc_one(&init_mm, PAGE_OFFSET + i * PGDIR_SIZE); + pud_t *pud = pud_alloc_one(&init_mm, + PAGE_OFFSET + i * PGDIR_SIZE); if (!pud) { WARN_ON(1); break; } + inc_zone_page_state(virt_to_page(pud), NR_KAISERTABLE); new_pgd = __pgd(_KERNPG_TABLE |__pa(pud)); /* * Make sure not to stomp on some other pgd entry. --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -131,8 +131,9 @@ enum zone_stat_item { NR_SLAB_RECLAIMABLE, NR_SLAB_UNRECLAIMABLE, NR_PAGETABLE, /* used for pagetables */ - NR_KERNEL_STACK, /* Second 128 byte cacheline */ + NR_KERNEL_STACK, + NR_KAISERTABLE, NR_UNSTABLE_NFS, /* NFS unstable pages */ NR_BOUNCE, NR_VMSCAN_WRITE, --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -736,6 +736,7 @@ const char * const vmstat_text[] = { "nr_slab_unreclaimable", "nr_page_table_pages", "nr_kernel_stack", + "nr_overhead", "nr_unstable", "nr_bounce", "nr_vmscan_write", Patches currently in stable-queue which might be from hughd(a)google.com are queue-4.4/kaiser-vmstat-show-nr_kaisertable-as-nr_overhead.patch queue-4.4/kaiser-add-nokaiser-boot-option-using-alternative.patch queue-4.4/kaiser-fix-unlikely-error-in-alloc_ldt_struct.patch queue-4.4/kaiser-_pgd_alloc-without-__gfp_repeat-to-avoid-stalls.patch queue-4.4/kaiser-kaiser_flush_tlb_on_return_to_user-check-pcid.patch queue-4.4/x86-paravirt-dont-patch-flush_tlb_single.patch queue-4.4/kaiser-merged-update.patch queue-4.4/kaiser-delete-kaiser_real_switch-option.patch queue-4.4/kaiser-kaiser_remove_mapping-move-along-the-pgd.patch queue-4.4/kaiser-fix-perf-crashes.patch queue-4.4/kaiser-drop-is_atomic-arg-to-kaiser_pagetable_walk.patch queue-4.4/kaiser-load_new_mm_cr3-let-switch_user_cr3-flush-user.patch queue-4.4/kaiser-enhanced-by-kernel-and-user-pcids.patch queue-4.4/kaiser-x86_cr3_pcid_noflush-and-x86_cr3_pcid_user.patch queue-4.4/kaiser-use-alternative-instead-of-x86_cr3_pcid_noflush.patch queue-4.4/kaiser-stack-map-page_size-at-thread_size-page_size.patch queue-4.4/kaiser-name-that-0x1000-kaiser_shadow_pgd_offset.patch queue-4.4/kaiser-fix-regs-to-do_nmi-ifndef-config_kaiser.patch queue-4.4/kaiser-do-not-set-_page_nx-on-pgd_none.patch queue-4.4/kaiser-tidied-up-asm-kaiser.h-somewhat.patch queue-4.4/kaiser-cleanups-while-trying-for-gold-link.patch queue-4.4/kaiser-tidied-up-kaiser_add-remove_mapping-slightly.patch queue-4.4/kaiser-fix-build-and-fixme-in-alloc_ldt_struct.patch queue-4.4/kaiser-kernel-address-isolation.patch queue-4.4/kaiser-enomem-if-kaiser_pagetable_walk-null.patch queue-4.4/kaiser-asm-tlbflush.h-handle-nopge-at-lower-level.patch queue-4.4/kaiser-paranoid_entry-pass-cr3-need-to-paranoid_exit.patch queue-4.4/kaiser-kaiser-depends-on-smp.patch queue-4.4/kaiser-pcid-0-for-kernel-and-128-for-user.patch

6 years, 11 months

1
0
0 0

[Linux-stable-mirror] Patch "kaiser: use ALTERNATIVE instead of x86_cr3_pcid_noflush" has been added to the 4.4-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled kaiser: use ALTERNATIVE instead of x86_cr3_pcid_noflush to the 4.4-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: kaiser-use-alternative-instead-of-x86_cr3_pcid_noflush.patch and it can be found in the queue-4.4 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From foo@baz Wed Jan 3 18:58:12 CET 2018 From: Hugh Dickins <hughd(a)google.com> Date: Tue, 3 Oct 2017 20:49:04 -0700 Subject: kaiser: use ALTERNATIVE instead of x86_cr3_pcid_noflush From: Hugh Dickins <hughd(a)google.com> Now that we're playing the ALTERNATIVE game, use that more efficient method: instead of user-mapping an extra page, and reading an extra cacheline each time for x86_cr3_pcid_noflush. Neel has found that __stringify(bts $X86_CR3_PCID_NOFLUSH_BIT, %rax) is a working substitute for the "bts $63, %rax" in these ALTERNATIVEs; but the one line with $63 in looks clearer, so let's stick with that. Worried about what happens with an ALTERNATIVE between the jump and jump label in another ALTERNATIVE? I was, but have checked the combinations in SWITCH_KERNEL_CR3_NO_STACK at entry_SYSCALL_64, and it does a good job. Signed-off-by: Hugh Dickins <hughd(a)google.com> Acked-by: Jiri Kosina <jkosina(a)suse.cz> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- arch/x86/entry/entry_64.S | 7 ++++--- arch/x86/include/asm/kaiser.h | 6 +++--- arch/x86/mm/kaiser.c | 11 +---------- 3 files changed, 8 insertions(+), 16 deletions(-) --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -1056,7 +1056,8 @@ ENTRY(paranoid_entry) jz 2f orl $2, %ebx andq $(~(X86_CR3_PCID_ASID_MASK | KAISER_SHADOW_PGD_OFFSET)), %rax - orq x86_cr3_pcid_noflush, %rax + /* If PCID enabled, set X86_CR3_PCID_NOFLUSH_BIT */ + ALTERNATIVE "", "bts $63, %rax", X86_FEATURE_PCID movq %rax, %cr3 2: #endif @@ -1318,7 +1319,7 @@ ENTRY(nmi) /* %rax is saved above, so OK to clobber here */ ALTERNATIVE "jmp 2f", "movq %cr3, %rax", X86_FEATURE_KAISER /* If PCID enabled, NOFLUSH now and NOFLUSH on return */ - orq x86_cr3_pcid_noflush, %rax + ALTERNATIVE "", "bts $63, %rax", X86_FEATURE_PCID pushq %rax /* mask off "user" bit of pgd address and 12 PCID bits: */ andq $(~(X86_CR3_PCID_ASID_MASK | KAISER_SHADOW_PGD_OFFSET)), %rax @@ -1562,7 +1563,7 @@ end_repeat_nmi: /* %rax is saved above, so OK to clobber here */ ALTERNATIVE "jmp 2f", "movq %cr3, %rax", X86_FEATURE_KAISER /* If PCID enabled, NOFLUSH now and NOFLUSH on return */ - orq x86_cr3_pcid_noflush, %rax + ALTERNATIVE "", "bts $63, %rax", X86_FEATURE_PCID pushq %rax /* mask off "user" bit of pgd address and 12 PCID bits: */ andq $(~(X86_CR3_PCID_ASID_MASK | KAISER_SHADOW_PGD_OFFSET)), %rax --- a/arch/x86/include/asm/kaiser.h +++ b/arch/x86/include/asm/kaiser.h @@ -25,7 +25,8 @@ .macro _SWITCH_TO_KERNEL_CR3 reg movq %cr3, \reg andq $(~(X86_CR3_PCID_ASID_MASK | KAISER_SHADOW_PGD_OFFSET)), \reg -orq x86_cr3_pcid_noflush, \reg +/* If PCID enabled, set X86_CR3_PCID_NOFLUSH_BIT */ +ALTERNATIVE "", "bts $63, \reg", X86_FEATURE_PCID movq \reg, %cr3 .endm @@ -39,7 +40,7 @@ movq \reg, %cr3 movq %cr3, \reg orq PER_CPU_VAR(x86_cr3_pcid_user), \reg js 9f -/* FLUSH this time, reset to NOFLUSH for next time (if PCID enabled) */ +/* If PCID enabled, FLUSH this time, reset to NOFLUSH for next time */ movb \regb, PER_CPU_VAR(x86_cr3_pcid_user+7) 9: movq \reg, %cr3 @@ -90,7 +91,6 @@ movq PER_CPU_VAR(unsafe_stack_register_b */ DECLARE_PER_CPU_USER_MAPPED(unsigned long, unsafe_stack_register_backup); -extern unsigned long x86_cr3_pcid_noflush; DECLARE_PER_CPU(unsigned long, x86_cr3_pcid_user); extern char __per_cpu_user_mapped_start[], __per_cpu_user_mapped_end[]; --- a/arch/x86/mm/kaiser.c +++ b/arch/x86/mm/kaiser.c @@ -32,7 +32,6 @@ DEFINE_PER_CPU_USER_MAPPED(unsigned long * This is also handy because systems that do not support PCIDs * just end up or'ing a 0 into their CR3, which does no harm. */ -unsigned long x86_cr3_pcid_noflush __read_mostly; DEFINE_PER_CPU(unsigned long, x86_cr3_pcid_user); /* @@ -357,10 +356,6 @@ void __init kaiser_init(void) kaiser_add_user_map_early(&debug_idt_table, sizeof(gate_desc) * NR_VECTORS, __PAGE_KERNEL); - - kaiser_add_user_map_early(&x86_cr3_pcid_noflush, - sizeof(x86_cr3_pcid_noflush), - __PAGE_KERNEL); } /* Add a mapping to the shadow mapping, and synchronize the mappings */ @@ -434,18 +429,14 @@ pgd_t kaiser_set_shadow_pgd(pgd_t *pgdp, void kaiser_setup_pcid(void) { - unsigned long kern_cr3 = 0; unsigned long user_cr3 = KAISER_SHADOW_PGD_OFFSET; - if (this_cpu_has(X86_FEATURE_PCID)) { - kern_cr3 |= X86_CR3_PCID_KERN_NOFLUSH; + if (this_cpu_has(X86_FEATURE_PCID)) user_cr3 |= X86_CR3_PCID_USER_NOFLUSH; - } /* * These variables are used by the entry/exit * code to change PCID and pgd and TLB flushing. */ - x86_cr3_pcid_noflush = kern_cr3; this_cpu_write(x86_cr3_pcid_user, user_cr3); } Patches currently in stable-queue which might be from hughd(a)google.com are queue-4.4/kaiser-vmstat-show-nr_kaisertable-as-nr_overhead.patch queue-4.4/kaiser-add-nokaiser-boot-option-using-alternative.patch queue-4.4/kaiser-fix-unlikely-error-in-alloc_ldt_struct.patch queue-4.4/kaiser-_pgd_alloc-without-__gfp_repeat-to-avoid-stalls.patch queue-4.4/kaiser-kaiser_flush_tlb_on_return_to_user-check-pcid.patch queue-4.4/x86-paravirt-dont-patch-flush_tlb_single.patch queue-4.4/kaiser-merged-update.patch queue-4.4/kaiser-delete-kaiser_real_switch-option.patch queue-4.4/kaiser-kaiser_remove_mapping-move-along-the-pgd.patch queue-4.4/kaiser-fix-perf-crashes.patch queue-4.4/kaiser-drop-is_atomic-arg-to-kaiser_pagetable_walk.patch queue-4.4/kaiser-load_new_mm_cr3-let-switch_user_cr3-flush-user.patch queue-4.4/kaiser-enhanced-by-kernel-and-user-pcids.patch queue-4.4/kaiser-x86_cr3_pcid_noflush-and-x86_cr3_pcid_user.patch queue-4.4/kaiser-use-alternative-instead-of-x86_cr3_pcid_noflush.patch queue-4.4/kaiser-stack-map-page_size-at-thread_size-page_size.patch queue-4.4/kaiser-name-that-0x1000-kaiser_shadow_pgd_offset.patch queue-4.4/kaiser-fix-regs-to-do_nmi-ifndef-config_kaiser.patch queue-4.4/kaiser-do-not-set-_page_nx-on-pgd_none.patch queue-4.4/kaiser-tidied-up-asm-kaiser.h-somewhat.patch queue-4.4/kaiser-cleanups-while-trying-for-gold-link.patch queue-4.4/kaiser-tidied-up-kaiser_add-remove_mapping-slightly.patch queue-4.4/kaiser-fix-build-and-fixme-in-alloc_ldt_struct.patch queue-4.4/kaiser-kernel-address-isolation.patch queue-4.4/kaiser-enomem-if-kaiser_pagetable_walk-null.patch queue-4.4/kaiser-asm-tlbflush.h-handle-nopge-at-lower-level.patch queue-4.4/kaiser-paranoid_entry-pass-cr3-need-to-paranoid_exit.patch queue-4.4/kaiser-kaiser-depends-on-smp.patch queue-4.4/kaiser-pcid-0-for-kernel-and-128-for-user.patch

6 years, 11 months

1
0
0 0

[Linux-stable-mirror] Patch "kaiser: tidied up kaiser_add/remove_mapping slightly" has been added to the 4.4-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled kaiser: tidied up kaiser_add/remove_mapping slightly to the 4.4-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: kaiser-tidied-up-kaiser_add-remove_mapping-slightly.patch and it can be found in the queue-4.4 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From foo@baz Wed Jan 3 18:58:12 CET 2018 From: Hugh Dickins <hughd(a)google.com> Date: Sun, 3 Sep 2017 19:23:08 -0700 Subject: kaiser: tidied up kaiser_add/remove_mapping slightly From: Hugh Dickins <hughd(a)google.com> Yes, unmap_pud_range_nofree()'s declaration ought to be in a header file really, but I'm not sure we want to use it anyway: so for now just declare it inside kaiser_remove_mapping(). And there doesn't seem to be such a thing as unmap_p4d_range(), even in a 5-level paging tree. Signed-off-by: Hugh Dickins <hughd(a)google.com> Acked-by: Jiri Kosina <jkosina(a)suse.cz> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- arch/x86/mm/kaiser.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) --- a/arch/x86/mm/kaiser.c +++ b/arch/x86/mm/kaiser.c @@ -286,8 +286,7 @@ void __init kaiser_init(void) __PAGE_KERNEL); } -extern void unmap_pud_range_nofree(pgd_t *pgd, unsigned long start, unsigned long end); -// add a mapping to the shadow-mapping, and synchronize the mappings +/* Add a mapping to the shadow mapping, and synchronize the mappings */ int kaiser_add_mapping(unsigned long addr, unsigned long size, unsigned long flags) { return kaiser_add_user_map((const void *)addr, size, flags); @@ -295,15 +294,13 @@ int kaiser_add_mapping(unsigned long add void kaiser_remove_mapping(unsigned long start, unsigned long size) { + extern void unmap_pud_range_nofree(pgd_t *pgd, + unsigned long start, unsigned long end); unsigned long end = start + size; unsigned long addr; for (addr = start; addr < end; addr += PGDIR_SIZE) { pgd_t *pgd = native_get_shadow_pgd(pgd_offset_k(addr)); - /* - * unmap_p4d_range() handles > P4D_SIZE unmaps, - * so no need to trim 'end'. - */ unmap_pud_range_nofree(pgd, addr, end); } } Patches currently in stable-queue which might be from hughd(a)google.com are queue-4.4/kaiser-vmstat-show-nr_kaisertable-as-nr_overhead.patch queue-4.4/kaiser-add-nokaiser-boot-option-using-alternative.patch queue-4.4/kaiser-fix-unlikely-error-in-alloc_ldt_struct.patch queue-4.4/kaiser-_pgd_alloc-without-__gfp_repeat-to-avoid-stalls.patch queue-4.4/kaiser-kaiser_flush_tlb_on_return_to_user-check-pcid.patch queue-4.4/x86-paravirt-dont-patch-flush_tlb_single.patch queue-4.4/kaiser-merged-update.patch queue-4.4/kaiser-delete-kaiser_real_switch-option.patch queue-4.4/kaiser-kaiser_remove_mapping-move-along-the-pgd.patch queue-4.4/kaiser-fix-perf-crashes.patch queue-4.4/kaiser-drop-is_atomic-arg-to-kaiser_pagetable_walk.patch queue-4.4/kaiser-load_new_mm_cr3-let-switch_user_cr3-flush-user.patch queue-4.4/kaiser-enhanced-by-kernel-and-user-pcids.patch queue-4.4/kaiser-x86_cr3_pcid_noflush-and-x86_cr3_pcid_user.patch queue-4.4/kaiser-use-alternative-instead-of-x86_cr3_pcid_noflush.patch queue-4.4/kaiser-stack-map-page_size-at-thread_size-page_size.patch queue-4.4/kaiser-name-that-0x1000-kaiser_shadow_pgd_offset.patch queue-4.4/kaiser-fix-regs-to-do_nmi-ifndef-config_kaiser.patch queue-4.4/kaiser-do-not-set-_page_nx-on-pgd_none.patch queue-4.4/kaiser-tidied-up-asm-kaiser.h-somewhat.patch queue-4.4/kaiser-cleanups-while-trying-for-gold-link.patch queue-4.4/kaiser-tidied-up-kaiser_add-remove_mapping-slightly.patch queue-4.4/kaiser-fix-build-and-fixme-in-alloc_ldt_struct.patch queue-4.4/kaiser-kernel-address-isolation.patch queue-4.4/kaiser-enomem-if-kaiser_pagetable_walk-null.patch queue-4.4/kaiser-asm-tlbflush.h-handle-nopge-at-lower-level.patch queue-4.4/kaiser-paranoid_entry-pass-cr3-need-to-paranoid_exit.patch queue-4.4/kaiser-kaiser-depends-on-smp.patch queue-4.4/kaiser-pcid-0-for-kernel-and-128-for-user.patch

6 years, 11 months

1
0
0 0

2024

2023

2022

2021

2020

2019

2018

2017

Linux-stable-mirror January 2018