- Linux-stable-mirror - lists.linaro.org

FAILED: patch "[PATCH] KVM: arm/arm64: Skip updating PTE entry if no change" failed to apply to 3.18-stable tree

by gregkh＠linuxfoundation.org

The patch below does not apply to the 3.18-stable tree. If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <stable(a)vger.kernel.org>. thanks, greg k-h ------------------ original commit in Linus's tree ------------------ >From 976d34e2dab10ece5ea8fe7090b7692913f89084 Mon Sep 17 00:00:00 2001 From: Punit Agrawal <punit.agrawal(a)arm.com> Date: Mon, 13 Aug 2018 11:43:51 +0100 Subject: [PATCH] KVM: arm/arm64: Skip updating PTE entry if no change When there is contention on faulting in a particular page table entry at stage 2, the break-before-make requirement of the architecture can lead to additional refaulting due to TLB invalidation. Avoid this by skipping a page table update if the new value of the PTE matches the previous value. Cc: stable(a)vger.kernel.org Fixes: d5d8184d35c9 ("KVM: ARM: Memory virtualization setup") Reviewed-by: Suzuki Poulose <suzuki.poulose(a)arm.com> Acked-by: Christoffer Dall <christoffer.dall(a)arm.com> Signed-off-by: Punit Agrawal <punit.agrawal(a)arm.com> Signed-off-by: Marc Zyngier <marc.zyngier(a)arm.com> diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c index 13dfe36501aa..91aaf73b00df 100644 --- a/virt/kvm/arm/mmu.c +++ b/virt/kvm/arm/mmu.c @@ -1147,6 +1147,10 @@ static int stage2_set_pte(struct kvm *kvm, struct kvm_mmu_memory_cache *cache, /* Create 2nd stage page table mapping - Level 3 */ old_pte = *pte; if (pte_present(old_pte)) { + /* Skip page table update if there is no change */ + if (pte_val(old_pte) == pte_val(*new_pte)) + return 0; + kvm_set_pte(pte, __pte(0)); kvm_tlb_flush_vmid_ipa(kvm, addr); } else {

6 years, 11 months

1
0
0 0

FAILED: patch "[PATCH] KVM: arm/arm64: Skip updating PTE entry if no change" failed to apply to 4.4-stable tree

by gregkh＠linuxfoundation.org

The patch below does not apply to the 4.4-stable tree. If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <stable(a)vger.kernel.org>. thanks, greg k-h ------------------ original commit in Linus's tree ------------------ >From 976d34e2dab10ece5ea8fe7090b7692913f89084 Mon Sep 17 00:00:00 2001 From: Punit Agrawal <punit.agrawal(a)arm.com> Date: Mon, 13 Aug 2018 11:43:51 +0100 Subject: [PATCH] KVM: arm/arm64: Skip updating PTE entry if no change When there is contention on faulting in a particular page table entry at stage 2, the break-before-make requirement of the architecture can lead to additional refaulting due to TLB invalidation. Avoid this by skipping a page table update if the new value of the PTE matches the previous value. Cc: stable(a)vger.kernel.org Fixes: d5d8184d35c9 ("KVM: ARM: Memory virtualization setup") Reviewed-by: Suzuki Poulose <suzuki.poulose(a)arm.com> Acked-by: Christoffer Dall <christoffer.dall(a)arm.com> Signed-off-by: Punit Agrawal <punit.agrawal(a)arm.com> Signed-off-by: Marc Zyngier <marc.zyngier(a)arm.com> diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c index 13dfe36501aa..91aaf73b00df 100644 --- a/virt/kvm/arm/mmu.c +++ b/virt/kvm/arm/mmu.c @@ -1147,6 +1147,10 @@ static int stage2_set_pte(struct kvm *kvm, struct kvm_mmu_memory_cache *cache, /* Create 2nd stage page table mapping - Level 3 */ old_pte = *pte; if (pte_present(old_pte)) { + /* Skip page table update if there is no change */ + if (pte_val(old_pte) == pte_val(*new_pte)) + return 0; + kvm_set_pte(pte, __pte(0)); kvm_tlb_flush_vmid_ipa(kvm, addr); } else {

6 years, 11 months

1
0
0 0

FAILED: patch "[PATCH] KVM: arm/arm64: Skip updating PMD entry if no change" failed to apply to 3.18-stable tree

by gregkh＠linuxfoundation.org

The patch below does not apply to the 3.18-stable tree. If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <stable(a)vger.kernel.org>. thanks, greg k-h ------------------ original commit in Linus's tree ------------------ >From 86658b819cd0a9aa584cd84453ed268a6f013770 Mon Sep 17 00:00:00 2001 From: Punit Agrawal <punit.agrawal(a)arm.com> Date: Mon, 13 Aug 2018 11:43:50 +0100 Subject: [PATCH] KVM: arm/arm64: Skip updating PMD entry if no change Contention on updating a PMD entry by a large number of vcpus can lead to duplicate work when handling stage 2 page faults. As the page table update follows the break-before-make requirement of the architecture, it can lead to repeated refaults due to clearing the entry and flushing the tlbs. This problem is more likely when - * there are large number of vcpus * the mapping is large block mapping such as when using PMD hugepages (512MB) with 64k pages. Fix this by skipping the page table update if there is no change in the entry being updated. Cc: stable(a)vger.kernel.org Fixes: ad361f093c1e ("KVM: ARM: Support hugetlbfs backed huge pages") Reviewed-by: Suzuki Poulose <suzuki.poulose(a)arm.com> Acked-by: Christoffer Dall <christoffer.dall(a)arm.com> Signed-off-by: Punit Agrawal <punit.agrawal(a)arm.com> Signed-off-by: Marc Zyngier <marc.zyngier(a)arm.com> diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c index 97d27cd9c654..13dfe36501aa 100644 --- a/virt/kvm/arm/mmu.c +++ b/virt/kvm/arm/mmu.c @@ -1044,19 +1044,35 @@ static int stage2_set_pmd_huge(struct kvm *kvm, struct kvm_mmu_memory_cache pmd = stage2_get_pmd(kvm, cache, addr); VM_BUG_ON(!pmd); - /* - * Mapping in huge pages should only happen through a fault. If a - * page is merged into a transparent huge page, the individual - * subpages of that huge page should be unmapped through MMU - * notifiers before we get here. - * - * Merging of CompoundPages is not supported; they should become - * splitting first, unmapped, merged, and mapped back in on-demand. - */ - VM_BUG_ON(pmd_present(*pmd) && pmd_pfn(*pmd) != pmd_pfn(*new_pmd)); - old_pmd = *pmd; if (pmd_present(old_pmd)) { + /* + * Multiple vcpus faulting on the same PMD entry, can + * lead to them sequentially updating the PMD with the + * same value. Following the break-before-make + * (pmd_clear() followed by tlb_flush()) process can + * hinder forward progress due to refaults generated + * on missing translations. + * + * Skip updating the page table if the entry is + * unchanged. + */ + if (pmd_val(old_pmd) == pmd_val(*new_pmd)) + return 0; + + /* + * Mapping in huge pages should only happen through a + * fault. If a page is merged into a transparent huge + * page, the individual subpages of that huge page + * should be unmapped through MMU notifiers before we + * get here. + * + * Merging of CompoundPages is not supported; they + * should become splitting first, unmapped, merged, + * and mapped back in on-demand. + */ + VM_BUG_ON(pmd_pfn(old_pmd) != pmd_pfn(*new_pmd)); + pmd_clear(pmd); kvm_tlb_flush_vmid_ipa(kvm, addr); } else {

6 years, 11 months

1
0
0 0

FAILED: patch "[PATCH] KVM: arm/arm64: Skip updating PMD entry if no change" failed to apply to 4.4-stable tree

by gregkh＠linuxfoundation.org

The patch below does not apply to the 4.4-stable tree. If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <stable(a)vger.kernel.org>. thanks, greg k-h ------------------ original commit in Linus's tree ------------------ >From 86658b819cd0a9aa584cd84453ed268a6f013770 Mon Sep 17 00:00:00 2001 From: Punit Agrawal <punit.agrawal(a)arm.com> Date: Mon, 13 Aug 2018 11:43:50 +0100 Subject: [PATCH] KVM: arm/arm64: Skip updating PMD entry if no change Contention on updating a PMD entry by a large number of vcpus can lead to duplicate work when handling stage 2 page faults. As the page table update follows the break-before-make requirement of the architecture, it can lead to repeated refaults due to clearing the entry and flushing the tlbs. This problem is more likely when - * there are large number of vcpus * the mapping is large block mapping such as when using PMD hugepages (512MB) with 64k pages. Fix this by skipping the page table update if there is no change in the entry being updated. Cc: stable(a)vger.kernel.org Fixes: ad361f093c1e ("KVM: ARM: Support hugetlbfs backed huge pages") Reviewed-by: Suzuki Poulose <suzuki.poulose(a)arm.com> Acked-by: Christoffer Dall <christoffer.dall(a)arm.com> Signed-off-by: Punit Agrawal <punit.agrawal(a)arm.com> Signed-off-by: Marc Zyngier <marc.zyngier(a)arm.com> diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c index 97d27cd9c654..13dfe36501aa 100644 --- a/virt/kvm/arm/mmu.c +++ b/virt/kvm/arm/mmu.c @@ -1044,19 +1044,35 @@ static int stage2_set_pmd_huge(struct kvm *kvm, struct kvm_mmu_memory_cache pmd = stage2_get_pmd(kvm, cache, addr); VM_BUG_ON(!pmd); - /* - * Mapping in huge pages should only happen through a fault. If a - * page is merged into a transparent huge page, the individual - * subpages of that huge page should be unmapped through MMU - * notifiers before we get here. - * - * Merging of CompoundPages is not supported; they should become - * splitting first, unmapped, merged, and mapped back in on-demand. - */ - VM_BUG_ON(pmd_present(*pmd) && pmd_pfn(*pmd) != pmd_pfn(*new_pmd)); - old_pmd = *pmd; if (pmd_present(old_pmd)) { + /* + * Multiple vcpus faulting on the same PMD entry, can + * lead to them sequentially updating the PMD with the + * same value. Following the break-before-make + * (pmd_clear() followed by tlb_flush()) process can + * hinder forward progress due to refaults generated + * on missing translations. + * + * Skip updating the page table if the entry is + * unchanged. + */ + if (pmd_val(old_pmd) == pmd_val(*new_pmd)) + return 0; + + /* + * Mapping in huge pages should only happen through a + * fault. If a page is merged into a transparent huge + * page, the individual subpages of that huge page + * should be unmapped through MMU notifiers before we + * get here. + * + * Merging of CompoundPages is not supported; they + * should become splitting first, unmapped, merged, + * and mapped back in on-demand. + */ + VM_BUG_ON(pmd_pfn(old_pmd) != pmd_pfn(*new_pmd)); + pmd_clear(pmd); kvm_tlb_flush_vmid_ipa(kvm, addr); } else {

6 years, 11 months

1
0
0 0

FAILED: patch "[PATCH] arm64: mm: check for upper PAGE_SHIFT bits in pfn_valid()" failed to apply to 3.18-stable tree

by gregkh＠linuxfoundation.org

The patch below does not apply to the 3.18-stable tree. If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <stable(a)vger.kernel.org>. thanks, greg k-h ------------------ original commit in Linus's tree ------------------ >From 5ad356eabc47d26a92140a0c4b20eba471c10de3 Mon Sep 17 00:00:00 2001 From: Greg Hackmann <ghackmann(a)android.com> Date: Wed, 15 Aug 2018 12:51:21 -0700 Subject: [PATCH] arm64: mm: check for upper PAGE_SHIFT bits in pfn_valid() ARM64's pfn_valid() shifts away the upper PAGE_SHIFT bits of the input before seeing if the PFN is valid. This leads to false positives when some of the upper bits are set, but the lower bits match a valid PFN. For example, the following userspace code looks up a bogus entry in /proc/kpageflags: int pagemap = open("/proc/self/pagemap", O_RDONLY); int pageflags = open("/proc/kpageflags", O_RDONLY); uint64_t pfn, val; lseek64(pagemap, [...], SEEK_SET); read(pagemap, &pfn, sizeof(pfn)); if (pfn & (1UL << 63)) { /* valid PFN */ pfn &= ((1UL << 55) - 1); /* clear flag bits */ pfn |= (1UL << 55); lseek64(pageflags, pfn * sizeof(uint64_t), SEEK_SET); read(pageflags, &val, sizeof(val)); } On ARM64 this causes the userspace process to crash with SIGSEGV rather than reading (1 << KPF_NOPAGE). kpageflags_read() treats the offset as valid, and stable_page_flags() will try to access an address between the user and kernel address ranges. Fixes: c1cc1552616d ("arm64: MMU initialisation") Cc: stable(a)vger.kernel.org Signed-off-by: Greg Hackmann <ghackmann(a)google.com> Signed-off-by: Will Deacon <will.deacon(a)arm.com> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 325cfb3b858a..811f9f8b3bb0 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -287,7 +287,11 @@ static void __init zone_sizes_init(unsigned long min, unsigned long max) #ifdef CONFIG_HAVE_ARCH_PFN_VALID int pfn_valid(unsigned long pfn) { - return memblock_is_map_memory(pfn << PAGE_SHIFT); + phys_addr_t addr = pfn << PAGE_SHIFT; + + if ((addr >> PAGE_SHIFT) != pfn) + return 0; + return memblock_is_map_memory(addr); } EXPORT_SYMBOL(pfn_valid); #endif

6 years, 11 months

1
0
0 0

FAILED: patch "[PATCH] arm64: Fix mismatched cache line size detection" failed to apply to 4.14-stable tree

by gregkh＠linuxfoundation.org

The patch below does not apply to the 4.14-stable tree. If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <stable(a)vger.kernel.org>. thanks, greg k-h ------------------ original commit in Linus's tree ------------------ >From 4c4a39dd5fe2d13e2d2fa5fceb8ef95d19fc389a Mon Sep 17 00:00:00 2001 From: Suzuki K Poulose <suzuki.poulose(a)arm.com> Date: Wed, 4 Jul 2018 23:07:45 +0100 Subject: [PATCH] arm64: Fix mismatched cache line size detection If there is a mismatch in the I/D min line size, we must always use the system wide safe value both in applications and in the kernel, while performing cache operations. However, we have been checking more bits than just the min line sizes, which triggers false negatives. We may need to trap the user accesses in such cases, but not necessarily patch the kernel. This patch fixes the check to do the right thing as advertised. A new capability will be added to check mismatches in other fields and ensure we trap the CTR accesses. Fixes: be68a8aaf925 ("arm64: cpufeature: Fix CTR_EL0 field definitions") Cc: <stable(a)vger.kernel.org> Cc: Mark Rutland <mark.rutland(a)arm.com> Cc: Catalin Marinas <catalin.marinas(a)arm.com> Reported-by: Will Deacon <will.deacon(a)arm.com> Signed-off-by: Suzuki K Poulose <suzuki.poulose(a)arm.com> Signed-off-by: Will Deacon <will.deacon(a)arm.com> diff --git a/arch/arm64/include/asm/cache.h b/arch/arm64/include/asm/cache.h index 5df5cfe1c143..5ee5bca8c24b 100644 --- a/arch/arm64/include/asm/cache.h +++ b/arch/arm64/include/asm/cache.h @@ -21,12 +21,16 @@ #define CTR_L1IP_SHIFT 14 #define CTR_L1IP_MASK 3 #define CTR_DMINLINE_SHIFT 16 +#define CTR_IMINLINE_SHIFT 0 #define CTR_ERG_SHIFT 20 #define CTR_CWG_SHIFT 24 #define CTR_CWG_MASK 15 #define CTR_IDC_SHIFT 28 #define CTR_DIC_SHIFT 29 +#define CTR_CACHE_MINLINE_MASK \ + (0xf << CTR_DMINLINE_SHIFT | 0xf << CTR_IMINLINE_SHIFT) + #define CTR_L1IP(ctr) (((ctr) >> CTR_L1IP_SHIFT) & CTR_L1IP_MASK) #define ICACHE_POLICY_VPIPT 0 diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index 1d2b6d768efe..5d1fa928ea4b 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -68,9 +68,11 @@ static bool has_mismatched_cache_line_size(const struct arm64_cpu_capabilities *entry, int scope) { + u64 mask = CTR_CACHE_MINLINE_MASK; + WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible()); - return (read_cpuid_cachetype() & arm64_ftr_reg_ctrel0.strict_mask) != - (arm64_ftr_reg_ctrel0.sys_val & arm64_ftr_reg_ctrel0.strict_mask); + return (read_cpuid_cachetype() & mask) != + (arm64_ftr_reg_ctrel0.sys_val & mask); } static void diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index f24892a40d2c..25d5cef00333 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -214,7 +214,7 @@ static const struct arm64_ftr_bits ftr_ctr[] = { * If we have differing I-cache policies, report it as the weakest - VIPT. */ ARM64_FTR_BITS(FTR_VISIBLE, FTR_NONSTRICT, FTR_EXACT, 14, 2, ICACHE_POLICY_VIPT), /* L1Ip */ - ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, 0, 4, 0), /* IminLine */ + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, CTR_IMINLINE_SHIFT, 4, 0), ARM64_FTR_END, };

6 years, 11 months

1
0
0 0

FAILED: patch "[PATCH] arm64: Handle mismatched cache type" failed to apply to 4.14-stable tree

by gregkh＠linuxfoundation.org

The patch below does not apply to the 4.14-stable tree. If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <stable(a)vger.kernel.org>. thanks, greg k-h ------------------ original commit in Linus's tree ------------------ >From 314d53d297980676011e6fd83dac60db4a01dc70 Mon Sep 17 00:00:00 2001 From: Suzuki K Poulose <suzuki.poulose(a)arm.com> Date: Wed, 4 Jul 2018 23:07:46 +0100 Subject: [PATCH] arm64: Handle mismatched cache type Track mismatches in the cache type register (CTR_EL0), other than the D/I min line sizes and trap user accesses if there are any. Fixes: be68a8aaf925 ("arm64: cpufeature: Fix CTR_EL0 field definitions") Cc: <stable(a)vger.kernel.org> Cc: Mark Rutland <mark.rutland(a)arm.com> Cc: Will Deacon <will.deacon(a)arm.com> Cc: Catalin Marinas <catalin.marinas(a)arm.com> Signed-off-by: Suzuki K Poulose <suzuki.poulose(a)arm.com> Signed-off-by: Will Deacon <will.deacon(a)arm.com> diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index 8a699c708fc9..be3bf3d08916 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -49,7 +49,8 @@ #define ARM64_HAS_CACHE_DIC 28 #define ARM64_HW_DBM 29 #define ARM64_SSBD 30 +#define ARM64_MISMATCHED_CACHE_TYPE 31 -#define ARM64_NCAPS 31 +#define ARM64_NCAPS 32 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index 5d1fa928ea4b..5d59ff9a8da9 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -65,11 +65,15 @@ is_kryo_midr(const struct arm64_cpu_capabilities *entry, int scope) } static bool -has_mismatched_cache_line_size(const struct arm64_cpu_capabilities *entry, - int scope) +has_mismatched_cache_type(const struct arm64_cpu_capabilities *entry, + int scope) { u64 mask = CTR_CACHE_MINLINE_MASK; + /* Skip matching the min line sizes for cache type check */ + if (entry->capability == ARM64_MISMATCHED_CACHE_TYPE) + mask ^= arm64_ftr_reg_ctrel0.strict_mask; + WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible()); return (read_cpuid_cachetype() & mask) != (arm64_ftr_reg_ctrel0.sys_val & mask); @@ -615,7 +619,14 @@ const struct arm64_cpu_capabilities arm64_errata[] = { { .desc = "Mismatched cache line size", .capability = ARM64_MISMATCHED_CACHE_LINE_SIZE, - .matches = has_mismatched_cache_line_size, + .matches = has_mismatched_cache_type, + .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, + .cpu_enable = cpu_enable_trap_ctr_access, + }, + { + .desc = "Mismatched cache type", + .capability = ARM64_MISMATCHED_CACHE_TYPE, + .matches = has_mismatched_cache_type, .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, .cpu_enable = cpu_enable_trap_ctr_access, },

6 years, 11 months

1
0
0 0

Resending: Please apply commit d1e20222d537 to 4.14.y and 4.18.y

by Jitendra Bhivare

Subject of the patch: iommu/arm-smmu: Error out only if not enough context interrupts Commit ID: 42b88ec6530fd76f1ae06de7f09830bcbca5bbd6 Why: ARM SMMU does not initialize for Broadcom Stingray SoC if bootloader reserves few contexts. Kernel should not error out if there are enough context interrupts. What kernel versions: 4.14.y and 4.18.y (applies cleanly). Before patching: Kernel does not boot using eMMC on Broadcom Stingray SoC with latest bootloader emitting this error "arm-smmu 64000000.mmu: found only 64 context interrupt(s) but 63 required" After patching: Kernel cleanly boots on the SoC. Thanks, JB

6 years, 11 months

2
1
0 0

Re: [PATCH] nohz: Fix missing tick reprog while interrupting inline timer softirq

by Grygorii Strashko

Hi On 07/31/2018 05:52 PM, Frederic Weisbecker wrote: > Before updating the full nohz tick or the idle time on IRQ exit, we > check first if we are not in a nesting interrupt, whether the inner > interrupt is a hard or a soft IRQ. > > There is a historical reason for that: the dyntick idle mode used to > reprogram the tick on IRQ exit, after softirq processing, and there was > no point in doing that job in the outer nesting interrupt because the > tick update will be performed through the end of the inner interrupt > eventually, with even potential new timer updates. > > One corner case could show up though: if an idle tick interrupts a softirq > executing inline in the idle loop (through a call to local_bh_enable()) > after we entered in dynticks mode, the IRQ won't reprogram the tick > because it assumes the softirq executes on an inner IRQ-tail. As a > result we might put the CPU in sleep mode with the tick completely > stopped whereas a timer can still be enqueued. Indeed there is no tick > reprogramming in local_bh_enable(). We probably asssumed there was no bh > disabled section in idle, although there didn't seem to be debug code > ensuring that. > > Nowadays the nesting interrupt optimization still stands but only concern > full dynticks. The tick is stopped on IRQ exit in full dynticks mode > and we want to wait for the end of the inner IRQ to reprogramm the tick. > But in_interrupt() doesn't make a difference between softirqs executing > on IRQ tail and those executing inline. What was to be considered a > corner case in dynticks-idle mode now becomes a serious opportunity for > a bug in full dynticks mode: if a tick interrupts a task executing > softirq inline, the tick reprogramming will be ignored and we may exit > to userspace after local_bh_enable() with an enqueued timer that will > never fire. > > To fix this, simply keep reprogramming the tick if we are in a hardirq > interrupting softirq. We can still figure out a way later to restore > this optimization while excluding inline softirq processing. > > Reported-by: Anna-Maria Gleixner <anna-maria(a)linutronix.de> > Signed-off-by: Frederic Weisbecker <frederic(a)kernel.org> > Cc: Thomas Gleixner <tglx(a)linutronix.de> > Cc: Ingo Molnar <mingo(a)kernel.org> > Tested-by: Anna-Maria Gleixner <anna-maria(a)linutronix.de> > --- > kernel/softirq.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/kernel/softirq.c b/kernel/softirq.c > index 900dcfe..0980a81 100644 > --- a/kernel/softirq.c > +++ b/kernel/softirq.c > @@ -386,7 +386,7 @@ static inline void tick_irq_exit(void) > > /* Make sure that timer wheel updates are propagated */ > if ((idle_cpu(cpu) && !need_resched()) || tick_nohz_full_cpu(cpu)) { > - if (!in_interrupt()) > + if (!in_irq()) > tick_nohz_irq_exit(); > } > #endif > This patch was back ported to the Stable linux-4.14.y and It causes regression - flood of "NOHZ: local_softirq_pending" messages on all TI boards during boot (NFS boot): [ 4.179796] NOHZ: local_softirq_pending 2c2 in sirq 256 [ 4.185051] NOHZ: local_softirq_pending 2c2 in sirq 256 the same is not reproducible with LKML - seems due to changes in tick-sched.c __tick_nohz_idle_enter()/tick_nohz_irq_exit(). I've generated backtrace from can_stop_idle_tick() (see below) and seems this patch makes tick_nohz_irq_exit() call unconditional in case of nested interrupt: gic_handle_irq |- irq_exit |- preempt_count_sub(HARDIRQ_OFFSET); <-- [1] |-__do_softirq <irqs enabled> |- gic_handle_irq() |- irq_exit() |- tick_irq_exit() if (!in_irq()) <-- My understanding is that this condition will be always true due to [1] tick_nohz_irq_exit(); |-__tick_nohz_idle_enter() |- can_stop_idle_tick() Sry, not sure if my conclusion is right and how can it be fixed. [ 3.842320] NOHZ: local_softirq_pending 40 in sirq 256 [ 3.847485] ------------[ cut here ]------------ [ 3.852133] WARNING: CPU: 0 PID: 0 at kernel/time/tick-sched.c:915 __tick_nohz_idle_enter+0x4b8/0x568 [ 3.861393] Modules linked in: [ 3.864469] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.14.66-01768-gc26f664-dirty #311 [ 3.872506] Hardware name: Generic DRA74X (Flattened Device Tree) [ 3.878623] Backtrace: [ 3.881091] [<c010c050>] (dump_backtrace) from [<c010c320>] (show_stack+0x18/0x1c) [ 3.888696] r7:00000009 r6:600f0193 r5:00000000 r4:c0c5fca4 [ 3.894386] [<c010c308>] (show_stack) from [<c07ad028>] (dump_stack+0x8c/0xa0) [ 3.901645] [<c07acf9c>] (dump_stack) from [<c012e558>] (__warn+0xec/0x104) [ 3.908638] r7:00000009 r6:c0996d08 r5:00000000 r4:00000000 [ 3.914329] [<c012e46c>] (__warn) from [<c012e628>] (warn_slowpath_null+0x28/0x30) [ 3.921933] r9:00000000 r8:e4e1f7de r7:c0c8c1d8 r6:c0c65180 r5:00000000 r4:eed408e8 [ 3.929715] [<c012e600>] (warn_slowpath_null) from [<c01a9780>] (__tick_nohz_idle_enter+0x4b8/0x568) [ 3.938890] [<c01a92c8>] (__tick_nohz_idle_enter) from [<c01a9c74>] (tick_nohz_irq_exit+0x2c/0x30) [ 3.947890] r10:c0c01f50 r9:c0c00000 r8:ee008000 r7:00000000 r6:c0c01ee0 r5:00000048 [ 3.955752] r4:c0b62afc [ 3.958301] [<c01a9c48>] (tick_nohz_irq_exit) from [<c0133748>] (irq_exit+0xf4/0x144) [ 3.966173] [<c0133654>] (irq_exit) from [<c0181e6c>] (__handle_domain_irq+0x68/0xbc) [ 3.974043] [<c0181e04>] (__handle_domain_irq) from [<c0101508>] (gic_handle_irq+0x44/0x80) [ 3.982432] r9:c0c00000 r8:fa213000 r7:fa212000 r6:c0c01dd0 r5:fa21200c r4:c0c03ff0 [ 3.990211] [<c01014c4>] (gic_handle_irq) from [<c010cf4c>] (__irq_svc+0x6c/0xa8) [ 3.997726] Exception stack(0xc0c01dd0 to 0xc0c01e18) [ 4.002800] 1dc0: 00000000 c0c65180 00000000 00000000 [ 4.011017] 1de0: 00000280 00000013 c0c00000 00000000 ee008000 c0c00000 c0c01f50 c0c01e7c [ 4.019232] 1e00: c0c01e80 c0c01e20 c0133730 c01015dc 600f0113 ffffffff [ 4.025877] r9:c0c00000 r8:ee008000 r7:c0c01e04 r6:ffffffff r5:600f0113 r4:c01015dc [ 4.033659] [<c0101548>] (__do_softirq) from [<c0133730>] (irq_exit+0xdc/0x144) [ 4.041002] r10:c0c01f50 r9:c0c00000 r8:ee008000 r7:00000000 r6:00000000 r5:00000013 [ 4.048863] r4:c0b62afc [ 4.051414] [<c0133654>] (irq_exit) from [<c0181e6c>] (__handle_domain_irq+0x68/0xbc) [ 4.059280] [<c0181e04>] (__handle_domain_irq) from [<c0101508>] (gic_handle_irq+0x44/0x80) [ 4.067670] r9:c0c00000 r8:fa213000 r7:fa212000 r6:c0c01ee0 r5:fa21200c r4:c0c03ff0 [ 4.075448] [<c01014c4>] (gic_handle_irq) from [<c010cf4c>] (__irq_svc+0x6c/0xa8) [ 4.082963] Exception stack(0xc0c01ee0 to 0xc0c01f28) [ 4.088041] 1ee0: 00000001 00000000 fe600000 00000000 c0c00000 c0c03cc8 c0c03c68 c0b623b8 [ 4.096258] 1f00: 00000000 00000000 c0c01f50 c0c01f3c c0c01f1c c0c01f30 c0120f14 c0108ae4 [ 4.104469] 1f20: 600f0013 ffffffff [ 4.107975] r9:c0c00000 r8:00000000 r7:c0c01f14 r6:ffffffff r5:600f0013 r4:c0108ae4 [ 4.115760] [<c0108abc>] (arch_cpu_idle) from [<c07c6a14>] (default_idle_call+0x28/0x34) [ 4.123894] [<c07c69ec>] (default_idle_call) from [<c016e4a4>] (do_idle+0x180/0x214) [ 4.131676] [<c016e324>] (do_idle) from [<c016e7fc>] (cpu_startup_entry+0x20/0x24) [ 4.139283] r10:effff7c0 r9:c0b48a30 r8:c0c64000 r7:c0c03c40 r6:ffffffff r5:00000002 [ 4.147146] r4:000000be [ 4.149698] [<c016e7dc>] (cpu_startup_entry) from [<c07c12e4>] (rest_init+0xd8/0xdc) [ 4.157483] [<c07c120c>] (rest_init) from [<c0b00d8c>] (start_kernel+0x3cc/0x3d8) [ 4.164997] r5:c0c64000 r4:c0c6404c [ 4.168592] [<c0b009c0>] (start_kernel) from [<8000807c>] (0x8000807c) [ 4.175148] ---[ end trace 9c10a64bf81ad3fe ]--- -- regards, -grygorii

6 years, 11 months

5
9
0 0

[PATCH v2] x86/nmi: Fix some races in NMI uaccess

by Andy Lutomirski

In NMI context, we might be in the middle of context switching or in the middle of switch_mm_irqs_off(). In either case, CR3 might not match current->mm, which could cause copy_from_user_nmi() and friends to read the wrong memory. Fix it by adding a new nmi_uaccess_okay() helper and checking it in copy_from_user_nmi() and in __copy_from_user_nmi()'s callers. Cc: stable(a)vger.kernel.org Cc: Peter Zijlstra <peterz(a)infradead.org> Cc: Nadav Amit <nadav.amit(a)gmail.com> Signed-off-by: Andy Lutomirski <luto(a)kernel.org> --- Nadav, this is intended for your series. Want to add it right before the use_temporary_mm() stuff? Changes from v1: - Improved comments - Add debugging - Fix bugs arch/x86/events/core.c | 2 +- arch/x86/include/asm/tlbflush.h | 44 +++++++++++++++++++++++++++++++++ arch/x86/lib/usercopy.c | 5 ++++ arch/x86/mm/tlb.c | 7 ++++++ 4 files changed, 57 insertions(+), 1 deletion(-) diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c index 5f4829f10129..dfb2f7c0d019 100644 --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -2465,7 +2465,7 @@ perf_callchain_user(struct perf_callchain_entry_ctx *entry, struct pt_regs *regs perf_callchain_store(entry, regs->ip); - if (!current->mm) + if (!nmi_uaccess_okay()) return; if (perf_callchain_user32(regs, entry)) diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h index 29c9da6c62fc..183d36a98b04 100644 --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -175,8 +175,16 @@ struct tlb_state { * are on. This means that it may not match current->active_mm, * which will contain the previous user mm when we're in lazy TLB * mode even if we've already switched back to swapper_pg_dir. + * + * During switch_mm_irqs_off(), loaded_mm will be set to + * LOADED_MM_SWITCHING during the brief interrupts-off window + * when CR3 and loaded_mm would otherwise be inconsistent. This + * is for nmi_uaccess_okay()'s benefit. */ struct mm_struct *loaded_mm; + +#define LOADED_MM_SWITCHING ((struct mm_struct *)1) + u16 loaded_mm_asid; u16 next_asid; /* last user mm's ctx id */ @@ -246,6 +254,42 @@ struct tlb_state { }; DECLARE_PER_CPU_SHARED_ALIGNED(struct tlb_state, cpu_tlbstate); +/* + * Blindly accessing user memory from NMI context can be dangerous + * if we're in the middle of switching the current user task or + * switching the loaded mm. It can also be dangerous if we + * interrupted some kernel code that was temporarily using a + * different mm. + */ +static inline bool nmi_uaccess_okay(void) +{ + struct mm_struct *loaded_mm = this_cpu_read(cpu_tlbstate.loaded_mm); + struct mm_struct *current_mm = current->mm; + +#ifdef CONFIG_DEBUG_VM + WARN_ON_ONCE(!loaded_mm); +#endif + + /* + * The condition we want to check is + * current_mm->pgd == __va(read_cr3_pa()). This may be slow, though, + * if we're running in a VM with shadow paging, and nmi_uaccess_okay() + * is supposed to be reasonably fast. + * + * Instead, we check the almost equivalent but somewhat conservative + * condition below, and we rely on the fact that switch_mm_irqs_off() + * sets loaded_mm to LOADED_MM_SWITCHING before writing to CR3. + */ + if (loaded_mm != current_mm) + return false; + +#ifdef CONFIG_DEBUG_VM + WARN_ON_ONCE(current_mm->pgd != __va(read_cr3_pa())); +#endif + + return true; +} + /* Initialize cr4 shadow for this CPU. */ static inline void cr4_init_shadow(void) { diff --git a/arch/x86/lib/usercopy.c b/arch/x86/lib/usercopy.c index c8c6ad0d58b8..3f435d7fca5e 100644 --- a/arch/x86/lib/usercopy.c +++ b/arch/x86/lib/usercopy.c @@ -7,6 +7,8 @@ #include <linux/uaccess.h> #include <linux/export.h> +#include <asm/tlbflush.h> + /* * We rely on the nested NMI work to allow atomic faults from the NMI path; the * nested NMI paths are careful to preserve CR2. @@ -19,6 +21,9 @@ copy_from_user_nmi(void *to, const void __user *from, unsigned long n) if (__range_not_ok(from, n, TASK_SIZE)) return n; + if (!nmi_uaccess_okay()) + return n; + /* * Even though this function is typically called from NMI/IRQ context * disable pagefaults so that its behaviour is consistent even when diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index e721cf2d4290..707d2b2a333f 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -304,6 +304,10 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next, choose_new_asid(next, next_tlb_gen, &new_asid, &need_flush); + /* Let nmi_uaccess_okay() know that we're changing CR3. */ + this_cpu_write(cpu_tlbstate.loaded_mm, LOADED_MM_SWITCHING); + barrier(); + if (need_flush) { this_cpu_write(cpu_tlbstate.ctxs[new_asid].ctx_id, next->context.ctx_id); this_cpu_write(cpu_tlbstate.ctxs[new_asid].tlb_gen, next_tlb_gen); @@ -334,6 +338,9 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next, if (next != &init_mm) this_cpu_write(cpu_tlbstate.last_ctx_id, next->context.ctx_id); + /* Make sure we write CR3 before loaded_mm. */ + barrier(); + this_cpu_write(cpu_tlbstate.loaded_mm, next); this_cpu_write(cpu_tlbstate.loaded_mm_asid, new_asid); } -- 2.17.1

6 years, 11 months

5
5
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-stable-mirror