- Linux-stable-mirror - lists.linaro.org

[Linux-stable-mirror] [PATCH for v4.9-stable] sched: fix softirq time accounting

by Rabin Vincent

From: Rabin Vincent <rabinv(a)axis.com> softirq time accounting is broken on v4.9.x if ksoftirqd runs. With CONFIG_IRQ_TIME_ACCOUNTING=y # CONFIG_VIRT_CPU_ACCOUNTING_GEN is not set this test code: struct tasklet_struct tasklet; static void delay_tasklet(unsigned long data) { udelay(10); tasklet_schedule(&tasklet); } tasklet_init(&tasklet, delay_tasklet, 0); tasklet_schedule(&tasklet); results in: $ while :; do grep cpu0 /proc/stat; done cpu0 5 0 80 25 16 107 1 0 0 0 cpu0 5 0 80 25 16 107 0 0 0 0 cpu0 5 0 80 25 16 107 0 0 0 0 cpu0 5 0 80 25 16 107 0 0 0 0 cpu0 5 0 81 25 16 107 0 0 0 0 cpu0 5 0 81 25 16 107 1 0 0 0 cpu0 5 0 81 25 16 108 18446744073709551615 0 0 0 cpu0 5 0 81 25 16 108 18446744073709551615 0 0 0 cpu0 5 0 81 25 16 108 18446744073709551615 0 0 0 cpu0 5 0 81 25 16 108 0 0 0 0 cpu0 6 0 81 25 16 108 0 0 0 0 cpu0 6 0 81 25 16 108 0 0 0 0 As can be seen, the softirq numbers are totally bogus. When ksoftirq is running, irqtime_account_process_tick() increments cpustat[CPUSTAT_SOFTIRQ]. This causes the "nsecs_to_cputime64(irqtime) - cpustat[CPUSTAT_SOFTIRQ]" calculation in irqtime_account_update() to underflow the next time a softirq is handled leading to the above values. The underflow bug was added by 57430218317e5b280 ("sched/cputime: Count actually elapsed irq & softirq time"). But ksoftirqd accounting was wrong even in earlier kernels. In earlier kernels, after a ksoftirq run, the kernel would simply stop accounting softirq time spent outside of ksoftirqd until that (accumulated) time exceeded the time for which ksofirqd previously had run. Fix both the underflow and the wrong accounting by using a counter specifically for the non-ksoftirqd softirq case. This code has been fixed in current mainline by a499a5a14db ("sched/cputime: Increment kcpustat directly on irqtime account") [note also the followup 25e2d8c1b9e327e ("sched/cputime: Fix ksoftirqd cputime accounting regression")], but that patch is a part of the many changes for eliminating of cputime_t so it does not seem suitable for backport. Signed-off-by: Rabin Vincent <rabinv(a)axis.com> --- include/linux/kernel_stat.h | 1 + kernel/sched/cputime.c | 9 ++++++++- 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/include/linux/kernel_stat.h b/include/linux/kernel_stat.h index 44fda64..d0826f1 100644 --- a/include/linux/kernel_stat.h +++ b/include/linux/kernel_stat.h @@ -33,6 +33,7 @@ enum cpu_usage_stat { struct kernel_cpustat { u64 cpustat[NR_STATS]; + u64 softirq_no_ksoftirqd; }; struct kernel_stat { diff --git a/kernel/sched/cputime.c b/kernel/sched/cputime.c index 5ebee31..1b5a9e6 100644 --- a/kernel/sched/cputime.c +++ b/kernel/sched/cputime.c @@ -73,12 +73,19 @@ EXPORT_SYMBOL_GPL(irqtime_account_irq); static cputime_t irqtime_account_update(u64 irqtime, int idx, cputime_t maxtime) { u64 *cpustat = kcpustat_this_cpu->cpustat; + u64 base = cpustat[idx]; cputime_t irq_cputime; - irq_cputime = nsecs_to_cputime64(irqtime) - cpustat[idx]; + if (idx == CPUTIME_SOFTIRQ) + base = kcpustat_this_cpu->softirq_no_ksoftirqd; + + irq_cputime = nsecs_to_cputime64(irqtime) - base; irq_cputime = min(irq_cputime, maxtime); cpustat[idx] += irq_cputime; + if (idx == CPUTIME_SOFTIRQ) + kcpustat_this_cpu->softirq_no_ksoftirqd += irq_cputime; + return irq_cputime; } -- 2.1.4

7 years, 5 months

2
2
0 0

[Linux-stable-mirror] FAILED: patch "[PATCH] libnvdimm, btt: Fix an incompatibility in the log layout" failed to apply to 4.9-stable tree

by gregkh＠linuxfoundation.org

The patch below does not apply to the 4.9-stable tree. If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <stable(a)vger.kernel.org>. thanks, greg k-h ------------------ original commit in Linus's tree ------------------ >From 24e3a7fb60a9187e5df90e5fa655ffc94b9c4f77 Mon Sep 17 00:00:00 2001 From: Vishal Verma <vishal.l.verma(a)intel.com> Date: Mon, 18 Dec 2017 09:28:39 -0700 Subject: [PATCH] libnvdimm, btt: Fix an incompatibility in the log layout Due to a spec misinterpretation, the Linux implementation of the BTT log area had different padding scheme from other implementations, such as UEFI and NVML. This fixes the padding scheme, and defaults to it for new BTT layouts. We attempt to detect the padding scheme in use when probing for an existing BTT. If we detect the older/incompatible scheme, we continue using it. Reported-by: Juston Li <juston.li(a)intel.com> Cc: Dan Williams <dan.j.williams(a)intel.com> Cc: <stable(a)vger.kernel.org> Fixes: 5212e11fde4d ("nd_btt: atomic sector updates") Signed-off-by: Vishal Verma <vishal.l.verma(a)intel.com> Signed-off-by: Dan Williams <dan.j.williams(a)intel.com> diff --git a/drivers/nvdimm/btt.c b/drivers/nvdimm/btt.c index e949e3302af4..c586bcdb5190 100644 --- a/drivers/nvdimm/btt.c +++ b/drivers/nvdimm/btt.c @@ -211,12 +211,12 @@ static int btt_map_read(struct arena_info *arena, u32 lba, u32 *mapping, return ret; } -static int btt_log_read_pair(struct arena_info *arena, u32 lane, - struct log_entry *ent) +static int btt_log_group_read(struct arena_info *arena, u32 lane, + struct log_group *log) { return arena_read_bytes(arena, - arena->logoff + (2 * lane * LOG_ENT_SIZE), ent, - 2 * LOG_ENT_SIZE, 0); + arena->logoff + (lane * LOG_GRP_SIZE), log, + LOG_GRP_SIZE, 0); } static struct dentry *debugfs_root; @@ -256,6 +256,8 @@ static void arena_debugfs_init(struct arena_info *a, struct dentry *parent, debugfs_create_x64("logoff", S_IRUGO, d, &a->logoff); debugfs_create_x64("info2off", S_IRUGO, d, &a->info2off); debugfs_create_x32("flags", S_IRUGO, d, &a->flags); + debugfs_create_u32("log_index_0", S_IRUGO, d, &a->log_index[0]); + debugfs_create_u32("log_index_1", S_IRUGO, d, &a->log_index[1]); } static void btt_debugfs_init(struct btt *btt) @@ -274,6 +276,11 @@ static void btt_debugfs_init(struct btt *btt) } } +static u32 log_seq(struct log_group *log, int log_idx) +{ + return le32_to_cpu(log->ent[log_idx].seq); +} + /* * This function accepts two log entries, and uses the * sequence number to find the 'older' entry. @@ -283,8 +290,10 @@ static void btt_debugfs_init(struct btt *btt) * * TODO The logic feels a bit kludge-y. make it better.. */ -static int btt_log_get_old(struct log_entry *ent) +static int btt_log_get_old(struct arena_info *a, struct log_group *log) { + int idx0 = a->log_index[0]; + int idx1 = a->log_index[1]; int old; /* @@ -292,23 +301,23 @@ static int btt_log_get_old(struct log_entry *ent) * the next time, the following logic works out to put this * (next) entry into [1] */ - if (ent[0].seq == 0) { - ent[0].seq = cpu_to_le32(1); + if (log_seq(log, idx0) == 0) { + log->ent[idx0].seq = cpu_to_le32(1); return 0; } - if (ent[0].seq == ent[1].seq) + if (log_seq(log, idx0) == log_seq(log, idx1)) return -EINVAL; - if (le32_to_cpu(ent[0].seq) + le32_to_cpu(ent[1].seq) > 5) + if (log_seq(log, idx0) + log_seq(log, idx1) > 5) return -EINVAL; - if (le32_to_cpu(ent[0].seq) < le32_to_cpu(ent[1].seq)) { - if (le32_to_cpu(ent[1].seq) - le32_to_cpu(ent[0].seq) == 1) + if (log_seq(log, idx0) < log_seq(log, idx1)) { + if ((log_seq(log, idx1) - log_seq(log, idx0)) == 1) old = 0; else old = 1; } else { - if (le32_to_cpu(ent[0].seq) - le32_to_cpu(ent[1].seq) == 1) + if ((log_seq(log, idx0) - log_seq(log, idx1)) == 1) old = 1; else old = 0; @@ -328,17 +337,18 @@ static int btt_log_read(struct arena_info *arena, u32 lane, { int ret; int old_ent, ret_ent; - struct log_entry log[2]; + struct log_group log; - ret = btt_log_read_pair(arena, lane, log); + ret = btt_log_group_read(arena, lane, &log); if (ret) return -EIO; - old_ent = btt_log_get_old(log); + old_ent = btt_log_get_old(arena, &log); if (old_ent < 0 || old_ent > 1) { dev_err(to_dev(arena), "log corruption (%d): lane %d seq [%d, %d]\n", - old_ent, lane, log[0].seq, log[1].seq); + old_ent, lane, log.ent[arena->log_index[0]].seq, + log.ent[arena->log_index[1]].seq); /* TODO set error state? */ return -EIO; } @@ -346,7 +356,7 @@ static int btt_log_read(struct arena_info *arena, u32 lane, ret_ent = (old_flag ? old_ent : (1 - old_ent)); if (ent != NULL) - memcpy(ent, &log[ret_ent], LOG_ENT_SIZE); + memcpy(ent, &log.ent[arena->log_index[ret_ent]], LOG_ENT_SIZE); return ret_ent; } @@ -360,17 +370,13 @@ static int __btt_log_write(struct arena_info *arena, u32 lane, u32 sub, struct log_entry *ent, unsigned long flags) { int ret; - /* - * Ignore the padding in log_entry for calculating log_half. - * The entry is 'committed' when we write the sequence number, - * and we want to ensure that that is the last thing written. - * We don't bother writing the padding as that would be extra - * media wear and write amplification - */ - unsigned int log_half = (LOG_ENT_SIZE - 2 * sizeof(u64)) / 2; - u64 ns_off = arena->logoff + (((2 * lane) + sub) * LOG_ENT_SIZE); + u32 group_slot = arena->log_index[sub]; + unsigned int log_half = LOG_ENT_SIZE / 2; void *src = ent; + u64 ns_off; + ns_off = arena->logoff + (lane * LOG_GRP_SIZE) + + (group_slot * LOG_ENT_SIZE); /* split the 16B write into atomic, durable halves */ ret = arena_write_bytes(arena, ns_off, src, log_half, flags); if (ret) @@ -453,7 +459,7 @@ static int btt_log_init(struct arena_info *arena) { size_t logsize = arena->info2off - arena->logoff; size_t chunk_size = SZ_4K, offset = 0; - struct log_entry log; + struct log_entry ent; void *zerobuf; int ret; u32 i; @@ -485,11 +491,11 @@ static int btt_log_init(struct arena_info *arena) } for (i = 0; i < arena->nfree; i++) { - log.lba = cpu_to_le32(i); - log.old_map = cpu_to_le32(arena->external_nlba + i); - log.new_map = cpu_to_le32(arena->external_nlba + i); - log.seq = cpu_to_le32(LOG_SEQ_INIT); - ret = __btt_log_write(arena, i, 0, &log, 0); + ent.lba = cpu_to_le32(i); + ent.old_map = cpu_to_le32(arena->external_nlba + i); + ent.new_map = cpu_to_le32(arena->external_nlba + i); + ent.seq = cpu_to_le32(LOG_SEQ_INIT); + ret = __btt_log_write(arena, i, 0, &ent, 0); if (ret) goto free; } @@ -594,6 +600,123 @@ static int btt_freelist_init(struct arena_info *arena) return 0; } +static bool ent_is_padding(struct log_entry *ent) +{ + return (ent->lba == 0) && (ent->old_map == 0) && (ent->new_map == 0) + && (ent->seq == 0); +} + +/* + * Detecting valid log indices: We read a log group (see the comments in btt.h + * for a description of a 'log_group' and its 'slots'), and iterate over its + * four slots. We expect that a padding slot will be all-zeroes, and use this + * to detect a padding slot vs. an actual entry. + * + * If a log_group is in the initial state, i.e. hasn't been used since the + * creation of this BTT layout, it will have three of the four slots with + * zeroes. We skip over these log_groups for the detection of log_index. If + * all log_groups are in the initial state (i.e. the BTT has never been + * written to), it is safe to assume the 'new format' of log entries in slots + * (0, 1). + */ +static int log_set_indices(struct arena_info *arena) +{ + bool idx_set = false, initial_state = true; + int ret, log_index[2] = {-1, -1}; + u32 i, j, next_idx = 0; + struct log_group log; + u32 pad_count = 0; + + for (i = 0; i < arena->nfree; i++) { + ret = btt_log_group_read(arena, i, &log); + if (ret < 0) + return ret; + + for (j = 0; j < 4; j++) { + if (!idx_set) { + if (ent_is_padding(&log.ent[j])) { + pad_count++; + continue; + } else { + /* Skip if index has been recorded */ + if ((next_idx == 1) && + (j == log_index[0])) + continue; + /* valid entry, record index */ + log_index[next_idx] = j; + next_idx++; + } + if (next_idx == 2) { + /* two valid entries found */ + idx_set = true; + } else if (next_idx > 2) { + /* too many valid indices */ + return -ENXIO; + } + } else { + /* + * once the indices have been set, just verify + * that all subsequent log groups are either in + * their initial state or follow the same + * indices. + */ + if (j == log_index[0]) { + /* entry must be 'valid' */ + if (ent_is_padding(&log.ent[j])) + return -ENXIO; + } else if (j == log_index[1]) { + ; + /* + * log_index[1] can be padding if the + * lane never got used and it is still + * in the initial state (three 'padding' + * entries) + */ + } else { + /* entry must be invalid (padding) */ + if (!ent_is_padding(&log.ent[j])) + return -ENXIO; + } + } + } + /* + * If any of the log_groups have more than one valid, + * non-padding entry, then the we are no longer in the + * initial_state + */ + if (pad_count < 3) + initial_state = false; + pad_count = 0; + } + + if (!initial_state && !idx_set) + return -ENXIO; + + /* + * If all the entries in the log were in the initial state, + * assume new padding scheme + */ + if (initial_state) + log_index[1] = 1; + + /* + * Only allow the known permutations of log/padding indices, + * i.e. (0, 1), and (0, 2) + */ + if ((log_index[0] == 0) && ((log_index[1] == 1) || (log_index[1] == 2))) + ; /* known index possibilities */ + else { + dev_err(to_dev(arena), "Found an unknown padding scheme\n"); + return -ENXIO; + } + + arena->log_index[0] = log_index[0]; + arena->log_index[1] = log_index[1]; + dev_dbg(to_dev(arena), "log_index_0 = %d\n", log_index[0]); + dev_dbg(to_dev(arena), "log_index_1 = %d\n", log_index[1]); + return 0; +} + static int btt_rtt_init(struct arena_info *arena) { arena->rtt = kcalloc(arena->nfree, sizeof(u32), GFP_KERNEL); @@ -650,8 +773,7 @@ static struct arena_info *alloc_arena(struct btt *btt, size_t size, available -= 2 * BTT_PG_SIZE; /* The log takes a fixed amount of space based on nfree */ - logsize = roundup(2 * arena->nfree * sizeof(struct log_entry), - BTT_PG_SIZE); + logsize = roundup(arena->nfree * LOG_GRP_SIZE, BTT_PG_SIZE); available -= logsize; /* Calculate optimal split between map and data area */ @@ -668,6 +790,10 @@ static struct arena_info *alloc_arena(struct btt *btt, size_t size, arena->mapoff = arena->dataoff + datasize; arena->logoff = arena->mapoff + mapsize; arena->info2off = arena->logoff + logsize; + + /* Default log indices are (0,1) */ + arena->log_index[0] = 0; + arena->log_index[1] = 1; return arena; } @@ -758,6 +884,13 @@ static int discover_arenas(struct btt *btt) arena->external_lba_start = cur_nlba; parse_arena_meta(arena, super, cur_off); + ret = log_set_indices(arena); + if (ret) { + dev_err(to_dev(arena), + "Unable to deduce log/padding indices\n"); + goto out; + } + mutex_init(&arena->err_lock); ret = btt_freelist_init(arena); if (ret) diff --git a/drivers/nvdimm/btt.h b/drivers/nvdimm/btt.h index 884fbbbdd18a..db3cb6d4d0d4 100644 --- a/drivers/nvdimm/btt.h +++ b/drivers/nvdimm/btt.h @@ -27,6 +27,7 @@ #define MAP_ERR_MASK (1 << MAP_ERR_SHIFT) #define MAP_LBA_MASK (~((1 << MAP_TRIM_SHIFT) | (1 << MAP_ERR_SHIFT))) #define MAP_ENT_NORMAL 0xC0000000 +#define LOG_GRP_SIZE sizeof(struct log_group) #define LOG_ENT_SIZE sizeof(struct log_entry) #define ARENA_MIN_SIZE (1UL << 24) /* 16 MB */ #define ARENA_MAX_SIZE (1ULL << 39) /* 512 GB */ @@ -50,12 +51,52 @@ enum btt_init_state { INIT_READY }; +/* + * A log group represents one log 'lane', and consists of four log entries. + * Two of the four entries are valid entries, and the remaining two are + * padding. Due to an old bug in the padding location, we need to perform a + * test to determine the padding scheme being used, and use that scheme + * thereafter. + * + * In kernels prior to 4.15, 'log group' would have actual log entries at + * indices (0, 2) and padding at indices (1, 3), where as the correct/updated + * format has log entries at indices (0, 1) and padding at indices (2, 3). + * + * Old (pre 4.15) format: + * +-----------------+-----------------+ + * | ent[0] | ent[1] | + * | 16B | 16B | + * | lba/old/new/seq | pad | + * +-----------------------------------+ + * | ent[2] | ent[3] | + * | 16B | 16B | + * | lba/old/new/seq | pad | + * +-----------------+-----------------+ + * + * New format: + * +-----------------+-----------------+ + * | ent[0] | ent[1] | + * | 16B | 16B | + * | lba/old/new/seq | lba/old/new/seq | + * +-----------------------------------+ + * | ent[2] | ent[3] | + * | 16B | 16B | + * | pad | pad | + * +-----------------+-----------------+ + * + * We detect during start-up which format is in use, and set + * arena->log_index[(0, 1)] with the detected format. + */ + struct log_entry { __le32 lba; __le32 old_map; __le32 new_map; __le32 seq; - __le64 padding[2]; +}; + +struct log_group { + struct log_entry ent[4]; }; struct btt_sb { @@ -126,6 +167,7 @@ struct aligned_lock { * @debugfs_dir: Debugfs dentry * @flags: Arena flags - may signify error states. * @err_lock: Mutex for synchronizing error clearing. + * @log_index: Indices of the valid log entries in a log_group * * arena_info is a per-arena handle. Once an arena is narrowed down for an * IO, this struct is passed around for the duration of the IO. @@ -158,6 +200,7 @@ struct arena_info { /* Arena flags */ u32 flags; struct mutex err_lock; + int log_index[2]; }; /**

7 years, 5 months

3
2
0 0

[Linux-stable-mirror] v3.18.92 build: 0 failures 1 warnings (v3.18.92)

by Build bot for Mark Brown

Tree/Branch: v3.18.92 Git describe: v3.18.92 Commit: a5d35deca2 Linux 3.18.92 Build Time: 61 min 7 sec Passed: 9 / 9 (100.00 %) Failed: 0 / 9 ( 0.00 %) Errors: 0 Warnings: 1 Section Mismatches: 0 ------------------------------------------------------------------------------- defconfigs with issues (other than build errors): 2 warnings 0 mismatches : x86_64-defconfig ------------------------------------------------------------------------------- Warnings Summary: 1 2 ../include/linux/ftrace.h:632:36: warning: calling '__builtin_return_address' with a nonzero argument is unsafe [-Wframe-address] =============================================================================== Detailed per-defconfig build reports below: ------------------------------------------------------------------------------- x86_64-defconfig : PASS, 0 errors, 2 warnings, 0 section mismatches Warnings: ../include/linux/ftrace.h:632:36: warning: calling '__builtin_return_address' with a nonzero argument is unsafe [-Wframe-address] ../include/linux/ftrace.h:632:36: warning: calling '__builtin_return_address' with a nonzero argument is unsafe [-Wframe-address] ------------------------------------------------------------------------------- Passed with no errors, warnings or mismatches: arm64-allnoconfig arm64-allmodconfig arm-multi_v5_defconfig arm-multi_v7_defconfig arm-allmodconfig arm-allnoconfig x86_64-allnoconfig arm64-defconfig close failed in file object destructor: sys.excepthook is missing lost sys.stderr

7 years, 5 months

1
0
0 0

[Linux-stable-mirror] Patch "x86/spectre: Add boot time option to select Spectre v2 mitigation" has been added to the 4.4-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled x86/spectre: Add boot time option to select Spectre v2 mitigation to the 4.4-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: x86-spectre-add-boot-time-option-to-select-spectre-v2-mitigation.patch and it can be found in the queue-4.4 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From da285121560e769cc31797bba6422eea71d473e0 Mon Sep 17 00:00:00 2001 From: David Woodhouse <dwmw(a)amazon.co.uk> Date: Thu, 11 Jan 2018 21:46:26 +0000 Subject: x86/spectre: Add boot time option to select Spectre v2 mitigation From: David Woodhouse <dwmw(a)amazon.co.uk> commit da285121560e769cc31797bba6422eea71d473e0 upstream. Add a spectre_v2= option to select the mitigation used for the indirect branch speculation vulnerability. Currently, the only option available is retpoline, in its various forms. This will be expanded to cover the new IBRS/IBPB microcode features. The RETPOLINE_AMD feature relies on a serializing LFENCE for speculation control. For AMD hardware, only set RETPOLINE_AMD if LFENCE is a serializing instruction, which is indicated by the LFENCE_RDTSC feature. [ tglx: Folded back the LFENCE/AMD fixes and reworked it so IBRS integration becomes simple ] Signed-off-by: David Woodhouse <dwmw(a)amazon.co.uk> Signed-off-by: Thomas Gleixner <tglx(a)linutronix.de> Cc: gnomes(a)lxorguk.ukuu.org.uk Cc: Rik van Riel <riel(a)redhat.com> Cc: Andi Kleen <ak(a)linux.intel.com> Cc: Josh Poimboeuf <jpoimboe(a)redhat.com> Cc: thomas.lendacky(a)amd.com Cc: Peter Zijlstra <peterz(a)infradead.org> Cc: Linus Torvalds <torvalds(a)linux-foundation.org> Cc: Jiri Kosina <jikos(a)kernel.org> Cc: Andy Lutomirski <luto(a)amacapital.net> Cc: Dave Hansen <dave.hansen(a)intel.com> Cc: Kees Cook <keescook(a)google.com> Cc: Tim Chen <tim.c.chen(a)linux.intel.com> Cc: Greg Kroah-Hartman <gregkh(a)linux-foundation.org> Cc: Paul Turner <pjt(a)google.com> Link: https://lkml.kernel.org/r/1515707194-20531-5-git-send-email-dwmw@amazon.co.… Signed-off-by: David Woodhouse <dwmw(a)amazon.co.uk> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- Documentation/kernel-parameters.txt | 28 ++++++ arch/x86/include/asm/nospec-branch.h | 10 ++ arch/x86/kernel/cpu/bugs.c | 158 ++++++++++++++++++++++++++++++++++- arch/x86/kernel/cpu/common.c | 4 4 files changed, 195 insertions(+), 5 deletions(-) --- a/Documentation/kernel-parameters.txt +++ b/Documentation/kernel-parameters.txt @@ -2452,6 +2452,11 @@ bytes respectively. Such letter suffixes nohugeiomap [KNL,x86] Disable kernel huge I/O mappings. + nospectre_v2 [X86] Disable all mitigations for the Spectre variant 2 + (indirect branch prediction) vulnerability. System may + allow data leaks with this option, which is equivalent + to spectre_v2=off. + noxsave [BUGS=X86] Disables x86 extended register state save and restore using xsave. The kernel will fallback to enabling legacy floating-point and sse state. @@ -3594,6 +3599,29 @@ bytes respectively. Such letter suffixes sonypi.*= [HW] Sony Programmable I/O Control Device driver See Documentation/laptops/sonypi.txt + spectre_v2= [X86] Control mitigation of Spectre variant 2 + (indirect branch speculation) vulnerability. + + on - unconditionally enable + off - unconditionally disable + auto - kernel detects whether your CPU model is + vulnerable + + Selecting 'on' will, and 'auto' may, choose a + mitigation method at run time according to the + CPU, the available microcode, the setting of the + CONFIG_RETPOLINE configuration option, and the + compiler with which the kernel was built. + + Specific mitigations can also be selected manually: + + retpoline - replace indirect branches + retpoline,generic - google's original retpoline + retpoline,amd - AMD-specific minimal thunk + + Not specifying this option is equivalent to + spectre_v2=auto. + spia_io_base= [HW,MTD] spia_fio_base= spia_pedr= --- a/arch/x86/include/asm/nospec-branch.h +++ b/arch/x86/include/asm/nospec-branch.h @@ -102,5 +102,15 @@ # define THUNK_TARGET(addr) [thunk_target] "rm" (addr) #endif +/* The Spectre V2 mitigation variants */ +enum spectre_v2_mitigation { + SPECTRE_V2_NONE, + SPECTRE_V2_RETPOLINE_MINIMAL, + SPECTRE_V2_RETPOLINE_MINIMAL_AMD, + SPECTRE_V2_RETPOLINE_GENERIC, + SPECTRE_V2_RETPOLINE_AMD, + SPECTRE_V2_IBRS, +}; + #endif /* __ASSEMBLY__ */ #endif /* __NOSPEC_BRANCH_H__ */ --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -10,6 +10,9 @@ #include <linux/init.h> #include <linux/utsname.h> #include <linux/cpu.h> + +#include <asm/nospec-branch.h> +#include <asm/cmdline.h> #include <asm/bugs.h> #include <asm/processor.h> #include <asm/processor-flags.h> @@ -20,6 +23,8 @@ #include <asm/pgtable.h> #include <asm/cacheflush.h> +static void __init spectre_v2_select_mitigation(void); + void __init check_bugs(void) { identify_boot_cpu(); @@ -29,6 +34,9 @@ void __init check_bugs(void) print_cpu_info(&boot_cpu_data); } + /* Select the proper spectre mitigation before patching alternatives */ + spectre_v2_select_mitigation(); + #ifdef CONFIG_X86_32 /* * Check whether we are able to run this kernel safely on SMP. @@ -61,6 +69,153 @@ void __init check_bugs(void) #endif } +/* The kernel command line selection */ +enum spectre_v2_mitigation_cmd { + SPECTRE_V2_CMD_NONE, + SPECTRE_V2_CMD_AUTO, + SPECTRE_V2_CMD_FORCE, + SPECTRE_V2_CMD_RETPOLINE, + SPECTRE_V2_CMD_RETPOLINE_GENERIC, + SPECTRE_V2_CMD_RETPOLINE_AMD, +}; + +static const char *spectre_v2_strings[] = { + [SPECTRE_V2_NONE] = "Vulnerable", + [SPECTRE_V2_RETPOLINE_MINIMAL] = "Vulnerable: Minimal generic ASM retpoline", + [SPECTRE_V2_RETPOLINE_MINIMAL_AMD] = "Vulnerable: Minimal AMD ASM retpoline", + [SPECTRE_V2_RETPOLINE_GENERIC] = "Mitigation: Full generic retpoline", + [SPECTRE_V2_RETPOLINE_AMD] = "Mitigation: Full AMD retpoline", +}; + +#undef pr_fmt +#define pr_fmt(fmt) "Spectre V2 mitigation: " fmt + +static enum spectre_v2_mitigation spectre_v2_enabled = SPECTRE_V2_NONE; + +static void __init spec2_print_if_insecure(const char *reason) +{ + if (boot_cpu_has_bug(X86_BUG_SPECTRE_V2)) + pr_info("%s\n", reason); +} + +static void __init spec2_print_if_secure(const char *reason) +{ + if (!boot_cpu_has_bug(X86_BUG_SPECTRE_V2)) + pr_info("%s\n", reason); +} + +static inline bool retp_compiler(void) +{ + return __is_defined(RETPOLINE); +} + +static inline bool match_option(const char *arg, int arglen, const char *opt) +{ + int len = strlen(opt); + + return len == arglen && !strncmp(arg, opt, len); +} + +static enum spectre_v2_mitigation_cmd __init spectre_v2_parse_cmdline(void) +{ + char arg[20]; + int ret; + + ret = cmdline_find_option(boot_command_line, "spectre_v2", arg, + sizeof(arg)); + if (ret > 0) { + if (match_option(arg, ret, "off")) { + goto disable; + } else if (match_option(arg, ret, "on")) { + spec2_print_if_secure("force enabled on command line."); + return SPECTRE_V2_CMD_FORCE; + } else if (match_option(arg, ret, "retpoline")) { + spec2_print_if_insecure("retpoline selected on command line."); + return SPECTRE_V2_CMD_RETPOLINE; + } else if (match_option(arg, ret, "retpoline,amd")) { + if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD) { + pr_err("retpoline,amd selected but CPU is not AMD. Switching to AUTO select\n"); + return SPECTRE_V2_CMD_AUTO; + } + spec2_print_if_insecure("AMD retpoline selected on command line."); + return SPECTRE_V2_CMD_RETPOLINE_AMD; + } else if (match_option(arg, ret, "retpoline,generic")) { + spec2_print_if_insecure("generic retpoline selected on command line."); + return SPECTRE_V2_CMD_RETPOLINE_GENERIC; + } else if (match_option(arg, ret, "auto")) { + return SPECTRE_V2_CMD_AUTO; + } + } + + if (!cmdline_find_option_bool(boot_command_line, "nospectre_v2")) + return SPECTRE_V2_CMD_AUTO; +disable: + spec2_print_if_insecure("disabled on command line."); + return SPECTRE_V2_CMD_NONE; +} + +static void __init spectre_v2_select_mitigation(void) +{ + enum spectre_v2_mitigation_cmd cmd = spectre_v2_parse_cmdline(); + enum spectre_v2_mitigation mode = SPECTRE_V2_NONE; + + /* + * If the CPU is not affected and the command line mode is NONE or AUTO + * then nothing to do. + */ + if (!boot_cpu_has_bug(X86_BUG_SPECTRE_V2) && + (cmd == SPECTRE_V2_CMD_NONE || cmd == SPECTRE_V2_CMD_AUTO)) + return; + + switch (cmd) { + case SPECTRE_V2_CMD_NONE: + return; + + case SPECTRE_V2_CMD_FORCE: + /* FALLTRHU */ + case SPECTRE_V2_CMD_AUTO: + goto retpoline_auto; + + case SPECTRE_V2_CMD_RETPOLINE_AMD: + if (IS_ENABLED(CONFIG_RETPOLINE)) + goto retpoline_amd; + break; + case SPECTRE_V2_CMD_RETPOLINE_GENERIC: + if (IS_ENABLED(CONFIG_RETPOLINE)) + goto retpoline_generic; + break; + case SPECTRE_V2_CMD_RETPOLINE: + if (IS_ENABLED(CONFIG_RETPOLINE)) + goto retpoline_auto; + break; + } + pr_err("kernel not compiled with retpoline; no mitigation available!"); + return; + +retpoline_auto: + if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD) { + retpoline_amd: + if (!boot_cpu_has(X86_FEATURE_LFENCE_RDTSC)) { + pr_err("LFENCE not serializing. Switching to generic retpoline\n"); + goto retpoline_generic; + } + mode = retp_compiler() ? SPECTRE_V2_RETPOLINE_AMD : + SPECTRE_V2_RETPOLINE_MINIMAL_AMD; + setup_force_cpu_cap(X86_FEATURE_RETPOLINE_AMD); + setup_force_cpu_cap(X86_FEATURE_RETPOLINE); + } else { + retpoline_generic: + mode = retp_compiler() ? SPECTRE_V2_RETPOLINE_GENERIC : + SPECTRE_V2_RETPOLINE_MINIMAL; + setup_force_cpu_cap(X86_FEATURE_RETPOLINE); + } + + spectre_v2_enabled = mode; + pr_info("%s\n", spectre_v2_strings[mode]); +} + +#undef pr_fmt + #ifdef CONFIG_SYSFS ssize_t cpu_show_meltdown(struct device *dev, struct device_attribute *attr, char *buf) @@ -85,6 +240,7 @@ ssize_t cpu_show_spectre_v2(struct devic { if (!boot_cpu_has_bug(X86_BUG_SPECTRE_V2)) return sprintf(buf, "Not affected\n"); - return sprintf(buf, "Vulnerable\n"); + + return sprintf(buf, "%s\n", spectre_v2_strings[spectre_v2_enabled]); } #endif --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -837,10 +837,6 @@ static void __init early_identify_cpu(st setup_force_cpu_bug(X86_BUG_SPECTRE_V1); setup_force_cpu_bug(X86_BUG_SPECTRE_V2); -#ifdef CONFIG_RETPOLINE - setup_force_cpu_cap(X86_FEATURE_RETPOLINE); -#endif - fpu__init_system(c); #ifdef CONFIG_X86_32 Patches currently in stable-queue which might be from dwmw(a)amazon.co.uk are queue-4.4/x86-spectre-add-boot-time-option-to-select-spectre-v2-mitigation.patch queue-4.4/x86-retpoline-irq32-convert-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-hyperv-convert-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-entry-convert-entry-assembler-indirect-jumps.patch queue-4.4/x86-cpu-amd-make-lfence-a-serializing-instruction.patch queue-4.4/x86-retpoline-ftrace-convert-ftrace-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-crypto-convert-crypto-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-xen-convert-xen-hypercall-indirect-jumps.patch queue-4.4/x86-retpoline-checksum32-convert-assembler-indirect-jumps.patch queue-4.4/x86-mm-32-move-setup_clear_cpu_cap-x86_feature_pcid-earlier.patch queue-4.4/x86-retpoline-fill-return-stack-buffer-on-vmexit.patch queue-4.4/x86-retpoline-remove-compile-time-warning.patch queue-4.4/x86-cpu-amd-use-lfence_rdtsc-in-preference-to-mfence_rdtsc.patch queue-4.4/x86-retpoline-add-initial-retpoline-support.patch

7 years, 5 months

1
0
0 0

[Linux-stable-mirror] Patch "x86/retpoline/xen: Convert Xen hypercall indirect jumps" has been added to the 4.4-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled x86/retpoline/xen: Convert Xen hypercall indirect jumps to the 4.4-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: x86-retpoline-xen-convert-xen-hypercall-indirect-jumps.patch and it can be found in the queue-4.4 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From ea08816d5b185ab3d09e95e393f265af54560350 Mon Sep 17 00:00:00 2001 From: David Woodhouse <dwmw(a)amazon.co.uk> Date: Thu, 11 Jan 2018 21:46:31 +0000 Subject: x86/retpoline/xen: Convert Xen hypercall indirect jumps From: David Woodhouse <dwmw(a)amazon.co.uk> commit ea08816d5b185ab3d09e95e393f265af54560350 upstream. Convert indirect call in Xen hypercall to use non-speculative sequence, when CONFIG_RETPOLINE is enabled. Signed-off-by: David Woodhouse <dwmw(a)amazon.co.uk> Signed-off-by: Thomas Gleixner <tglx(a)linutronix.de> Acked-by: Arjan van de Ven <arjan(a)linux.intel.com> Acked-by: Ingo Molnar <mingo(a)kernel.org> Reviewed-by: Juergen Gross <jgross(a)suse.com> Cc: gnomes(a)lxorguk.ukuu.org.uk Cc: Rik van Riel <riel(a)redhat.com> Cc: Andi Kleen <ak(a)linux.intel.com> Cc: Josh Poimboeuf <jpoimboe(a)redhat.com> Cc: thomas.lendacky(a)amd.com Cc: Peter Zijlstra <peterz(a)infradead.org> Cc: Linus Torvalds <torvalds(a)linux-foundation.org> Cc: Jiri Kosina <jikos(a)kernel.org> Cc: Andy Lutomirski <luto(a)amacapital.net> Cc: Dave Hansen <dave.hansen(a)intel.com> Cc: Kees Cook <keescook(a)google.com> Cc: Tim Chen <tim.c.chen(a)linux.intel.com> Cc: Greg Kroah-Hartman <gregkh(a)linux-foundation.org> Cc: Paul Turner <pjt(a)google.com> Link: https://lkml.kernel.org/r/1515707194-20531-10-git-send-email-dwmw@amazon.co… Signed-off-by: David Woodhouse <dwmw(a)amazon.co.uk> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- arch/x86/include/asm/xen/hypercall.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) --- a/arch/x86/include/asm/xen/hypercall.h +++ b/arch/x86/include/asm/xen/hypercall.h @@ -44,6 +44,7 @@ #include <asm/page.h> #include <asm/pgtable.h> #include <asm/smap.h> +#include <asm/nospec-branch.h> #include <xen/interface/xen.h> #include <xen/interface/sched.h> @@ -215,9 +216,9 @@ privcmd_call(unsigned call, __HYPERCALL_5ARG(a1, a2, a3, a4, a5); stac(); - asm volatile("call *%[call]" + asm volatile(CALL_NOSPEC : __HYPERCALL_5PARAM - : [call] "a" (&hypercall_page[call]) + : [thunk_target] "a" (&hypercall_page[call]) : __HYPERCALL_CLOBBER5); clac(); Patches currently in stable-queue which might be from dwmw(a)amazon.co.uk are queue-4.4/x86-spectre-add-boot-time-option-to-select-spectre-v2-mitigation.patch queue-4.4/x86-retpoline-irq32-convert-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-hyperv-convert-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-entry-convert-entry-assembler-indirect-jumps.patch queue-4.4/x86-cpu-amd-make-lfence-a-serializing-instruction.patch queue-4.4/x86-retpoline-ftrace-convert-ftrace-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-crypto-convert-crypto-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-xen-convert-xen-hypercall-indirect-jumps.patch queue-4.4/x86-retpoline-checksum32-convert-assembler-indirect-jumps.patch queue-4.4/x86-mm-32-move-setup_clear_cpu_cap-x86_feature_pcid-earlier.patch queue-4.4/x86-retpoline-fill-return-stack-buffer-on-vmexit.patch queue-4.4/x86-retpoline-remove-compile-time-warning.patch queue-4.4/x86-cpu-amd-use-lfence_rdtsc-in-preference-to-mfence_rdtsc.patch queue-4.4/x86-retpoline-add-initial-retpoline-support.patch

7 years, 5 months

1
0
0 0

[Linux-stable-mirror] Patch "x86/retpoline: Remove compile time warning" has been added to the 4.4-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled x86/retpoline: Remove compile time warning to the 4.4-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: x86-retpoline-remove-compile-time-warning.patch and it can be found in the queue-4.4 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From b8b9ce4b5aec8de9e23cabb0a26b78641f9ab1d6 Mon Sep 17 00:00:00 2001 From: Thomas Gleixner <tglx(a)linutronix.de> Date: Sun, 14 Jan 2018 22:13:29 +0100 Subject: x86/retpoline: Remove compile time warning From: Thomas Gleixner <tglx(a)linutronix.de> commit b8b9ce4b5aec8de9e23cabb0a26b78641f9ab1d6 upstream. Remove the compile time warning when CONFIG_RETPOLINE=y and the compiler does not have retpoline support. Linus rationale for this is: It's wrong because it will just make people turn off RETPOLINE, and the asm updates - and return stack clearing - that are independent of the compiler are likely the most important parts because they are likely the ones easiest to target. And it's annoying because most people won't be able to do anything about it. The number of people building their own compiler? Very small. So if their distro hasn't got a compiler yet (and pretty much nobody does), the warning is just annoying crap. It is already properly reported as part of the sysfs interface. The compile-time warning only encourages bad things. Fixes: 76b043848fd2 ("x86/retpoline: Add initial retpoline support") Requested-by: Linus Torvalds <torvalds(a)linux-foundation.org> Signed-off-by: Thomas Gleixner <tglx(a)linutronix.de> Cc: David Woodhouse <dwmw(a)amazon.co.uk> Cc: Peter Zijlstra (Intel) <peterz(a)infradead.org> Cc: gnomes(a)lxorguk.ukuu.org.uk Cc: Rik van Riel <riel(a)redhat.com> Cc: Andi Kleen <ak(a)linux.intel.com> Cc: Josh Poimboeuf <jpoimboe(a)redhat.com> Cc: thomas.lendacky(a)amd.com Cc: Linus Torvalds <torvalds(a)linux-foundation.org> Cc: Jiri Kosina <jikos(a)kernel.org> Cc: Andy Lutomirski <luto(a)amacapital.net> Cc: Dave Hansen <dave.hansen(a)intel.com> Cc: Kees Cook <keescook(a)google.com> Cc: Tim Chen <tim.c.chen(a)linux.intel.com> Cc: Greg Kroah-Hartman <gregkh(a)linux-foundation.org> Link: https://lkml.kernel.org/r/CA+55aFzWgquv4i6Mab6bASqYXg3ErV3XDFEYf=GEcCDQg5uA… Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- arch/x86/Makefile | 2 -- 1 file changed, 2 deletions(-) --- a/arch/x86/Makefile +++ b/arch/x86/Makefile @@ -194,8 +194,6 @@ ifdef CONFIG_RETPOLINE RETPOLINE_CFLAGS += $(call cc-option,-mindirect-branch=thunk-extern -mindirect-branch-register) ifneq ($(RETPOLINE_CFLAGS),) KBUILD_CFLAGS += $(RETPOLINE_CFLAGS) -DRETPOLINE - else - $(warning CONFIG_RETPOLINE=y, but not supported by the compiler. Toolchain update recommended.) endif endif Patches currently in stable-queue which might be from tglx(a)linutronix.de are queue-4.4/x86-spectre-add-boot-time-option-to-select-spectre-v2-mitigation.patch queue-4.4/x86-retpoline-irq32-convert-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-hyperv-convert-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-entry-convert-entry-assembler-indirect-jumps.patch queue-4.4/x86-asm-use-register-variable-to-get-stack-pointer-value.patch queue-4.4/x86-cpu-amd-make-lfence-a-serializing-instruction.patch queue-4.4/x86-retpoline-ftrace-convert-ftrace-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-crypto-convert-crypto-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-xen-convert-xen-hypercall-indirect-jumps.patch queue-4.4/x86-retpoline-checksum32-convert-assembler-indirect-jumps.patch queue-4.4/x86-mm-32-move-setup_clear_cpu_cap-x86_feature_pcid-earlier.patch queue-4.4/x86-retpoline-fill-return-stack-buffer-on-vmexit.patch queue-4.4/x86-retpoline-remove-compile-time-warning.patch queue-4.4/x86-cpu-amd-use-lfence_rdtsc-in-preference-to-mfence_rdtsc.patch queue-4.4/x86-retpoline-add-initial-retpoline-support.patch queue-4.4/x86-asm-make-asm-alternative.h-safe-from-assembly.patch

7 years, 5 months

1
0
0 0

[Linux-stable-mirror] Patch "x86/retpoline/irq32: Convert assembler indirect jumps" has been added to the 4.4-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled x86/retpoline/irq32: Convert assembler indirect jumps to the 4.4-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: x86-retpoline-irq32-convert-assembler-indirect-jumps.patch and it can be found in the queue-4.4 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From 7614e913db1f40fff819b36216484dc3808995d4 Mon Sep 17 00:00:00 2001 From: Andi Kleen <ak(a)linux.intel.com> Date: Thu, 11 Jan 2018 21:46:33 +0000 Subject: x86/retpoline/irq32: Convert assembler indirect jumps From: Andi Kleen <ak(a)linux.intel.com> commit 7614e913db1f40fff819b36216484dc3808995d4 upstream. Convert all indirect jumps in 32bit irq inline asm code to use non speculative sequences. Signed-off-by: Andi Kleen <ak(a)linux.intel.com> Signed-off-by: Thomas Gleixner <tglx(a)linutronix.de> Acked-by: Arjan van de Ven <arjan(a)linux.intel.com> Acked-by: Ingo Molnar <mingo(a)kernel.org> Cc: gnomes(a)lxorguk.ukuu.org.uk Cc: Rik van Riel <riel(a)redhat.com> Cc: Josh Poimboeuf <jpoimboe(a)redhat.com> Cc: thomas.lendacky(a)amd.com Cc: Peter Zijlstra <peterz(a)infradead.org> Cc: Linus Torvalds <torvalds(a)linux-foundation.org> Cc: Jiri Kosina <jikos(a)kernel.org> Cc: Andy Lutomirski <luto(a)amacapital.net> Cc: Dave Hansen <dave.hansen(a)intel.com> Cc: Kees Cook <keescook(a)google.com> Cc: Tim Chen <tim.c.chen(a)linux.intel.com> Cc: Greg Kroah-Hartman <gregkh(a)linux-foundation.org> Cc: Paul Turner <pjt(a)google.com> Link: https://lkml.kernel.org/r/1515707194-20531-12-git-send-email-dwmw@amazon.co… Signed-off-by: David Woodhouse <dwmw(a)amazon.co.uk> Signed-off-by: Razvan Ghitulete <rga(a)amazon.de> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- arch/x86/kernel/irq_32.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) --- a/arch/x86/kernel/irq_32.c +++ b/arch/x86/kernel/irq_32.c @@ -20,6 +20,7 @@ #include <linux/mm.h> #include <asm/apic.h> +#include <asm/nospec-branch.h> #ifdef CONFIG_DEBUG_STACKOVERFLOW @@ -55,11 +56,11 @@ DEFINE_PER_CPU(struct irq_stack *, softi static void call_on_stack(void *func, void *stack) { asm volatile("xchgl %%ebx,%%esp \n" - "call *%%edi \n" + CALL_NOSPEC "movl %%ebx,%%esp \n" : "=b" (stack) : "0" (stack), - "D"(func) + [thunk_target] "D"(func) : "memory", "cc", "edx", "ecx", "eax"); } @@ -95,11 +96,11 @@ static inline int execute_on_irq_stack(i call_on_stack(print_stack_overflow, isp); asm volatile("xchgl %%ebx,%%esp \n" - "call *%%edi \n" + CALL_NOSPEC "movl %%ebx,%%esp \n" : "=a" (arg1), "=b" (isp) : "0" (desc), "1" (isp), - "D" (desc->handle_irq) + [thunk_target] "D" (desc->handle_irq) : "memory", "cc", "ecx"); return 1; } Patches currently in stable-queue which might be from ak(a)linux.intel.com are queue-4.4/x86-spectre-add-boot-time-option-to-select-spectre-v2-mitigation.patch queue-4.4/x86-retpoline-irq32-convert-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-hyperv-convert-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-entry-convert-entry-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-ftrace-convert-ftrace-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-crypto-convert-crypto-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-xen-convert-xen-hypercall-indirect-jumps.patch queue-4.4/x86-retpoline-checksum32-convert-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-fill-return-stack-buffer-on-vmexit.patch queue-4.4/x86-retpoline-remove-compile-time-warning.patch queue-4.4/x86-retpoline-add-initial-retpoline-support.patch

7 years, 5 months

1
0
0 0

[Linux-stable-mirror] Patch "x86/retpoline/hyperv: Convert assembler indirect jumps" has been added to the 4.4-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled x86/retpoline/hyperv: Convert assembler indirect jumps to the 4.4-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: x86-retpoline-hyperv-convert-assembler-indirect-jumps.patch and it can be found in the queue-4.4 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From e70e5892b28c18f517f29ab6e83bd57705104b31 Mon Sep 17 00:00:00 2001 From: David Woodhouse <dwmw(a)amazon.co.uk> Date: Thu, 11 Jan 2018 21:46:30 +0000 Subject: x86/retpoline/hyperv: Convert assembler indirect jumps From: David Woodhouse <dwmw(a)amazon.co.uk> commit e70e5892b28c18f517f29ab6e83bd57705104b31 upstream. Convert all indirect jumps in hyperv inline asm code to use non-speculative sequences when CONFIG_RETPOLINE is enabled. Signed-off-by: David Woodhouse <dwmw(a)amazon.co.uk> Signed-off-by: Thomas Gleixner <tglx(a)linutronix.de> Acked-by: Arjan van de Ven <arjan(a)linux.intel.com> Acked-by: Ingo Molnar <mingo(a)kernel.org> Cc: gnomes(a)lxorguk.ukuu.org.uk Cc: Rik van Riel <riel(a)redhat.com> Cc: Andi Kleen <ak(a)linux.intel.com> Cc: Josh Poimboeuf <jpoimboe(a)redhat.com> Cc: thomas.lendacky(a)amd.com Cc: Peter Zijlstra <peterz(a)infradead.org> Cc: Linus Torvalds <torvalds(a)linux-foundation.org> Cc: Jiri Kosina <jikos(a)kernel.org> Cc: Andy Lutomirski <luto(a)amacapital.net> Cc: Dave Hansen <dave.hansen(a)intel.com> Cc: Kees Cook <keescook(a)google.com> Cc: Tim Chen <tim.c.chen(a)linux.intel.com> Cc: Greg Kroah-Hartman <gregkh(a)linux-foundation.org> Cc: Paul Turner <pjt(a)google.com> Link: https://lkml.kernel.org/r/1515707194-20531-9-git-send-email-dwmw@amazon.co.… [ backport to 4.4, hopefully correct, not tested... - gregkh ] Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- drivers/hv/hv.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) --- a/drivers/hv/hv.c +++ b/drivers/hv/hv.c @@ -31,6 +31,7 @@ #include <linux/clockchips.h> #include <asm/hyperv.h> #include <asm/mshyperv.h> +#include <asm/nospec-branch.h> #include "hyperv_vmbus.h" /* The one and only */ @@ -103,9 +104,10 @@ static u64 do_hypercall(u64 control, voi return (u64)ULLONG_MAX; __asm__ __volatile__("mov %0, %%r8" : : "r" (output_address) : "r8"); - __asm__ __volatile__("call *%3" : "=a" (hv_status) : + __asm__ __volatile__(CALL_NOSPEC : + "=a" (hv_status) : "c" (control), "d" (input_address), - "m" (hypercall_page)); + THUNK_TARGET(hypercall_page)); return hv_status; @@ -123,11 +125,12 @@ static u64 do_hypercall(u64 control, voi if (!hypercall_page) return (u64)ULLONG_MAX; - __asm__ __volatile__ ("call *%8" : "=d"(hv_status_hi), + __asm__ __volatile__ (CALL_NOSPEC : "=d"(hv_status_hi), "=a"(hv_status_lo) : "d" (control_hi), "a" (control_lo), "b" (input_address_hi), "c" (input_address_lo), "D"(output_address_hi), - "S"(output_address_lo), "m" (hypercall_page)); + "S"(output_address_lo), + THUNK_TARGET(hypercall_page)); return hv_status_lo | ((u64)hv_status_hi << 32); #endif /* !x86_64 */ Patches currently in stable-queue which might be from dwmw(a)amazon.co.uk are queue-4.4/x86-spectre-add-boot-time-option-to-select-spectre-v2-mitigation.patch queue-4.4/x86-retpoline-irq32-convert-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-hyperv-convert-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-entry-convert-entry-assembler-indirect-jumps.patch queue-4.4/x86-cpu-amd-make-lfence-a-serializing-instruction.patch queue-4.4/x86-retpoline-ftrace-convert-ftrace-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-crypto-convert-crypto-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-xen-convert-xen-hypercall-indirect-jumps.patch queue-4.4/x86-retpoline-checksum32-convert-assembler-indirect-jumps.patch queue-4.4/x86-mm-32-move-setup_clear_cpu_cap-x86_feature_pcid-earlier.patch queue-4.4/x86-retpoline-fill-return-stack-buffer-on-vmexit.patch queue-4.4/x86-retpoline-remove-compile-time-warning.patch queue-4.4/x86-cpu-amd-use-lfence_rdtsc-in-preference-to-mfence_rdtsc.patch queue-4.4/x86-retpoline-add-initial-retpoline-support.patch

7 years, 5 months

1
0
0 0

[Linux-stable-mirror] Patch "x86/retpoline/ftrace: Convert ftrace assembler indirect jumps" has been added to the 4.4-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled x86/retpoline/ftrace: Convert ftrace assembler indirect jumps to the 4.4-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: x86-retpoline-ftrace-convert-ftrace-assembler-indirect-jumps.patch and it can be found in the queue-4.4 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From 9351803bd803cdbeb9b5a7850b7b6f464806e3db Mon Sep 17 00:00:00 2001 From: David Woodhouse <dwmw(a)amazon.co.uk> Date: Thu, 11 Jan 2018 21:46:29 +0000 Subject: x86/retpoline/ftrace: Convert ftrace assembler indirect jumps From: David Woodhouse <dwmw(a)amazon.co.uk> commit 9351803bd803cdbeb9b5a7850b7b6f464806e3db upstream. Convert all indirect jumps in ftrace assembler code to use non-speculative sequences when CONFIG_RETPOLINE is enabled. Signed-off-by: David Woodhouse <dwmw(a)amazon.co.uk> Signed-off-by: Thomas Gleixner <tglx(a)linutronix.de> Acked-by: Arjan van de Ven <arjan(a)linux.intel.com> Acked-by: Ingo Molnar <mingo(a)kernel.org> Cc: gnomes(a)lxorguk.ukuu.org.uk Cc: Rik van Riel <riel(a)redhat.com> Cc: Andi Kleen <ak(a)linux.intel.com> Cc: Josh Poimboeuf <jpoimboe(a)redhat.com> Cc: thomas.lendacky(a)amd.com Cc: Peter Zijlstra <peterz(a)infradead.org> Cc: Linus Torvalds <torvalds(a)linux-foundation.org> Cc: Jiri Kosina <jikos(a)kernel.org> Cc: Andy Lutomirski <luto(a)amacapital.net> Cc: Dave Hansen <dave.hansen(a)intel.com> Cc: Kees Cook <keescook(a)google.com> Cc: Tim Chen <tim.c.chen(a)linux.intel.com> Cc: Greg Kroah-Hartman <gregkh(a)linux-foundation.org> Cc: Paul Turner <pjt(a)google.com> Link: https://lkml.kernel.org/r/1515707194-20531-8-git-send-email-dwmw@amazon.co.… Signed-off-by: David Woodhouse <dwmw(a)amazon.co.uk> Signed-off-by: Razvan Ghitulete <rga(a)amazon.de> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- arch/x86/entry/entry_32.S | 5 +++-- arch/x86/kernel/mcount_64.S | 7 ++++--- 2 files changed, 7 insertions(+), 5 deletions(-) --- a/arch/x86/entry/entry_32.S +++ b/arch/x86/entry/entry_32.S @@ -862,7 +862,8 @@ trace: movl 0x4(%ebp), %edx subl $MCOUNT_INSN_SIZE, %eax - call *ftrace_trace_function + movl ftrace_trace_function, %ecx + CALL_NOSPEC %ecx popl %edx popl %ecx @@ -897,7 +898,7 @@ return_to_handler: movl %eax, %ecx popl %edx popl %eax - jmp *%ecx + JMP_NOSPEC %ecx #endif #ifdef CONFIG_TRACING --- a/arch/x86/kernel/mcount_64.S +++ b/arch/x86/kernel/mcount_64.S @@ -7,7 +7,7 @@ #include <linux/linkage.h> #include <asm/ptrace.h> #include <asm/ftrace.h> - +#include <asm/nospec-branch.h> .code64 .section .entry.text, "ax" @@ -285,8 +285,9 @@ trace: * ip and parent ip are used and the list function is called when * function tracing is enabled. */ - call *ftrace_trace_function + movq ftrace_trace_function, %r8 + CALL_NOSPEC %r8 restore_mcount_regs jmp fgraph_trace @@ -329,5 +330,5 @@ GLOBAL(return_to_handler) movq 8(%rsp), %rdx movq (%rsp), %rax addq $24, %rsp - jmp *%rdi + JMP_NOSPEC %rdi #endif Patches currently in stable-queue which might be from dwmw(a)amazon.co.uk are queue-4.4/x86-spectre-add-boot-time-option-to-select-spectre-v2-mitigation.patch queue-4.4/x86-retpoline-irq32-convert-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-hyperv-convert-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-entry-convert-entry-assembler-indirect-jumps.patch queue-4.4/x86-cpu-amd-make-lfence-a-serializing-instruction.patch queue-4.4/x86-retpoline-ftrace-convert-ftrace-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-crypto-convert-crypto-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-xen-convert-xen-hypercall-indirect-jumps.patch queue-4.4/x86-retpoline-checksum32-convert-assembler-indirect-jumps.patch queue-4.4/x86-mm-32-move-setup_clear_cpu_cap-x86_feature_pcid-earlier.patch queue-4.4/x86-retpoline-fill-return-stack-buffer-on-vmexit.patch queue-4.4/x86-retpoline-remove-compile-time-warning.patch queue-4.4/x86-cpu-amd-use-lfence_rdtsc-in-preference-to-mfence_rdtsc.patch queue-4.4/x86-retpoline-add-initial-retpoline-support.patch

7 years, 5 months

1
0
0 0

[Linux-stable-mirror] Patch "x86/retpoline: Fill return stack buffer on vmexit" has been added to the 4.4-stable tree

by gregkh＠linuxfoundation.org

This is a note to let you know that I've just added the patch titled x86/retpoline: Fill return stack buffer on vmexit to the 4.4-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=sum… The filename of the patch is: x86-retpoline-fill-return-stack-buffer-on-vmexit.patch and it can be found in the queue-4.4 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable(a)vger.kernel.org> know about it. >From 117cc7a908c83697b0b737d15ae1eb5943afe35b Mon Sep 17 00:00:00 2001 From: David Woodhouse <dwmw(a)amazon.co.uk> Date: Fri, 12 Jan 2018 11:11:27 +0000 Subject: x86/retpoline: Fill return stack buffer on vmexit From: David Woodhouse <dwmw(a)amazon.co.uk> commit 117cc7a908c83697b0b737d15ae1eb5943afe35b upstream. In accordance with the Intel and AMD documentation, we need to overwrite all entries in the RSB on exiting a guest, to prevent malicious branch target predictions from affecting the host kernel. This is needed both for retpoline and for IBRS. [ak: numbers again for the RSB stuffing labels] Signed-off-by: David Woodhouse <dwmw(a)amazon.co.uk> Signed-off-by: Thomas Gleixner <tglx(a)linutronix.de> Tested-by: Peter Zijlstra (Intel) <peterz(a)infradead.org> Cc: gnomes(a)lxorguk.ukuu.org.uk Cc: Rik van Riel <riel(a)redhat.com> Cc: Andi Kleen <ak(a)linux.intel.com> Cc: Josh Poimboeuf <jpoimboe(a)redhat.com> Cc: thomas.lendacky(a)amd.com Cc: Linus Torvalds <torvalds(a)linux-foundation.org> Cc: Jiri Kosina <jikos(a)kernel.org> Cc: Andy Lutomirski <luto(a)amacapital.net> Cc: Dave Hansen <dave.hansen(a)intel.com> Cc: Kees Cook <keescook(a)google.com> Cc: Tim Chen <tim.c.chen(a)linux.intel.com> Cc: Greg Kroah-Hartman <gregkh(a)linux-foundation.org> Cc: Paul Turner <pjt(a)google.com> Link: https://lkml.kernel.org/r/1515755487-8524-1-git-send-email-dwmw@amazon.co.uk Signed-off-by: David Woodhouse <dwmw(a)amazon.co.uk> Signed-off-by: Razvan Ghitulete <rga(a)amazon.de> Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> --- arch/x86/include/asm/nospec-branch.h | 76 ++++++++++++++++++++++++++++++++++- arch/x86/kvm/svm.c | 4 + arch/x86/kvm/vmx.c | 4 + 3 files changed, 83 insertions(+), 1 deletion(-) --- a/arch/x86/include/asm/nospec-branch.h +++ b/arch/x86/include/asm/nospec-branch.h @@ -7,6 +7,48 @@ #include <asm/alternative-asm.h> #include <asm/cpufeature.h> +/* + * Fill the CPU return stack buffer. + * + * Each entry in the RSB, if used for a speculative 'ret', contains an + * infinite 'pause; jmp' loop to capture speculative execution. + * + * This is required in various cases for retpoline and IBRS-based + * mitigations for the Spectre variant 2 vulnerability. Sometimes to + * eliminate potentially bogus entries from the RSB, and sometimes + * purely to ensure that it doesn't get empty, which on some CPUs would + * allow predictions from other (unwanted!) sources to be used. + * + * We define a CPP macro such that it can be used from both .S files and + * inline assembly. It's possible to do a .macro and then include that + * from C via asm(".include <asm/nospec-branch.h>") but let's not go there. + */ + +#define RSB_CLEAR_LOOPS 32 /* To forcibly overwrite all entries */ +#define RSB_FILL_LOOPS 16 /* To avoid underflow */ + +/* + * Google experimented with loop-unrolling and this turned out to be + * the optimal version — two calls, each with their own speculation + * trap should their return address end up getting used, in a loop. + */ +#define __FILL_RETURN_BUFFER(reg, nr, sp) \ + mov $(nr/2), reg; \ +771: \ + call 772f; \ +773: /* speculation trap */ \ + pause; \ + jmp 773b; \ +772: \ + call 774f; \ +775: /* speculation trap */ \ + pause; \ + jmp 775b; \ +774: \ + dec reg; \ + jnz 771b; \ + add $(BITS_PER_LONG/8) * nr, sp; + #ifdef __ASSEMBLY__ /* @@ -61,6 +103,19 @@ #endif .endm + /* + * A simpler FILL_RETURN_BUFFER macro. Don't make people use the CPP + * monstrosity above, manually. + */ +.macro FILL_RETURN_BUFFER reg:req nr:req ftr:req +#ifdef CONFIG_RETPOLINE + ALTERNATIVE "jmp .Lskip_rsb_\@", \ + __stringify(__FILL_RETURN_BUFFER(\reg,\nr,%_ASM_SP)) \ + \ftr +.Lskip_rsb_\@: +#endif +.endm + #else /* __ASSEMBLY__ */ #if defined(CONFIG_X86_64) && defined(RETPOLINE) @@ -97,7 +152,7 @@ X86_FEATURE_RETPOLINE) # define THUNK_TARGET(addr) [thunk_target] "rm" (addr) -#else /* No retpoline */ +#else /* No retpoline for C / inline asm */ # define CALL_NOSPEC "call *%[thunk_target]\n" # define THUNK_TARGET(addr) [thunk_target] "rm" (addr) #endif @@ -112,5 +167,24 @@ enum spectre_v2_mitigation { SPECTRE_V2_IBRS, }; +/* + * On VMEXIT we must ensure that no RSB predictions learned in the guest + * can be followed in the host, by overwriting the RSB completely. Both + * retpoline and IBRS mitigations for Spectre v2 need this; only on future + * CPUs with IBRS_ATT *might* it be avoided. + */ +static inline void vmexit_fill_RSB(void) +{ +#ifdef CONFIG_RETPOLINE + unsigned long loops = RSB_CLEAR_LOOPS / 2; + + asm volatile (ALTERNATIVE("jmp 910f", + __stringify(__FILL_RETURN_BUFFER(%0, RSB_CLEAR_LOOPS, %1)), + X86_FEATURE_RETPOLINE) + "910:" + : "=&r" (loops), ASM_CALL_CONSTRAINT + : "r" (loops) : "memory" ); +#endif +} #endif /* __ASSEMBLY__ */ #endif /* __NOSPEC_BRANCH_H__ */ --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -37,6 +37,7 @@ #include <asm/desc.h> #include <asm/debugreg.h> #include <asm/kvm_para.h> +#include <asm/nospec-branch.h> #include <asm/virtext.h> #include "trace.h" @@ -3904,6 +3905,9 @@ static void svm_vcpu_run(struct kvm_vcpu #endif ); + /* Eliminate branch target predictions from guest mode */ + vmexit_fill_RSB(); + #ifdef CONFIG_X86_64 wrmsrl(MSR_GS_BASE, svm->host.gs_base); #else --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -47,6 +47,7 @@ #include <asm/kexec.h> #include <asm/apic.h> #include <asm/irq_remapping.h> +#include <asm/nospec-branch.h> #include "trace.h" #include "pmu.h" @@ -8701,6 +8702,9 @@ static void __noclone vmx_vcpu_run(struc #endif ); + /* Eliminate branch target predictions from guest mode */ + vmexit_fill_RSB(); + /* MSR_IA32_DEBUGCTLMSR is zeroed on vmexit. Restore it if needed */ if (debugctlmsr) update_debugctlmsr(debugctlmsr); Patches currently in stable-queue which might be from dwmw(a)amazon.co.uk are queue-4.4/x86-spectre-add-boot-time-option-to-select-spectre-v2-mitigation.patch queue-4.4/x86-retpoline-irq32-convert-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-hyperv-convert-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-entry-convert-entry-assembler-indirect-jumps.patch queue-4.4/x86-cpu-amd-make-lfence-a-serializing-instruction.patch queue-4.4/x86-retpoline-ftrace-convert-ftrace-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-crypto-convert-crypto-assembler-indirect-jumps.patch queue-4.4/x86-retpoline-xen-convert-xen-hypercall-indirect-jumps.patch queue-4.4/x86-retpoline-checksum32-convert-assembler-indirect-jumps.patch queue-4.4/x86-mm-32-move-setup_clear_cpu_cap-x86_feature_pcid-earlier.patch queue-4.4/x86-retpoline-fill-return-stack-buffer-on-vmexit.patch queue-4.4/x86-retpoline-remove-compile-time-warning.patch queue-4.4/x86-cpu-amd-use-lfence_rdtsc-in-preference-to-mfence_rdtsc.patch queue-4.4/x86-retpoline-add-initial-retpoline-support.patch

7 years, 5 months

1
0
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-stable-mirror