Thomas,
Andi and I have made an update to our draft of the Spectre admin guide. We may be out on Christmas vacation for a while. But we want to send it out for everyone to take a look.
Thanks.
Tim
From: Andi Kleen ak@linux.intel.com
There are no document in admin guides describing Spectre v1 and v2 side channels and their mitigations in Linux.
Create a document to describe Spectre and the mitigation methods used in the kernel.
Signed-off-by: Andi Kleen ak@linux.intel.com Signed-off-by: Tim Chen tim.c.chen@linux.intel.com --- Documentation/admin-guide/spectre.rst | 502 ++++++++++++++++++++++++++++++++++ 1 file changed, 502 insertions(+) create mode 100644 Documentation/admin-guide/spectre.rst
diff --git a/Documentation/admin-guide/spectre.rst b/Documentation/admin-guide/spectre.rst new file mode 100644 index 0000000..0ba708e --- /dev/null +++ b/Documentation/admin-guide/spectre.rst @@ -0,0 +1,502 @@ +Spectre side channels +===================== + +Spectre is a class of side channel attacks against modern CPUs that +exploit branch prediction and speculative execution to read memory, +possibly bypassing access controls. These exploits do not modify memory. + +This document covers Spectre variant 1 and 2. + +Affected processors +------------------- + +The vulnerability affects a wide range of modern high performance +processors, since most modern high speed processors use branch prediction +and speculative execution. + +The following CPUs are vulnerable: + + - Intel Core, Atom, Pentium, Xeon CPUs + - AMD CPUs like Phenom, EPYC, Zen. + - IBM processors like POWER and zSeries + - Higher end ARM processors + - Apple CPUs + - Higher end MIPS CPUs + - Likely most other high performance CPUs. Contact your CPU vendor for details. + +This document describes the mitigations on Intel CPUs. Mitigations +on other architectures may be different. + +Related CVEs +------------ + +The following CVE entries describe Spectre variants: + + ============= ======================= ========== + CVE-2017-5753 Bounds check bypass Spectre-V1 + CVE-2017-5715 Branch target injection Spectre-V2 + +Problem +------- + +CPUs have shared caches, such as buffers for branch prediction, which are +later used to guide speculative execution. These buffers are not flushed +over context switches or change in privilege levels. Malicious software +might influence these buffers and trigger specific speculative execution +in the kernel or different user processes. This speculative execution can +then be used to read data in memory and cause side effects, such as displacing +data in a data cache. The side effect can then later be measured by the +malicious software, and used to determine the memory values read speculatively. + +Spectre attacks allow tricking other software to disclose +values in their memory. + +In a typical Spectre variant 1 attack, the attacker passes an parameter +to a victim. The victim boundary checks the parameter and rejects illegal +values. However due to speculation over branch prediction the code path +for correct values might be speculatively executed, then reference memory +controlled by the input parameter and leave measurable side effects in +the caches. The attacker could then measure these side effects +and determine the leaked value. + +There are some extensions of Spectre variant 1 attacks for reading +data over the network, see [2]. However the attacks are very +difficult, low bandwidth and fragile and considered low risk. + +For Spectre variant 2 the attacker poisons the indirect branch +predictors of the CPU. Then control is passed to the victim, which +executes indirect branches. Due to the poisoned branch predictor data +the CPU can speculatively execute arbitrary code in the victim's +address space, such as a code sequence ("disclosure gadget") that +reads arbitrary data on some input parameter and causes a measurable +cache side effect based on the value. The attacker can then measure +this side effect after gaining control again and determine the value. + +The most useful gadgets take an attacker-controlled input parameter so +that the memory read can be controlled. Gadgets without input parameters +might be possible, but the attacker would have very little control over what +memory can be read, reducing the risk of the attack revealing useful data. + +Attack scenarios +---------------- + +Here is a list of attack scenarios that have been anticipated, but +may not cover all possible attack patterns. Reduing the occurrences of +attack pre-requisites listed can reduce the risk that a spectre attack +leaks useful data. + +1. Local User process attacking kernel +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Code in system calls often enforces access controls with conditional +branches based on user data. These branches are potential targets for +Spectre v2 exploits. Interrupt handlers, on the other hand, rarely +handle user data or enforce access controls, which makes them unlikely +exploit targets. + +For typical variant 2 attack, the attacker may poison the CPU branch +buffers first, and then enter the kernel and trick it into jumping to a +disclosure gadget through an indirect branch. If the attacker wants to control the +memory addresses leaked, it would also need to pass a parameter +to the gadget, either through a register or through a known address in +memory. Finally when it executes again it can measure the side effect. + +Necessary Prequisites: +1. Malicious local process passing parameters to kernel +2. Kernel has secrets. + +2. User process attacking another user process +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In this scenario an malicious user process wants to attack another +user process through a context switch. + +For variant 1 this generally requires passing some parameter between +the processes, which needs a data passing relationship, such a remote +procedure calls (RPC). + +For variant 2 the poisoning can happen through a context switch, or +on CPUs with simultaneous multi-threading (SMT) potentially on the +thread sibling executing in parallel on the same core. In either case, +controlling the memory leaked by the disclosure gadget also requires a data +passing relationship to the victim process, otherwise while it may +observe values through side effects, it won't know which memory +addresses they relate to. + +Necessary Prerequisites: +1. Malicious code running as local process +2. Victim processes containing secrets running on same core. + +3. User sandbox attacking runtime in process +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +A process, such as a web browser, might be running interpreted or JITed +untrusted code, such as javascript code downloaded from a website. +It uses restrictions in the JIT code generator and checks in a run time +to prevent the untrusted code from attacking the hosting process. + +The untrusted code might either use variant 1 or 2 to trick +a disclosure gadget in the run time to read memory inside the process. + +Necessary Prerequisites: +1. Sandbox in process running untrusted code. +2. Runtime in same process containing secrets. + +4. Kernel sandbox attacking kernel +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The kernel has support for running user-supplied programs within the +kernel. Specific rules (such as bounds checking) are enforced on these +programs by the kernel to ensure that they do not violate access controls. + +eBPF is a kernel sub-system that uses user-supplied program +to execute JITed untrusted byte code inside the kernel. eBPF is used +for manipulating and examining network packets, examining system call +parameters for sand boxes and other uses. + +A malicious local process could upload and trigger an malicious +eBPF script to the kernel, with the script attacking the kernel +using variant 1 or 2 and reading memory. + +Necessary Prerequisites: +1. Malicious local process +2. eBPF JIT enabled for unprivileged users, attacking kernel with secrets +on the same machine. + +5. Virtualization guest attacking host +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +An untrusted guest might attack the host through a hyper call +or other virtualization exit. + +Necessary Prerequisites: +1. Untrusted guest attacking host +2. Host has secrets on local machine. + +For variant 1 VM exits use appropriate mitigations +("bounds clipping") to prevent speculation leaking data +in kernel code. For variant 2 the kernel flushes the branch buffer. + +6. Virtualization guest attacking other guest +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +An untrusted guest attacking another guest containing +secrets. Mitigations are similar to when a guest attack +the host. + +Runtime vulnerability information +--------------------------------- + +The kernel reports the vulnerability and mitigation status in +/sys/devices/system/cpu/vulnerabilities/* + +The spectre_v1 file describes the always enabled variant 1 +mitigation: + +/sys/devices/system/cpu/vulnerabilities/spectre_v1 + +The value in this file: + + ======================================= ================================= + 'Mitigation: __user pointer sanitation' Protection in kernel on a case by + case base with explicit pointer + sanitation. + ======================================= ================================= + +The spectre_v2 kernel file reports if the kernel has been compiled with a +retpoline aware compiler, if the CPU has hardware mitigation, and if the +CPU has microcode support for additional process specific mitigations. + +It also reports CPU features enabled by microcode to mitigate attack +between user processes: + +1. Indirect Branch Prediction Barrier (IBPB) to add additional + isolation between processes of different users +2. Single Thread Indirect Branch Prediction (STIBP) to additional + isolation between CPU threads running on the same core. + +These CPU features may impact performance when used and can +be enabled per process on a case-by-case base. + +/sys/devices/system/cpu/vulnerabilities/spectre_v2 + +The values in this file: + + - Kernel status: + + ==================================== ================================= + 'Not affected' The processor is not vulnerable + 'Vulnerable' Vulnerable, no mitigation + 'Mitigation: Full generic retpoline' Software-focused mitigation + 'Mitigation: Full AMD retpoline' AMD-specific software mitigation + 'Mitigation: Enhanced IBRS' Hardware-focused mitigation + ==================================== ================================= + + - Firmware status: + + ========== ============================================================= + 'IBRS_FW' Protection against user program attacks when calling firmware + ========== ============================================================= + + - Indirect branch prediction barrier (IBPB) status for protection between + processes of different users. This feature can be controlled through + prctl per process, or through kernel command line options. For more details + see below. + + =================== ======================================================== + 'IBPB: disabled' IBPB unused + 'IBPB: always-on' Use IBPB on all tasks + 'IBPB: conditional' Use IBPB on SECCOMP or indirect branch restricted tasks + =================== ======================================================== + + - Single threaded indirect branch prediction (STIBP) status for protection + between different hyper threads. This feature can be controlled through + prctl per process, or through kernel command line options. For more details + see below. + + ==================== ======================================================== + 'STIBP: disabled' STIBP unused + 'STIBP: forced' Use STIBP on all tasks + 'STIBP: conditional' Use STIBP on SECCOMP or indirect branch restricted tasks + ==================== ======================================================== + + - Return stack buffer (RSB) protection status: + + ============= =========================================== + 'RSB filling' Protection of RSB on context switch enabled + ============= =========================================== + +Full mitigations might require an microcode update from the CPU +vendor. When the necessary microcode is not available the kernel +will report vulnerability. + +Kernel mitigation +----------------- + +The kernel has default on mitigations for Variant 1 and Variant 2 +against attacks from user programs or guests. For variant 1 it +annotates vulnerable kernel code (as determined by the sparse code +scanning tool and code audits) to use "bounds clipping" to avoid any +usable disclosure gadgets. + +For variant 2 the kernel employs "retpoline" with compiler help to secure +the indirect branches inside the kernel, when CONFIG_RETPOLINE is enabled +and the compiler supports retpoline. On Intel Skylake-era systems the +mitigation covers most, but not all, cases, see [1] for more details. + +On CPUs with hardware mitigations for variant 2, retpoline is +automatically disabled at runtime. + +Using kernel address space randomization (CONFIG_RANDOMIZE_SLAB=y +and CONFIG_SLAB_FREELIST_RANDOM=y in the kernel configuration) +makes attacks on the kernel generally more difficult. + +Host mitigation +--------------- + +The Linux kernel uses retpoline to eliminate attacks on indirect +branches. It also flushes the Return Branch Stack on every VM exit to +prevent guests from attacking the host kernel when retpoline is +enabled. + +Variant 1 attacks are mitigated unconditionally. + +The kernel also allows guests to use any microcode based mitigations +they chose to use (such as IBPB or STIBP), assuming the +host has an updated microcode and reports the feature in +/sys/devices/system/cpu/vulnerabilities/spectre_v2. + +Mitigation control at kernel build time +--------------------------------------- + +When the CONFIG_RETPOLINE option is enabled the kernel uses special +code sequences to avoid attacks on indirect branches through +Variant 2 attacks. + +The compiler also needs to support retpoline and support the +-mindirect-branch=thunk-extern -mindirect-branch-register options +for gcc, or -mretpoline-external-thunk option for clang. + +When the compiler doesn't support these options the kernel +will report that it is vulnerable. + +Variant 1 mitigations and other side channel related user APIs are +enabled unconditionally. + +Hardware mitigation +------------------- + +Some CPUs have hardware mitigations (e.g. enhanced IBRS) for Spectre +variant 2. The 4.19 kernel has support for detecting this capability +and automatically disable any unnecessary workarounds at runtime. + +User program mitigation +----------------------- + +For variant 1 user programs can use LFENCE or bounds clipping. For more +details see [3]. + +For variant 2 user programs can be compiled with retpoline or +restricting its indirect branch speculation via prctl. (See +Documenation/speculation.txt for detailed API.) + +User programs should use address space randomization +(/proc/sys/kernel/randomize_va_space = 1 or 2) to make any attacks +more difficult. + +Mitigation control on the kernel command line +--------------------------------------------- + +Spectre v2 mitigations can be disabled and force enabled at the kernel +command line. + + nospectre_v2 [X86] Disable all mitigations for the Spectre variant 2 + (indirect branch prediction) vulnerability. System may + allow data leaks with this option, which is equivalent + to spectre_v2=off. + + + spectre_v2= [X86] Control mitigation of Spectre variant 2 + (indirect branch speculation) vulnerability. + The default operation protects the kernel from + user space attacks. + + on - unconditionally enable, implies + spectre_v2_user=on + off - unconditionally disable, implies + spectre_v2_user=off + auto - kernel detects whether your CPU model is + vulnerable + + Selecting 'on' will, and 'auto' may, choose a + mitigation method at run time according to the + CPU, the available microcode, the setting of the + CONFIG_RETPOLINE configuration option, and the + compiler with which the kernel was built. + + Selecting 'on' will also enable the mitigation + against user space to user space task attacks. + + Selecting 'off' will disable both the kernel and + the user space protections. + + Specific mitigations can also be selected manually: + + retpoline - replace indirect branches + retpoline,generic - google's original retpoline + retpoline,amd - AMD-specific minimal thunk + + Not specifying this option is equivalent to + spectre_v2=auto. + +For user space mitigation: + + spectre_v2_user= + [X86] Control mitigation of Spectre variant 2 + (indirect branch speculation) vulnerability between + user space tasks + + on - Unconditionally enable mitigations. Is + enforced by spectre_v2=on + + off - Unconditionally disable mitigations. Is + enforced by spectre_v2=off + + prctl - Indirect branch speculation is enabled, + but mitigation can be enabled via prctl + per thread. The mitigation control state + is inherited on fork. + + prctl,ibpb + - Like "prctl" above, but only STIBP is + controlled per thread. IBPB is issued + always when switching between different user + space processes. + + seccomp + - Same as "prctl" above, but all seccomp + threads will enable the mitigation unless + they explicitly opt out. + + seccomp,ibpb + - Like "seccomp" above, but only STIBP is + controlled per thread. IBPB is issued + always when switching between different + user space processes. + + auto - Kernel selects the mitigation depending on + the available CPU features and vulnerability. + + Default mitigation: + If CONFIG_SECCOMP=y then "seccomp", otherwise "prctl" + + Not specifying this option is equivalent to + spectre_v2_user=auto. + + In general the kernel by default selects + reasonable mitigations for the current CPU. To + disable Spectre v2 mitigations boot with + spectre_v2=off. Spectre v1 mitigations cannot + be disabled. + +APIs for mitigation control of user process +------------------------------------------- + +When enabling the "prctl" option for spectre_v2_user boot parameter, +prctl can be used to restrict indirect branch speculation on a process. +See Documenation/speculation.txt for detailed API. + +Processes containing secrets, such as cryptographic keys, may invoke +this prctl for extra protection against Spectre v2. + +Before running untrusted processes, restricting their indirect branch +speculation will prevent such processes from launching Spectre v2 attacks. + +Restricting indirect branch speuclation on a process should be only used +as needed, as restricting speculation reduces both performance of the +process, and also process running on the sibling CPU thread. + +Under the "seccomp" option, the processes sandboxed with SECCOMP will +have indirect branch speculation restricted automatically. + +References +---------- + +Intel white papers and documents on Spectre: + +https://newsroom.intel.com/wp-content/uploads/sites/11/2018/01/Intel-Analysi... + +[1] +https://software.intel.com/security-software-guidance/api-app/sites/default/... + +https://www.intel.com/content/www/us/en/architecture-and-technology/facts-ab... + +[3] https://software.intel.com/security-software-guidance/ + +https://software.intel.com/security-software-guidance/insights/deep-dive-sin... + +AMD white papers: + +https://developer.amd.com/wp-content/resources/90343-B_SoftwareTechniquesfor... + +https://www.amd.com/en/corporate/security-updates + +ARM white papers: + +https://developer.arm.com/support/arm-security-updates/speculative-processor... + +https://developer.arm.com/support/arm-security-updates/speculative-processor... + +MIPS: + +https://www.mips.com/blog/mips-response-on-speculative-execution-and-side-ch... + +Academic papers: + +https://spectreattack.com/spectre.pdf [original spectre paper] + +[2] https://arxiv.org/abs/1807.10535 [NetSpectre] + +https://arxiv.org/abs/1811.05441 [generalization of Spectre] + +https://arxiv.org/abs/1807.07940 [Spectre RSB, a variant of Spectre v2]