Hardware assisted tracing families such as ARM Coresight, Intel PT provides rich tracing capabilities including instruction level tracing and accurate timestamps which are very useful for profiling and also pose a significant security risk. One such example of security risk is when kernel mode tracing is not excluded and these hardware assisted tracing can be used to analyze cryptographic code execution. In this case, even the root user must not be able to infer anything.

To explain it more clearly in the words of a security team member (credits: Mattias Nissler),

"Consider a system where disk contents are encrypted and the encryption key is set up by the user when mounting the file system. From that point on the encryption key resides in the kernel. It seems reasonable to expect that the disk encryption key be protected from exfiltration even if the system later suffers a root compromise (or even against insiders that have root access), at least as long as the attacker doesn't manage to compromise the kernel."

Here the idea is to protect such important information from all users including root users since root privileges does not have to mean full control over the kernel [1] and root compromise does not have to be the end of the world.

Currently we can exclude kernel mode tracing via perf_event_paranoid sysctl but it has following limitations,

* It is applicable to all PMUs and not just the ones supporting instruction tracing. * No option to restrict kernel mode instruction tracing by the root user. * Not possible to restrict kernel mode instruction tracing when the hardware assisted tracing IPs like ARM Coresight ETMs use an additional interface via sysfs for tracing in addition to perf interface.

So introduce a new config CONFIG_EXCLUDE_KERNEL_HW_ITRACE to exclude kernel mode instruction tracing which will be generic and applicable to all hardware tracing families and which can also be used with other interfaces like sysfs in case of ETMs.

[1] https://lwn.net/Articles/796866/

Suggested-by: Suzuki K Poulose suzuki.poulose@arm.com Suggested-by: Al Grant al.grant@arm.com Tested-by: Denis Nikitin denik@chromium.org Link: https://lore.kernel.org/lkml/20201015124522.1876-1-saiprakash.ranjan@codeaur... Signed-off-by: Sai Prakash Ranjan saiprakash.ranjan@codeaurora.org --- init/Kconfig | 12 ++++++++++++ kernel/events/core.c | 6 ++++++ 2 files changed, 18 insertions(+)

diff --git a/init/Kconfig b/init/Kconfig index af454a51f3c5..31b4d1f26bce 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1832,6 +1832,18 @@ config DEBUG_PERF_USE_VMALLOC

endmenu

+config EXCLUDE_KERNEL_HW_ITRACE + bool "Exclude kernel mode hardware assisted instruction tracing" + depends on PERF_EVENTS + help + Exclude kernel mode instruction tracing by hardware tracing + family such as ARM Coresight ETM, Intel PT and so on. + + This option allows to disable kernel mode instruction tracing + offered by hardware assisted tracing for all users(including root) + especially for production systems where only userspace tracing might + be preferred for security reasons. + config VM_EVENT_COUNTERS default y bool "Enable VM event counters for /proc/vmstat" if EXPERT diff --git a/kernel/events/core.c b/kernel/events/core.c index aece2fe19693..044a774cef6d 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -11866,6 +11866,12 @@ SYSCALL_DEFINE5(perf_event_open, goto err_task; }

+ if (IS_ENABLED(CONFIG_EXCLUDE_KERNEL_HW_ITRACE) && + (event->pmu->capabilities & PERF_PMU_CAP_ITRACE) && !attr.exclude_kernel) { + err = -EACCES; + goto err_alloc; + } + if (is_sampling_event(event)) { if (event->pmu->capabilities & PERF_PMU_CAP_NO_INTERRUPT) { err = -EOPNOTSUPP;

On production systems with ETMs enabled, it is preferred to exclude kernel mode(NS EL1) tracing for security concerns and support only userspace(NS EL0) tracing. Perf subsystem interface for ETMs use the newly introduced kernel config CONFIG_EXCLUDE_KERNEL_HW_ITRACE to exclude kernel mode tracing, but there is an additional interface via sysfs for ETMs which also needs to be handled to exclude kernel mode tracing. So we use this same generic kernel config to handle the sysfs mode of tracing. This config is disabled by default and would not affect the current configuration which has both kernel and userspace tracing enabled by default.

Tested-by: Denis Nikitin denik@chromium.org Signed-off-by: Sai Prakash Ranjan saiprakash.ranjan@codeaurora.org --- drivers/hwtracing/coresight/coresight-etm4x-core.c | 14 +++++++++++++- .../hwtracing/coresight/coresight-etm4x-sysfs.c | 3 ++- 2 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c index b20b6ff17cf6..f94143057bb8 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c @@ -1052,12 +1052,16 @@ static void etm4_set_default(struct etmv4_config *config) return;

/* - * Make default initialisation trace everything + * Make default initialisation trace everything when + * CONFIG_EXCLUDE_KERNEL_HW_ITRACE is disabled. * * This is done by a minimum default config sufficient to enable * full instruction trace - with a default filter for trace all * achieved by having no filtering. */ + if (IS_ENABLED(CONFIG_EXCLUDE_KERNEL_HW_ITRACE)) + config->mode |= ETM_MODE_EXCL_KERN; + etm4_set_default_config(config); etm4_set_default_filter(config); } @@ -1195,6 +1199,7 @@ static int etm4_set_event_filters(struct etmv4_drvdata *drvdata, void etm4_config_trace_mode(struct etmv4_config *config) { u32 mode; + struct etmv4_drvdata *drvdata = container_of(config, struct etmv4_drvdata, config);

mode = config->mode; mode &= (ETM_MODE_EXCL_KERN | ETM_MODE_EXCL_USER); @@ -1202,6 +1207,13 @@ void etm4_config_trace_mode(struct etmv4_config *config) /* excluding kernel AND user space doesn't make sense */ WARN_ON_ONCE(mode == (ETM_MODE_EXCL_KERN | ETM_MODE_EXCL_USER));

+ if (!(mode & ETM_MODE_EXCL_KERN) && IS_ENABLED(CONFIG_EXCLUDE_KERNEL_HW_ITRACE)) { + dev_err(&drvdata->csdev->dev, + "Kernel mode tracing is not allowed, check your kernel config\n"); + config->mode |= ETM_MODE_EXCL_KERN; + return; + } + /* nothing to do if neither flags are set */ if (!(mode & ETM_MODE_EXCL_KERN) && !(mode & ETM_MODE_EXCL_USER)) return; diff --git a/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c b/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c index 989ce7b8ade7..f1d19d69d151 100644 --- a/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c +++ b/drivers/hwtracing/coresight/coresight-etm4x-sysfs.c @@ -426,7 +426,8 @@ static ssize_t mode_store(struct device *dev, else config->vinst_ctrl &= ~BIT(11);

- if (config->mode & (ETM_MODE_EXCL_KERN | ETM_MODE_EXCL_USER)) + if ((config->mode & (ETM_MODE_EXCL_KERN | ETM_MODE_EXCL_USER)) || + IS_ENABLED(CONFIG_EXCLUDE_KERNEL_HW_ITRACE)) etm4_config_trace_mode(config);

spin_unlock(&drvdata->spinlock);

Hi,

Thanks for taking a look, comments inline.

On 2021-02-23 01:44, Doug Anderson wrote:

Hi,

On Fri, Jan 29, 2021 at 11:08 AM Sai Prakash Ranjan saiprakash.ranjan@codeaurora.org wrote:

...

@@ -1202,6 +1207,13 @@ void etm4_config_trace_mode(struct etmv4_config *config) /* excluding kernel AND user space doesn't make sense */ WARN_ON_ONCE(mode == (ETM_MODE_EXCL_KERN | ETM_MODE_EXCL_USER));

•   if (!(mode & ETM_MODE_EXCL_KERN) && 
  

IS_ENABLED(CONFIG_EXCLUDE_KERNEL_HW_ITRACE)) {

•           dev_err(&drvdata->csdev->dev,
  

•                   "Kernel mode tracing is not allowed, check 
  

your kernel config\n");

•           config->mode |= ETM_MODE_EXCL_KERN;
  

•           return;
  

So I'm not an expert on this code, but the above looks suspicious to me. Specifically you are still modifying "config->mode" even though printing an "error" (dev_err, not dev_warn) and then skipping the rest of this function. Since you're skipping the rest of this function you're not applying the access, right? Naively I'd have expected one of these:

1. Maybe the "dev_err" should be a "dev_warn" and then you shouldn't

"return". In this case you're just implicitly adding "ETM_MODE_EXCL_KERN" (and shouting) but then making things work. Of course, then what happens if the user already specified "ETM_MODE_EXCL_USER" too? As per the comment above that "doesn't make sense". ...so maybe the code wouldn't behave properly...

2. Maybe you should be modifying this function to return an error code.

mode_store() which is the caller of this function sets the config->mode based on the value passed in the sysfs, so if the user passes the mode which doesn't exclude the kernel even though the kernel config is enabled and the code just sets it, then that is an error and the user should be warned about, so I used dev_err, but I can change it to dev_warn if that is preferred. And to make sysfs mode show the original mode after failure, I set the mode in etm4_config_trace_mode().

But you are right, I am skipping to set the config for other mode bits and returning which is wrong, will fix it as you suggest below.

3. Maybe you should just be updating the one caller of this function

to error check this right at the beginning of the function and then fail the sysfs write if the user did the wrong thing. Then in etm4_config_trace_mode you could just have a WARN_ON_ONCE if the kernel wasn't excluded...

Right, will do this.

Thanks, Sai