Hi Suzuki,
thanks for the reply! The CPUs of the boards I am using are all based on Arm-v8(.2), but I found the components' addresses in the manuals of the SoCs.
I managed to modify the Devicetree by writing my own .dtsi file (see attachment) and finally got the CoreSight devices in /sys/devices/.
However, dmesg shows the following:
[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Linux version 4.9.253-coresight (user@user-desktop) (gcc version 7.5.0 (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) ) #1 SMP PREEMPT Wed Jan 1 18:45:04 CET 2025
[ 0.000000] Boot CPU: AArch64 Processor [411fd071]
(omitted 87 lines)
[ 0.212039] DTS File Name: /home/user/Downloads/Linux_for_Tegra/source/public/kernel/kernel-4.9/arch/arm64/boot/dts/../../../../../../hardware/nvidia/platform/t210/porg/kernel-dts/tegra210-p3448-0000-p3449-0000-a02.dts
[ 0.212045] DTB Build time: Jan 1 2025 16:04:45
(omitted 35 lines)
[ 0.420616] DTS File Name: /home/user/Downloads/Linux_for_Tegra/source/public/kernel/kernel-4.9/arch/arm64/boot/dts/../../../../../../hardware/nvidia/platform/t210/porg/kernel-dts/tegra210-p3448-0000-p3449-0000-a02.dts
[ 0.420622] DTB Build time: Jan 1 2025 16:04:45
(omitted 75 lines)
[ 0.524166] OF: amba_device_add() failed (-19) for /funnel_bccplex@73001000
(omitted 367 lines)
[ 1.330484] OF: graph: no port node found in /etf@72030000
[ 1.330757] OF: graph: no port node found in /etr@72050000
[ 1.330987] OF: graph: no port node found in /funnel_major@72010000
[ 1.331238] OF: graph: no port node found in /ptm0@73440000
[ 1.331451] coresight-etm4x 73440000.ptm0: CPU0: Cortex-A57 ETM v4.0 initialized
[ 1.331482] OF: graph: no port node found in /ptm1@73540000
[ 1.331689] coresight-etm4x 73540000.ptm1: CPU1: Cortex-A57 ETM v4.0 initialized
[ 1.331719] OF: graph: no port node found in /ptm2@73640000
[ 1.331938] coresight-etm4x 73640000.ptm2: CPU2: Cortex-A57 ETM v4.0 initialized
[ 1.331944] extcon-disp-state extcon:disp-state: cable 47 state 0
[ 1.331946] Extcon AUX1(HDMI) disable
[ 1.331976] OF: graph: no port node found in /ptm3@73740000
[ 1.332192] coresight-etm4x 73740000.ptm3: CPU3: Cortex-A57 ETM v4.0 initialized
[ 1.332250] OF: graph: no port node found in /replicator@72040000
[ 1.332305] coresight-replicator-qcom 72040000.replicator: REPLICATOR 1.0 initialized
[ 1.332350] OF: graph: no port node found in /stm@72070000
[ 1.332386] coresight-stm 72070000.stm: stm_register_device failed, probing deffered
(omitted 64 lines)
[ 1.411751] OF: graph: no port node found in /stm@72070000
[ 1.412025] coresight-stm 72070000.stm: STM32 initialized
(omitted 212 lines)
Do you have an idea what I did wrong? In the end, I want to be able to follow the steps described here:
https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3275/index.html#pa…
Best regards,
Vincent
(P.S. There was a problem sending this email a first time, but it should work now)
On 07/01/2025 13:01, Yeoreum Yun wrote:
> Hi Suzuki,
>
>> Hi Levi
>>
>> On 23/12/2024 18:53, Yeoreum Yun wrote:
>>> While enable active config via cscfg_csdev_enable_active_config(),
>>> active config could be deactivated via configfs' sysfs interface.
>>> This could make UAF issue in below scenario:
>>>
>>> CPU0 CPU1
>>> (sysfs enable) load module
>>> cscfg_load_config_sets()
>>> activate config. // sysfs
>>> (sys_active_cnt == 1)
>>> ...
>>> cscfg_csdev_enable_active_config()
>>> lock(csdev->cscfg_csdev_lock)
>>> // here load config activate by CPU1
>>> unlock(csdev->cscfg_csdev_lock)
>>>
>>> deactivate config // sysfs
>>> (sys_activec_cnt == 0)
>>> cscfg_unload_config_sets()
>>> unload module
>>>
>>> // access to config_desc which freed
>>> // while unloading module.
>>> cfs_csdev_enable_config
>>>
>>> To address this, introduce sys_enable_cnt in cscfg_mgr to prevent
>>> deactivate while there is enabled configuration.
>>
>> Thanks for the finding the problem and the detailed description + patch. I
>> have some concerns on the fix, please find it below.
>>
>>>
>>
>>
>> So we have 3 atomic counters now !
>> cscfg_mgr->sys_active_cnt // Global count
>> config->active_cnt // Per config count,
>>
>> And another one which this one introduces.
>>
>> cscfg_mgr->sys_enable_cnt // ?
>>
>>
>> And config->active_cnt is always ever 0 or 1. i.e., it is not really a
>> reference counter at the moment but it indicates whether it is active or
>> not. Could we not use that for tracking the references on the specific
>> config ?
>>
>> i.e., every time we "enable_active_config" atomic_inc(config->active_cnt)
>>
>> and disable_active_config() always decrements the config. These could be
>> wrapped in cscfg_get() cscfg_put() which would, inc/dec the refcounts
>> and also drop the "module" reference when the active_cnt == 0
>
> This action is done via _cscfg_activate_config() already but its
> activation is done via "sysfs".
> and the checking active_cnt, I think it would increase lots of complex.
I don't understand. We have this today :
cscfg_config_desc which is getting de-activated. This has reference to
the module (owner). If someone is using this config_desc in running
session (be it perf or sysfg), it must be refcounted. As such I don't
see that the cscfg is refcounted anywhere. (The active_cnt indicates if
this has been activated, but not used in a session). If we have a
refcount on the "cscfg_config_desc" for each active session, we could
prevent/delay unloading the module until the last one drops.
Something like this is what I was checking ?
coresight: cscfg: Hold refcount on the config for active
sessions
Signed-off-by: Suzuki K Poulose <suzuki.poulose(a)arm.com>
---
.../hwtracing/coresight/coresight-config.h | 2 +-
.../hwtracing/coresight/coresight-syscfg.c | 24 +++++++++++++------
2 files changed, 18 insertions(+), 8 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-config.h
b/drivers/hwtracing/coresight/coresight-config.h
index 6ba013975741..84cdde6f0e4d 100644
--- a/drivers/hwtracing/coresight/coresight-config.h
+++ b/drivers/hwtracing/coresight/coresight-config.h
@@ -228,7 +228,7 @@ struct cscfg_feature_csdev {
* @feats_csdev:references to the device features to enable.
*/
struct cscfg_config_csdev {
- const struct cscfg_config_desc *config_desc;
+ struct cscfg_config_desc *config_desc;
struct coresight_device *csdev;
bool enabled;
struct list_head node;
diff --git a/drivers/hwtracing/coresight/coresight-syscfg.c
b/drivers/hwtracing/coresight/coresight-syscfg.c
index 11138a9762b0..53baeaaf907f 100644
--- a/drivers/hwtracing/coresight/coresight-syscfg.c
+++ b/drivers/hwtracing/coresight/coresight-syscfg.c
@@ -914,15 +914,20 @@ static int _cscfg_activate_config(unsigned long
cfg_hash)
return err;
}
+static void cscfg_config_desc_put(struct cscfg_config_desc *config_desc)
+{
+ if (!atomic_dec_return(&config_desc->active_cnt))
+ cscfg_owner_put(config_desc->load_owner);
+}
+
static void _cscfg_deactivate_config(unsigned long cfg_hash)
{
struct cscfg_config_desc *config_desc;
list_for_each_entry(config_desc, &cscfg_mgr->config_desc_list, item) {
if ((unsigned long)config_desc->event_ea->var == cfg_hash) {
- atomic_dec(&config_desc->active_cnt);
atomic_dec(&cscfg_mgr->sys_active_cnt);
- cscfg_owner_put(config_desc->load_owner);
+ cscfg_config_desc_put(config_desc);
dev_dbg(cscfg_device(), "Deactivate config %s.\n", config_desc->name);
break;
}
@@ -1047,7 +1052,7 @@ int cscfg_csdev_enable_active_config(struct
coresight_device *csdev,
unsigned long cfg_hash, int preset)
{
struct cscfg_config_csdev *config_csdev_active = NULL,
*config_csdev_item;
- const struct cscfg_config_desc *config_desc;
+ struct cscfg_config_desc *config_desc;
unsigned long flags;
int err = 0;
@@ -1062,8 +1067,8 @@ int cscfg_csdev_enable_active_config(struct
coresight_device *csdev,
spin_lock_irqsave(&csdev->cscfg_csdev_lock, flags);
list_for_each_entry(config_csdev_item, &csdev->config_csdev_list, node) {
config_desc = config_csdev_item->config_desc;
- if ((atomic_read(&config_desc->active_cnt)) &&
- ((unsigned long)config_desc->event_ea->var == cfg_hash)) {
+ if ((unsigned long)config_desc->event_ea->var == cfg_hash &&
+ atomic_inc_not_zero(&config_desc->active_cnt)) {
config_csdev_active = config_csdev_item;
csdev->active_cscfg_ctxt = (void *)config_csdev_active;
break;
@@ -1091,12 +1096,15 @@ int cscfg_csdev_enable_active_config(struct
coresight_device *csdev,
* Set enabled if OK, err if not.
*/
spin_lock_irqsave(&csdev->cscfg_csdev_lock, flags);
- if (csdev->active_cscfg_ctxt)
+ if (csdev->active_cscfg_ctxt == config_csdev_active)
config_csdev_active->enabled = true;
else
err = -EBUSY;
spin_unlock_irqrestore(&csdev->cscfg_csdev_lock, flags);
}
+ if (err)
+ cscfg_config_desc_put(config_desc);
+
}
return err;
}
@@ -1136,8 +1144,10 @@ void cscfg_csdev_disable_active_config(struct
coresight_device *csdev)
spin_unlock_irqrestore(&csdev->cscfg_csdev_lock, flags);
/* true if there was an enabled active config */
- if (config_csdev)
+ if (config_csdev) {
cscfg_csdev_disable_config(config_csdev);
+ cscfg_config_desc_put(config_csdev->config_desc);
+ }
}
EXPORT_SYMBOL_GPL(cscfg_csdev_disable_active_config);
--
> because, if so, it should iterate all config in each csdev.
> So, I believe it is the reason why the activation and module_cnt get via "sysfs"
> to prevent iterating every config in csdev when config unload.
>
> although, active_cnt in each config added to list in csdev be 0 or 1,
> the module could be >= 1 (by sum of active_cnt which have the same
> module owner). So I'm skeptical to use active_cnt like "reference cnt"
> and That's why I decide to use "sys_enable_cnt"
>>
>>> Signed-off-by: Yeoreum Yun <yeoreum.yun(a)arm.com>
>>> ---
>>> from v1 to v2:
>>> - modify commit message.
>>> ---
>>> .../hwtracing/coresight/coresight-etm4x-core.c | 3 +++
>>> drivers/hwtracing/coresight/coresight-syscfg.c | 18 ++++++++++++++++--
>>> drivers/hwtracing/coresight/coresight-syscfg.h | 2 ++
>>> 3 files changed, 21 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c
>>> index 86893115df17..6218ef40acbc 100644
>>> --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c
>>> +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c
>>> @@ -986,6 +986,9 @@ static void etm4_disable_sysfs(struct coresight_device *csdev)
>>> smp_call_function_single(drvdata->cpu, etm4_disable_hw, drvdata, 1);
>>>
>>> raw_spin_unlock(&drvdata->spinlock);
>>> +
>>> + cscfg_csdev_disable_active_config(csdev);
>>
>> This looks like a separate "fix" from what you are trying to address. Please
>> could split this ?
>
> I don't think so, because without this calling, the "sys_enable_cnt"
> never down, It makes error.
My point is, we need to "disable_active_config" irrespective of this
race, in the sysfs path. So this should be a fix apart from fixing the race.
Suzuki
>
>> Also, would like to hear what Mike has to say about this change.
>
> IIRC, I followed his suggestion.
>
> Thanks!
>
Hi Levi
On 23/12/2024 18:53, Yeoreum Yun wrote:
> While enable active config via cscfg_csdev_enable_active_config(),
> active config could be deactivated via configfs' sysfs interface.
> This could make UAF issue in below scenario:
>
> CPU0 CPU1
> (sysfs enable) load module
> cscfg_load_config_sets()
> activate config. // sysfs
> (sys_active_cnt == 1)
> ...
> cscfg_csdev_enable_active_config()
> lock(csdev->cscfg_csdev_lock)
> // here load config activate by CPU1
> unlock(csdev->cscfg_csdev_lock)
>
> deactivate config // sysfs
> (sys_activec_cnt == 0)
> cscfg_unload_config_sets()
> unload module
>
> // access to config_desc which freed
> // while unloading module.
> cfs_csdev_enable_config
>
> To address this, introduce sys_enable_cnt in cscfg_mgr to prevent
> deactivate while there is enabled configuration.
Thanks for the finding the problem and the detailed description + patch.
I have some concerns on the fix, please find it below.
>
So we have 3 atomic counters now !
cscfg_mgr->sys_active_cnt // Global count
config->active_cnt // Per config count,
And another one which this one introduces.
cscfg_mgr->sys_enable_cnt // ?
And config->active_cnt is always ever 0 or 1. i.e., it is not really a
reference counter at the moment but it indicates whether it is active or
not. Could we not use that for tracking the references on the specific
config ?
i.e., every time we "enable_active_config" atomic_inc(config->active_cnt)
and disable_active_config() always decrements the config. These could be
wrapped in cscfg_get() cscfg_put() which would, inc/dec the refcounts
and also drop the "module" reference when the active_cnt == 0
> Signed-off-by: Yeoreum Yun <yeoreum.yun(a)arm.com>
> ---
> from v1 to v2:
> - modify commit message.
> ---
> .../hwtracing/coresight/coresight-etm4x-core.c | 3 +++
> drivers/hwtracing/coresight/coresight-syscfg.c | 18 ++++++++++++++++--
> drivers/hwtracing/coresight/coresight-syscfg.h | 2 ++
> 3 files changed, 21 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/hwtracing/coresight/coresight-etm4x-core.c b/drivers/hwtracing/coresight/coresight-etm4x-core.c
> index 86893115df17..6218ef40acbc 100644
> --- a/drivers/hwtracing/coresight/coresight-etm4x-core.c
> +++ b/drivers/hwtracing/coresight/coresight-etm4x-core.c
> @@ -986,6 +986,9 @@ static void etm4_disable_sysfs(struct coresight_device *csdev)
> smp_call_function_single(drvdata->cpu, etm4_disable_hw, drvdata, 1);
>
> raw_spin_unlock(&drvdata->spinlock);
> +
> + cscfg_csdev_disable_active_config(csdev);
This looks like a separate "fix" from what you are trying to address.
Please could split this ?
Also, would like to hear what Mike has to say about this change.
Suzuki
> +
> cpus_read_unlock();
>
> /*
> diff --git a/drivers/hwtracing/coresight/coresight-syscfg.c b/drivers/hwtracing/coresight/coresight-syscfg.c
> index a70c1454b410..dfa7dcbaf25d 100644
> --- a/drivers/hwtracing/coresight/coresight-syscfg.c
> +++ b/drivers/hwtracing/coresight/coresight-syscfg.c
> @@ -953,7 +953,8 @@ int cscfg_config_sysfs_activate(struct cscfg_config_desc *config_desc, bool acti
> cscfg_mgr->sysfs_active_config = cfg_hash;
> } else {
> /* disable if matching current value */
> - if (cscfg_mgr->sysfs_active_config == cfg_hash) {
> + if (cscfg_mgr->sysfs_active_config == cfg_hash &&
> + !atomic_read(&cscfg_mgr->sys_enable_cnt)) {
> _cscfg_deactivate_config(cfg_hash);
> cscfg_mgr->sysfs_active_config = 0;
> } else
> @@ -1055,6 +1056,12 @@ int cscfg_csdev_enable_active_config(struct coresight_device *csdev,
> if (!atomic_read(&cscfg_mgr->sys_active_cnt))
> return 0;
>
> + /*
> + * increment sys_enable_cnt first to prevent deactivate the config
> + * while enable active config.
> + */
> + atomic_inc(&cscfg_mgr->sys_enable_cnt);
> +
> /*
> * Look for matching configuration - set the active configuration
> * context if found.
> @@ -1098,6 +1105,10 @@ int cscfg_csdev_enable_active_config(struct coresight_device *csdev,
> raw_spin_unlock_irqrestore(&csdev->cscfg_csdev_lock, flags);
> }
> }
> +
> + if (!config_csdev_active || err)
> + atomic_dec(&cscfg_mgr->sys_enable_cnt);
> +
> return err;
> }
> EXPORT_SYMBOL_GPL(cscfg_csdev_enable_active_config);
> @@ -1129,8 +1140,10 @@ void cscfg_csdev_disable_active_config(struct coresight_device *csdev)
> if (config_csdev) {
> if (!config_csdev->enabled)
> config_csdev = NULL;
> - else
> + else {
> config_csdev->enabled = false;
> + atomic_dec(&cscfg_mgr->sys_enable_cnt);
> + }
> }
> csdev->active_cscfg_ctxt = NULL;
> raw_spin_unlock_irqrestore(&csdev->cscfg_csdev_lock, flags);
> @@ -1179,6 +1192,7 @@ static int cscfg_create_device(void)
> INIT_LIST_HEAD(&cscfg_mgr->config_desc_list);
> INIT_LIST_HEAD(&cscfg_mgr->load_order_list);
> atomic_set(&cscfg_mgr->sys_active_cnt, 0);
> + atomic_set(&cscfg_mgr->sys_enable_cnt, 0);
> cscfg_mgr->load_state = CSCFG_NONE;
>
> /* setup the device */
> diff --git a/drivers/hwtracing/coresight/coresight-syscfg.h b/drivers/hwtracing/coresight/coresight-syscfg.h
> index 66e2db890d82..2fc397919985 100644
> --- a/drivers/hwtracing/coresight/coresight-syscfg.h
> +++ b/drivers/hwtracing/coresight/coresight-syscfg.h
> @@ -38,6 +38,7 @@ enum cscfg_load_ops {
> * @config_desc_list: List of system configuration descriptors to load into registered devices.
> * @load_order_list: Ordered list of owners for dynamically loaded configurations.
> * @sys_active_cnt: Total number of active config descriptor references.
> + * @sys_enable_cnt: Total number of enable of active config descriptor references.
> * @cfgfs_subsys: configfs subsystem used to manage configurations.
> * @sysfs_active_config:Active config hash used if CoreSight controlled from sysfs.
> * @sysfs_active_preset:Active preset index used if CoreSight controlled from sysfs.
> @@ -50,6 +51,7 @@ struct cscfg_manager {
> struct list_head config_desc_list;
> struct list_head load_order_list;
> atomic_t sys_active_cnt;
> + atomic_t sys_enable_cnt;
> struct configfs_subsystem cfgfs_subsys;
> u32 sysfs_active_config;
> int sysfs_active_preset;
> --
> LEVI:{C3F47F37-75D8-414A-A8BA-3980EC8A46D7}
>
On 06/01/2025 2:48 pm, Marc Zyngier wrote:
> On Mon, 06 Jan 2025 14:24:35 +0000,
> James Clark <james.clark(a)linaro.org> wrote:
>>
>> FEAT_TRF is a Coresight feature that allows trace capture to be
>> completely filtered at different exception levels, unlike the existing
>> TRCVICTLR controls which may still emit target addresses of branches,
>> even if the following trace is filtered.
>>
>> Without FEAT_TRF, it was possible to start a trace session on a host and
>> also collect trace from the guest as TRCVICTLR was never programmed to
>> exclude guests (and it could still emit target addresses even if it
>> was).
>>
>> With FEAT_TRF, the current behavior of trace in guests exists depends on
>> whether nVHE or VHE are being used. Both of the examples below are from
>> the host's point of view, as Coresight isn't accessible from guests.
>> This patchset is only relevant to when FEAT_TRF exists, otherwise there
>> is no change.
>>
>> Current behavior:
>>
>> nVHE/pKVM:
>>
>> Because the host and the guest are both using TRFCR_EL1, trace will be
>> generated in guests depending on the same filter rules the host is
>> using. For example if the host is tracing userspace only, then guest
>> userspace trace will also be collected.
>>
>> (This is further limited by whether TRBE is used because an issue
>> with TRBE means that it's completely disabled in nVHE guests, but it's
>> possible to have other tracing components.)
>>
>> VHE:
>>
>> With VHE, the host filters will be in TRFCR_EL2, but the filters in
>> TRFCR_EL1 will be active when the guest is running. Because we don't
>> write to TRFCR_EL1, guest trace will be completely disabled.
>>
>> New behavior:
>>
>> The guest filtering rules from the Perf session are now honored for both
>> nVHE and VHE modes. This is done by either writing to TRFCR_EL12 at the
>> start of the Perf session and doing nothing else further, or caching the
>> guest value and writing it at guest switch for nVHE. In pKVM, trace is
>> now be disabled for both protected and unprotected guests.
>>
>> There is also an optimization where the Coresight drivers pass their
>> enabled state to KVM. This means in the common case KVM doesn't have to
>> touch any sysregs when the feature isn't in use.
>>
>> Applies to kvmarm/next (00163be8bb59).
>
> Can you *PLEASE* stop this absolutely nonsense of posting patches
> based on top of random commits? Please look at how we integrate new
> developments: they are *always* based on an early -rc tag (usually
> -rc3).
>
> If you depend on other patches, add them to your series and post the
> whole thing.
>
> Thanks,
>
> M.
>
Sure, I re-posted it on the latest -rc with a few commits picked up.
Thanks
James
FEAT_TRF is a Coresight feature that allows trace capture to be
completely filtered at different exception levels, unlike the existing
TRCVICTLR controls which may still emit target addresses of branches,
even if the following trace is filtered.
Without FEAT_TRF, it was possible to start a trace session on a host and
also collect trace from the guest as TRCVICTLR was never programmed to
exclude guests (and it could still emit target addresses even if it
was).
With FEAT_TRF, the current behavior of trace in guests exists depends on
whether nVHE or VHE are being used. Both of the examples below are from
the host's point of view, as Coresight isn't accessible from guests.
This patchset is only relevant to when FEAT_TRF exists, otherwise there
is no change.
Current behavior:
nVHE/pKVM:
Because the host and the guest are both using TRFCR_EL1, trace will be
generated in guests depending on the same filter rules the host is
using. For example if the host is tracing userspace only, then guest
userspace trace will also be collected.
(This is further limited by whether TRBE is used because an issue
with TRBE means that it's completely disabled in nVHE guests, but it's
possible to have other tracing components.)
VHE:
With VHE, the host filters will be in TRFCR_EL2, but the filters in
TRFCR_EL1 will be active when the guest is running. Because we don't
write to TRFCR_EL1, guest trace will be completely disabled.
New behavior:
The guest filtering rules from the Perf session are now honored for both
nVHE and VHE modes. This is done by either writing to TRFCR_EL12 at the
start of the Perf session and doing nothing else further, or caching the
guest value and writing it at guest switch for nVHE. In pKVM, trace is
now be disabled for both protected and unprotected guests.
There is also an optimization where the Coresight drivers pass their
enabled state to KVM. This means in the common case KVM doesn't have to
touch any sysregs when the feature isn't in use.
Applies to 6.13-rc6
---
Changes since V9 [9]:
* Rebase onto 6.13-rc6 and pickup commit 38131c02a53f ("KVM: arm64:
Track presence of SPE/TRBE in kvm_host_data instead of vCPU") to get
host_data_set_flag(). Also requires two intertwined parent commits.
Changes since V8 [8]:
* Rename guest_trfcr_el1 -> trfcr_while_in_guest
* Rename GUEST_FILTER -> EL1_TRACING_CONFIGURED
* Rename kvm_set_trfcr() -> kvm_tracing_set_el1_configuration()
* #include ordering
* Reorder Coresight driver to remove need for preempt_disable()
to avoid the warning
* Force EL1_TRACING_CONFIGURED on for pKVM which drops an additional
special case but still disables trace
* Change set/clear trfcr to a single function that disables swapping
if it has the same value as the host
* Make the drain condition a bit clearer with __trace_needs_drain()
instead of host trfcr != 0 (Or checking individual E*TRE bits)
* Drain is only really required on switch to guest so move it there
* Only for pKVM, restore the original behavior for draining whenever
trbe is enabled. This prevents hypothetical case where a host has
the filters disabled but hasn't drained yet which we had by only
looking at host trfcr != 0
Changes since V7 [6]:
* Drop SPE changes
* Change the interface to be based on intent, i.e kvm_enable_trbe()
rather than passing the raw register value
* Drop change to re-use vcpu_flags mechanism in favour of [7]
* Simplify by using the same switch function to and from guest
Changes since V6 [5]:
* Implement a better "do nothing" case where both the SPE and Coresight
drivers give the enabled state to KVM, allowing some register
reads to be dropped.
* Move the state and feature flags out of the vCPU into the per-CPU
host_debug_state.
* Simplify the switch logic by adding a new flag HOST_STATE_SWAP_TRFCR
and only storing a single TRFCR value.
* Rename vcpu flag macros to a more generic kvm_flag...
Changes since V5 [4]:
* Sort new sysreg entries by encoding
* Add a comment about sorting arch/arm64/tools/sysreg
* Warn on preemptible() before calling smp_processor_id()
* Pickup tags
* Change TRFCR_EL2 from SysregFields to Sysreg because it was only
used once
Changes since V4 [3]:
* Remove all V3 changes that made it work in pKVM and just disable
trace there instead
* Restore PMU host/hyp state sharing back to how it was
(kvm_pmu_update_vcpu_events())
* Simplify some of the duplication in the comments and function docs
* Add a WARN_ON_ONCE() if kvm_etm_set_guest_trfcr() is called when
the trace filtering feature doesn't exist.
* Split sysreg change into a tools update followed by the new register
addition
Changes since V3:
* Create a new shared area to store the host state instead of copying
it before each VCPU run
* Drop commit that moved SPE and trace registers from host_debug_state
into the kvm sysregs array because the guest values were never used
* Document kvm_etm_set_guest_trfcr()
* Guard kvm_etm_set_guest_trfcr() with a feature check
* Drop Mark B and Suzuki's review tags on the sysreg patch because it
turned out that broke the Perf build and needed some unconventional
changes to fix it (as in: to update the tools copy of the headers in
the same commit as the kernel changes)
Changes since V2:
* Add a new iflag to signify presence of FEAT_TRF and keep the
existing TRBE iflag. This fixes the issue where TRBLIMITR_EL1 was
being accessed even if TRBE didn't exist
* Reword a commit message
Changes since V1:
* Squashed all the arm64/tools/sysreg changes into the first commit
* Add a new commit to move SPE and TRBE regs into the kvm sysreg array
* Add a comment above the TRFCR global that it's per host CPU rather
than vcpu
Changes since nVHE RFC [1]:
* Re-write just in terms of the register value to be written for the
host and the guest. This removes some logic from the hyp code and
a value of kvm_vcpu_arch:trfcr_el1 = 0 no longer means "don't
restore".
* Remove all the conditional compilation and new files.
* Change the kvm_etm_update_vcpu_events macro to a function.
* Re-use DEBUG_STATE_SAVE_TRFCR so iflags don't need to be expanded
anymore.
* Expand the cover letter.
Changes since VHE v3 [2]:
* Use the same interface as nVHE mode so TRFCR_EL12 is now written by
kvm.
[1]: https://lore.kernel.org/kvmarm/20230804101317.460697-1-james.clark@arm.com/
[2]: https://lore.kernel.org/kvmarm/20230905102117.2011094-1-james.clark@arm.com/
[3]: https://lore.kernel.org/linux-arm-kernel/20240104162714.1062610-1-james.cla…
[4]: https://lore.kernel.org/all/20240220100924.2761706-1-james.clark@arm.com/
[5]: https://lore.kernel.org/linux-arm-kernel/20240226113044.228403-1-james.clar…
[6]: https://lore.kernel.org/kvmarm/20241112103717.589952-1-james.clark@linaro.o…
[7]: https://lore.kernel.org/kvmarm/20241115224924.2132364-4-oliver.upton@linux.…
[8]: https://lore.kernel.org/linux-arm-kernel/20241127100130.1162639-1-james.cla…
[9]: https://lore.kernel.org/linux-arm-kernel/20250106142446.628923-1-james.clar…
James Clark (7):
arm64/sysreg: Add a comment that the sysreg file should be sorted
tools: arm64: Update sysreg.h header files
arm64/sysreg/tools: Move TRFCR definitions to sysreg
coresight: trbe: Remove redundant disable call
KVM: arm64: coresight: Give TRBE enabled state to KVM
KVM: arm64: Support trace filtering for guests
coresight: Pass guest TRFCR value to KVM
Oliver Upton (3):
KVM: arm64: Drop MDSCR_EL1_DEBUG_MASK
KVM: arm64: Get rid of __kvm_get_mdcr_el2() and related warts
KVM: arm64: Track presence of SPE/TRBE in kvm_host_data instead of
vCPU
arch/arm64/include/asm/kvm_asm.h | 5 +-
arch/arm64/include/asm/kvm_host.h | 37 +-
arch/arm64/include/asm/sysreg.h | 12 -
arch/arm64/kvm/arm.c | 5 +-
arch/arm64/kvm/debug.c | 94 ++--
arch/arm64/kvm/hyp/nvhe/debug-sr.c | 72 +--
arch/arm64/kvm/hyp/nvhe/hyp-main.c | 6 -
arch/arm64/kvm/hyp/vhe/debug-sr.c | 5 -
arch/arm64/tools/sysreg | 38 ++
.../coresight/coresight-etm4x-core.c | 49 ++-
drivers/hwtracing/coresight/coresight-etm4x.h | 2 +-
drivers/hwtracing/coresight/coresight-priv.h | 3 +
.../coresight/coresight-self-hosted-trace.h | 9 -
drivers/hwtracing/coresight/coresight-trbe.c | 15 +-
tools/arch/arm64/include/asm/sysreg.h | 410 +++++++++++++++++-
tools/include/linux/kasan-tags.h | 15 +
16 files changed, 626 insertions(+), 151 deletions(-)
create mode 100644 tools/include/linux/kasan-tags.h
--
2.34.1
Change since V5:
1. Fix the &CPUn vs &cpun issue in device tree file.
Change since V4:
1. Use ^ete(-[0-9]+)?$ for the pattern of node name -- comments from Krzysztof Kozlowski <krzk(a)kernel.org>
2. Update commit message --- comments from Rob Herring <robh(a)kernel.org>
Change since V3:
1. Use ^ete-[0-9]+$ for the pattern of node name -- comments from Rob Herring
Change since V2:
1. Change the name in binding as 'ete'.
Change since V1:
1. Remove the pattern match of ETE node name.
2. Update the tmc-etr node name in DT.
Mao Jinlong (2):
dt-bindings: arm: coresight: Update the pattern of ete node name
arm64: dts: qcom: sm8450: Add coresight nodes
.../arm/arm,embedded-trace-extension.yaml | 6 +-
arch/arm64/boot/dts/qcom/sm8450.dtsi | 726 ++++++++++++++++++
2 files changed, 729 insertions(+), 3 deletions(-)
--
2.17.1
FEAT_TRF is a Coresight feature that allows trace capture to be
completely filtered at different exception levels, unlike the existing
TRCVICTLR controls which may still emit target addresses of branches,
even if the following trace is filtered.
Without FEAT_TRF, it was possible to start a trace session on a host and
also collect trace from the guest as TRCVICTLR was never programmed to
exclude guests (and it could still emit target addresses even if it
was).
With FEAT_TRF, the current behavior of trace in guests exists depends on
whether nVHE or VHE are being used. Both of the examples below are from
the host's point of view, as Coresight isn't accessible from guests.
This patchset is only relevant to when FEAT_TRF exists, otherwise there
is no change.
Current behavior:
nVHE/pKVM:
Because the host and the guest are both using TRFCR_EL1, trace will be
generated in guests depending on the same filter rules the host is
using. For example if the host is tracing userspace only, then guest
userspace trace will also be collected.
(This is further limited by whether TRBE is used because an issue
with TRBE means that it's completely disabled in nVHE guests, but it's
possible to have other tracing components.)
VHE:
With VHE, the host filters will be in TRFCR_EL2, but the filters in
TRFCR_EL1 will be active when the guest is running. Because we don't
write to TRFCR_EL1, guest trace will be completely disabled.
New behavior:
The guest filtering rules from the Perf session are now honored for both
nVHE and VHE modes. This is done by either writing to TRFCR_EL12 at the
start of the Perf session and doing nothing else further, or caching the
guest value and writing it at guest switch for nVHE. In pKVM, trace is
now be disabled for both protected and unprotected guests.
There is also an optimization where the Coresight drivers pass their
enabled state to KVM. This means in the common case KVM doesn't have to
touch any sysregs when the feature isn't in use.
Applies to kvmarm/next (00163be8bb59).
---
Changes since V8 [8]:
* Rename guest_trfcr_el1 -> trfcr_while_in_guest
* Rename GUEST_FILTER -> EL1_TRACING_CONFIGURED
* Rename kvm_set_trfcr() -> kvm_tracing_set_el1_configuration()
* #include ordering
* Reorder Coresight driver to remove need for preempt_disable()
to avoid the warning
* Force EL1_TRACING_CONFIGURED on for pKVM which drops an additional
special case but still disables trace
* Change set/clear trfcr to a single function that disables swapping
if it has the same value as the host
* Make the drain condition a bit clearer with __trace_needs_drain()
instead of host trfcr != 0 (Or checking individual E*TRE bits)
* Drain is only really required on switch to guest so move it there
* Only for pKVM, restore the original behavior for draining whenever
trbe is enabled. This prevents hypothetical case where a host has
the filters disabled but hasn't drained yet which we had by only
looking at host trfcr != 0
Changes since V7 [6]:
* Drop SPE changes
* Change the interface to be based on intent, i.e kvm_enable_trbe()
rather than passing the raw register value
* Drop change to re-use vcpu_flags mechanism in favour of [7]
* Simplify by using the same switch function to and from guest
Changes since V6 [5]:
* Implement a better "do nothing" case where both the SPE and Coresight
drivers give the enabled state to KVM, allowing some register
reads to be dropped.
* Move the state and feature flags out of the vCPU into the per-CPU
host_debug_state.
* Simplify the switch logic by adding a new flag HOST_STATE_SWAP_TRFCR
and only storing a single TRFCR value.
* Rename vcpu flag macros to a more generic kvm_flag...
Changes since V5 [4]:
* Sort new sysreg entries by encoding
* Add a comment about sorting arch/arm64/tools/sysreg
* Warn on preemptible() before calling smp_processor_id()
* Pickup tags
* Change TRFCR_EL2 from SysregFields to Sysreg because it was only
used once
Changes since V4 [3]:
* Remove all V3 changes that made it work in pKVM and just disable
trace there instead
* Restore PMU host/hyp state sharing back to how it was
(kvm_pmu_update_vcpu_events())
* Simplify some of the duplication in the comments and function docs
* Add a WARN_ON_ONCE() if kvm_etm_set_guest_trfcr() is called when
the trace filtering feature doesn't exist.
* Split sysreg change into a tools update followed by the new register
addition
Changes since V3:
* Create a new shared area to store the host state instead of copying
it before each VCPU run
* Drop commit that moved SPE and trace registers from host_debug_state
into the kvm sysregs array because the guest values were never used
* Document kvm_etm_set_guest_trfcr()
* Guard kvm_etm_set_guest_trfcr() with a feature check
* Drop Mark B and Suzuki's review tags on the sysreg patch because it
turned out that broke the Perf build and needed some unconventional
changes to fix it (as in: to update the tools copy of the headers in
the same commit as the kernel changes)
Changes since V2:
* Add a new iflag to signify presence of FEAT_TRF and keep the
existing TRBE iflag. This fixes the issue where TRBLIMITR_EL1 was
being accessed even if TRBE didn't exist
* Reword a commit message
Changes since V1:
* Squashed all the arm64/tools/sysreg changes into the first commit
* Add a new commit to move SPE and TRBE regs into the kvm sysreg array
* Add a comment above the TRFCR global that it's per host CPU rather
than vcpu
Changes since nVHE RFC [1]:
* Re-write just in terms of the register value to be written for the
host and the guest. This removes some logic from the hyp code and
a value of kvm_vcpu_arch:trfcr_el1 = 0 no longer means "don't
restore".
* Remove all the conditional compilation and new files.
* Change the kvm_etm_update_vcpu_events macro to a function.
* Re-use DEBUG_STATE_SAVE_TRFCR so iflags don't need to be expanded
anymore.
* Expand the cover letter.
Changes since VHE v3 [2]:
* Use the same interface as nVHE mode so TRFCR_EL12 is now written by
kvm.
[1]: https://lore.kernel.org/kvmarm/20230804101317.460697-1-james.clark@arm.com/
[2]: https://lore.kernel.org/kvmarm/20230905102117.2011094-1-james.clark@arm.com/
[3]: https://lore.kernel.org/linux-arm-kernel/20240104162714.1062610-1-james.cla…
[4]: https://lore.kernel.org/all/20240220100924.2761706-1-james.clark@arm.com/
[5]: https://lore.kernel.org/linux-arm-kernel/20240226113044.228403-1-james.clar…
[6]: https://lore.kernel.org/kvmarm/20241112103717.589952-1-james.clark@linaro.o…
[7]: https://lore.kernel.org/kvmarm/20241115224924.2132364-4-oliver.upton@linux.…
[8]: https://lore.kernel.org/linux-arm-kernel/20241127100130.1162639-1-james.cla…
James Clark (7):
arm64/sysreg: Add a comment that the sysreg file should be sorted
tools: arm64: Update sysreg.h header files
arm64/sysreg/tools: Move TRFCR definitions to sysreg
coresight: trbe: Remove redundant disable call
KVM: arm64: coresight: Give TRBE enabled state to KVM
KVM: arm64: Support trace filtering for guests
coresight: Pass guest TRFCR value to KVM
arch/arm64/include/asm/kvm_host.h | 11 +
arch/arm64/include/asm/sysreg.h | 12 -
arch/arm64/kvm/debug.c | 50 ++-
arch/arm64/kvm/hyp/nvhe/debug-sr.c | 63 +--
arch/arm64/tools/sysreg | 38 ++
.../coresight/coresight-etm4x-core.c | 49 ++-
drivers/hwtracing/coresight/coresight-etm4x.h | 2 +-
drivers/hwtracing/coresight/coresight-priv.h | 3 +
.../coresight/coresight-self-hosted-trace.h | 9 -
drivers/hwtracing/coresight/coresight-trbe.c | 15 +-
tools/arch/arm64/include/asm/sysreg.h | 410 +++++++++++++++++-
tools/include/linux/kasan-tags.h | 15 +
12 files changed, 599 insertions(+), 78 deletions(-)
create mode 100644 tools/include/linux/kasan-tags.h
--
2.34.1
When multiple ETMs are enabled simultaneously, the time required
to complete a flush in the process of reading the TMC device node
may exceed the default wait time of 100us. If the TMC capture is
stopped while any ETM has not completed its flush, it can cause
the corresponding CPU to hang.
Fix the by checking the TMCReady bit after the flush. If TMCReady
bit is set, TraceCaptEn bit can be clear; otherwise, return directly
and stop the TMC read.
Signed-off-by: Yuanfang Zhang <quic_yuanfang(a)quicinc.com>
---
drivers/hwtracing/coresight/coresight-tmc-etf.c | 17 +++++++++++++++--
drivers/hwtracing/coresight/coresight-tmc-etr.c | 22 +++++++++++++++++-----
2 files changed, 32 insertions(+), 7 deletions(-)
diff --git a/drivers/hwtracing/coresight/coresight-tmc-etf.c b/drivers/hwtracing/coresight/coresight-tmc-etf.c
index d4f641cd9de69488fe3d1c1dc9b5a9eafb55ed59..bded290c42891d782344d9a6e63ebdbed6719133 100644
--- a/drivers/hwtracing/coresight/coresight-tmc-etf.c
+++ b/drivers/hwtracing/coresight/coresight-tmc-etf.c
@@ -80,11 +80,21 @@ static void tmc_etb_dump_hw(struct tmc_drvdata *drvdata)
return;
}
-static void __tmc_etb_disable_hw(struct tmc_drvdata *drvdata)
+static int __tmc_etb_disable_hw(struct tmc_drvdata *drvdata)
{
+ int rc;
+
CS_UNLOCK(drvdata->base);
tmc_flush_and_stop(drvdata);
+
+ rc = tmc_wait_for_tmcready(drvdata);
+ if (rc) {
+ dev_err(&drvdata->csdev->dev,
+ "Failed to disable : TMC is not ready\n");
+ CS_LOCK(drvdata->base);
+ return rc;
+ }
/*
* When operating in sysFS mode the content of the buffer needs to be
* read before the TMC is disabled.
@@ -94,6 +104,7 @@ static void __tmc_etb_disable_hw(struct tmc_drvdata *drvdata)
tmc_disable_hw(drvdata);
CS_LOCK(drvdata->base);
+ return 0;
}
static void tmc_etb_disable_hw(struct tmc_drvdata *drvdata)
@@ -650,7 +661,9 @@ int tmc_read_prepare_etb(struct tmc_drvdata *drvdata)
ret = -EINVAL;
goto out;
}
- __tmc_etb_disable_hw(drvdata);
+ ret = __tmc_etb_disable_hw(drvdata);
+ if (ret)
+ goto out;
}
drvdata->reading = true;
diff --git a/drivers/hwtracing/coresight/coresight-tmc-etr.c b/drivers/hwtracing/coresight/coresight-tmc-etr.c
index a48bb85d0e7f44a25b813f3c828cc3d705d16012..63a1f7501562fa0b5c2fe6ea53dce4d82842bec3 100644
--- a/drivers/hwtracing/coresight/coresight-tmc-etr.c
+++ b/drivers/hwtracing/coresight/coresight-tmc-etr.c
@@ -1135,11 +1135,21 @@ static void tmc_etr_sync_sysfs_buf(struct tmc_drvdata *drvdata)
}
}
-static void __tmc_etr_disable_hw(struct tmc_drvdata *drvdata)
+static int __tmc_etr_disable_hw(struct tmc_drvdata *drvdata)
{
+ int rc;
+
CS_UNLOCK(drvdata->base);
tmc_flush_and_stop(drvdata);
+
+ rc = tmc_wait_for_tmcready(drvdata);
+ if (rc) {
+ dev_err(&drvdata->csdev->dev,
+ "Failed to disable : TMC is not ready\n");
+ CS_LOCK(drvdata->base);
+ return rc;
+ }
/*
* When operating in sysFS mode the content of the buffer needs to be
* read before the TMC is disabled.
@@ -1150,7 +1160,7 @@ static void __tmc_etr_disable_hw(struct tmc_drvdata *drvdata)
tmc_disable_hw(drvdata);
CS_LOCK(drvdata->base);
-
+ return 0;
}
void tmc_etr_disable_hw(struct tmc_drvdata *drvdata)
@@ -1779,9 +1789,11 @@ int tmc_read_prepare_etr(struct tmc_drvdata *drvdata)
}
/* Disable the TMC if we are trying to read from a running session. */
- if (coresight_get_mode(drvdata->csdev) == CS_MODE_SYSFS)
- __tmc_etr_disable_hw(drvdata);
-
+ if (coresight_get_mode(drvdata->csdev) == CS_MODE_SYSFS) {
+ ret = __tmc_etr_disable_hw(drvdata);
+ if (ret)
+ goto out;
+ }
drvdata->reading = true;
out:
spin_unlock_irqrestore(&drvdata->spinlock, flags);
---
base-commit: fac04efc5c793dccbd07e2d59af9f90b7fc0dca4
change-id: 20250103-fix_cpu_hung-b5a95179ada4
Best regards,
--
Yuanfang Zhang <quic_yuanfang(a)quicinc.com>