Cc: Sudeep
Hi Steve
On 28/08/2023 17:35, Steve Clevenger wrote:
Hi Suzuki,
On 8/27/2023 2:35 PM, Suzuki K Poulose wrote:
Hi Steve
On 26/08/2023 01:14, Steve Clevenger wrote:
Unfortunately, I tested with the original patch not [PATCH V2]. I've remedied this. My results below:
[root@sut01sys-b212 linux]# cat /sys/devices/system/cpu/cpu123/ARMHC501:23/tmc_etr35/buf_modes_available auto flat catu [root@sut01sys-b212 linux]# cat /sys/devices/system/cpu/cpu123/ARMHC501:23/tmc_etr35/buf_mode_preferred auto [root@sut01sys-b212 linux]# echo "catu" > /sys/devices/system/cpu/cpu123/ARMHC501:23/tmc_etr35/buf_mode_preferred [root@sut01sys-b212 linux]# cat /sys/devices/system/cpu/cpu123/ARMHC501:23/tmc_etr35/buf_mode_preferred catu
As with the V1 patch, auto defaults to catu.
I expected to see tmc-sg (former default) as an available mode, but do not. As I recall, the buffer mode defaulted to ETR scatter-gather prior to this patch. Must this capability now be explicitly advertised? I've seen this done as "arm,scatter-gather" in device trees, but not used by Ampere. Perhaps someone can enlighten me.
Yes, you must add that property to the TMC-ETR node (for both DT and ACPI). In the past, almost all of the TMC-ETRs (except Juno board) locked up the system while using the SG mode (due to the interconnect issues, something to do with the transaction). Thus, we decided to add a property explicitly enabling this for a given platform.
When you mentioned, it was using TMC-ETR SG mode, how did you verify this ? Please be aware that the table allocation code etc are shared by both TMC-SG and CATU.
You might recall how this started. I had no way to test the CATU due to the order the ETR modes defaulted (Flat, ETR-SG, CATU). For test purposes, I programmatically swapped the ETR-SG/CATU order and could then verify CATU operation by the driver calling into CATU code. This
So, were you using the DT based boot for the above runs ?
suggests Flat mode was bypassed, and the driver defaulted to ETR-SG prior to this hack. This didn't offer the user any control, hence my feature request. Note that most of the early Ampere self-hosted trace collection used ETR-SG. Now I can't select it.
How is this property described in the ACPI? The "ACPI for CoreSight™ 1.1 Platform Design Document" (DEN0067) doesn't describe this.
This is not specified in the ACPI platform design document. I can get it fixed. Ideally we need a property describing that the scatter-gather mode is safe to use.
DT uses "arm,scatter-gather" property [0] and this is what we now expect in the ACPI based systems too.
https://elixir.bootlin.com/linux/latest/source/Documentation/devicetree/bind...
Does it sound fine ?
Suzuki
Thanks, Steve
Kind regards Suzuki
Steve C.
On 8/23/2023 4:10 PM, Steve Clevenger wrote:
Here's some quick feedback. My system shows two modes available; auto catu
etr_buf_mode_current is writable. I expected to see tmc-sg (former default) listed in etr_buf_modes_available but it doesn't show up.
Note that both the auto and catu etr_buf_mode_current settings default to catu. My understanding is auto should revert to the default behavior. On my system the default was tmc-sg.
More later.
[root@sut01sys-b212 kernel]# cat /sys/devices/system/cpu/cpu20/ARMHC501:60/tmc_etr96/etr_buf_modes_available
auto catu [root@sut01sys-b212 kernel]# cat /sys/devices/system/cpu/cpu20/ARMHC501:60/tmc_etr96/etr_buf_mode_current catu [root@sut01sys-b212 kernel]# echo "catu" > /sys/devices/system/cpu/cpu20/ARMHC501:60/tmc_etr96/etr_buf_mode_current [root@sut01sys-b212 kernel]# cat /sys/devices/system/cpu/cpu20/ARMHC501:60/tmc_etr96/etr_buf_mode_current catu
Steve C.
On 8/21/2023 12:40 PM, Steve Clevenger wrote:
Hi Suzuki,
I may be able to test it this week. You've already pointed me at the patch thread(s). The main holdup is I need to merge the 6.6 pending platform work in order to use the Ampere ACPI. I couldn't get these patches to apply directly to 6.4 last I tried.
Steve C.
On 8/18/2023 2:39 AM, Suzuki K Poulose wrote:
Cc: Steve
Steve,
Are you able to test this with CATU ?
On 18/08/2023 09:21, Anshuman Khandual wrote: > Currently TMC-ETR automatically selects the buffer mode from all > available > methods in the following sequentially fallback manner - also in that > order. > > 1. FLAT mode with or without IOMMU > 2. TMC-ETR-SG (scatter gather) mode when available > 3. CATU mode when available > > But this order might not be ideal for all situations. For example if > there > is a CATU connected to ETR, it may be better to use TMC-ETR scatter > gather > method, rather than CATU. But hard coding such order changes will > prevent > us from testing or using a particular mode. This change provides > following > new sysfs tunables for the user to control TMC-ETR buffer mode > explicitly, > if required. This adds following new sysfs files for buffer mode > selection > purpose explicitly in the user space. > > /sys/bus/coresight/devices/tmc_etr<N>/buf_modes_available > /sys/bus/coresight/devices/tmc_etr<N>/buf_mode_preferred > > $ cat buf_modes_available > auto flat tmc-sg catu ------------------> Supported TMC-ETR buffer > modes > > $ echo catu > buf_mode_preferred -------> Explicit buffer mode > request > > But explicit user request has to be within supported ETR buffer modes > only. > These sysfs interface files are exclussive to ETR, and hence these > are > not > available for other TMC devices such as ETB or ETF etc. > > A new auto' mode (i.e ETR_MODE_AUTO) has been added to help fallback > to the > existing default behaviour, when user provided preferred buffer mode > fails. > ETR_MODE_FLAT and ETR_MODE_AUTO are always available as preferred > modes. > > Cc: Suzuki K Poulose suzuki.poulose@arm.com > Cc: Mike Leach mike.leach@linaro.org > Cc: James Clark james.clark@arm.com > Cc: Leo Yan leo.yan@linaro.org > Cc: Alexander Shishkin alexander.shishkin@linux.intel.com > Cc: coresight@lists.linaro.org > Cc: linux-arm-kernel@lists.infradead.org > Cc: linux-kernel@vger.kernel.org > Signed-off-by: Anshuman Khandual anshuman.khandual@arm.com > --- > This applies on v6.5-rc6 > > Changes in V2: > > - Renamed sysfs file etr_buf_modes_available as buf_modes_available > - Renamed sysfs file buf_mode_current as buf_mode_preferred > - Renamed etr_supports_flat_mode() as etr_can_use_flat_mode() > - Renamed coresight_tmc_groups[] as coresight_etf_groups[] > - Reused coresight_tmc_group[] for trigger_cntr and buffer_size > - Fallback trying ETR_MODE_AUTO when user preferred mode fails > - Moved ETR sysfs details into coresight-tmc-etr.c > - Dropped etr_can_use_flat_mode() check while offering ETR_MODE_FLAT > in sysfs > - Moved struct etr_buf_hw inside coresight-tmc-etr.c > - Moved get_etr_buf_hw() and etr_can_use_flat_mode() inside > coresight-tmc-etr.c > - Updated month in > Documentation/ABI/testing/sysfs-bus-coresight-devices-tmc > > Changes in V1: > > https://lore.kernel.org/all/20230728084837.276551-1-anshuman.khandual@arm.co... > > .../testing/sysfs-bus-coresight-devices-tmc | 16 +++ > .../hwtracing/coresight/coresight-tmc-core.c | 15 ++- > .../hwtracing/coresight/coresight-tmc-etr.c | 111 > ++++++++++++++++-- > drivers/hwtracing/coresight/coresight-tmc.h | 3 + > 4 files changed, 131 insertions(+), 14 deletions(-)
Looks good to me.
Suzuki
Hi Steve
On 30/08/2023 17:04, Suzuki K Poulose wrote:
Cc: Sudeep
Hi Steve
On 28/08/2023 17:35, Steve Clevenger wrote:
Hi Suzuki,
On 8/27/2023 2:35 PM, Suzuki K Poulose wrote:
Hi Steve
On 26/08/2023 01:14, Steve Clevenger wrote:
Unfortunately, I tested with the original patch not [PATCH V2]. I've remedied this. My results below:
[root@sut01sys-b212 linux]# cat /sys/devices/system/cpu/cpu123/ARMHC501:23/tmc_etr35/buf_modes_available auto flat catu [root@sut01sys-b212 linux]# cat /sys/devices/system/cpu/cpu123/ARMHC501:23/tmc_etr35/buf_mode_preferred auto [root@sut01sys-b212 linux]# echo "catu" > /sys/devices/system/cpu/cpu123/ARMHC501:23/tmc_etr35/buf_mode_preferred [root@sut01sys-b212 linux]# cat /sys/devices/system/cpu/cpu123/ARMHC501:23/tmc_etr35/buf_mode_preferred catu
As with the V1 patch, auto defaults to catu.
I expected to see tmc-sg (former default) as an available mode, but do not. As I recall, the buffer mode defaulted to ETR scatter-gather prior to this patch. Must this capability now be explicitly advertised? I've seen this done as "arm,scatter-gather" in device trees, but not used by Ampere. Perhaps someone can enlighten me.
Yes, you must add that property to the TMC-ETR node (for both DT and ACPI). In the past, almost all of the TMC-ETRs (except Juno board) locked up the system while using the SG mode (due to the interconnect issues, something to do with the transaction). Thus, we decided to add a property explicitly enabling this for a given platform.
When you mentioned, it was using TMC-ETR SG mode, how did you verify this ? Please be aware that the table allocation code etc are shared by both TMC-SG and CATU.
You might recall how this started. I had no way to test the CATU due to the order the ETR modes defaulted (Flat, ETR-SG, CATU). For test purposes, I programmatically swapped the ETR-SG/CATU order and could then verify CATU operation by the driver calling into CATU code. This
So, were you using the DT based boot for the above runs ?
suggests Flat mode was bypassed, and the driver defaulted to ETR-SG prior to this hack. This didn't offer the user any control, hence my feature request. Note that most of the early Ampere self-hosted trace collection used ETR-SG. Now I can't select it.
How is this property described in the ACPI? The "ACPI for CoreSight™ 1.1 Platform Design Document" (DEN0067) doesn't describe this.
This is not specified in the ACPI platform design document. I can get it fixed. Ideally we need a property describing that the scatter-gather mode is safe to use.
Looks like this is not straight forward copying of DT property. We are investigating this on our side and will get back to you.
Suzuki
DT uses "arm,scatter-gather" property [0] and this is what we now expect in the ACPI based systems too.
https://elixir.bootlin.com/linux/latest/source/Documentation/devicetree/bind...
Does it sound fine ?
Suzuki
Thanks, Steve
Kind regards Suzuki
Steve C.
On 8/23/2023 4:10 PM, Steve Clevenger wrote:
Here's some quick feedback. My system shows two modes available; auto catu
etr_buf_mode_current is writable. I expected to see tmc-sg (former default) listed in etr_buf_modes_available but it doesn't show up.
Note that both the auto and catu etr_buf_mode_current settings default to catu. My understanding is auto should revert to the default behavior. On my system the default was tmc-sg.
More later.
[root@sut01sys-b212 kernel]# cat /sys/devices/system/cpu/cpu20/ARMHC501:60/tmc_etr96/etr_buf_modes_available
auto catu [root@sut01sys-b212 kernel]# cat /sys/devices/system/cpu/cpu20/ARMHC501:60/tmc_etr96/etr_buf_mode_current catu [root@sut01sys-b212 kernel]# echo "catu" > /sys/devices/system/cpu/cpu20/ARMHC501:60/tmc_etr96/etr_buf_mode_current [root@sut01sys-b212 kernel]# cat /sys/devices/system/cpu/cpu20/ARMHC501:60/tmc_etr96/etr_buf_mode_current catu
Steve C.
On 8/21/2023 12:40 PM, Steve Clevenger wrote:
Hi Suzuki,
I may be able to test it this week. You've already pointed me at the patch thread(s). The main holdup is I need to merge the 6.6 pending platform work in order to use the Ampere ACPI. I couldn't get these patches to apply directly to 6.4 last I tried.
Steve C.
On 8/18/2023 2:39 AM, Suzuki K Poulose wrote: > Cc: Steve > > Steve, > > Are you able to test this with CATU ? > > > On 18/08/2023 09:21, Anshuman Khandual wrote: >> Currently TMC-ETR automatically selects the buffer mode from all >> available >> methods in the following sequentially fallback manner - also in >> that >> order. >> >> 1. FLAT mode with or without IOMMU >> 2. TMC-ETR-SG (scatter gather) mode when available >> 3. CATU mode when available >> >> But this order might not be ideal for all situations. For >> example if >> there >> is a CATU connected to ETR, it may be better to use TMC-ETR scatter >> gather >> method, rather than CATU. But hard coding such order changes will >> prevent >> us from testing or using a particular mode. This change provides >> following >> new sysfs tunables for the user to control TMC-ETR buffer mode >> explicitly, >> if required. This adds following new sysfs files for buffer mode >> selection >> purpose explicitly in the user space. >> >> /sys/bus/coresight/devices/tmc_etr<N>/buf_modes_available >> /sys/bus/coresight/devices/tmc_etr<N>/buf_mode_preferred >> >> $ cat buf_modes_available >> auto flat tmc-sg catu ------------------> Supported TMC-ETR >> buffer >> modes >> >> $ echo catu > buf_mode_preferred -------> Explicit buffer mode >> request >> >> But explicit user request has to be within supported ETR buffer >> modes >> only. >> These sysfs interface files are exclussive to ETR, and hence these >> are >> not >> available for other TMC devices such as ETB or ETF etc. >> >> A new auto' mode (i.e ETR_MODE_AUTO) has been added to help >> fallback >> to the >> existing default behaviour, when user provided preferred buffer >> mode >> fails. >> ETR_MODE_FLAT and ETR_MODE_AUTO are always available as preferred >> modes. >> >> Cc: Suzuki K Poulose suzuki.poulose@arm.com >> Cc: Mike Leach mike.leach@linaro.org >> Cc: James Clark james.clark@arm.com >> Cc: Leo Yan leo.yan@linaro.org >> Cc: Alexander Shishkin alexander.shishkin@linux.intel.com >> Cc: coresight@lists.linaro.org >> Cc: linux-arm-kernel@lists.infradead.org >> Cc: linux-kernel@vger.kernel.org >> Signed-off-by: Anshuman Khandual anshuman.khandual@arm.com >> --- >> This applies on v6.5-rc6 >> >> Changes in V2: >> >> - Renamed sysfs file etr_buf_modes_available as buf_modes_available >> - Renamed sysfs file buf_mode_current as buf_mode_preferred >> - Renamed etr_supports_flat_mode() as etr_can_use_flat_mode() >> - Renamed coresight_tmc_groups[] as coresight_etf_groups[] >> - Reused coresight_tmc_group[] for trigger_cntr and buffer_size >> - Fallback trying ETR_MODE_AUTO when user preferred mode fails >> - Moved ETR sysfs details into coresight-tmc-etr.c >> - Dropped etr_can_use_flat_mode() check while offering >> ETR_MODE_FLAT >> in sysfs >> - Moved struct etr_buf_hw inside coresight-tmc-etr.c >> - Moved get_etr_buf_hw() and etr_can_use_flat_mode() inside >> coresight-tmc-etr.c >> - Updated month in >> Documentation/ABI/testing/sysfs-bus-coresight-devices-tmc >> >> Changes in V1: >> >> https://lore.kernel.org/all/20230728084837.276551-1-anshuman.khandual@arm.co... >> >> .../testing/sysfs-bus-coresight-devices-tmc | 16 +++ >> .../hwtracing/coresight/coresight-tmc-core.c | 15 ++- >> .../hwtracing/coresight/coresight-tmc-etr.c | 111 >> ++++++++++++++++-- >> drivers/hwtracing/coresight/coresight-tmc.h | 3 + >> 4 files changed, 131 insertions(+), 14 deletions(-) > > > Looks good to me. > > Suzuki > >
-
Suzuki K Poulose