Hi Christoph
On Tue, 27 Dec 2022 at 17:09, Christoph Hellwig <hch(a)infradead.org> wrote:
>
> On Mon, Dec 19, 2022 at 11:46:35PM +0000, Mike Leach wrote:
> > Add in functionality and binary attribute to load and unload
> > configurations as binary data.
> >
> > Files are loaded via the 'load' binary attribute. System reads the incoming
> > file, which must be formatted correctly as defined in the file reader code.
> > This will create configuration(s) and/or feature(s) and load them
> > into the system.
>
> Binary attributes are intended to pass things such as firmware
> through. Defining your own structured file format seems like a
> major abuse of the configfs design. What's the advantage of this
> over simply using an ioctl?
The coresight configurations loaded here, represent programming of the
entire coresight subsystem - possibly tens of registers (especially on
the ETM), across multiple devices, in ways that are not possible using
the limited parameters of the perf command line.
The ETM can be programmed in ways that use counters. sequencers, and
optionally interact with other components such as CTI / CTM to send
conditional hardware triggers, all of which control the when and how
the trace is collected. Additionally there are system trace components
that might need to run at the same time - such as ELA, and other
system monitors that output data on the trace bus currently being
introduce by some chip designers.
As such the configuration must be loaded into the coresight system as
a single operation - with the individual drivers validating the
requested programming, where any error will fail the configuration
load. The individual drivers are also responsible for defining which
device registers user configurations can use - these being limited to
those that affect the scope of trace collected, with other operational
functions being reserved to the drivers themselves.
To achieve this a variable sized table of programming descriptors is
defined, that are transferred into the individual devices. The very
limited set of built in configurations - where the programming
descriptors are compiled into the driver modules themselves use the
same set of descriptors. however, recompiling kernel modules to
program new configurations is neither scaleable, flexible or desirable
- so we need a way of loading configurations at runtime. So the file
structure is simple a serialisation of these descriptor tables - with
a header defining the input type and overall size..
The advantage of these runtime configurations is that they can be
portable - and dependent on the hardware in the system, not the kernel
build version. Moreover, there are trace scenarios when we want to
trace what is present, not recompile a module / kernel to achieve
this, especially investigating issues on production systems.
1) using configfs to load these configurations keeps all the coresight
configuration ABI in a single file system - configfs. The current
builtin configurations can be viewed, enabled, and parameters
configrured in the current configfs that we have upstreamed. Adding
load / unload here is a logical extension of this.
2) because of the nature of configurations described above - we would
need a separate device to represent the whole subsystem - in order to
provide the ioctl support for loading. This device approach to
managing configurations has been previously rejected by the Coresight
maintainers, who suggested that configfs was the correct way to
configure a complex sub-system.
3) configurations are variable in size, An ioctl command would provide
a pointer to the configuration data - but the kernel would have to
trust that the underlying data is correctly formed. With a configfs
file write we get the buffer _and_ its size which is a good deal safer
from an input perspective.
4) ioctl use would require a loader program - configfs allows load
directly from the command line.
I agree that ioctls certainly have there uses, especially with small,
fixed size data types - but configfs is far better suited to this
complex use case.
Indeed the ioctl documentation suggests using configfs for
configuration cases that are too complex for sysfs, when an ioctl may
not be suitable.
This use of binary attributes is based on the existing use of a
configfs binary attribute is for the ACPI tables - the ACPI driver
here takes the buffer, does some initial validation of the file size
etc, then calls the inslall ./ validate ACPI routines where the table
is added to an internal list of tables maintained by the kernel. These
tables may well be shared by firmware - but are also used by the
kernel.
There appears to be nothing in the configfs documentation limiting the
use of binary attributes for passing firmware, Even the sysfs docs
suggest that this is an expected use but it is not a hard and fast
rule if there are no alternatives.
Granted in the vast majority of cases there are better alternatives.
I believe that loading via configfs is the best and safest engineering
solution for this particular use case.
Regards
Mike
--
Mike Leach
Principal Engineer, ARM Ltd.
Manchester Design Centre. UK
Hi Bagas,
On Wed, 21 Dec 2022 at 03:55, Bagas Sanjaya <bagasdotme(a)gmail.com> wrote:
>
> On Mon, Dec 19, 2022 at 11:46:38PM +0000, Mike Leach wrote:
> > diff --git a/Documentation/trace/coresight/coresight-config.rst b/Documentation/trace/coresight/coresight-config.rst
> > index 6d5ffa6f7347..109053eb1b93 100644
> > --- a/Documentation/trace/coresight/coresight-config.rst
> > +++ b/Documentation/trace/coresight/coresight-config.rst
> > @@ -141,11 +141,11 @@ Mount configfs as normal and the 'cs-syscfg' subsystem will appear::
> > $ ls /config
> > cs-syscfg stp-policy
> >
> > -This has two sub-directories::
> > +This has two sub-directories, with the load and unload attribute files::
> >
> > $ cd cs-syscfg/
> > $ ls
> > - configurations features
> > + configurations features load unload
> >
> > The system has the configuration 'autofdo' built in. It may be examined as
> > follows::
> > @@ -278,9 +278,16 @@ Creating and Loading Custom Configurations
> > ==========================================
> >
> > Custom configurations and / or features can be dynamically loaded into the
> > -system by using a loadable module.
> > +system by using a loadable module, or by loading a binary configuration
> > +file in configfs.
> >
> > -An example of a custom configuration is found in ./samples/coresight.
> > +Loaded configurations can use previously loaded features. The system will
> > +ensure that it is not possible to unload a feature that is currently in
> > +use, by enforcing the unload order as the strict reverse of the load order.
> > +
> > +
> > +Using a Loadable Module
> > +-----------------------
> >
> > This creates a new configuration that uses the existing built in
> > strobing feature, but provides a different set of presets.
> > @@ -289,6 +296,187 @@ When the module is loaded, then the configuration appears in the configfs
> > file system and is selectable in the same way as the built in configuration
> > described above.
> >
> > -Configurations can use previously loaded features. The system will ensure
> > -that it is not possible to unload a feature that is currently in use, by
> > -enforcing the unload order as the strict reverse of the load order.
> > +The file 'coresight-cfg-sample.c' contains the configuration and module
> > +initialisation code needed to create the loadable module.
> > +
> > +This will be built alongside the kernel modules if select in KConfig.
>
> What config options (CONFIG_) are required for above to work?
>
> > +
> > +An example of a custom configuration module is found in './samples/coresight'.
> > +
> > +Using a Binary Configuration File
> > +---------------------------------
> > +
> > +The './tools/coresight' directory contains example programs to generate and
> > +read and print binary configuration files.
> > +
> > +Building the tools creates the 'coresight-cfg-file-gen' program that will
> > +generate a configuration binary 'example1.cscfg' that can be loaded into the
> > +system using configfs. The configuration declared in the source file
> > +'coresight-cfg-example1.c' is named 'autofdo3' - the name that will be used
> > +once loaded.
> > +
> > +The source files 'coresight-cfg-bufw.h' and 'coresight-cfg-bufw.c' provide a
> > +standard function to convert a configuration declared in 'C' into the correct
> > +binary buffer format. These files can be re-used to create new custom
> > +configurations. Alternatively, addition examples can be added to the
>
> s/addition/additional/
>
> > +'coresight-cfg-file-gen' program::
> > +
> > + $ ./coresight-cfg-file-gen
> > + Coresight Configuration file Generator
> > +
> > + Generating example1 example
> > + Generating example2 example
> > +
> > +The program 'coresight-cfg-file-read' can read back and print a configuration
> > +binary. This is built using the file reader from the driver code
> > +(coresight-config-file.c), which is copied over into './tools/coresight' at
> > +build time.::
> > +
> > + ./coresight-cfg-file-read example1.cscfg
> > + CoreSight Configuration file reader
> > + ============================================
> > +
> > + Configuration 1
> > + Name:- autofdo3
> > + Description:-
> > + Setup ETMs with strobing for autofdo
> > + Supplied presets allow experimentation with mark-space ratio for various loads
> > +
> > + Uses 1 features:-
> > + Feature-1: strobing
> > +
> > + Provides 4 sets of preset values, 2 presets per set
> > + set[0]: 0x7d0, 0x64,
> > + set[1]: 0x7d0, 0x3e8,
> > + set[2]: 0x7d0, 0x1388,
> > + set[3]: 0x7d0, 0x2710,
> > +
> > + ============================================
> > + File contains no features
> > +
> > +There are additional attributes in the cs-syscfg directory - load and
> > +unload that can be used to load and unload configuration binary files. To
> > +load, 'cat' the binary config into the load attribute::
> > +
> > + $ ls /config/cs-syscfg
> > + configurations features load unload
> > + $ cat example1.cscfg > /config/cs-syscfg/load
> > + $ ls /config/cs-syscfg/configurations/
> > + autofdo autofdo3
> > +
> > +To unload, use the same file in the unload attribute::
> > +
> > + $ cat example1.cscfg > /config/cs-syscfg/unload
> > + ls /config/cs-syscfg/configurations/
> > + autofdo
> > +
> > +
> > +
> > +Binary Configuration File Format
> > +--------------------------------
> > +
> > +The file format is defined in the source file **coresight-config-file.h**
>
> Use single-quote for identifier names for consistency here.
>
> > +
> > +The source reader and generator examples produce a binary of this format.
> > +
> > +This arrangement is reproduced below:-
> > +
> > +Overall File structure
> > +~~~~~~~~~~~~~~~~~~~~~~
> > +
> > +::
> > +
> > + [cscfg_file_header] // Mandatory
> > + [CONFIG_ELEM]* // Optional - multiple, defined by cscfg_file_header.nr_configs
> > + [FEATURE_ELEM]* // Optional - multiple, defined by cscfg_file_header.nr_features
> > +
> > +File is invalid if both [CONFIG_ELEM] and [FEATURE_ELEM] are omitted.
> > +
> > +A file that contains only [FEATURE_ELEM] may be loaded, and the features used
> > +by subsequently loaded files with [CONFIG_ELEM] elements.
> > +
> > +Element Name Strings
> > +~~~~~~~~~~~~~~~~~~~~
> > +
> > +Configuration name strings are required to consist of alphanumeric characters and '_' only. Other special characters are not permitted.
> > +
> > +::
> > + my_config_2 // is a valid name.
> > + this-bad-config#5 // this will not work
>
> Instead of using code-block for name examples, what about "... For
> example, foo_bar is a valid name where as foo-bar# is not."?
>
> > +
> > +This is in order to comply with the requirements of the perf command line.
> > +
> > +It is recommended that Feature and Parameter names use the same convention to allow for future enhancements to the command line syntax.
> > +
> > +CONFIG_ELEM element
> > +~~~~~~~~~~~~~~~~~~~
> > +
> > +::
> > +
> > + [cscfg_file_elem_header] // header length value to end of feature strings.
> > + [cscfg_file_elem_str] // name of the configuration.
> > + // (see element string name requirements)
> > + [cscfg_file_elem_str] // description of configuration.
> > + [u16 value](nr_presets) // number of defined sets presets values.
> > + [u32 value](nr_total_params) // total parameters defined by all used features.
> > + [u16 value](nr_feat_refs) // number of features referenced by the configuration
> > + [u64 values] * (nr_presets * nr_total_params) // the preset values.
> > + [cscfg_file_elem_str] * (nr_feat_refs) // names of features used in the configurations.
> > +
> > +FEATURE_ELEM element
> > +~~~~~~~~~~~~~~~~~~~~
> > +
> > +::
> > +
> > + [cscfg_file_elem_header] // header length is total bytes to end of param structures.
> > + [cscfg_file_elem_str] // feature name.
> > + [cscfg_file_elem_str] // feature description.
> > + [u32 value](match_flags) // flags to associate the feature with a device.
> > + [u16 value](nr_regs) // number of registers.
> > + [u16 value](nr_params) // number of parameters.
> > + [cscfg_regval_desc struct] * (nr_regs) // register definitions
> > + [PARAM_ELEM] * (nr_params) // parameters definitions
> > +
> > +PARAM_ELEM element
> > +~~~~~~~~~~~~~~~~~~
> > +
> > +::
> > +
> > + [cscfg_file_elem_str] // parameter name.
> > + [u64 value](param_value) // initial value.
> > +
> > +Additional definitions.
> > +~~~~~~~~~~~~~~~~~~~~~~~
>
> Trim trailing period for section names
>
> > +
> > +The following structures are defined in **coresight-config-file.h**
> > +
> > + * **struct cscfg_file_header** : This structure contains an initial magic number, the total
> > + length of the file, and the number of configurations and features in the file.
> > + * **struct cscfg_file_elem_header**: This defines the total length and type of a CONFIG_ELEM
> > + or a FEATURE_ELEM.
> > + * **struct cscfg_file_elem_str**: This defines a string and its length.
>
> Again, for consistency, wrap identifier names in single-quotes.
>
> > +
> > +The magic number in cscfg_file_header is defined as two bitfields::
> > +
> > + [31:8] Fixed magic number to identify file type.
> > + [7:0] Current file format version.
> > +
> > +The following defines determine the maximum overall file size and maximum individual
> > +string size::
> > +
> > + CSCFG_FILE_MAXSIZE // maximum overall file size.
> > + CSCFG_FILE_STR_MAXSIZE // maximum individual string size.
>
> For parameter lists in elements, use bullet lists instead.
>
> > +
> > +Load Dependencies.
> > +~~~~~~~~~~~~~~~~~~
>
> Trim trailing period for section names.
> > +
> > +Files may be unloaded only in the strict reverse order of loading. This is enforced by the
> > +configuration system.
> > +
> > +This is to ensure that any load dependencies are maintained.
> > +
> > +A configuration file that contains a CONFIG_ELEM that references named features "feat_A" and "feat_B" will load only if either:-
> > +a) "feat_A" and/or "feat_B" has been loaded previously, or are present as built-in / module loaded features.
> > +b) "feat_A" and/or "feat_B" are declared as FEAT_ELEM in the same file as the CONFIG_ELEM.
>
> Separate the preceding paragraph and the list with a blank line in order
> for the list to be rendered as list.
>
The next version will contain all the adjustments that you suggested,
though rather than using single quotes, I have changed all the
occurrences to consistently the double back tick to make the filenames
etc fixed width font.
Thanks for the review
Mike
> Thanks.
>
> --
> An old man doll... just what I always wanted! - Clara
--
Mike Leach
Principal Engineer, ARM Ltd.
Manchester Design Centre. UK
I'll rework this in the next set.
Thanks for the review
Mike
On Tue, 27 Dec 2022 at 17:10, Christoph Hellwig <hch(a)infradead.org> wrote:
>
> On Mon, Dec 19, 2022 at 11:46:36PM +0000, Mike Leach wrote:
> > Update coresight-config.h and the coresight-config-file.c & .h
> > to allow use in userspace programs.
> >
> > Use __KERNEL__ defines to filter out driver only structures and
> > elements so that user space programs can use the descriptor structures.
> >
> > Abstract memory allocation in coresight-config-file.c to allow read
> > file functions to be run in userspace and kernel drivers.
>
> That's now how kernel code is written.
--
Mike Leach
Principal Engineer, ARM Ltd.
Manchester Design Centre. UK
Add support for UltraSoc System Memory Buffer.
Change since v15:
- Modify document based on Bagas and Suzuki's comments.
- Link: https://lore.kernel.org/lkml/20221220092945.30722-1-hejunhao3@huawei.com/
Change since v14:
- Add some helpers to the buffer and simplify dumping data according to Jonathan's comment.
- Link: https://lore.kernel.org/lkml/20221123123823.27973-1-hejunhao3@huawei.com/
Change since v13:
- Modify document and the patches description according to Bagas's comment.
- Add dependency on config ACPI, drop redundant enable hw in smb_update_buffer(),
Modify smb_purge_data() description according to Suzuki's comment.
- Link: https://lore.kernel.org/lkml/20221114090316.63157-1-hejunhao3@huawei.com/
Change since v12:
- Modify the code style and add "#ifdef CONFIG_ACPI" according to Jonathan's comment.
- Address the comments from Yicong, included drop "buf_base"__iomem attribute,
modify the "reading" type to bool and fix FIELD_PREP.
- Link: https://lore.kernel.org/lkml/20221109135008.9485-1-hejunhao3@huawei.com/
Change since v11:
- Modify the code style and rename the macro according to Jonathan's comment.
- Link: https://lore.kernel.org/lkml/20221107130624.59886-1-hejunhao3@huawei.com/
Change since v10:
- Rebase onto v6.1-rc4, included similar sysfs register accessors (as same as James's patch)
- Link: https://lore.kernel.org/lkml/20221022115929.7503-1-hejunhao3@huawei.com/
Change since v9:
- Update the Contact tag in SMB document.
- Replace the spinlock with mutex.
- Do some clean-ups in "smb_enable()" and "smb_release()".
- Use classic memory mapped interface.
- Link: https://lore.kernel.org/linux-arm-kernel/20220818132231.28240-1-hejunhao3@h…
Change since v8:
- Insert a blank line at the end of the config tag in Kconfig according to Randy's comment.
- Link: https://lore.kernel.org/linux-arm-kernel/20220816131634.38195-1-hejunhao3@h…
Change since v7:
- Use the macros for register bit flags and numbers of resource.
- Cleanup punctuation.
- Update the Date tag and the KernelVersion tag in the document.
- Link: https://lore.kernel.org/lkml/20220712091353.34540-1-hejunhao3@huawei.com/
Change since v6:
- Modify the code style and driver description according to Suzuki's comment.
- Modify configuration of "drvdata->reading", to void problems in open/read
concurrency scenario.
- Rename the macro of "SMB_FLOW_MASK".
- Use the "handle->head" to determine the page number and offset.
- Link: https://lore.kernel.org/linux-arm-kernel/20220606130223.57354-1-liuqi115@hu…
Change since v5:
- Address the comments from Suzuki, add some comments in SMB document, and modify
configuration of "drvdata->reading", to void problems in multi-core concurrency scenario
- Link: https://lore.kernel.org/linux-arm-kernel/20220416083953.52610-1-liuqi115@hu…
Change since v4:
- Add a simple document of SMB driver according to Suzuki's comment.
- Address the comments from Suzuki.
- Link: https://lore.kernel.org/linux-arm-kernel/20220128061755.31909-1-liuqi115@hu…
Change since v3:
- Modify the file header according to community specifications.
- Address the comments from Mathieu.
- Link: https://lore.kernel.org/linux-arm-kernel/20211118110016.40398-1-liuqi115@hu…
Change since v2:
- Move ultrasoc driver to drivers/hwtracing/coresight by Mathieu's comment.
- Link: https://lists.linaro.org/pipermail/coresight/2021-November/007310.html
Change since v1:
- Drop the document of UltraSoc according to Mathieu's comment.
- Add comments to explain some private hardware settings.
- Address the comments from Mathieu.
- Link: https://lists.linaro.org/pipermail/coresight/2021-August/006842.html
Change since RFC:
- Move driver to drivers/hwtracing/coresight/ultrasoc.
- Remove ultrasoc-axi-com.c, as AXI-COM doesn't need to be configured in
basic tracing function.
- Remove ultrasoc.c as SMB does not need to register with the ultrasoc core.
- Address the comments from Mathieu and Suzuki.
- Link: https://lists.linaro.org/pipermail/coresight/2021-June/006535.html
Qi Liu (2):
drivers/coresight: Add UltraSoc System Memory Buffer driver
Documentation: Add document for UltraSoc SMB driver
.../sysfs-bus-coresight-devices-ultra_smb | 31 +
.../trace/coresight/ultrasoc-smb.rst | 83 +++
drivers/hwtracing/coresight/Kconfig | 12 +
drivers/hwtracing/coresight/Makefile | 1 +
drivers/hwtracing/coresight/ultrasoc-smb.c | 648 ++++++++++++++++++
drivers/hwtracing/coresight/ultrasoc-smb.h | 125 ++++
6 files changed, 900 insertions(+)
create mode 100644 Documentation/ABI/testing/sysfs-bus-coresight-devices-ultra_smb
create mode 100644 Documentation/trace/coresight/ultrasoc-smb.rst
create mode 100644 drivers/hwtracing/coresight/ultrasoc-smb.c
create mode 100644 drivers/hwtracing/coresight/ultrasoc-smb.h
--
2.33.0
Changes since v2:
* Reword first commit message and add fixes tag
* Pickup Jinlong's tested-by tags
----
This should be a slight improvement on Jinlong's previous version.
Now it's not possible to trigger the error message from
pm_runtime_put() by calling disable twice.
It's also similar to the original pre-breaking change version where
pm_runtime_put() was only called if the device was actually disabled,
but with one difference: Previously pm_runtime_put() was only called
once for the last disable call, but because of the reference counting
in pm_runtime, it should have been called once for each enable call.
This meant that the clock would have never been disabled if there were
ever multiple enable calls. This is now fixed.
The third commit is a refactor and doesn't need to be backported. I
removed one of the atomic types because it didn't appear to be
required. Maybe it was added for a reason which I'm not aware of, if
so it should be pretty easy to drop that change.
James Clark (2):
coresight: cti: Prevent negative values of enable count
coresight: cti: Remove atomic type from enable_req_count
Mao Jinlong (1):
coresight: cti: Add PM runtime call in enable_store
.../hwtracing/coresight/coresight-cti-core.c | 23 ++++++++++++-------
.../hwtracing/coresight/coresight-cti-sysfs.c | 15 +++++++++---
drivers/hwtracing/coresight/coresight-cti.h | 2 +-
3 files changed, 28 insertions(+), 12 deletions(-)
base-commit: 88603b6dc419445847923fcb7fe5080067a30f98
--
2.25.1
This should be a slight improvement on Jinlong's previous version.
Now it's not possible to trigger the error message from
pm_runtime_put() by calling disable twice.
It's also similar to the original pre-breaking change version where
pm_runtime_put() was only called if the device was actually disabled,
but with one difference: Previously pm_runtime_put() was only called
once for the last disable call, but because of the reference counting
in pm_runtime, it should have been called once for each enable call.
This meant that the clock would have never been disabled if there were
ever multiple enable calls. This is now fixed.
The third commit is a refactor and doesn't need to be backported. I
removed one of the atomic types because it didn't appear to be
required. Maybe it was added for a reason which I'm not aware of, if
so it should be pretty easy to drop that change.
James Clark (2):
coresight: cti: Prevent negative values of enable count
coresight: cti: Remove atomic type from enable_req_count
Mao Jinlong (1):
coresight: cti: Add PM runtime call in enable_store
.../hwtracing/coresight/coresight-cti-core.c | 23 ++++++++++++-------
.../hwtracing/coresight/coresight-cti-sysfs.c | 15 +++++++++---
drivers/hwtracing/coresight/coresight-cti.h | 2 +-
3 files changed, 28 insertions(+), 12 deletions(-)
base-commit: c767c34740132ffc478226864a7461493cdc2413
--
2.25.1