Dear EBBR stakeholders,
I hope you all had a nice summer break.
Our next EBBR call is approaching as it will take place next Monday, Aug 26
at 14h30 BST (13h30 UTC; BST is active).
In the mean time, I would like to remind you that we have a poll[1]
on-going, to try to find an even better schedule for our EBBR call.
Thank you to those who added their availabilities already; for those of
you, who have not yet completed this first poll, could you please do so?
At the time of writing, we have a couple of pull requests on the agenda[2]:
- #131: Update UEFI version to 2.10 A
- #132: Boot Manager requirements
This an attempt at fixing issue #130: Explicitly require boot manager,
from Heinrich
Please have a look, and feel free to add to the agenda, directly on the
wiki page or by e-mail.
Best regards,
Vincent Stehlé
System Architect - Arm
[1] https://framadate.org/6jm8P3jHqAzmv8Xo
[2] https://github.com/ARM-software/ebbr/wiki/EBBR-Meetings
[130] https://github.com/ARM-software/ebbr/issues/130
[131] https://github.com/ARM-software/ebbr/pull/131
[132] https://github.com/ARM-software/ebbr/pull/132
>> Could this section be updated in order to reduce the number of different
>> interpretations?
>
> Yes, it can be clarified. As Daniel says, this section is only
> intended for platforms where there is no other place to store
> firmware. If the OS and the firmware need to share a logical block
> device, and there is no way to protect the firmware bits, then this
> section describes how they can co-exist. e.g., you don't want an OS
> deployment to accidentally wipe out firmware.
>
> It is preferred and encouraged to have firmware contained entirely in
> something separate from the main block storage. For example, in eMMC
> boot areas (separate from the primary area), or on a separate device
> entirely (SPI flash). I predict that a future version of EBBR will
> require this and drop the shared storage option entirely because it is
> necessary to protect against attacks against firmware (replacement or
> deletion)
>
> g.
>
Thank you all contributing to this thread!
While most of the points in this thread are self-explanatory and should
not need a summary, I want to extract the main points to avoid
misinterpretations:
1. Chapters '4.1 Partitioning of Shared Storage' and '4.2 Firmware
Partition Filesystem' refer to cases when shared storage is used for OS
and firmware images. In this case, the block device needs to be
partitioned using GPT and the firmware binaries should be stored in a
FAT partition. The folder organization for this partition is described
in the '4.2.1 The firmware directory hierarchy' chapter. Ideally, the
BootROM should be able to load images from a FAT partition. However, a
feature release of the EBBR is likely to drop this option in the future.
2. For scenarios involving dedicated storage, the organization of
the underlying storage for firmware falls outside the scope of the EBBR
and can be arranged according to the SoC vendor's discretion. This can
include options such as offset addressing, without GPT, with MRB, with
or without a partition, based on the platform's requirements or limitations.
Could you please confirm that this understanding is correct?
> If I recall correctly, on the Arm platform, A-BL1 (BootROM) and A-BL2
> are responsible only for
> loading the images. SCP_BL2 is loaded by BL2 into trusted SRAM and then
> transferred to the SCP using the MHU protocol and I believe it is
> followed this
> way for other platforms also (?).
> In summary, A-BL1(BootROM) and A-BL2 need to be aware of their next
> stages to load those
> images properly:
> BootROM -> BL2 -> BL31 -> ...
> |
> v
> SCP
If the above summary points are valid, then in the case of a shared
device, the BL2 should be able to load the next image(s) (either FIP or
BL31) from a FAT partition, I suppose.
Regards,
Ghennadi
Greetings everyone,
I came across the following paragraph while reading the 'Firmware Partition Filesystem' chapter from EBBR v2.2.0
and I would like to clarify my understanding:
Where possible, firmware images and data should be stored in a filesystem. Firmware can be stored either in a
dedicated firmware partition, or in certain circumstances in the UEFI System Partition (ESP). Using a filesystem
makes it simpler to manage multiple firmware files and makes it possible for a single disk image to contain firmware
for multiple platforms.
Dedicated firmware partitions should be formatted with a FAT filesystem as defined in UEFI § 13.3 File System
Format. Dedicated firmware partitions should use the same /FIRMWARE directory hierarchy. OS tools shall ignore
dedicated firmware partitions, and shall not attempt to use a dedicated firmware partition as an ESP
Questions:
1. Does the above paragraph mean that, if the device allows, all firmware binaries (TF-A, U-Boot, and some others)
must be stored in a dedicated firmware partition formatted with a FAT filesystem and GUID Partition Table (GPT)
disk layout?
2. If so, would this also mean that, where possible, the BootROM or a (first stage) bootloader running before TF-A
should have GPT and FAT support embedded support to load one of the TF-A stages (BL1/BL2)?
Regards,
Ghennadi
Hi,
We have nothing on today's EBBR call's agenda [1]; I would thus be inclined to
cancel it.
If you have an urgent topic to discuss today, please let us know before 1pm UTC,
otherwise we will adjourn the call.
Best regards,
Vincent Stehlé
System Architect - Arm
[1]: https://github.com/ARM-software/ebbr/wiki/EBBR-Meetings
Hi,
The question of moving the EBBR call 1/2 hour earlier was raised during our last
call [1].
Some people have colliding meetings now and it would make their lives easier.
Also, this seemed practical for all the people present.
Could all the EBBR attendees please tick their available hours on the following
poll?
https://framadate.org/FWI2Ff4Tmyz9yOfR
Thank you in advance!
If possible we will move the EBBR call to 14:30 UTC/BST starting with our next
call on Jul 15.
Best regards,
Vincent Stehlé
System Architect - Arm
[1] https://github.com/ARM-software/ebbr/wiki/EBBR-Notes-2024.07.01
Device manufacturers frequently ship multiple boards or SKUs under a
single software package. These software packages will ship multiple
devicetree blobs and require some mechanism to pick the correct DTB for
the board the software package was deployed. Introduce a common
definition for adding board identifiers to device trees. board-id
provides a mechanism for bootloaders to select the appropriate DTB which
is vendor/OEM-agnostic.
This series is based off a talk I gave at EOSS NA 2024 [1]. There is
some further discussion about how to do devicetree selection in the
boot-architecture mailing list [2].
[1]: https://sched.co/1aBFy
[2]: https://lists.linaro.org/archives/list/boot-architecture@lists.linaro.org/t…
Quick summary
-------------
This series introduces a new subnode in the root:
/ {
board-id {
some-hw-id = <value>;
other-hw-id = <val1>, <val2>;
};
};
Firmware provides a mechanism to fetch the values of "some-hw-id" and
"other-hw-id" based on the property name. I'd like to leave exact
mechanism data out of the scope of this proposal to keep this proposal
flexible because it seems architecture specific, although I think we we
should discuss possible approaches. A DTB matches if firmware can
provide a matching value for every one of the properties under
/board-id. In the above example, val1 and val2 are both valid values and
firmware only provides the one that actually describes the board.
It's expected that devicetree's board-id don't describe all the
properties firmware could provide. For instance, a devicetree overlay
may only care about "other-hw-id" and not "some-hw-id". Thus, it need
only mention "other-hw-id" in its board-id node.
Isn't that what the compatible property is for?
-----------------------------------------------
The compatible property can be used for board matching, but requires
bootloaders and/or firmware to maintain a database of possible strings
to match against or implement complex compatible string matching.
Compatible string matching becomes complicated when there are multiple
versions of board: the device tree selector should recognize a DTB that
cares to distinguish between v1/v2 and a DTB that doesn't make the
distinction. An eeprom either needs to store the compatible strings
that could match against the board or the bootloader needs to have
vendor-specific decoding logic for the compatible string. Neither
increasing eeprom storage nor adding vendor-specific decoding logic is
desirable.
How is this better than Qualcomm's qcom,msm-id/qcom,board-id?
-------------------------------------------------------------
The selection process for devicetrees was Qualcomm-specific and not
useful for other devices and bootloaders that were not developed by
Qualcomm because a complex algorithm was used to implement. Board-ids
provide a matching solution that can be implemented by bootloaders
without introducing vendor-specific code. Qualcomm uses three
devicetree properties: msm-id (interchangeably: soc-id), board-id, and
pmic-id. This does not scale well for use casese which use identifiers,
for example, to distinguish between a display panel. For a display
panel, an approach could be to add a new property: display-id, but now
bootloaders need to be updated to also read this property. We want to
avoid requiring to update bootloaders with new hardware identifiers: a
bootloader need only recognize the identifiers it can handle.
Notes about the patches
-----------------------
In my opinion, most of the patches in this series should be submitted to
libfdt and/or dtschema project. I've made them apply on the kernel tree
to be easier for other folks to pick them up and play with them. As the
patches evolve, I can send them to the appropriate projects.
Signed-off-by: Elliot Berman <quic_eberman(a)quicinc.com>
---
Changes in v3:
- Follow new "/board-id {}" approach, which uses key-value pairs
- Add match algorithm in libfdt and some examples to demo how the
selection could work in tools/board-id
Changes in V2:
- Addressed few comments related to board-id, and DDR type.
- Link to V2: https://lore.kernel.org/all/a930a3d6-0846-a709-8fe9-44335fec92ca@quicinc.co…
---
Amrit Anand (1):
dt-bindings: arm: qcom: Update Devicetree identifiers
Elliot Berman (8):
libfdt: board-id: Implement board-id scoring
dt-bindings: board: Introduce board-id
fdt-select-board: Add test tool for selecting dtbs based on board-id
dt-bindings: board: Document board-ids for Qualcomm devices
arm64: boot: dts: sm8650: Add board-id
arm64: boot: dts: qcom: Use phandles for thermal_zones
arm64: boot: dts: qcom: sm8550: Split into overlays
tools: board-id: Add test suite
.../devicetree/bindings/board/board-id.yaml | 24 ++++
.../devicetree/bindings/board/qcom,board-id.yaml | 144 ++++++++++++++++++++
arch/arm64/boot/dts/qcom/Makefile | 4 +
arch/arm64/boot/dts/qcom/pm8010.dtsi | 62 ++++-----
arch/arm64/boot/dts/qcom/pm8550.dtsi | 32 ++---
arch/arm64/boot/dts/qcom/pm8550b.dtsi | 36 +++--
arch/arm64/boot/dts/qcom/pm8550ve.dtsi | 38 +++---
arch/arm64/boot/dts/qcom/pm8550vs.dtsi | 128 +++++++++--------
arch/arm64/boot/dts/qcom/pmr735d_a.dtsi | 38 +++---
arch/arm64/boot/dts/qcom/pmr735d_b.dtsi | 38 +++---
.../dts/qcom/{sm8550-mtp.dts => sm8550-mtp.dtso} | 24 +++-
.../dts/qcom/{sm8550-qrd.dts => sm8550-qrd.dtso} | 22 ++-
.../boot/dts/qcom/{sm8550.dtsi => sm8550.dts} | 10 +-
arch/arm64/boot/dts/qcom/sm8650-mtp.dts | 6 +
arch/arm64/boot/dts/qcom/sm8650-qrd.dts | 6 +
arch/arm64/boot/dts/qcom/sm8650.dtsi | 2 +-
include/dt-bindings/arm/qcom,ids.h | 86 ++++++++++--
scripts/dtc/.gitignore | 1 +
scripts/dtc/Makefile | 3 +-
scripts/dtc/fdt-select-board.c | 126 +++++++++++++++++
scripts/dtc/libfdt/fdt_ro.c | 76 +++++++++++
scripts/dtc/libfdt/libfdt.h | 54 ++++++++
tools/board-id/test.py | 151 +++++++++++++++++++++
23 files changed, 901 insertions(+), 210 deletions(-)
---
base-commit: e8f897f4afef0031fe618a8e94127a0934896aba
change-id: 20240112-board-ids-809ff0281ee5
Best regards,
--
Elliot Berman <quic_eberman(a)quicinc.com>
Hi all. Several EBBR meetings ago, I introduced the need for allowing OS provided device trees [1]. Please find below the proposal I am delinquent on sending.
Hopefully, we can discuss this in the next meeting.
Thanks
Jon
[1] https://github.com/ARM-software/ebbr/wiki/EBBR-Notes-2024.02.12
Problem statement:
==================
Device trees are in theory a pure description of the hardware, and since the hardware
doesn't change, the device tree describing the hardware likewise never changes.
With this, a device tree could then be burned into the hardware's ROM to be
queried by software for hardware discovery. In practice, though, device trees
evolve over time. They evolve for many reasons, including
- support for previously unsupported hardware
- device driver improvements that require additional hardware information
- bug fixes
Linux's device tree source is maintained with the kernel source, and kernel builds
include building the device trees too. This ensures that the device tree
matching the kernel's usage is always kept in sync. Often, embedded distros will
include the matching device tree blobs.
The EBBR mandates that the device tree blob is provided by the firmware.
Thus it is likely that the device tree provided by the firmware and given to the
operating system is not the matching device tree blob for that kernel. This can
cause hardware to be missing, buggy, or non-functional.
Proposal:
=========
A key goal of the EBBR is to define the contract between the firmware and the OS
so that the OS doesn't need to be built specifically for the hardware, and the
firmware can boot any compliant OS. Thus, any solution that requires the OS to
know specifics about the hardware beyond the EBBR contract would violate the
EBBR goals. This precludes any solution where the OS, having the matching DTBs,
would pick the DTB, because this requires the OS to know what hardware it is
being run on. Likewise, any solution where the firmware is aware of the OS
matching DTBs would require the firmware to be aware of the particular OS it is
booting.
What can be known:
- The firmware knows what board it is running on, and thus knows what device
tree to use. But it doesn't know what version of the device tree to use,
because it doesn't know what OS is being booted.
- The OS knows what version of DTBs matches it's kernel, but does not know which
specific device tree to use.
This proposal then has the firmware choose the device tree by name, or some
other identifier that can be used to match the device tree for the board [1]. It
has the OS-provided OS loader select the location of the matching versions of
DTBs for it.
The firmware would pass the device tree filename/id to the OS loader, instead of
the DTB itself. The OS loader would determine the location of the matching DTBs
based on the chosen OS to boot, load the matching DTB from that location, and
pass to the kernel.
Considerations:
- often a DTB requires fixups. The EFI_DT_FIXUP_PROTOCOL could be utilized.
- device tree overlays could be indicated with a scheme using the device tree ID
passed to the OS loader
- authenticating the DTB would be the responsibility of the OS distribution and
handled in the same way as the kernel itself is authenticated. The OS is the
entity responsible for signing the DTB, as it should be.
This proposal should be in addition to supporting the standard way of passing in
a firmware-provided DT, in cases where the OS doesn't provide or have a need to
provide a matching DT.
[1] Rather than using the device tree source filename, to have more flexibility,
one can conceive an ID or compatible string that the OS could then scan the DTBs
to find a match.
Hi,
I am happy to announce that v2.2.0 of the EBBR specification has now been
released. [1]
The following people (listed in alphabetical order) have participated to this
EBBR release directly or indirectly, with patches, github issues, pull requests,
reviews, comments, suggestions, or by attending the zoom calls:
Ilias Apalodimas
Joakim Bech
Ard Biesheuvel
Etienne Carrière
Wei Chen
Patrick Delaunay
Samuel Dionne-Riel
Samer El-Haj-Mahmoud
Andreas Färber
Ahmad Fatoum
Sughosh Ganu
Simon Glass
Alexander Graf
Rob Herring
Jon Humphreys
Jan Kiszka
Jan Lübbe
Jose Marinho
Loïc Minier
Weizhao Ouyang
François-Frédéric Ozog
Stephen Paulger
Tom Rini
Peter Robinson
Heinrich Schuchardt
Michal Simek
Vincent Stehlé
Daniel Thompson
Ivan T. Ivanov
Stuart Yoder
A big thank you to all the EBBR contributors!
Best regards,
Vincent Stehlé
System Architect - Arm
[1]: https://github.com/ARM-software/ebbr/releases/tag/v2.2.0
On Sat, May 25, 2024 at 05:54:52PM +0100, Conor Dooley wrote:
> On Wed, May 22, 2024 at 04:54:23PM -0700, Elliot Berman wrote:
> > Device manufcturers frequently ship multiple boards or SKUs under a
> > single softwre package. These software packages ship multiple devicetree
> > blobs and require some mechanims to pick the correct DTB for the boards
> > that use the software package. This patch introduces a common language
> > for adding board identifiers to devicetrees.
> >
> > Signed-off-by: Elliot Berman <quic_eberman(a)quicinc.com>
> > ---
> > .../devicetree/bindings/board/board-id.yaml | 71 ++++++++++++++++++++++
> > 1 file changed, 71 insertions(+)
> >
> > diff --git a/Documentation/devicetree/bindings/board/board-id.yaml b/Documentation/devicetree/bindings/board/board-id.yaml
> > new file mode 100644
> > index 000000000000..894c1e310cbd
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/board/board-id.yaml
> > @@ -0,0 +1,71 @@
> > +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
> > +%YAML 1.2
> > +---
> > +$id: http://devicetree.org/schemas/board/board-id.yaml#
> > +$schema: http://devicetree.org/meta-schemas/core.yaml#
> > +
> > +title: Board identifiers
> > +description: |
> > + This node contains a list of identifier values for the board(s) supported by
> > + this devicetree. Identifier values are either N-tuples of integers or a
> > + string. The number of items for an N-tuple identifer is determined by the
> > + property name. String identifiers must be suffixed with "-string".
> > +
> > + Every identifier in the devicetree must have a matching value from the board
> > + to be considered a valid devicetree for the board. In other words: if
> > + multiple identifiers are present in the board-id and one identifier doesn't
> > + match against the board, then the devicetree is not applicable. Note this is
> > + not the case where the the board can provide more identifiers than the
> > + devicetree describes: those additional identifers can be ignored.
> > +
> > + Identifiers in the devicetree can describe multiple possible valid values,
> > + such as revision 1 and revision 2.
> > +
> > +maintainers:
> > + - Elliot Berman <quic_eberman(a)quicinc.com>
> > +
> > +properties:
> > + $nodename:
> > + const: '/'
> > + board-id:
>
>
> Does this need to be
> properties:
> $nodename:
> const: board-id
> ? That's the pattern I see for all top level nodes.
>
> > + type: object
> > + patternProperties:
> > + "^.*(?<!-string)$":
>
> At least this regex now actually works :)
>
> > + $ref: /schemas/types.yaml#/definitions/uint32-matrix
> > + description: |
> > + List of values that match boards this devicetree applies to.
> > + A bootloader checks whether any of the values in this list
> > + match against the board's value.
> > +
> > + The number of items per tuple is determined by the property name,
> > + see the vendor-specific board-id bindings.
> > + "^.*-string$":
> > + $ref: /schemas/types.yaml#/definitions/string-array
>
> Your description above doesn't match a string-array, just a single
> string. That said I'm far from sold on the "thou shalt have -string"
> edict. If every vendor is expected to go and define their own set of
> properties (and provide their own callback in your libfdt PoC) there's
> little to no reason to inflict property naming on them, AFAICT all that
> is gained is a being able to share
> if (string) {
> return fdt_stringlist_contains(prop->data,
> fdt32_to_cpu(prop->len),
> data);
> } else {
> // exact data comparison. data_len is the size of each entry
> if (fdt32_to_cpu(prop->len) % data_len || data_len % 4)
> return -FDT_ERR_BADVALUE;
>
> for (int i = 0; i < fdt32_to_cpu(prop->len); i += data_len) {
> if (!memcmp(&prop->data[i], data, data_len))
> return 1;
> }
>
> return 0;
> }
> in the libfdt PoC? I'd be expecting that a common mechanism would use
> the same "callback" for boards shipped by both Qualcomm and
> $other_vendor. Every vendor having different properties and only sharing
> the board-id node name seems a wee bit like paying lip-service to a
> common mechanism to me. What am I missing?
One way I thought to get the real board-id values from firmware to OS
loader is via DT itself. A firmware-provided DT provides the real
board-id values. In this case, firmware doesn't have any way to say the
board-id property is a string or a number, so I put that info in the DT
property name.
Another way I thought to get the real board-id values from firmware is
via a UEFI protocol. In that case, we could easily share whether the
value is a string or number and we can drop the "-string" suffix bit.
Thanks,
Elliot
On Sat, May 25, 2024 at 06:21:32PM +0100, Conor Dooley wrote:
> On Tue, May 21, 2024 at 11:38:01AM -0700, Elliot Berman wrote:
> > #define QCOM_BOARD_ID(a, major, minor) \
> > - (((major & 0xff) << 16) | ((minor & 0xff) << 8) | QCOM_BOARD_ID_##a)
> > + (((major & 0xff) << 16) | ((minor & 0xff) << 8) | ((QCOM_BOARD_ID_##a) & 0xff))
>
> I assume there's no devices that have a >8 bit QCOM_BOARD_ID that would
> end up with a different value in their dtb due to this change?
That's correct.
On Mon, May 27, 2024 at 09:19:59AM +0200, Michal Simek wrote:
> Hi,
>
> thanks for CCing me.
>
> On 5/24/24 17:51, Konrad Dybcio wrote:
> > On 21.05.2024 9:00 PM, Dmitry Baryshkov wrote:
> > > Hi Elliot,
> > >
> > > On Tue, 21 May 2024 at 21:41, Elliot Berman <quic_eberman(a)quicinc.com> wrote:
> > > >
> > > > Device manufacturers frequently ship multiple boards or SKUs under a
> > > > single software package. These software packages will ship multiple
> > > > devicetree blobs and require some mechanism to pick the correct DTB for
> > > > the board the software package was deployed. Introduce a common
> > > > definition for adding board identifiers to device trees. board-id
> > > > provides a mechanism for bootloaders to select the appropriate DTB which
> > > > is vendor/OEM-agnostic.
> > >
> > > This is a v3 of the RFC, however it is still a qcom-only series. Might
> > > I suggest gaining an actual interest from any other hardware vendor
> > > (in the form of the patches) before posting v4? Otherwise it might
> > > still end up being a Qualcomm solution which is not supported and/or
> > > used by other hardware vendors.
> >
> > AMD should be onboard [1].
> >
> > Konrad
> >
> > [1] https://resources.linaro.org/en/resource/q7U3Rr7m3ZbZmXzYK7A9u3
>
> I am trying to wrap my head around this and I have also looked at that EOSS
> presentation.
> I don't think I fully understand your case.
> There are multiple components which you need to detect. SOC - I expect
> reading by some regs, board - I expect you have any eeprom, OTP, adc, gpio,
> etc way how to detect board ID and revision.
> And then you mentioned displays - how do you detect them?
We have a similar mechanism to what you mention below: we have a ROM
which encodes information about the platform and that can be read by
firmware/bootloader/OS.
>
> In our Kria platform we have eeproms on SOM and CC cards (or FMC/extension
> cards) which we read and decode and based on information from it we are
> composing "unique" string. And then we are having DTBs in FIT image where
> description of configuration it taken as regular expression. That's why it
> is up to you how you want to combine them.
I don't think this is a fundamentally different approach from my
proposal. Instead of composing a "unique" string and using regex to
match, I'm proposing that the information (bytes) that are in your
eeprom can be matched without going through regex/string conversion.
Instead of firmware/bootloader doing a conversion to the strings, it
provides the values via board-id. I have concerns about having
bootloaders to contain a regex library -- probably easily addressed by
standardizing what terms the regex processor needs to support. I'm also
not sure if regex strings are an appropriate use of compatible strings.
Using strings limits the usefulness of comaptible strings to the
consumers of DTB, since the compatible string has to describe only the
boards the DTB is applicable to, you can't mention compatible strings
"this board is like" such as some generic SoC compatible.
> Currently we are merging them offline and we are not applying any DT overlay
> at run time but can be done (we are missing one missing piece in U-Boot for
> it).
>
> In presentation you mentioned also that applying overlay can fail but not
> sure how you can reach it. Because Linux kernel has the whole infrastructure
> to cover all combinations with base DT + overlays. It means if you cover all
> working combinations there you should see if they don't apply properly.
Mostly, I was referring to a situation where firmware provides an
overlay. Firmware doesn't know the DTB that OS has and I don't see any
way to gaurantee that firmware knows how to fix up the OS DTB.
>
> Also do you really need to detect everything from firmware side? Or isn't it
> enough to have just "some" devices and then load the rest where you are in
> OS?
> I think that's pretty much another way to go to have bare minimum
> functionality provided by firmware and deal with the rest in OS.
Agreed, although not all devices can be loaded once you are in the OS.
All nondiscoverable devices would need to be desribed in the DT.
Thanks,
Elliot
On Sat, May 25, 2024 at 06:08:46PM +0100, Conor Dooley wrote:
> On Tue, May 21, 2024 at 11:38:02AM -0700, Elliot Berman wrote:
> > +
> > +allOf:
> > + # either describe soc or soc-version; it's confusing to have both
>
> Why not just use the one that has the most information and discard the
> others? If your dtb picker for this platform doesn't care about the soc
> version, then just don't look at that cell?
The dtb picker for the platform doesn't know whether to care about the
SoC version/platform version/whatever. That's a property of the DTB
itself and I don't think it makes much sense to bake that into the DTB
picker which would otherwise be unaware of this.
>
> Likewise for platform and PMIC, why can't you ignore the cells you don't
> care about, rather than having a new property for each variant? Nothing
> in this patch explains why multiple variants are required rather than
> just dealing with the most informational.
>
Sure, I will explain in future revision.
- Elliot
Hi Conor,
Thanks for taking the time to look at the patch.
On Tue, May 21, 2024 at 08:21:45PM +0100, Conor Dooley wrote:
> On Tue, May 21, 2024 at 11:37:59AM -0700, Elliot Berman wrote:
> > Device manufcturers frequently ship multiple boards or SKUs under a
> > single softwre package. These software packages ship multiple devicetree
> > blobs and require some mechanims to pick the correct DTB for the boards
> > that use the software package.
>
> Okay, you've got the problem statement here, nice.
>
> > This patch introduces a common language
> > for adding board identifiers to devicetrees.
>
> But then a completely useless remainder of the commit message.
> I open this patch, see the regexes, say "wtf", look at the commit
> message and there is absolutely no explanation of what these properties
> are for. That's quite frankly just not good enough - even for an RFC.
>
Understood, I've been trying to walk the line of getting the idea across
to have conversation about the board-ids, while not getting into too
much of the weeds. I was hoping the example and the matching code in the
first patch would get enough of the idea across, but I totally
empathize that might not be enough. I'll reply here shortly with a
version of this patch which adds more details.
> >
> > Signed-off-by: Elliot Berman <quic_eberman(a)quicinc.com>
> > ---
> > .../devicetree/bindings/board/board-id.yaml | 24 ++++++++++++++++++++++
> > 1 file changed, 24 insertions(+)
> >
> > diff --git a/Documentation/devicetree/bindings/board/board-id.yaml b/Documentation/devicetree/bindings/board/board-id.yaml
> > new file mode 100644
> > index 000000000000..99514aef9718
> > --- /dev/null
> > +++ b/Documentation/devicetree/bindings/board/board-id.yaml
> > @@ -0,0 +1,24 @@
> > +# SPDX-License-Identifier: BSD-2-Clause
> > +%YAML 1.2
> > +---
> > +$id: http://devicetree.org/schemas/board/board-id.yaml#
> > +$schema: http://devicetree.org/meta-schemas/core.yaml#
> > +
> > +title: board identifiers
> > +description: Common property for board-id subnode
>
> s/property/properties/
>
> > +
> > +maintainers:
> > + - Elliot Berman <quic_eberman(a)quicinc.com>
> > +
> > +properties:
> > + $nodename:
> > + const: '/'
> > + board-id:
> > + type: object
> > + patternProperties:
> > + "^.*(?!_str)$":
>
> Does this regex even work? Take "foo_str" as an example - doesn't "^.*"
> consume all of the string, leaving the negative lookahead with nothing
> to object to? I didn't properly test this with an example and the dt
> tooling, but I lazily threw it into regex101 and both the python and
> emcascript versions agree with me. Did you test this?
Right, it should be a lookbehind, not a lookahead.
>
> And while I am here, no underscores in property names please. And if
> "str" means string, I suggest not saving 3 characters.
>
> > + $ref: /schemas/types.yaml#/definitions/uint32-matrix
> > + "^.*_str$":
> > + $ref: /schemas/types.yaml#/definitions/string-array
>
> Why do we even need two methods? Commit message tells me nothing and
> there's no description at all... Why do we need regexes here, rather
> than explicitly defined properties? Your commit message should explain
> the justification for that and the property descriptions (as comments if
> needs be for patternProperties) should explain why this is intended to
> be used.
>
> How is anyone supposed to look at this binding and understand how it
> should be used?
I was thinking that firmware may only provide the data without being
able to provide the context whether the value is a number or a string.
I think this is posisble if firmware provides the device's board
identifier in the format of a DT itself. It seems natural to me in the
EBBR flow. There is example of this in example in patches 3
(fdt-select-board) and 9 (the test suite). DTB doesn't inherently
provide instruction on how to interpret a property's value, so I created
a rule that strings have to be suffixed with "-string".
One other note -- I (QCOM) don't currently have a need for board-ids to
be strings. I thought it was likely that someone might want that though.
Thanks,
Elliot
On Tue, May 21, 2024 at 08:28:01PM +0100, Conor Dooley wrote:
> On Tue, May 21, 2024 at 11:37:58AM -0700, Elliot Berman wrote:
> > The devicetree spec introduced a mechanism to match devicetree blobs to
> > boards using firmware-provided identifiers.
>
> Can you share a link to where the devicetree spec introduced this
> mechanism? I don't recall seeing a PR to dt-schema for it nor did a
> quick check of the devicetree specification repo show a PR adding it.
>
> What am I missing?
My thinking is that the next patch would probably go to dt-schema or
devicetree specification repo.
Thanks,
Elliot
On Tue, May 21, 2024 at 2:25 PM Conor Dooley <conor(a)kernel.org> wrote:
>
> On Tue, May 21, 2024 at 08:21:45PM +0100, Conor Dooley wrote:
> > On Tue, May 21, 2024 at 11:37:59AM -0700, Elliot Berman wrote:
> > > Device manufcturers frequently ship multiple boards or SKUs under a
> > > single softwre package. These software packages ship multiple devicetree
> > > blobs and require some mechanims to pick the correct DTB for the boards
> > > that use the software package.
> >
> > Okay, you've got the problem statement here, nice.
> >
> > > This patch introduces a common language
> > > for adding board identifiers to devicetrees.
> >
> > But then a completely useless remainder of the commit message.
> > I open this patch, see the regexes, say "wtf", look at the commit
> > message and there is absolutely no explanation of what these properties
> > are for. That's quite frankly just not good enough - even for an RFC.
> >
> > >
> > > Signed-off-by: Elliot Berman <quic_eberman(a)quicinc.com>
> > > ---
> > > .../devicetree/bindings/board/board-id.yaml | 24 ++++++++++++++++++++++
> > > 1 file changed, 24 insertions(+)
> > >
> > > diff --git a/Documentation/devicetree/bindings/board/board-id.yaml b/Documentation/devicetree/bindings/board/board-id.yaml
> > > new file mode 100644
> > > index 000000000000..99514aef9718
> > > --- /dev/null
> > > +++ b/Documentation/devicetree/bindings/board/board-id.yaml
> > > @@ -0,0 +1,24 @@
> > > +# SPDX-License-Identifier: BSD-2-Clause
> > > +%YAML 1.2
> > > +---
> > > +$id: http://devicetree.org/schemas/board/board-id.yaml#
> > > +$schema: http://devicetree.org/meta-schemas/core.yaml#
> > > +
> > > +title: board identifiers
> > > +description: Common property for board-id subnode
> >
> > s/property/properties/
> >
> > > +
> > > +maintainers:
> > > + - Elliot Berman <quic_eberman(a)quicinc.com>
> > > +
> > > +properties:
> > > + $nodename:
> > > + const: '/'
> > > + board-id:
> > > + type: object
> > > + patternProperties:
> > > + "^.*(?!_str)$":
> >
> > Does this regex even work? Take "foo_str" as an example - doesn't "^.*"
> > consume all of the string, leaving the negative lookahead with nothing
> > to object to? I didn't properly test this with an example and the dt
> > tooling, but I lazily threw it into regex101 and both the python and
> > emcascript versions agree with me. Did you test this?
> >
> > And while I am here, no underscores in property names please. And if
> > "str" means string, I suggest not saving 3 characters.
> >
> > > + $ref: /schemas/types.yaml#/definitions/uint32-matrix
> > > + "^.*_str$":
> > > + $ref: /schemas/types.yaml#/definitions/string-array
> >
> > Why do we even need two methods? Commit message tells me nothing and
> > there's no description at all... Why do we need regexes here, rather
> > than explicitly defined properties? Your commit message should explain
> > the justification for that and the property descriptions (as comments if
> > needs be for patternProperties) should explain why this is intended to
> > be used.
> >
> > How is anyone supposed to look at this binding and understand how it
> > should be used?
>
> Also, please do not CC private mailing lists on your postings, I do not
> want to get spammed by linaro's mailman :(
boot-architecture is not private[0]. It is where EBBR gets discussed
amongst other things. This came up in a thread there[1].
Rob
[0] https://lists.linaro.org/mailman3/lists/boot-architecture.lists.linaro.org/
[1] https://lists.linaro.org/archives/list/boot-architecture@lists.linaro.org/t…
Hi,
To the folks on this mailing-list attending the Linaro Connect in Madrid,
it was suggested we could seize the opportunity to meet in person.
If you like the idea, let's meet on the terrace were we took the group picture,
on Thursday 18:30, before the social evening.
See you there!
Best regards,
Vincent.
On Wed, May 8, 2024 at 7:19 PM Elliot Berman <quic_eberman(a)quicinc.com> wrote:
>
> On Thu, May 02, 2024 at 09:00:47AM -0500, Rob Herring wrote:
> > On Wed, May 1, 2024 at 4:18 PM Humphreys, Jonathan <j-humphreys(a)ti.com> wrote:
> > > [1] Rather than using the device tree source filename, to have more flexibility,
> > > one can conceive an ID or compatible string that the OS could then scan the DTBs
> > > to find a match.
> >
> > I agree with Daniel that we should use the root node compatible for
> > this. We discussed this a while back on this list (or u-boot?). To
> > summarize, both using the filename or root node compatible were
> > proposed. Several folks (myself included) don't like making the
> > filename an ABI. However, there are some cases where the filename is
> > more unique than the root node compatible. We should fix those root
> > node compatibles in that case IMO.
>
> I think firmware-provided compatible string can cause headaches for both
> firmware and OS developers. I gave a talk about this at EOSS [1,2] and
> we've been posting some proposals [3,4] to introduce a board-id, which
> allows DTBs to have varying degrees of precision about describing what
> hardware they are applicable to.
>
> Compatible strings should be a mapping of some identifier
> registers/storage into a string. Today, bootloader has to figure out
> that mapping and I understood Jon's proposal as wanting to get firmware
> to provide the compatible string. However, the compatible string for a
> DTB could need to describe only a subset of those identifiers
> (compatible string) to get a DTB that works. This would be especially
> true for DT overlays, although there are other real and hypothetical
> situations where a DTB shouldn't/can't describe the complete set of
> identifiers. Firmware either needs to provide every possible combination
> of compatible string or knowledge needs to be baked into the OS about
> interpreting the compatible string. In simple terms, the proposal is to
> split out the identifers that are baked into the compatible string into
> separate "board-id" properties.
I don't think there is any way the OS (or OS loader) would be able
handle these properties and the logic to parse them. It all looks to
be platform specific. This could only work if the OS says to the
firmware "here's the 1000 DTB files, which one should I use? That's
quite different from the current proposals of how this would work.
Rob
On Tue, Mar 26, 2024 at 4:29 PM Daniel Golle <daniel(a)makrotopia.org> wrote:
>
> Hi Rob,
>
> On Tue, Mar 26, 2024 at 03:24:49PM -0500, Rob Herring wrote:
> > +boot-architecture list
>
> Good idea, thank you :)
Now really adding it. :(
Will reply to rest later.
> >
> > On Mon, Mar 25, 2024 at 03:38:19PM +0000, Daniel Golle wrote:
> > > On Mon, Mar 25, 2024 at 10:10:46AM -0500, Rob Herring wrote:
> > > > On Thu, Mar 21, 2024 at 07:31:48PM +0000, Daniel Golle wrote:
> > > > > On embedded devices using an eMMC it is common that one or more (hw/sw)
> > > > > partitions on the eMMC are used to store MAC addresses and Wi-Fi
> > > > > calibration EEPROM data.
> > > > >
> > > > > Implement an NVMEM provider backed by a block device as typically the
> > > > > NVMEM framework is used to have kernel drivers read and use binary data
> > > > > from EEPROMs, efuses, flash memory (MTD), ...
> > > > >
> > > > > In order to be able to reference hardware partitions on an eMMC, add code
> > > > > to bind each hardware partition to a specific firmware subnode.
> > > > >
> > > > > Overall, this enables uniform handling across practially all flash
> > > > > storage types used for this purpose (MTD, UBI, and now also MMC).
> > > > >
> > > > > As part of this series it was necessary to define a device tree schema
> > > > > for block devices and partitions on them, which (similar to how it now
> > > > > works also for UBI volumes) can be matched by one or more properties.
> > > > >
> > > > > ---
> > > > > This series has previously been submitted as RFC on July 19th 2023[1]
> > > > > and most of the basic idea did not change since. Another round of RFC
> > > > > was submitted on March 5th 2024[2] which has received overall positive
> > > > > feedback and only minor corrections have been done since (see
> > > > > changelog below).
> > > >
> > > > I don't recall giving positive feedback.
> > > >
> > > > I still think this should use offsets rather than partition specific
> > > > information. Not wanting to have to update the offsets if they change is
> > > > not reason enough to not use them.
> > >
> > > Using raw offsets on the block device (rather than the partition)
> > > won't work for most existing devices and boot firmware out there. They
> > > always reference the partition, usually by the name of a GPT
> > > partition (but sometimes also PARTUUID or even PARTNO) which is then
> > > used in the exact same way as an MTD partition or UBI volume would be
> > > on devices with NOR or NAND flash.
> >
> > MTD normally uses offsets hence why I'd like some alignment. UBI is
> > special because raw NAND is, well, special.
>
> I get the point and in a way this is also already intended and
> supported by this series. You can already just add an 'nvmem-layout'
> node directly to a disk device rather than to a partition and define a
> layout in this way.
>
> Making this useful in practice will require some improvements to the
> nvmem system in Linux though, because that currently uses signed 32-bit
> integers as addresses which is not sufficient for the size of the
> user-part of an eMMC. However, that needs to be done then and should
> of course not be read as an excuse.
>
> >
> > > Just on eMMC we usually use a GPT
> > > or MBR partition table rather than defining partitions in DT or cmdline,
> > > which is rather rare (for historic reasons, I suppose, but it is what it
> > > is now).
> >
> > Yes, I understand how eMMC works. I don't understand why if you have
> > part #, uuid, or name you can't get to the offset or vice-versa. You
> > need only 1 piece of identification to map partition table entries to DT
> > nodes.
>
> Yes, either of them (or a combination) is fine. In practise I've mostly
> seen PARTNAME as identifier used in userland scripts, and only adding
> this for now will probably cover most devices (and existing boot firmware)
> out there. Notable exceptions are devices which are using MBR partitions
> because the BootROM expects the bootloader to be at the same block as
> we would usually have the primary GPT. In this case we can only use the
> PARTNO, of course, and it stinks.
> MediaTek's MT7623A/N is such an example, but it's a slingly outdated
> and pretty weird niche SoC I admit.
>
> > Sure, offsets can change, but surely the firmware can handle
> > adjusting the DT?
>
> Future firmware may be able to do this, of course. Current existing
> firmware already out there on devices such as the quite popular
> GL.iNet MT-6000, Netgear's Orbi and Orbi Pro series as well as all
> Adtran SmartRG devices does not. Updating or changing the boot
> firmware of devices already out there is not intended and quite
> challenging, and will make the device incompatible with its vendor
> firmware. Hence it would be better to support replacing only the
> Linux-based firmware (eg. with OpenWrt or even Debian or any
> general-purpose Linux, the eMMC is large enough...) while not having
> to touch the boot firmware (and risking to brick the device if that
> goes wrong).
>
> Personally, I'm rather burdened and unhappy with vendor attempts to
> have the boot firmware mess around too much in (highly customized,
> downstream) DT, it may look like a good solution at the moment, but
> can totally become an obstacle in an unpredictable future (no offense
> ASUS...)
>
> >
> > An offset would also work for the case of random firmware data on the
> > disk that may or may not have a partition associated with it. There are
> > certainly cases of that. I don't think we have much of a solution for
> > that other than trying to educate vendors to not do that or OS
> > installers only supporting installing to something other than eMMC. This
> > is something EBBR[1] is trying to address.
>
> Absolutely. Actually *early* GL-iNet devices did exactly that: Use the
> eMMC boot hw-partitions to store boot firmware as well as MAC
> addresses and potentially also Wi-Fi calibration data.
>
> The MT-2500 is the example I'm aware of and got sitting on my desk for
> testing with this very series (which allows to also reference eMMC
> hardware partitions, see "[7/8] mmc: block: set fwnode of disk
> devices").
> Unfortunately later devices such the the flag-ship MT-6000 moved MAC
> addresses and WiFi-EEPROMs into a GPT partition on the user-part of
> the eMMC.
>
> >
> > > Depending on the eMMC chip used, that partition may not even be at the
> > > same offset for different batches of the same device and hence I'd
> > > like to just do it in the same way vendor firmware does it as well.
> >
> > Often vendor firmware is not a model to follow...
>
> I totally agree. However, I don't see a good reason for not supporting
> those network-appliance-type embedded devices which even ship with
> (outdated, downstream) Linux by default while going through great
> lengths for things like broken ACPI tables in many laptops which
> require lots of work-arounds to have features like suspend-to-disk
> working, or even be able to run Linux at all.
>
> >
> > > Chad of Adtran has previously confirmed that [1], which was the
> > > positive feedback I was refering to. Other vendors like GL-iNet or
> > > Netgear are doing the exact same thing.
> > >
> > > As of now, we support this in OpenWrt by adding a lot of
> > > board-specific knowledge to userland, which is ugly and also prevents
> > > using things like PXE-initiated nfsroot on those devices.
> > >
> > > The purpose of this series is to be able to properly support such devices
> > > (ie. practially all consumer-grade routers out there using an eMMC for
> > > storing firmware).
> > >
> > > Also, those devices have enough resources to run a general purpose
> > > distribution like Debian instead of OpenWrt, and all the userland
> > > hacks to set MAC addresses and extract WiFi-EEPROM-data in a
> > > board-specific ways will most certainly never find their way into
> > > Debian. It's just not how embedded Linux works, unless you are looking
> > > only at the RaspberryPi which got that data stored in a textfile
> > > which is shipped by the distribution -- something very weird and very
> > > different from literally all of-the-shelf routers, access-points or
> > > switches I have ever seen (and I've seen many). Maybe Felix who has
> > > seen even more of them can tell us more about that.
> >
> > General purpose distros want to partition the disk themselves. Adding
> > anything to the DT for disk partitions would require the installer to be
> > aware of it. There's various distro folks on the boot-arch list, so
> > maybe one of them can comment.
>
> Usually the installers are already aware to not touch partitions when
> unaware of their purpose. Repartitioning the disk from scratch is not
> what (modern) distributions are doing, at least the EFI System
> partition is kept, as well as typical rescue/recovery partitions many
> vendors put on their (Windows, Mac) laptops to allow to "factory
> reset" them.
>
> Installers usually offer to replace (or resize) the "large" partition
> used by the currently installed OS instead.
>
> And well, the DT reference to a partition holding e.g. MAC addresses
> does make the installer aware of it, obviously.
>
>
> Thank you for the constructive debate!
>
>
> Cheers
>
>
> Daniel
>
>
> >
> > Rob
> >
> > [1] https://arm-software.github.io/ebbr/index.html#document-chapter4-firmware-m…