Re: [TF-A] Proposal: TF-A to adopt hand-off blocks (HOBs) for information passing between boot stages
+Boot Architecture Mailman List boot-architecture@lists.linaro.org
standardization is very much welcomed here and need to accommodate a very diverse set of situations. For example, TEE OS may need to pass memory reservations to BL33 or "capture" a device for the secure world.
I have observed a number of architectures: 1) pass information from BLx to BLy in the form of a specific object 2) BLx called by BLy by a platform specific SMC to get information 3) BLx called by BLy by a platform specific SMC to perform Device Tree fixups
I also imagined a standardized "broadcast" FF-A call so that any firmware element can either provide information or "do something".
My understanding of your proposal is about standardizing on architecture 1) with the HOB format.
The advantage of the HOB is simplicity but it may be difficult to implement schemes such as pruning a DT because device assignment in the secure world.
In any case, it looks feasible to have TF-A and OP-TEE complement the list of HOBs to pass information downstream (the bootflow).
It would be good to start with building the comprehensive list of information that need to be conveyed between firmware elements:
information. | authoritative entity | reporting entity | information exchanged: dram | TFA | TFA | <format to be detailed, NUMA topology to build the SRAT table or DT equivalent?> PSCI | SCP | TFA? | SCMI | SCP or TEE-OS | TFA? TEE-OS?| secure SRAM | TFA. | TFA. | secure DRAM | TFA? TEE-OS? | TFA? TEE-OS? | other? | | |
Cheers
FF
On Mon, 22 Mar 2021 at 09:34, Harb Abdulhamid OS via TF-A < tf-a@lists.trustedfirmware.org> wrote:
Hello Folks,
I'm emailing to start an open discussion about the adoption of a concept known as "hand-off blocks" or HOB to become a part of the TF-A Firmware Framework Architecture (FFA). This is something that is a pretty major pain point when it comes to the adoption of TF-A in ARM Server SoC’s designed to enable a broad range of highly configurable datacenter platforms.
What is a HOB (Background)?
UEFI PI spec describes a particular definition for how HOB may be used for transitioning between the PEI and DXE boot phases, which is a good reference point for this discussion, but not necessarily the exact solution appropriate for TF-A.
A HOB is simply a dynamically generated data structure passed in between two boot phases. This is information that was obtained through discovery and needs to be passed forward to the next boot phase *once*, with no API needed to call back (e.g. no call back into previous firmware phase is needed to fetch this information at run-time - it is simply passed one time during boot).
There may be one or more HOBs passed in between boot phases. If there are more than one HOB that needs to be passed, this can be in a form of a "HOB table", which (for example) could be a UUID indexed array of pointers to HOB structures, used to locate a HOB of interest (based on UUID). In such cases, instead of passing a single HOB, the boot phases may rely on passing the pointer to the HOB table.
This has been extremely useful concept to employ on highly configurable systems that must rely on flexible discovery mechanisms to initialize and boot the system. This is especially helpful when you have multiple
Why do we need HOBs in TF-A?:
It is desirable that EL3 firmware (e.g. TF-A) built for ARM Server SoC in a way that is SoC specific *but* platform agnostic. This means that a single ARM SoC that a SiP may deliver to customers may provide a single TF-A binary (e.g. BL1, BL2, BL31) that could be used to support a broad range of platform designs and configurations in order to boot a platform specific firmware (e.g. BL33 and possibly even BL32 code). In order to achieve this, the platform configuration must be *discovered* instead of statically compiled as it is today in TF-A via device tree based enumeration. The mechanisms of discovery may differ broadly depending on the relevant industry standard, or in some cases may have rely on SiP specific discovery flows.
For example: On server systems that support a broad range DIMM memory population/topologies, all the necessary information required to boot is fully discovered via standard JEDEC Serial Presence Detect (SPD) over an I2C bus. Leveraging the SPD bus, may platform variants could be supported with a single TF-A binary. Not only is this information required to initialize memory in early boot phases (e.g. BL2), the subsequent boot phases will also need this SPD info to construct a system physical address map and properly initialize the MMU based on the memory present, and where the memory may be present. Subsequent boot phases (e.g. BL33 / UEFI) may need to generate standard firmware tables to the operating systems, such as SMBIOS tables describing DIMM topology and various ACPI tables (e.g. SLIT, SRAT, even NFIT if NVDIMM's are present).
In short, it all starts with a standardized or vendor specific discovery flow in an early boot stage (e.g. BL1/BL2), followed by the passing of information to the next boot stages (e.g. BL31/BL32/BL33).
Today, every HOB may be a vendor specific structure, but in the future there may be benefit of defining standard HOBs. This may be useful for memory discovery, passing the system physical address map, enabling TPM measured boot, and potentially many other common HOB use-cases.
It would be extremely beneficial to the datacenter market segment if the TF-A community would adopt this concept of information passing between all boot phases as opposed to rely solely on device tree enumeration. This is not intended to replace device tree, rather intended as an alternative way to describe the info that must be discovered and dynamically generated.
Conclusion:
We are proposing that the TF-A community begin pursuing the adoption of HOBs as a mechanism used for information exchange between each boot stage (e.g. BL1->BL2, BL2->BL31, BL31->BL32, and BL31->BL33)? Longer term we want to explore standardizing some HOB structures for the BL33 phase (e.g. UEFI HOB structures), but initially would like to agree on this being a useful mechanism used to pass information between each boot stage.
Thanks,
--Harb
-- TF-A mailing list TF-A@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/tf-a
Hi,
Can I suggest using bloblist for this instead? It is lightweight, easier to parse, doesn't have GUIDs and is already used within U-Boot for passing info between SPL/U-Boot, etc.
Docs here: https://github.com/u-boot/u-boot/blob/master/doc/README.bloblist Header file describes the format: https://github.com/u-boot/u-boot/blob/master/include/bloblist.h
Full set of unit tests: https://github.com/u-boot/u-boot/blob/master/test/bloblist.c
Regards, Simon
On Mon, 22 Mar 2021 at 23:58, François Ozog francois.ozog@linaro.org wrote:
+Boot Architecture Mailman List boot-architecture@lists.linaro.org
standardization is very much welcomed here and need to accommodate a very diverse set of situations. For example, TEE OS may need to pass memory reservations to BL33 or "capture" a device for the secure world.
I have observed a number of architectures:
- pass information from BLx to BLy in the form of a specific object
- BLx called by BLy by a platform specific SMC to get information
- BLx called by BLy by a platform specific SMC to perform Device Tree
fixups
I also imagined a standardized "broadcast" FF-A call so that any firmware element can either provide information or "do something".
My understanding of your proposal is about standardizing on architecture 1) with the HOB format.
The advantage of the HOB is simplicity but it may be difficult to implement schemes such as pruning a DT because device assignment in the secure world.
In any case, it looks feasible to have TF-A and OP-TEE complement the list of HOBs to pass information downstream (the bootflow).
It would be good to start with building the comprehensive list of information that need to be conveyed between firmware elements:
information. | authoritative entity | reporting entity | information exchanged: dram | TFA | TFA | <format to be detailed, NUMA topology to build the SRAT table or DT equivalent?> PSCI | SCP | TFA? | SCMI | SCP or TEE-OS | TFA? TEE-OS?| secure SRAM | TFA. | TFA. | secure DRAM | TFA? TEE-OS? | TFA? TEE-OS? | other? | | |
Cheers
FF
On Mon, 22 Mar 2021 at 09:34, Harb Abdulhamid OS via TF-A < tf-a@lists.trustedfirmware.org> wrote:
Hello Folks,
I'm emailing to start an open discussion about the adoption of a concept known as "hand-off blocks" or HOB to become a part of the TF-A Firmware Framework Architecture (FFA). This is something that is a pretty major pain point when it comes to the adoption of TF-A in ARM Server SoC’s designed to enable a broad range of highly configurable datacenter platforms.
What is a HOB (Background)?
UEFI PI spec describes a particular definition for how HOB may be used for transitioning between the PEI and DXE boot phases, which is a good reference point for this discussion, but not necessarily the exact solution appropriate for TF-A.
A HOB is simply a dynamically generated data structure passed in between two boot phases. This is information that was obtained through discovery and needs to be passed forward to the next boot phase *once*, with no API needed to call back (e.g. no call back into previous firmware phase is needed to fetch this information at run-time - it is simply passed one time during boot).
There may be one or more HOBs passed in between boot phases. If there are more than one HOB that needs to be passed, this can be in a form of a "HOB table", which (for example) could be a UUID indexed array of pointers to HOB structures, used to locate a HOB of interest (based on UUID). In such cases, instead of passing a single HOB, the boot phases may rely on passing the pointer to the HOB table.
This has been extremely useful concept to employ on highly configurable systems that must rely on flexible discovery mechanisms to initialize and boot the system. This is especially helpful when you have multiple
Why do we need HOBs in TF-A?:
It is desirable that EL3 firmware (e.g. TF-A) built for ARM Server SoC in a way that is SoC specific *but* platform agnostic. This means that a single ARM SoC that a SiP may deliver to customers may provide a single TF-A binary (e.g. BL1, BL2, BL31) that could be used to support a broad range of platform designs and configurations in order to boot a platform specific firmware (e.g. BL33 and possibly even BL32 code). In order to achieve this, the platform configuration must be *discovered* instead of statically compiled as it is today in TF-A via device tree based enumeration. The mechanisms of discovery may differ broadly depending on the relevant industry standard, or in some cases may have rely on SiP specific discovery flows.
For example: On server systems that support a broad range DIMM memory population/topologies, all the necessary information required to boot is fully discovered via standard JEDEC Serial Presence Detect (SPD) over an I2C bus. Leveraging the SPD bus, may platform variants could be supported with a single TF-A binary. Not only is this information required to initialize memory in early boot phases (e.g. BL2), the subsequent boot phases will also need this SPD info to construct a system physical address map and properly initialize the MMU based on the memory present, and where the memory may be present. Subsequent boot phases (e.g. BL33 / UEFI) may need to generate standard firmware tables to the operating systems, such as SMBIOS tables describing DIMM topology and various ACPI tables (e.g. SLIT, SRAT, even NFIT if NVDIMM's are present).
In short, it all starts with a standardized or vendor specific discovery flow in an early boot stage (e.g. BL1/BL2), followed by the passing of information to the next boot stages (e.g. BL31/BL32/BL33).
Today, every HOB may be a vendor specific structure, but in the future there may be benefit of defining standard HOBs. This may be useful for memory discovery, passing the system physical address map, enabling TPM measured boot, and potentially many other common HOB use-cases.
It would be extremely beneficial to the datacenter market segment if the TF-A community would adopt this concept of information passing between all boot phases as opposed to rely solely on device tree enumeration. This is not intended to replace device tree, rather intended as an alternative way to describe the info that must be discovered and dynamically generated.
Conclusion:
We are proposing that the TF-A community begin pursuing the adoption of HOBs as a mechanism used for information exchange between each boot stage (e.g. BL1->BL2, BL2->BL31, BL31->BL32, and BL31->BL33)? Longer term we want to explore standardizing some HOB structures for the BL33 phase (e.g. UEFI HOB structures), but initially would like to agree on this being a useful mechanism used to pass information between each boot stage.
Thanks,
--Harb
-- TF-A mailing list TF-A@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/tf-a
-- François-Frédéric Ozog | *Director Linaro Edge & Fog Computing Group* T: +33.67221.6485 francois.ozog@linaro.org | Skype: ffozog _______________________________________________ boot-architecture mailing list boot-architecture@lists.linaro.org https://lists.linaro.org/mailman/listinfo/boot-architecture
Hi,
I propose we cover the topic at the next Trusted Substrate https://collaborate.linaro.org/display/TS/Trusted+Substrate+Home zoom call https://linaro-org.zoom.us/j/94563644892 on April 8th 4pm CET.
The agenda: ABI between non-secure firmware and the rest of firmware (EL3, S-EL1, S-EL2, SCP) to adapt hardware description to some runtime conditions. runtime conditions here relates to DRAM size and topology detection, secure DRAM memory carvings, PSCI and SCMI interface publishing.
For additional background on existing metadata: UEFI Platform Initialization Specification Version 1.7 https://uefi.org/sites/default/files/resources/PI_Spec_1_7_final_Jan_2019.pdf , 5.5 Resource Descriptor HOB Out of the ResourceType we care about is EFI_RESOURCE_SYSTEM_MEMORY. This HOB lacks memory NUMA attachment or something that could be related to fill SRAT table for ACPI or relevant DT proximity domains. HOB is not consistent accros platforms: some platforms (Arm) lists memory from the booting NUMA node, other platforms (x86) lists all memory from all NUMA nodes. (At least this is the case on the two platforms I tested).
There are two proposals to use memory structures from SPL/BLx up to the handover function (as defined in the Device Tree technical report https://docs.google.com/document/d/1CLkhLRaz_zcCq44DLGmPZQFPbYHOC6nzPowaL0XmRk0/edit?usp=sharing) which can be U-boot (BL33 or just U-Boot in case of SPL/U-Boot scheme) or EDK2. I would propose we also discuss possibility of FF-A interface to actually query information or request actions to be done (this is a model actually used in some SoCs with proprietary SMC calls).
Requirements (to be validated): - ACPI and DT hardware descriptions. - agnostic to boot framework (SPL/U-Boot, TF-A/U-Boot, TF-A/EDK2) - at least allows complete DRAM description and "persistent" usage (reserved areas for secure world or other usages) - support secure world device assignment
Cheers
FF
On Mon, 22 Mar 2021 at 19:56, Simon Glass sjg@chromium.org wrote:
Hi,
Can I suggest using bloblist for this instead? It is lightweight, easier to parse, doesn't have GUIDs and is already used within U-Boot for passing info between SPL/U-Boot, etc.
Docs here: https://github.com/u-boot/u-boot/blob/master/doc/README.bloblist Header file describes the format: https://github.com/u-boot/u-boot/blob/master/include/bloblist.h
Full set of unit tests: https://github.com/u-boot/u-boot/blob/master/test/bloblist.c
Regards, Simon
On Mon, 22 Mar 2021 at 23:58, François Ozog francois.ozog@linaro.org wrote:
+Boot Architecture Mailman List boot-architecture@lists.linaro.org
standardization is very much welcomed here and need to accommodate a very diverse set of situations. For example, TEE OS may need to pass memory reservations to BL33 or "capture" a device for the secure world.
I have observed a number of architectures:
- pass information from BLx to BLy in the form of a specific object
- BLx called by BLy by a platform specific SMC to get information
- BLx called by BLy by a platform specific SMC to perform Device Tree
fixups
I also imagined a standardized "broadcast" FF-A call so that any firmware element can either provide information or "do something".
My understanding of your proposal is about standardizing on architecture
with the HOB format.
The advantage of the HOB is simplicity but it may be difficult to
implement
schemes such as pruning a DT because device assignment in the secure
world.
In any case, it looks feasible to have TF-A and OP-TEE complement the
list
of HOBs to pass information downstream (the bootflow).
It would be good to start with building the comprehensive list of information that need to be conveyed between firmware elements:
information. | authoritative entity | reporting entity | information exchanged: dram | TFA | TFA | <format to be detailed, NUMA topology to build the SRAT table or DT equivalent?> PSCI | SCP | TFA? | SCMI | SCP or TEE-OS | TFA? TEE-OS?| secure SRAM | TFA. | TFA. | secure DRAM | TFA? TEE-OS? | TFA? TEE-OS? | other? | | |
Cheers
FF
On Mon, 22 Mar 2021 at 09:34, Harb Abdulhamid OS via TF-A < tf-a@lists.trustedfirmware.org> wrote:
Hello Folks,
I'm emailing to start an open discussion about the adoption of a
concept
known as "hand-off blocks" or HOB to become a part of the TF-A Firmware Framework Architecture (FFA). This is something that is a pretty major pain point when it comes to the adoption of TF-A in ARM Server SoC’s designed to enable a broad range of highly configurable datacenter platforms.
What is a HOB (Background)?
UEFI PI spec describes a particular definition for how HOB may be used
for
transitioning between the PEI and DXE boot phases, which is a good reference point for this discussion, but not necessarily the exact
solution
appropriate for TF-A.
A HOB is simply a dynamically generated data structure passed in
between
two boot phases. This is information that was obtained through
discovery
and needs to be passed forward to the next boot phase *once*, with no
API
needed to call back (e.g. no call back into previous firmware phase is needed to fetch this information at run-time - it is simply passed one
time
during boot).
There may be one or more HOBs passed in between boot phases. If there
are
more than one HOB that needs to be passed, this can be in a form of a
"HOB
table", which (for example) could be a UUID indexed array of pointers
to
HOB structures, used to locate a HOB of interest (based on UUID). In
such
cases, instead of passing a single HOB, the boot phases may rely on
passing
the pointer to the HOB table.
This has been extremely useful concept to employ on highly configurable systems that must rely on flexible discovery mechanisms to initialize
and
boot the system. This is especially helpful when you have multiple
Why do we need HOBs in TF-A?:
It is desirable that EL3 firmware (e.g. TF-A) built for ARM Server SoC
in
a way that is SoC specific *but* platform agnostic. This means that a single ARM SoC that a SiP may deliver to customers may provide a single TF-A binary (e.g. BL1, BL2, BL31) that could be used to support a broad range of platform designs and configurations in order to boot a
platform
specific firmware (e.g. BL33 and possibly even BL32 code). In order to achieve this, the platform configuration must be *discovered* instead
of
statically compiled as it is today in TF-A via device tree based enumeration. The mechanisms of discovery may differ broadly depending
on
the relevant industry standard, or in some cases may have rely on SiP specific discovery flows.
For example: On server systems that support a broad range DIMM memory population/topologies, all the necessary information required to boot
is
fully discovered via standard JEDEC Serial Presence Detect (SPD) over
an
I2C bus. Leveraging the SPD bus, may platform variants could be
supported
with a single TF-A binary. Not only is this information required to initialize memory in early boot phases (e.g. BL2), the subsequent boot phases will also need this SPD info to construct a system physical
address
map and properly initialize the MMU based on the memory present, and
where
the memory may be present. Subsequent boot phases (e.g. BL33 / UEFI)
may
need to generate standard firmware tables to the operating systems,
such as
SMBIOS tables describing DIMM topology and various ACPI tables (e.g.
SLIT,
SRAT, even NFIT if NVDIMM's are present).
In short, it all starts with a standardized or vendor specific
discovery
flow in an early boot stage (e.g. BL1/BL2), followed by the passing of information to the next boot stages (e.g. BL31/BL32/BL33).
Today, every HOB may be a vendor specific structure, but in the future there may be benefit of defining standard HOBs. This may be useful for memory discovery, passing the system physical address map, enabling TPM measured boot, and potentially many other common HOB use-cases.
It would be extremely beneficial to the datacenter market segment if
the
TF-A community would adopt this concept of information passing between
all
boot phases as opposed to rely solely on device tree enumeration.
This is
not intended to replace device tree, rather intended as an alternative
way
to describe the info that must be discovered and dynamically generated.
Conclusion:
We are proposing that the TF-A community begin pursuing the adoption of HOBs as a mechanism used for information exchange between each boot
stage
(e.g. BL1->BL2, BL2->BL31, BL31->BL32, and BL31->BL33)? Longer term we want to explore standardizing some HOB structures for the BL33 phase
(e.g.
UEFI HOB structures), but initially would like to agree on this being a useful mechanism used to pass information between each boot stage.
Thanks,
--Harb
-- TF-A mailing list TF-A@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/tf-a
-- François-Frédéric Ozog | *Director Linaro Edge & Fog Computing Group* T: +33.67221.6485 francois.ozog@linaro.org | Skype: ffozog _______________________________________________ boot-architecture mailing list boot-architecture@lists.linaro.org https://lists.linaro.org/mailman/listinfo/boot-architecture
boot-architecture@lists.linaro.org
-
François Ozog -
Simon Glass