Hi Bill,
Improve search algorithm performance:
We will need data to show the problem. I suppose this would best be done when unflatening the data at runtime? What is the expected gain in boot time? Are there any measurements of how much time is spent in the search
routines today?
Actually, It is nessary to search and modify device tree in boot loader, and the device tree has not be unflatened.
We need to search device tree node int dtb by libfdt, however libfdt algorithm is pretty horrible, eg:fdt_node_offset_by_compatible.
It takes about 10ms on average to search for a node.
I suppose the dtb structure and libfdt algorithm can be optimized to reduce boot time?
The time for searching for a node should be less than 1 ms.
Reduce DTB space:
What is the goal of the use case?
1) Fit in limited storage ( ex: 256MB )
2) Conserve more space of modest storage for container data ( 1GB eMMC)
3) Improve boot time
For 3, the load time will be reduced but the decompression time will be added. These need to be balanced based on the CPU.
One pet peeve I have in most of our boot loaders today is that they do loading and decompression serially. During loading the IO is 100% loaded and the CPU is very lightly loaded. During decomoression the CPU is 100% loaded and the IO is 0%.
It makes sense to pipleline / overlap these things which means that it needs to go into the loader. To optimize boot time the decompression algorithm needs to be chosen correctly. On smaller CPUs the time taken to decompress
newer algorithms can greatly outweigh the time taken to load the decompressed data. Ideally the time to decompress 1 block == time to load one block. The dynamics shift with CPU and IO performance.
Today, a lot of people focused on boot speed just use decompressed data but I think we could do better if we pipeline
Since one dtbo image contains of hundreds of dtbs, the iamge is too large.
The goal of reducing dtb space is to fit in limited storage without increasing boot time.
Pipleline / overlap loading and decompression is a good idea.
We can try to overlap them in boot loader, and discuss it further if there's a problem.
Define specific rule for properties:
This is harder.In 2019 I had proposed an ATOM based DTB enhancement [2]. I was told Frank Rowand had other proposals for format changes.
OK.I am very interested in the ATOM based DTB enhancement.
I will learn about it and other format changes, then make some detailed discussions with you.
Thanks very much, Bill.
Happy Chinese New Year
Regards,
Zhangpeng
发件人: Jammy Zhou [mailto:jammy.zhou@linaro.org]
发送时间: 2021年2月9日 16:45
收件人: Xiamingliang (XML, Hisilicon) <xiamingliang(a)huawei.com>
抄送: Bill Mills <bill.mills(a)linaro.org>; boot-architecture(a)lists.linaro.org; Frank Rowand <frowand.list(a)gmail.com>; Zhangpeng (Parker, Kirin) <zhangpeng55(a)huawei.com>; Wangjun (U) <wangjundrv.wang(a)huawei.com>
主题: Re: Ideas for DT improvements
Hi Bill,
Thanks very much for your comments. Since we're close to the Chinese New Year holiday, I would assume there will be some delay for the response by Zhangpeng.
Regards,
Jammy
On Sun, 7 Feb 2021 at 09:35, Xiamingliang (XML, Hisilicon) <xiamingliang(a)huawei.com<mailto:xiamingliang@huawei.com>> wrote:
+ zhangpeng, owner of DT in Hisilicon
-----Original Message-----
From: Bill Mills [mailto:bill.mills@linaro.org<mailto:bill.mills@linaro.org>]
Sent: 2021年2月7日 1:39
To: Jammy Zhou <jammy.zhou(a)linaro.org<mailto:jammy.zhou@linaro.org>>; boot-architecture(a)lists.linaro.org<mailto:boot-architecture@lists.linaro.org>; Frank Rowand <frowand.list(a)gmail.com<mailto:frowand.list@gmail.com>>
Cc: Xiamingliang (XML, Hisilicon) <xiamingliang(a)huawei.com<mailto:xiamingliang@huawei.com>>
Subject: Re: Ideas for DT improvements
Hi Jammy & Mingliang,
On 2/5/21 2:59 AM, Jammy Zhou wrote:
> Hi,
>
> There are several ideas for DT improvements. Please check if they are
> reasonable, and any comments are welcome. I would let Mingliang (CCed)
> share more details if needed.
>
> 1) Improve search algorithm performance: Is the binary search tree or
> other algorithm better than current algorithm?
>
We will need data to show the problem. I suppose this would best be done when unflatening the data at runtime? What is the expected gain in boot time? Are there any measurements of how much time is spent in the search routines today?
> 2) Reduce DTB space: when use one DTB to support multiple boards, the
> image is quite big (e.g, ~39MB space for 100 configurations), and some
> compression algorithm can reduce the space a lot (e.g, from 39MB to 7MB).
> Shall we have such compression support for DTB? And it can be helpful
> if we can have more efficient compression algorithm.
>
This could be done as an enhancement to the DTB loader instead of the DTB format itself.
Compressing each DTB (boardx.dtb.xz) will get you gains but compressing a set of boards (vmlinux-5.4.0-65-generic-dbt-set-20.tar.xz) might give you more.
To be significant, the number of boards would need to be large and the size of the rootfs would need to be modest. A 200 to 300 MB minimal image would make an interesting comparison point. (A rootfs of 10s of MB would probably only target a few boards.)
What is the goal of the use case?
1) Fit in limited storage ( ex: 256MB )
2) Conserve more space of modest storage for container data ( 1GB eMMC)
3) Improve boot time
For 3, the load time will be reduced but the decompression time will be added. These need to be balanced based on the CPU.
One pet peeve I have in most of our boot loaders today is that they do loading and decompression serially. During loading the IO is 100% loaded and the CPU is very lightly loaded. During decomoression the CPU is 100% loaded and the IO is 0%. It makes sense to pipleline / overlap these things which means that it needs to go into the loader. To optimize boot time the decompression algorithm needs to be chosen correctly. On smaller CPUs the time taken to decompress newer algorithms can greatly outweigh the time taken to load the decompressed data. Ideally the time to decompress 1 block == time to load one block.
The dynamics shift with CPU and IO performance.
Today, a lot of people focused on boot speed just use decompressed data but I think we could do better if we pipeline
> 3) Define specific rule for properties: The property value
> (FDT_PROP_DATA) itself occupies only ~50% of the total DTB space. And
> the property of each node is different and the private property name
> length is too long, for
> example: “freq-autodown-baseaddress-num” in dt_strings. It seems more
> reasonable that the property value should occupies more than 70% of
> the total DTB space. It can probably be achieved to define some rules
> to restrict the length of property name, etc.
>
This is harder. In 2019 I had proposed an ATOM based DTB enhancement [2]. I was told Frank Rowand had other proposals for format changes.
Thanks,
Bill
[2]
https://docs.google.com/document/d/19XbxN-zX-GYwOXdF78lGnp0j7UNx1MT3wzyCjai…
> Thanks,
> Jammy
> _______________________________________________
> boot-architecture mailing list
> boot-architecture(a)lists.linaro.org<mailto:boot-architecture@lists.linaro.org>
> https://lists.linaro.org/mailman/listinfo/boot-architecture
>
--
Bill Mills
Principal Technical Consultant, Linaro
+1-240-643-0836
TZ: US Eastern
Work Schedule: Tues/Wed/Thur
Hi,
There are several ideas for DT improvements. Please check if they are
reasonable, and any comments are welcome. I would let Mingliang (CCed)
share more details if needed.
1) Improve search algorithm performance: Is the binary search tree or other
algorithm better than current algorithm?
2) Reduce DTB space: when use one DTB to support multiple boards, the image
is quite big (e.g, ~39MB space for 100 configurations), and some
compression algorithm can reduce the space a lot (e.g, from 39MB to 7MB).
Shall we have such compression support for DTB? And it can be helpful if we
can have more efficient compression algorithm.
3) Define specific rule for properties: The property value (FDT_PROP_DATA)
itself occupies only ~50% of the total DTB space. And the property of each
node is different and the private property name length is too long, for
example: “freq-autodown-baseaddress-num” in dt_strings. It seems more
reasonable that the property value should occupies more than 70% of the
total DTB space. It can probably be achieved to define some rules to
restrict the length of property name, etc.
Thanks,
Jammy
All,
I hope we have finally settled on a standing meeting time for the DT Evo
call. We will have the call every other Monday alternating with EBBR in
the same time slot. If I have done the TZ math correctly this is 16:00
UTC, and 11 AM US Eastern, 8AM US Pacific.
I have sent a google calendar invite to all those listed on the previous
call. If you would like to be added to that please email me directly at
bill.mills(a)linaro.org
Topic: Devicetree Evolution
Time: Jan 25, 2021 11:00 AM Eastern Time (US and Canada)
Every 2 weeks on Mon
(Alternates with EBBR meeting in same time slot)
Join Zoom Meeting
https://linaro-org.zoom.us/j/96170428801?pwd=elBJNFdVMFJub0UzanFUcVQxTHBqdz…
Meeting ID: 961 7042 8801
Passcode: 8250
One tap mobile
+13017158592,,96170428801# US (Washington D.C)
+16465588656,,96170428801# US (New York)
Dial by your location
+1 301 715 8592 US (Washington D.C)
888 788 0099 US Toll-free
877 853 5247 US Toll-free
+44 203 481 5237 United Kingdom
0 800 031 5717 United Kingdom Toll-free
Meeting ID: 961 7042 8801
Find your local number: https://linaro-org.zoom.us/u/acQEZ30MEP
--
Bill Mills
Principal Technical Consultant, Linaro
+1-240-643-0836
TZ: US Eastern
Work Schedule: Tues/Wed/Thur
If the platform has an RTC, then EFI_GET_TIME and EFI_SET_TIME are required
before ExitBootServices(). Clarify this in the spec.
Also specify that EFI_{GET,SET}_WAKEUP_TIME are required before
ExitBootService() if the RTC can wakeup the platform.
Signed-off-by: Grant Likely <grant.likely(a)arm.com>
---
Reading through the RTC text it didn't seem clear to me. How's this for
a clarification?
g.
source/chapter2-uefi.rst | 15 +++++++++------
1 file changed, 9 insertions(+), 6 deletions(-)
diff --git a/source/chapter2-uefi.rst b/source/chapter2-uefi.rst
index 9906fd9..ab22932 100644
--- a/source/chapter2-uefi.rst
+++ b/source/chapter2-uefi.rst
@@ -159,16 +159,16 @@ are required to be implemented during boot services and runtime services.
- Before ExitBootServices()
- After ExitBootServices()
* - `EFI_GET_TIME`
- - Optional
+ - Required if RTC present
- Optional
* - `EFI_SET_TIME`
- - Optional
+ - Required if RTC present
- Optional
* - `EFI_GET_WAKEUP_TIME`
- - Optional
+ - Required if wakeup supported
- Optional
* - `EFI_SET_WAKEUP_TIME`
- - Optional
+ - Required if wakeup supported
- Optional
* - `EFI_SET_VIRTUAL_ADDRESS_MAP`
- N/A
@@ -227,8 +227,11 @@ it may not be possible to access the RTC from runtime services.
e.g., The RTC may be on a shared I2C bus which runtime services cannot access
because it will conflict with the OS.
-If firmware does not support access to the RTC, then GetTime() and
-SetTime() shall return EFI_UNSUPPORTED,
+If an RTC is present, then GetTime() and SetTime() must be supported
+before ExitBootServices() is called.
+
+However, if firmware does not support access to the RTC after
+ExitBootServices(), then GetTime() and SetTime() shall return EFI_UNSUPPORTED
and the OS must use a device driver to control the RTC.
UEFI Reset and Shutdown
--
2.20.1
Hi
I assume this needs to be analyzed from System Device Tree perspective:
https://trustedfirmware-a.readthedocs.io/en/latest/components/psa-ffa-manif…
And this is to be included in the DT Technical Report.
Cheers
FF
--
François-Frédéric Ozog | *Director Linaro Edge & Fog Computing Group*
T: +33.67221.6485
francois.ozog(a)linaro.org | Skype: ffozog
Hi everyone.
I have to do this, but I have another unexpected conflict for the EBBR biweekly on the 14th.
Rather than cancelling outright, does anyone else want to chair the meeting? The major planned orientatio item on the agenda was to talk about EBBR testing, with Heinrich sharing what he is currently doing.
If I don't hear anything by about 1pm GMT tomorrow then I'll just cancel. Our next meeting will be in January as I believe most of us will already be on Christmas holiday on the 21st
g.
Get Outlook for Android<https://aka.ms/ghei36>
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
EFI_UPDATE_CAPSULE is the industry standard method for applying firmware
updates. Make it a requirement in EBBR so that fwupd, Windows Update,
and any other generic firmware update service can support EBBR platforms.
This is made required because the ability to update firmware is a
critical part of building secure platforms.
Fixes: #69
Signed-off-by: Grant Likely <grant.likely(a)arm.com>
---
source/chapter2-uefi.rst | 32 +++++++++++++++++++++++++++++++-
1 file changed, 31 insertions(+), 1 deletion(-)
diff --git a/source/chapter2-uefi.rst b/source/chapter2-uefi.rst
index 7b5eb24..b1182a8 100644
--- a/source/chapter2-uefi.rst
+++ b/source/chapter2-uefi.rst
@@ -167,7 +167,10 @@ are required to be implemented during boot services and runtime services.
EFI_SET_VARIABLE Required Optional
EFI_GET_NEXT_HIGH_MONO_COUNT N/A Optional
EFI_RESET_SYSTEM Required Optional
- EFI_UPDATE_CAPSULE Optional Optional
+ EFI_UPDATE_CAPSULE Required Optional
+ for in-band
+ firmware
+ update
EFI_QUERY_CAPSULE_CAPABILITIES Optional Optional
EFI_QUERY_VARIABLE_INFO Optional Optional
============================== ============= ================
@@ -243,6 +246,25 @@ Even when SetVariable() is not supported during runtime services, firmware
should cache variable names and values in EfiRuntimeServicesData memory so
that GetVariable() and GetNextVeriableName() can behave as specified.
+Firmware Update
+---------------
+
+Being able to update firmware to address security issues is a key feature of secure platforms.
+EBBR platforms are required to implement either an in-band or an out-of-band firmware update mechanism.
+
+If firmware update is performed in-band (firmware on the application processor updates itself),
+then the firmware shall implement EFI_UPDATE_CAPSULE and accept updates in the
+"Firmware Management Protocol Data Capsule Structure" format as described in [UEFI]_ § 23.3,
+"Delivering Capsules Containing Updates to Firmware Management Protocol. [#FMPNote]_
+Firmware is also required to provide an EFI System Resource Table (ESRT). [UEFI]_ § 23.4
+Every firmware image that is updated in-band must be described in the ESRT.
+
+If firmware update is performed out-of-band (e.g., by an independent Board Management Controller,
+or firmware is provided by a hypervisor), then the platform is not required to implement EFI_UPDATE_CAPSULE.
+
+EFI_UPDATE_CAPSULE is only required before ExitBootServices() is called.
+
+
.. [#OPTEESupplicant] It is worth noting that OP-TEE has a similar problem
regarding secure storage.
OP-TEE's chosen solution is to rely on an OS supplicant agent to perform
@@ -253,3 +275,11 @@ that GetVariable() and GetNextVeriableName() can behave as specified.
during runtime services.
https://optee.readthedocs.io/en/latest/architecture/secure_storage.html
+
+.. [#FMPNote] The `EFI_UPDATE_CAPSULE` implementation is expected to be suitable
+ for use by generic firmware update services like fwupd and Windows Update.
+ Both fwupd and Windows Update read the ESRT table to determine what firmware
+ can be updated, and use an EFI helper application to call `EFI_UPDATE_CAPSULE`
+ before ExitBootServices() is called.
+
+ https://fwupd.org/
--
2.20.1
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hi,
I thought perhaps it might be worth starting a thread on this, as
despite Grant and Heinrich kinding spending a bit of time talking
about this, I am still very much in the dark about how 'embedded' and
distro/other boot flows are going to come together with EBBR. Of
course this would be easier f2f.
Case 1:
Firmware loads the kernel to a particular address, selects DT and
boots it. The kernel may require EFI boot services, or may not, but in
the general case the firmware provides them.
Case 2:
Firmware loads EFI app and provides EFI boot services to it. How the
system actually boots is under control of the app.
I feel that a lot of the confusion about verified boot, DT selections,
boot menus, etc. is coming from the introduction of an EFI app which
has no specification (it can be grub, shim or something else, as I
understand it). Certainly this is very flexible and future-proof, but
it is also arbitrarily complex, unpredictable and hard to secure.
I am wondering if we can come up with a way to deterministically
specify how a system will boot and how to make it boot a different way
(i.e. with a different kernel, initrd, DT).
Heinrich mentioned EFI variables as a way of selecting
kernel/initrd/DT. Then the problem becomes just a case of being able
to change those variables from Linux userspace. Is that right?
We are talking about having a 'secure' part of EBBR, which allows for
secure boot. Should we have a 'defined boot' part of EBBR, that
defines how the kernel/DT/initrd are selected, based on EFI variables?
Unfortunately I just don't know enough about all the different boot
flows used by the different distros. It seems like crazy town. Does
anyone have some pointers so I can do some study?
Regards,
SImon
All,
On the Devicetree evolution call Wednesday I promised to finish my
comparison of u-boot DT vs kernel DT.
The script is not perfect but the results are still interesting.
For each dts and dtsi file in the tip of the u-boot tree, it tries to
correlate it to the kernel tip.
It compares git SHA1 signatures or falls back to filenames.
The results were surprising to me but perhaps they should not have been.
I have checked in the script[1] and the full results here [2]
The full file lists (with some diff stats) are in the root dir.
Example [3]
I also looked at the line count of the u-boot override files.
Even though we don't expect these to correlate, we do expect reasonable
usage to result in small files. Big files are an indication of possible
abuse of the system. (I don't think the idea was to have wholesale new
versions of the DTS as an override.)
I plan to redo the script in python. It will be much easier to be more
precise and to look deeper. (For example figure out how old the u-boot
version is in number of change sets and number of days. Or if no
content sync now were they ever synced?)
Here is the scripts output: (from summary.txt)
Devicetree sync status for u-boot v2021.01-rc5-7-gb8c725e736
Compared to kernel v5.11-rc2-156-g71c061d24438
14% (255) are completely synced
253 arm
2 riscv
0 mips
0 powerpc
0 x86
0 68k
0 microblaze
0 sh
0 arc
23% (416) content has appeared in the kernel but is not up to date
411 arm
0 riscv
1 mips
0 powerpc
1 x86
0 68k
0 microblaze
0 sh
1 arc
33% (584) filename appears in kernel but content never has
467 arm
1 riscv
12 mips
91 powerpc
0 x86
0 68k
0 microblaze
0 sh
8 arc
28% (510) neither filename nor content appears in kernel
305 arm
4 riscv
48 mips
35 powerpc
44 x86
0 68k
1 microblaze
1 sh
6 arc
n/a (510) U-Boot specific, no correlation expected
7 sandbox
358 override
211 test
histogram of override size (in raw lines)
10 61
20 53
30 38
40 33
50 23
60 14
70 12
80 7
90 5
100 4
110 4
120 5
130 6
140 4
150 0
160 2
170 0
180 0
190 4
200 0
210 2
220 2
230 1
240 2
250 1
260 1
270 1
280 0
290 0
300 0
310 0
320 1
[1]
https://github.com/wmamills/devicetree-source/blob/master/scripts/correlate…
[2] https://github.com/wmamills/devicetree-source
[3]
https://github.com/wmamills/devicetree-source/blob/master/dts-somewhere.txt
--
Bill Mills
Principal Technical Consultant, Linaro
+1-240-643-0836
TZ: US Eastern
Work Schedule: Tues/Wed/Thur
Hello,
Thanks to all that have participated in the doodle poll. We don't yet
have all key stakeholders so please add your info if you have not.
As expected there is no perfect time but the current leader is
Wednesdays at 4PM UK, 11 AM US Eastern.
I have scheduled a meeting for tomorrow as this first one.
Possible topics for tomorrow include:
* More Conformance testing of DT source
* Keeping multiple DT projects in sync (w/o moving the DT source)
* DT overlay source in the kernel source tree (for bootloader applied
overlays)
If we stay with Wednesdays the meeting after this one would be Jan 27.
(Wednesdays require working around Linaro TSC calls)
Thanks,
Bill
***
Bill Mills is inviting you to a scheduled Zoom meeting.
Topic: DT Evolution
Time: Jan 6, 2021 04:00 PM London
Join Zoom Meeting
https://linaro-org.zoom.us/j/94413146152?pwd=NEs1Ym1xbnRBS0U4ZWNsaXFzbm1Ndz…
Meeting ID: 944 1314 6152
Passcode: 8250
One tap mobile
+13017158592,,94413146152# US (Washington D.C)
+13126266799,,94413146152# US (Chicago)
Dial by your location
+1 301 715 8592 US (Washington D.C)
+1 312 626 6799 US (Chicago)
+1 646 558 8656 US (New York)
+1 669 900 9128 US (San Jose)
+1 253 215 8782 US (Tacoma)
+1 346 248 7799 US (Houston)
877 853 5247 US Toll-free
888 788 0099 US Toll-free
Meeting ID: 944 1314 6152
Find your local number: https://linaro-org.zoom.us/u/aesZr3aPDG
--
Bill Mills
Principal Technical Consultant, Linaro
+1-240-643-0836
TZ: US Eastern
Work Schedule: Tues/Wed/Thur
