Hi There is bit of discussion on linux-efi too , to handle DT update
I guess some members of this forum are active there too.
https://www.spinics.net/lists/linux-efi/msg13700.html
To summaries 1/ Ownership of DTB IMO should be firmware and we should retain this ownership in EBBR as well, Any objections/thoughts ?
Update 1/ Updating whole device tree from OS [Capsule update can be used ]
2/ Just modifying the device tree DTBO
My preferred way to handle DTBO in firmware will be https://source.android.com/devices/architecture/dto/multiple See picture Runtime DTO implementation for multiple DTs
To store this information in partition, options we have 1/ Run time variables 2/ Some driver in Linux writing to DTBO partition 3/ Some other way ??
I am not sure, if distro are updating device tree which is default shipped with board ??
Thanks Udit
On 27.04.18 08:24, Udit Kumar wrote:
Hi There is bit of discussion on linux-efi too , to handle DT update
I guess some members of this forum are active there too.
https://www.spinics.net/lists/linux-efi/msg13700.html
To summaries 1/ Ownership of DTB IMO should be firmware and we should retain this ownership in EBBR as well, Any objections/thoughts ?
I fully agree. On top of that we need to make clear that backward and forward compatibility are not optional.
For that I think we may need to actually give people workable solutions to create device trees that are compatible with multiple levels of kernel support. The main areas I'm aware of that keep breaking are:
* fine grained interrupt controller support * clock support * power domain support * pinctrl support
Every time a device tree changes in any of the above, that usually ends up in backwards incompatibility.
My idea to solve that would be to basically create a device tree that has self-contained overlays that only trigger when certain feature strings are available. That way the base device tree could for example contain fixed clocks, but an overlay can get applied when the clock driver is enabled in the kernel configuration. That overlay would then enable the kernel to drive clocks.
Further down, we could even extend dtc with annotations that indicate "this property should only be exposed when feature string X is available" to not force people to write overlays inside the device tree.
Update 1/ Updating whole device tree from OS [Capsule update can be used ]
I think the device tree should be part of firmware. If you need to update it, update your firmware (or a firmware specific method, not specified by EBBR).
2/ Just modifying the device tree DTBO
Yes, dtbo support in the boot chain definitely makes sense.
My preferred way to handle DTBO in firmware will be https://source.android.com/devices/architecture/dto/multiple See picture Runtime DTO implementation for multiple DTs
To store this information in partition, options we have 1/ Run time variables
You mean EFI variables? We could certainly have a driver in firmware that reads certain EFI variables to apply dtbos from.
2/ Some driver in Linux writing to DTBO partition
What is a DTBO partition?
3/ Some other way ??
In a lot of cases I think we will end up with enumerable extensions to systems that will want to have a dtbo applied by their respective driver. As outlined in the notes from last week, that driver could expose availability of such an extension to the OS (grub) which could then apply an OS provided dtbo.
I am not sure, if distro are updating device tree which is default shipped with board ??
I would prefer they didn't :). Right now we leave people little chance, because device trees keep changing incompatibly. If we get rid of that problem, there will be very little incentive by distros to ship device trees.
Alex
On Fri, Apr 27, 2018 at 2:47 AM, Alexander Graf agraf@suse.de wrote:
On 27.04.18 08:24, Udit Kumar wrote:
Hi There is bit of discussion on linux-efi too , to handle DT update
I guess some members of this forum are active there too.
https://www.spinics.net/lists/linux-efi/msg13700.html
To summaries 1/ Ownership of DTB IMO should be firmware and we should retain this ownership in EBBR as well, Any objections/thoughts ?
I fully agree. On top of that we need to make clear that backward and forward compatibility are not optional.
For that I think we may need to actually give people workable solutions to create device trees that are compatible with multiple levels of kernel support. The main areas I'm aware of that keep breaking are:
- fine grained interrupt controller support
Do you have an example of that? The only thing I can think of is people switching interrupts from the GIC to an always-on, low-power mode custom interrupt controller.
- clock support
Are there cases other than going from fixed, fake clocks to a real clock controller node. I'm inclined to stop allowing people to do that. A better way this could be done is just provide a clock controller driver with a bunch of fixed clocks. Then the switch from the dumb driver to the real driver is just a kernel change.
- power domain support
Example?
- pinctrl support
This would be the firmware initially does all the pin setup, then you move it to DT and drop the setup from the firmware? Otherwise I don't understand the problem in this case. We'd start with no pinctrl and then add it to the DT. Why wouldn't the kernel just ignore it?
Every time a device tree changes in any of the above, that usually ends up in backwards incompatibility.
TBC, you're talking about new dtb with old kernels. We've mainly cared about old dtbs and new kernels. So first we should agree the former is important too. I do, because simply you wouldn't want a BIOS update to make your PC stop booting your already installed OS.
I'd like to solve this with policy and good practice before we try to apply technical solutions on top of a mess.
My idea to solve that would be to basically create a device tree that has self-contained overlays that only trigger when certain feature strings are available. That way the base device tree could for example contain fixed clocks, but an overlay can get applied when the clock driver is enabled in the kernel configuration. That overlay would then enable the kernel to drive clocks.
The number of combinations that would create makes me run away. Then we're going to combine that with all the other ways people want to use overlays.
Further down, we could even extend dtc with annotations that indicate "this property should only be exposed when feature string X is available" to not force people to write overlays inside the device tree.
How would that work with clocks where you are changing:
clocks = <&fake_fixed_clk>;
to:
clocks = <&soc_clock_ctrlr 123>;
Update 1/ Updating whole device tree from OS [Capsule update can be used ]
I think the device tree should be part of firmware. If you need to update it, update your firmware (or a firmware specific method, not specified by EBBR).
2/ Just modifying the device tree DTBO
Yes, dtbo support in the boot chain definitely makes sense.
My preferred way to handle DTBO in firmware will be https://source.android.com/devices/architecture/dto/multiple See picture Runtime DTO implementation for multiple DTs
The Android way of handling overlays is very much rooted in how the Android ecosystem works.
We should probably have wider discussion and decision on to what extent does EBBR address/care about/work with Android? On the one hand, I don't think Android requires anything that's specifically incompatible with EBBR if some wants to follow EBBR and use Android. OTOH, we can't define any requirements for Android in EBBR. Google will define things to the extent they want and vendors will follow that only to the extent they have to.
To store this information in partition, options we have 1/ Run time variables
You mean EFI variables? We could certainly have a driver in firmware that reads certain EFI variables to apply dtbos from.
2/ Some driver in Linux writing to DTBO partition
What is a DTBO partition?
The Android way. Everything can be solved with another partition. :)
Rob
Hi Rob,
On 27.04.18 18:40, Rob Herring wrote:
On Fri, Apr 27, 2018 at 2:47 AM, Alexander Graf agraf@suse.de wrote:
On 27.04.18 08:24, Udit Kumar wrote:
Hi There is bit of discussion on linux-efi too , to handle DT update
I guess some members of this forum are active there too.
https://www.spinics.net/lists/linux-efi/msg13700.html
To summaries 1/ Ownership of DTB IMO should be firmware and we should retain this ownership in EBBR as well, Any objections/thoughts ?
I fully agree. On top of that we need to make clear that backward and forward compatibility are not optional.
For that I think we may need to actually give people workable solutions to create device trees that are compatible with multiple levels of kernel support. The main areas I'm aware of that keep breaking are:
- fine grained interrupt controller support
Do you have an example of that? The only thing I can think of is people switching interrupts from the GIC to an always-on, low-power mode custom interrupt controller.
The last time I've seen that breakage was:
https://github.com/raspberrypi/linux/blob/rpi-4.14.y/arch/arm/boot/dts/bcm27...
which indeed does switch interrupts from the GIC to an interrupt muxer behind the GIC.
The problem is that once support for that lands upstream, you will have very little option but to break backwards compatibility today.
- clock support
Are there cases other than going from fixed, fake clocks to a real clock controller node. I'm inclined to stop allowing people to do that. A better way this could be done is just provide a clock controller driver with a bunch of fixed clocks. Then the switch from the dumb driver to the real driver is just a kernel change.
Going from fixed to in-kernel clock driver is one case I'm aware of, yes.
The other one I can think of right now is going from one clock interface to another. For example if you configure clocks using a firmware interface and that firmware interface moves to a newer, more shiny interface (say proprietary to SCMI). That switch may also regress users of the device tree.
I think Andre had other examples of where DT compatibility broke, so I'll let him chime in as well.
- power domain support
Example?
https://github.com/Xilinx/linux-xlnx/blob/master/arch/arm64/boot/dts/xilinx/...
Booting with the power-domains properties included in the device tree fails for me, because the driver framework considers them critical to load a device and just errors out with -EDEFER.
- pinctrl support
This would be the firmware initially does all the pin setup, then you move it to DT and drop the setup from the firmware? Otherwise I don't understand the problem in this case. We'd start with no pinctrl and then add it to the DT. Why wouldn't the kernel just ignore it?
Because it doesn't. The linux driver framework as it works today looks at the pinctrl parent for a specific device and if it can't find it loaded, throws -EDEFER for the driver getting loaded. I suppose the rationale behind that is that pinctrl drivers could be modules that get loaded after the driver you want to load.
Every time a device tree changes in any of the above, that usually ends up in backwards incompatibility.
TBC, you're talking about new dtb with old kernels. We've mainly cared
Yes. Both really.
about old dtbs and new kernels. So first we should agree the former is important too. I do, because simply you wouldn't want a BIOS update to make your PC stop booting your already installed OS.
Yes.
I'd like to solve this with policy and good practice before we try to apply technical solutions on top of a mess.
I'm not holding my hopes up, but we can try ;). Another full dimension of messiness are downstream, unapproved bindings that some times can't live in the same device tree as their upstream counterparts once upstream chose a different route.
For pinctrl and power domain, I suppose we could have a compatible whitelist in the kernel and instead of -EDEFER just consider the targets immutable if we hit none of the whitelisted strings.
My idea to solve that would be to basically create a device tree that has self-contained overlays that only trigger when certain feature strings are available. That way the base device tree could for example contain fixed clocks, but an overlay can get applied when the clock driver is enabled in the kernel configuration. That overlay would then enable the kernel to drive clocks.
The number of combinations that would create makes me run away. Then we're going to combine that with all the other ways people want to use overlays.
I'm happy to see better alternatives, but the current state is just plain painful.
Further down, we could even extend dtc with annotations that indicate "this property should only be exposed when feature string X is available" to not force people to write overlays inside the device tree.
How would that work with clocks where you are changing:
clocks = <&fake_fixed_clk>;
to:
clocks = <&soc_clock_ctrlr 123>;
I haven't made up my mind if we need to come up with a legacy dtc compatible syntax for this. But assume we don't, then it would be something as easy as:
clocks$can_handle_soc_clock_ctrlr = <&soc_clock_ctrlr 123>; clocks$!can_handle_soc_clock_ctrlr = <&fake_fixed_clk>;
which would result in 2 overlays, one that matches when the kernel understands the compatible for soc_clock_ctrlr (feature string "can_handle_sock_clock_ctrlr"), one that only matches when it doesn't.
Alex
On Fri, Apr 27, 2018 at 4:39 PM, Alexander Graf agraf@suse.de wrote:
Hi Rob,
On 27.04.18 18:40, Rob Herring wrote:
On Fri, Apr 27, 2018 at 2:47 AM, Alexander Graf agraf@suse.de wrote:
On 27.04.18 08:24, Udit Kumar wrote:
Hi There is bit of discussion on linux-efi too , to handle DT update
I guess some members of this forum are active there too.
https://www.spinics.net/lists/linux-efi/msg13700.html
To summaries 1/ Ownership of DTB IMO should be firmware and we should retain this ownership in EBBR as well, Any objections/thoughts ?
I fully agree. On top of that we need to make clear that backward and forward compatibility are not optional.
For that I think we may need to actually give people workable solutions to create device trees that are compatible with multiple levels of kernel support. The main areas I'm aware of that keep breaking are:
- fine grained interrupt controller support
Do you have an example of that? The only thing I can think of is people switching interrupts from the GIC to an always-on, low-power mode custom interrupt controller.
The last time I've seen that breakage was:
https://github.com/raspberrypi/linux/blob/rpi-4.14.y/arch/arm/boot/dts/bcm27...
which indeed does switch interrupts from the GIC to an interrupt muxer behind the GIC.
The problem is that once support for that lands upstream, you will have very little option but to break backwards compatibility today.
This one is unfortunate. It could have been handled better. An interrupt-map property in the aux ctrlr could have mapped the interrupts to GIC without any aux driver. Then when the aux driver lands, it just needs to remove the interrupt-map (on boot). Alternatively, I skimmed thru some discussions of the issue, but I'm not clear why the devices behind the aux controller can't all just treat their interrupts as shared. But that would be a simple change to the drivers' irq handlers, so I'm probably missing something. If that worked, then the DT would never need to change.
I guess whether this could have been handled better depends if folks knowingly ignored the issue or this was found after upstreaming support. The latter case may be unavoidable, but maybe we can make it rare enough we only need overlays in some exceptions. Whether we try to be stricter and do better up front or have some overlay based solution, either one is going to require folks be aware of the issues and effort to avoid them.
- clock support
Are there cases other than going from fixed, fake clocks to a real clock controller node. I'm inclined to stop allowing people to do that. A better way this could be done is just provide a clock controller driver with a bunch of fixed clocks. Then the switch from the dumb driver to the real driver is just a kernel change.
Going from fixed to in-kernel clock driver is one case I'm aware of, yes.
The other one I can think of right now is going from one clock interface to another. For example if you configure clocks using a firmware interface and that firmware interface moves to a newer, more shiny interface (say proprietary to SCMI). That switch may also regress users of the device tree.
If not only the clocks but the whole firmware interface changes, then I don't see how a stable DT matters at that point.
I think Andre had other examples of where DT compatibility broke, so I'll let him chime in as well.
- power domain support
Example?
https://github.com/Xilinx/linux-xlnx/blob/master/arch/arm64/boot/dts/xilinx/...
Booting with the power-domains properties included in the device tree fails for me, because the driver framework considers them critical to load a device and just errors out with -EDEFER.
- pinctrl support
This would be the firmware initially does all the pin setup, then you move it to DT and drop the setup from the firmware? Otherwise I don't understand the problem in this case. We'd start with no pinctrl and then add it to the DT. Why wouldn't the kernel just ignore it?
Because it doesn't. The linux driver framework as it works today looks at the pinctrl parent for a specific device and if it can't find it loaded, throws -EDEFER for the driver getting loaded. I suppose the rationale behind that is that pinctrl drivers could be modules that get loaded after the driver you want to load.
Every time a device tree changes in any of the above, that usually ends up in backwards incompatibility.
TBC, you're talking about new dtb with old kernels. We've mainly cared
Yes. Both really.
about old dtbs and new kernels. So first we should agree the former is important too. I do, because simply you wouldn't want a BIOS update to make your PC stop booting your already installed OS.
Yes.
I'd like to solve this with policy and good practice before we try to apply technical solutions on top of a mess.
I'm not holding my hopes up, but we can try ;). Another full dimension of messiness are downstream, unapproved bindings that some times can't live in the same device tree as their upstream counterparts once upstream chose a different route.
For pinctrl and power domain, I suppose we could have a compatible whitelist in the kernel and instead of -EDEFER just consider the targets immutable if we hit none of the whitelisted strings.
Yes, I had similar thought as well, but was thinking a blacklist rather. We could modify the DT on boot to disable the listed nodes. Or various subsystems could have lists of compatibles to not return -EPROBE_DEFER. The one complexity with a blacklist is we'd need patches for stable that may not be in mainline unless we add the compatible to the blacklist and then immediately remove it when adding the real driver.
Also, this is not just a old vs. new compatibility issue. It can also be an on-going kernel config issue. If you fail to enable all the right drivers that are dependencies because you've added them to the DT, then you won't boot any more. A whitelist based on the kernel config would solve that problem. The challenge is how to create the list. The information is already present, just not easily accessed.
Anyway, now I have a clear picture of this problem and need to think some more about it. This particular issue is bit outside of EBBR and this audience. We should discuss with more kernel folks.
Rob
On 04/30/2018 08:36 PM, Rob Herring wrote:
On Fri, Apr 27, 2018 at 4:39 PM, Alexander Graf agraf@suse.de wrote:
Hi Rob,
On 27.04.18 18:40, Rob Herring wrote:
On Fri, Apr 27, 2018 at 2:47 AM, Alexander Graf agraf@suse.de wrote:
On 27.04.18 08:24, Udit Kumar wrote:
Hi There is bit of discussion on linux-efi too , to handle DT update
I guess some members of this forum are active there too.
https://www.spinics.net/lists/linux-efi/msg13700.html
To summaries 1/ Ownership of DTB IMO should be firmware and we should retain this ownership in EBBR as well, Any objections/thoughts ?
I fully agree. On top of that we need to make clear that backward and forward compatibility are not optional.
For that I think we may need to actually give people workable solutions to create device trees that are compatible with multiple levels of kernel support. The main areas I'm aware of that keep breaking are:
- fine grained interrupt controller support
Do you have an example of that? The only thing I can think of is people switching interrupts from the GIC to an always-on, low-power mode custom interrupt controller.
The last time I've seen that breakage was:
https://github.com/raspberrypi/linux/blob/rpi-4.14.y/arch/arm/boot/dts/bcm27...
which indeed does switch interrupts from the GIC to an interrupt muxer behind the GIC.
The problem is that once support for that lands upstream, you will have very little option but to break backwards compatibility today.
This one is unfortunate. It could have been handled better. An interrupt-map property in the aux ctrlr could have mapped the interrupts to GIC without any aux driver. Then when the aux driver lands, it just needs to remove the interrupt-map (on boot).
To do that you would've needed to know that you need the change in the first place ;).
Alternatively, I skimmed thru some discussions of the issue, but I'm not clear why the devices behind the aux controller can't all just treat their interrupts as shared. But that would be a simple change to the drivers' irq handlers, so I'm probably missing something. If that worked, then the DT would never need to change.
I guess whether this could have been handled better depends if folks knowingly ignored the issue or this was found after upstreaming support. The latter case may be unavoidable, but maybe we can make it
I think most of these cases are the latter.
rare enough we only need overlays in some exceptions. Whether we try to be stricter and do better up front or have some overlay based solution, either one is going to require folks be aware of the issues and effort to avoid them.
Yes, but with the overlay approach we can temper it up after the fact :).
I personally think what we need to do is have a flag day event. I think ideally that would be a move of the dts file out of the Linux tree into a common git repository. Once it's there, we'd need scripts that ensure backward compatibility all the way back to whatever was current at the flag day.
Projects like U-Boot (or maybe even other OSs) could then have a git submodule on that repo and be sure that they can rely on these device trees to be stable.
- clock support
Are there cases other than going from fixed, fake clocks to a real clock controller node. I'm inclined to stop allowing people to do that. A better way this could be done is just provide a clock controller driver with a bunch of fixed clocks. Then the switch from the dumb driver to the real driver is just a kernel change.
Going from fixed to in-kernel clock driver is one case I'm aware of, yes.
The other one I can think of right now is going from one clock interface to another. For example if you configure clocks using a firmware interface and that firmware interface moves to a newer, more shiny interface (say proprietary to SCMI). That switch may also regress users of the device tree.
If not only the clocks but the whole firmware interface changes, then I don't see how a stable DT matters at that point.
Firmware could expose both interfaces, but if the DT only mentions the "new" one, OSs that used to work will cease to work.
I think Andre had other examples of where DT compatibility broke, so I'll let him chime in as well.
- power domain support
Example?
https://github.com/Xilinx/linux-xlnx/blob/master/arch/arm64/boot/dts/xilinx/...
Booting with the power-domains properties included in the device tree fails for me, because the driver framework considers them critical to load a device and just errors out with -EDEFER.
- pinctrl support
This would be the firmware initially does all the pin setup, then you move it to DT and drop the setup from the firmware? Otherwise I don't understand the problem in this case. We'd start with no pinctrl and then add it to the DT. Why wouldn't the kernel just ignore it?
Because it doesn't. The linux driver framework as it works today looks at the pinctrl parent for a specific device and if it can't find it loaded, throws -EDEFER for the driver getting loaded. I suppose the rationale behind that is that pinctrl drivers could be modules that get loaded after the driver you want to load.
Every time a device tree changes in any of the above, that usually ends up in backwards incompatibility.
TBC, you're talking about new dtb with old kernels. We've mainly cared
Yes. Both really.
about old dtbs and new kernels. So first we should agree the former is important too. I do, because simply you wouldn't want a BIOS update to make your PC stop booting your already installed OS.
Yes.
I'd like to solve this with policy and good practice before we try to apply technical solutions on top of a mess.
I'm not holding my hopes up, but we can try ;). Another full dimension of messiness are downstream, unapproved bindings that some times can't live in the same device tree as their upstream counterparts once upstream chose a different route.
For pinctrl and power domain, I suppose we could have a compatible whitelist in the kernel and instead of -EDEFER just consider the targets immutable if we hit none of the whitelisted strings.
Yes, I had similar thought as well, but was thinking a blacklist rather. We could modify the DT on boot to disable the listed nodes. Or various subsystems could have lists of compatibles to not return -EPROBE_DEFER. The one complexity with a blacklist is we'd need patches for stable that may not be in mainline unless we add the compatible to the blacklist and then immediately remove it when adding the real driver.
Patches to old kernels won't work for us, as the "installer" kernel is fixed at the time it gets put onto an iso. So while we can update the kernel later on, initial installation is expected to be possible with an old, unpatched kernel.
Also, this is not just a old vs. new compatibility issue. It can also be an on-going kernel config issue. If you fail to enable all the right drivers that are dependencies because you've added them to the DT, then you won't boot any more. A whitelist based on the kernel config would solve that problem. The challenge is how to create the list. The information is already present, just not easily accessed.
Yeah, especially when we add modules to the list :(.
Anyway, now I have a clear picture of this problem and need to think some more about it. This particular issue is bit outside of EBBR and this audience. We should discuss with more kernel folks.
Yes, thanks a lot for kicking that off!
Alex
On Thu, May 3, 2018 at 9:29 AM, Alexander Graf agraf@suse.de wrote:
On 04/30/2018 08:36 PM, Rob Herring wrote:
On Fri, Apr 27, 2018 at 4:39 PM, Alexander Graf agraf@suse.de wrote:
Hi Rob,
On 27.04.18 18:40, Rob Herring wrote:
On Fri, Apr 27, 2018 at 2:47 AM, Alexander Graf agraf@suse.de wrote:
On 27.04.18 08:24, Udit Kumar wrote:
Hi There is bit of discussion on linux-efi too , to handle DT update
I guess some members of this forum are active there too.
https://www.spinics.net/lists/linux-efi/msg13700.html
To summaries 1/ Ownership of DTB IMO should be firmware and we should retain this ownership in EBBR as well, Any objections/thoughts ?
I fully agree. On top of that we need to make clear that backward and forward compatibility are not optional.
For that I think we may need to actually give people workable solutions to create device trees that are compatible with multiple levels of kernel support. The main areas I'm aware of that keep breaking are:
- fine grained interrupt controller support
Do you have an example of that? The only thing I can think of is people switching interrupts from the GIC to an always-on, low-power mode custom interrupt controller.
The last time I've seen that breakage was:
https://github.com/raspberrypi/linux/blob/rpi-4.14.y/arch/arm/boot/dts/bcm27...
which indeed does switch interrupts from the GIC to an interrupt muxer behind the GIC.
The problem is that once support for that lands upstream, you will have very little option but to break backwards compatibility today.
This one is unfortunate. It could have been handled better. An interrupt-map property in the aux ctrlr could have mapped the interrupts to GIC without any aux driver. Then when the aux driver lands, it just needs to remove the interrupt-map (on boot).
To do that you would've needed to know that you need the change in the first place ;).
Actually, I think we can still solve this. Add the interrupt-map now. Then when the aux driver lands it has to fixup the interrupt-map.
I think I actually hit the problem when testing my deferred probe patch. I saw a 30 sec delay in the console output when the pl011 driver probed and using the pl011 as the console (they really made a mess of the serial console on RPi 3).
Alternatively, I skimmed thru some discussions of the issue, but I'm not clear why the devices behind the aux controller can't all just treat their interrupts as shared. But that would be a simple change to the drivers' irq handlers, so I'm probably missing something. If that worked, then the DT would never need to change.
I guess whether this could have been handled better depends if folks knowingly ignored the issue or this was found after upstreaming support. The latter case may be unavoidable, but maybe we can make it
I think most of these cases are the latter.
rare enough we only need overlays in some exceptions. Whether we try to be stricter and do better up front or have some overlay based solution, either one is going to require folks be aware of the issues and effort to avoid them.
Yes, but with the overlay approach we can temper it up after the fact :).
I was assuming you did not want to be the one to find all the issues and fix them. You do want upstream to do a better job and avoid some of the issues to begin with, right?
I personally think what we need to do is have a flag day event. I think ideally that would be a move of the dts file out of the Linux tree into a common git repository. Once it's there, we'd need scripts that ensure backward compatibility all the way back to whatever was current at the flag day.
Well, we wouldn't move over platforms that don't have pinctrl, clocks, etc. because they aren't stable and many of your problems go away. You would also still have the same problem that the DT declared stable at the flag day is newer than the kernel you are on and may not work with it.
The problem also is how do you define when things are stable? Lots of folks ask for a way to declare bindings stable vs. unstable. My concern with that is things will never get moved to stable and anything unstable is free to change at will.
Can I have those compatibility checking scripts now please. I'll add them to the kernel because I'd like to know when we break things even if we don't enforce not breaking. Really, I don't see how we'd check compatibility in a script. What's compatible or not depends on the OS.
Projects like U-Boot (or maybe even other OSs) could then have a git submodule on that repo and be sure that they can rely on these device trees to be stable.
- clock support
Are there cases other than going from fixed, fake clocks to a real clock controller node. I'm inclined to stop allowing people to do that. A better way this could be done is just provide a clock controller driver with a bunch of fixed clocks. Then the switch from the dumb driver to the real driver is just a kernel change.
Going from fixed to in-kernel clock driver is one case I'm aware of, yes.
The other one I can think of right now is going from one clock interface to another. For example if you configure clocks using a firmware interface and that firmware interface moves to a newer, more shiny interface (say proprietary to SCMI). That switch may also regress users of the device tree.
If not only the clocks but the whole firmware interface changes, then I don't see how a stable DT matters at that point.
Firmware could expose both interfaces, but if the DT only mentions the "new" one, OSs that used to work will cease to work.
For the declared stable, post flag day DT, the clocks in DT will never change from one controller to another. Changing them doesn't sound stable to me.
I think Andre had other examples of where DT compatibility broke, so I'll let him chime in as well.
- power domain support
Example?
https://github.com/Xilinx/linux-xlnx/blob/master/arch/arm64/boot/dts/xilinx/...
Booting with the power-domains properties included in the device tree fails for me, because the driver framework considers them critical to load a device and just errors out with -EDEFER.
- pinctrl support
This would be the firmware initially does all the pin setup, then you move it to DT and drop the setup from the firmware? Otherwise I don't understand the problem in this case. We'd start with no pinctrl and then add it to the DT. Why wouldn't the kernel just ignore it?
Because it doesn't. The linux driver framework as it works today looks at the pinctrl parent for a specific device and if it can't find it loaded, throws -EDEFER for the driver getting loaded. I suppose the rationale behind that is that pinctrl drivers could be modules that get loaded after the driver you want to load.
Every time a device tree changes in any of the above, that usually ends up in backwards incompatibility.
TBC, you're talking about new dtb with old kernels. We've mainly cared
Yes. Both really.
about old dtbs and new kernels. So first we should agree the former is important too. I do, because simply you wouldn't want a BIOS update to make your PC stop booting your already installed OS.
Yes.
I'd like to solve this with policy and good practice before we try to apply technical solutions on top of a mess.
I'm not holding my hopes up, but we can try ;). Another full dimension of messiness are downstream, unapproved bindings that some times can't live in the same device tree as their upstream counterparts once upstream chose a different route.
For pinctrl and power domain, I suppose we could have a compatible whitelist in the kernel and instead of -EDEFER just consider the targets immutable if we hit none of the whitelisted strings.
Yes, I had similar thought as well, but was thinking a blacklist rather. We could modify the DT on boot to disable the listed nodes. Or various subsystems could have lists of compatibles to not return -EPROBE_DEFER. The one complexity with a blacklist is we'd need patches for stable that may not be in mainline unless we add the compatible to the blacklist and then immediately remove it when adding the real driver.
Patches to old kernels won't work for us, as the "installer" kernel is fixed at the time it gets put onto an iso. So while we can update the kernel later on, initial installation is expected to be possible with an old, unpatched kernel.
Okay, I'll keep that in mind. So any kernel fixes (like my patch) will only fix future installers.
I was considering whether for things like pinctrl, if we should just add a DT property to say the default configuration works. Default could be reset state or firmware initialized state (which could have been the firmware processing the DT pinctrl and configuring everything). Maybe EBBR should define that firmware does setup the pins (most platforms probably do already, just maybe not based on DT) and then should set the flag. I'm trying to be careful and define this as a reflection of the h/w, not a Linux property to disable deferred probe or not. The main problem I see is "works" is not a binary property. I guess it could be fine grained per device, but then that assumes people know if the default works or not. Of course, this would still need a kernel change and won't fix your installers now.
Rob
On Thu, May 3, 2018 at 11:11 AM, Rob Herring robh@kernel.org wrote:
On Thu, May 3, 2018 at 9:29 AM, Alexander Graf agraf@suse.de wrote:
On 04/30/2018 08:36 PM, Rob Herring wrote:
On Fri, Apr 27, 2018 at 4:39 PM, Alexander Graf agraf@suse.de wrote:
Hi Rob,
On 27.04.18 18:40, Rob Herring wrote:
On Fri, Apr 27, 2018 at 2:47 AM, Alexander Graf agraf@suse.de wrote:
On 27.04.18 08:24, Udit Kumar wrote: > > Hi > There is bit of discussion on linux-efi too , to handle DT update > > I guess some members of this forum are active there too. > > https://www.spinics.net/lists/linux-efi/msg13700.html > > To summaries > 1/ Ownership of DTB > IMO should be firmware and we should retain this > ownership in EBBR as well, Any objections/thoughts ?
I fully agree. On top of that we need to make clear that backward and forward compatibility are not optional.
For that I think we may need to actually give people workable solutions to create device trees that are compatible with multiple levels of kernel support. The main areas I'm aware of that keep breaking are:
- fine grained interrupt controller support
Do you have an example of that? The only thing I can think of is people switching interrupts from the GIC to an always-on, low-power mode custom interrupt controller.
The last time I've seen that breakage was:
https://github.com/raspberrypi/linux/blob/rpi-4.14.y/arch/arm/boot/dts/bcm27...
which indeed does switch interrupts from the GIC to an interrupt muxer behind the GIC.
The problem is that once support for that lands upstream, you will have very little option but to break backwards compatibility today.
This one is unfortunate. It could have been handled better. An interrupt-map property in the aux ctrlr could have mapped the interrupts to GIC without any aux driver. Then when the aux driver lands, it just needs to remove the interrupt-map (on boot).
To do that you would've needed to know that you need the change in the first place ;).
Actually, I think we can still solve this. Add the interrupt-map now. Then when the aux driver lands it has to fixup the interrupt-map.
Scrap that...
I think I actually hit the problem when testing my deferred probe patch. I saw a 30 sec delay in the console output when the pl011 driver probed and using the pl011 as the console (they really made a mess of the serial console on RPi 3).
Alternatively, I skimmed thru some discussions of the issue, but I'm not clear why the devices behind the aux controller can't all just treat their interrupts as shared. But that would be a simple change to the drivers' irq handlers, so I'm probably missing something. If that worked, then the DT would never need to change.
Fix posted[1]. :) In fact the IRQ handlers are shared already, there's just a bug in the handler.
Rob
On 03.05.18 18:11, Rob Herring wrote:
On Thu, May 3, 2018 at 9:29 AM, Alexander Graf agraf@suse.de wrote:
On 04/30/2018 08:36 PM, Rob Herring wrote:
On Fri, Apr 27, 2018 at 4:39 PM, Alexander Graf agraf@suse.de wrote:
Hi Rob,
On 27.04.18 18:40, Rob Herring wrote:
On Fri, Apr 27, 2018 at 2:47 AM, Alexander Graf agraf@suse.de wrote:
On 27.04.18 08:24, Udit Kumar wrote: > > Hi > There is bit of discussion on linux-efi too , to handle DT update > > I guess some members of this forum are active there too. > > https://www.spinics.net/lists/linux-efi/msg13700.html > > To summaries > 1/ Ownership of DTB > IMO should be firmware and we should retain this > ownership in EBBR as well, Any objections/thoughts ?
I fully agree. On top of that we need to make clear that backward and forward compatibility are not optional.
For that I think we may need to actually give people workable solutions to create device trees that are compatible with multiple levels of kernel support. The main areas I'm aware of that keep breaking are:
- fine grained interrupt controller support
Do you have an example of that? The only thing I can think of is people switching interrupts from the GIC to an always-on, low-power mode custom interrupt controller.
The last time I've seen that breakage was:
https://github.com/raspberrypi/linux/blob/rpi-4.14.y/arch/arm/boot/dts/bcm27...
which indeed does switch interrupts from the GIC to an interrupt muxer behind the GIC.
The problem is that once support for that lands upstream, you will have very little option but to break backwards compatibility today.
This one is unfortunate. It could have been handled better. An interrupt-map property in the aux ctrlr could have mapped the interrupts to GIC without any aux driver. Then when the aux driver lands, it just needs to remove the interrupt-map (on boot).
To do that you would've needed to know that you need the change in the first place ;).
Actually, I think we can still solve this. Add the interrupt-map now. Then when the aux driver lands it has to fixup the interrupt-map> I think I actually hit the problem when testing my deferred probe patch. I saw a 30 sec delay in the console output when the pl011 driver probed and using the pl011 as the console (they really made a mess of the serial console on RPi 3).
Alternatively, I skimmed thru some discussions of the issue, but I'm not clear why the devices behind the aux controller can't all just treat their interrupts as shared. But that would be a simple change to the drivers' irq handlers, so I'm probably missing something. If that worked, then the DT would never need to change.
I guess whether this could have been handled better depends if folks knowingly ignored the issue or this was found after upstreaming support. The latter case may be unavoidable, but maybe we can make it
I think most of these cases are the latter.
rare enough we only need overlays in some exceptions. Whether we try to be stricter and do better up front or have some overlay based solution, either one is going to require folks be aware of the issues and effort to avoid them.
Yes, but with the overlay approach we can temper it up after the fact :).
I was assuming you did not want to be the one to find all the issues and fix them. You do want upstream to do a better job and avoid some of the issues to begin with, right?
Oh I don't want to be the one, but I don't believe in humans finding 100% of all errors. So I want to also have an answer for the 10% that fall through the cracks.
I personally think what we need to do is have a flag day event. I think ideally that would be a move of the dts file out of the Linux tree into a common git repository. Once it's there, we'd need scripts that ensure backward compatibility all the way back to whatever was current at the flag day.
Well, we wouldn't move over platforms that don't have pinctrl, clocks, etc. because they aren't stable and many of your problems go away. You
That's something we can easily define, true.
would also still have the same problem that the DT declared stable at the flag day is newer than the kernel you are on and may not work with it.
Yes, but that's normal for distros. SLES10 doesn't run all that great on Skylake either. People accept the fact that their new system may need a new OS. What they don't like is when we regress - so their hardware worked before and now doesn't anymore. Either by updating firmware or the OS.
The problem also is how do you define when things are stable? Lots of folks ask for a way to declare bindings stable vs. unstable. My concern with that is things will never get moved to stable and anything unstable is free to change at will.
We can always mandate the DT has to be in the stable tree to claim EBBR compatibility. Then distros could claim they only provide "good" support for EBBR platforms. And then people will have an incentive to make their DTs stable.
For SLES for example, we could tell SoC vendors we won't support them until their DT is stable. Today we don't have any handle on that. With a policy like the above it would be something easy for them to understand and follow.
Can I have those compatibility checking scripts now please. I'll add them to the kernel because I'd like to know when we break things even if we don't enforce not breaking. Really, I don't see how we'd check compatibility in a script. What's compatible or not depends on the OS.
Well, we have the list in this thread already. Changing a compatible of a device for example should at least spawn a checkpatch warning. If we don't manage to fix the clock and pinctrl dependency issue, we can create checkpatch errors when adding clock dependencies.
Projects like U-Boot (or maybe even other OSs) could then have a git submodule on that repo and be sure that they can rely on these device trees to be stable.
- clock support
Are there cases other than going from fixed, fake clocks to a real clock controller node. I'm inclined to stop allowing people to do that. A better way this could be done is just provide a clock controller driver with a bunch of fixed clocks. Then the switch from the dumb driver to the real driver is just a kernel change.
Going from fixed to in-kernel clock driver is one case I'm aware of, yes.
The other one I can think of right now is going from one clock interface to another. For example if you configure clocks using a firmware interface and that firmware interface moves to a newer, more shiny interface (say proprietary to SCMI). That switch may also regress users of the device tree.
If not only the clocks but the whole firmware interface changes, then I don't see how a stable DT matters at that point.
Firmware could expose both interfaces, but if the DT only mentions the "new" one, OSs that used to work will cease to work.
For the declared stable, post flag day DT, the clocks in DT will never change from one controller to another. Changing them doesn't sound stable to me.
I agree, but FWIW at least one SoC vendor is planning exactly that ;). It basically means they can only declare their DT stable after the transition or not change the fw interface at all I guess.
I think Andre had other examples of where DT compatibility broke, so I'll let him chime in as well.
- power domain support
Example?
https://github.com/Xilinx/linux-xlnx/blob/master/arch/arm64/boot/dts/xilinx/...
Booting with the power-domains properties included in the device tree fails for me, because the driver framework considers them critical to load a device and just errors out with -EDEFER.
- pinctrl support
This would be the firmware initially does all the pin setup, then you move it to DT and drop the setup from the firmware? Otherwise I don't understand the problem in this case. We'd start with no pinctrl and then add it to the DT. Why wouldn't the kernel just ignore it?
Because it doesn't. The linux driver framework as it works today looks at the pinctrl parent for a specific device and if it can't find it loaded, throws -EDEFER for the driver getting loaded. I suppose the rationale behind that is that pinctrl drivers could be modules that get loaded after the driver you want to load.
Every time a device tree changes in any of the above, that usually ends up in backwards incompatibility.
TBC, you're talking about new dtb with old kernels. We've mainly cared
Yes. Both really.
about old dtbs and new kernels. So first we should agree the former is important too. I do, because simply you wouldn't want a BIOS update to make your PC stop booting your already installed OS.
Yes.
I'd like to solve this with policy and good practice before we try to apply technical solutions on top of a mess.
I'm not holding my hopes up, but we can try ;). Another full dimension of messiness are downstream, unapproved bindings that some times can't live in the same device tree as their upstream counterparts once upstream chose a different route.
For pinctrl and power domain, I suppose we could have a compatible whitelist in the kernel and instead of -EDEFER just consider the targets immutable if we hit none of the whitelisted strings.
Yes, I had similar thought as well, but was thinking a blacklist rather. We could modify the DT on boot to disable the listed nodes. Or various subsystems could have lists of compatibles to not return -EPROBE_DEFER. The one complexity with a blacklist is we'd need patches for stable that may not be in mainline unless we add the compatible to the blacklist and then immediately remove it when adding the real driver.
Patches to old kernels won't work for us, as the "installer" kernel is fixed at the time it gets put onto an iso. So while we can update the kernel later on, initial installation is expected to be possible with an old, unpatched kernel.
Okay, I'll keep that in mind. So any kernel fixes (like my patch) will only fix future installers.
Yes, but that's ok. Few people expect new hardware to work flawlessly with old OSs.
What we shouldn't do is block the road for enabling boards or SoCs that would easily work with an older kernel though.
On x86 for example, while old SLES releases may not have all drivers needed to drive all devices of a new system, most parts are generic enough or handled via ACPI that you can make a system work by adding kernel modules.
I was considering whether for things like pinctrl, if we should just add a DT property to say the default configuration works. Default
That's tricky with overlays, no?
could be reset state or firmware initialized state (which could have been the firmware processing the DT pinctrl and configuring everything). Maybe EBBR should define that firmware does setup the pins (most platforms probably do already, just maybe not based on DT) and then should set the flag. I'm trying to be careful and define this as a reflection of the h/w, not a Linux property to disable deferred probe or not. The main problem I see is "works" is not a binary property. I guess it could be fine grained per device, but then that assumes people know if the default works or not. Of course, this would still need a kernel change and won't fix your installers now.
Yes, the problem is terribly complicated.
Honestly, if you want my opinion, I would move pinctrl and clock frameworks all the way up to EL3 and leave it to firmware to take care of them always. That interface would then need to be 100% stable.
That way we get back to the point above where enabling a new SoC becomes a matter of adding drivers to an older OS, because we don't rely on in-kernel drivers to drive simple things.
Alex
Hi Rob
The Android way of handling overlays is very much rooted in how the Android ecosystem works.
Yes, idea is if we can leverage something from this.
We should probably have wider discussion and decision on to what extent does EBBR address/care about/work with Android? On the one hand, I don't think Android requires anything that's specifically incompatible with EBBR if some wants to follow EBBR and use Android.
On part of requirement, IMO, we should define how device-tree update will be handled. The EBBR document should mention, -how kernel will provide overlays. - where and how those will be stored - and how those will be applied
Defined EBBR may/may not work for Android. But I guess this is not primary goal of EBBR.
OTOH, we can't define any requirements for Android in EBBR. Google will define things to the extent they want and vendors will follow that only to the extent they have to.
Yes
To store this information in partition, options we have 1/ Run time variables
You mean EFI variables? We could certainly have a driver in firmware that reads certain EFI variables to apply dtbos from.
2/ Some driver in Linux writing to DTBO partition
What is a DTBO partition?
The Android way. Everything can be solved with another partition. :)
Rob
Hi Alex
-----Original Message----- From: Alexander Graf [mailto:agraf@suse.de] Sent: Friday, April 27, 2018 1:18 PM To: Udit Kumar udit.kumar@nxp.com; William Mills wmills@ti.com Cc: boot-architecture@lists.linaro.org; nd@arm.com; Rod Dorris rod.dorris@nxp.com; arm.ebbr-discuss@arm.com Subject: Re: DT handling, [Ref Linux-Efi]
On 27.04.18 08:24, Udit Kumar wrote:
Hi There is bit of discussion on linux-efi too , to handle DT update
I guess some members of this forum are active there too.
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww .spinics.net%2Flists%2Flinux-efi%2Fmsg13700.html&data=02%7C01%7Cudit.k
umar%40nxp.com%7Cd320a10fef8446e2aec308d5ac132cd3%7C686ea1d3bc2b 4c6fa9
2cd99c5c301635%7C0%7C0%7C636604120716683231&sdata=USXfARVLtgi2umi %2BSw
LxLwJPqiDztVuNGpTGz09T0q0%3D&reserved=0
To summaries 1/ Ownership of DTB IMO should be firmware and we should retain this ownership in EBBR as well, Any objections/thoughts ?
I fully agree. On top of that we need to make clear that backward and forward compatibility are not optional.
This will be on kernel drivers, not to break the contract.
For that I think we may need to actually give people workable solutions to create device trees that are compatible with multiple levels of kernel support. The main areas I'm aware of that keep breaking are:
- fine grained interrupt controller support
- clock support
- power domain support
- pinctrl support
In-line with similar problems I am facing currently, new kernel doesn't boot with old uefi firmware. Kernel device tree is updated and therefore driver is, the combination of old device tree and new kernel is not working, This comes down to mainly on management of device trees. Here if we consider, firmware and kernel development is independent then driver should run with same level of functions with new kernel and old device tree. New functions/binding added in driver couldn't be used.
Every time a device tree changes in any of the above, that usually ends up in backwards incompatibility.
My idea to solve that would be to basically create a device tree that has self- contained overlays that only trigger when certain feature strings are available.
This will be good, but can we predict what will be overlays ?
That way the base device tree could for example contain fixed clocks, but an overlay can get applied when the clock driver is enabled in the kernel configuration. That overlay would then enable the kernel to drive clocks.
With assumption, firmware has nothing to fix for overlays. i.e clock-frequency (if given in overlay)
Further down, we could even extend dtc with annotations that indicate "this property should only be exposed when feature string X is available" to not force people to write overlays inside the device tree.
Update 1/ Updating whole device tree from OS [Capsule update can be used ]
I think the device tree should be part of firmware. If you need to update it, update your firmware (or a firmware specific method, not specified by EBBR).
Edk2, supports DTB as part of firmware and separate data blob as well.. I think, this is work in progress by Bhupesh to update only DTB.
2/ Just modifying the device tree DTBO
Yes, dtbo support in the boot chain definitely makes sense.
My preferred way to handle DTBO in firmware will be https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsou
rce.android.com%2Fdevices%2Farchitecture%2Fdto%2Fmultiple&data=02%7C0 1
%7Cudit.kumar%40nxp.com%7Cd320a10fef8446e2aec308d5ac132cd3%7C686e a1d3b
c2b4c6fa92cd99c5c301635%7C0%7C0%7C636604120716683231&sdata=a8rseb EngRA
dKP%2Fs3wFSfqMHYrOf4hn6JNfQpXdFgzU%3D&reserved=0 See picture Runtime DTO implementation for multiple DTs
To store this information in partition, options we have 1/ Run time variables
You mean EFI variables? We could certainly have a driver in firmware that reads certain EFI variables to apply dtbos from.
Yes, there will be a need of generic driver which reads variable and merge DTB and DTBO.
2/ Some driver in Linux writing to DTBO partition
What is a DTBO partition?
This is defined in android, if there is agreement we can define something like this or other.
3/ Some other way ??
In a lot of cases I think we will end up with enumerable extensions to systems that will want to have a dtbo applied by their respective driver. As outlined in the notes from last week, that driver could expose availability of such an extension to the OS (grub) which could then apply an OS provided dtbo.
Could be, if grub does not need platform information for these DTBO.
I am not sure, if distro are updating device tree which is default shipped with
board ??
I would prefer they didn't :). Right now we leave people little chance, because device trees keep changing incompatibly. If we get rid of that problem, there will be very little incentive by distros to ship device trees.
Thanks Udit
Some thoughts,
-----Original Message----- From: arm.ebbr-discuss-bounces@arm.com [mailto:arm.ebbr-discuss- bounces@arm.com] On Behalf Of Udit Kumar Sent: Saturday, April 28, 2018 4:07 PM To: Alexander Graf agraf@suse.de; William Mills wmills@ti.com Cc: boot-architecture@lists.linaro.org; nd@arm.com; Rod Dorris rod.dorris@nxp.com; arm.ebbr-discuss@arm.com Subject: Re: [Arm.ebbr-discuss] DT handling, [Ref Linux-Efi]
Hi Alex
-----Original Message----- From: Alexander Graf [mailto:agraf@suse.de] Sent: Friday, April 27, 2018 1:18 PM To: Udit Kumar udit.kumar@nxp.com; William Mills wmills@ti.com Cc: boot-architecture@lists.linaro.org; nd@arm.com; Rod Dorris rod.dorris@nxp.com; arm.ebbr-discuss@arm.com Subject: Re: DT handling, [Ref Linux-Efi]
On 27.04.18 08:24, Udit Kumar wrote:
Hi There is bit of discussion on linux-efi too , to handle DT update
I guess some members of this forum are active there too.
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw
ww .spinics.net%2Flists%2Flinux-
efi%2Fmsg13700.html&data=02%7C01%7Cudit
.k
umar%40nxp.com%7Cd320a10fef8446e2aec308d5ac132cd3%7C686ea1d3bc2 b
4c6fa9
2cd99c5c301635%7C0%7C0%7C636604120716683231&sdata=USXfARVLtgi2um i
%2BSw
LxLwJPqiDztVuNGpTGz09T0q0%3D&reserved=0
To summaries 1/ Ownership of DTB IMO should be firmware and we should retain this ownership in EBBR as well, Any objections/thoughts ?
I fully agree. On top of that we need to make clear that backward and forward compatibility are not optional.
This will be on kernel drivers, not to break the contract.
For that I think we may need to actually give people workable solutions to create device trees that are compatible with multiple levels of
kernel support.
The main areas I'm aware of that keep breaking are:
- fine grained interrupt controller support
- clock support
- power domain support
- pinctrl support
In-line with similar problems I am facing currently, new kernel doesn't boot with old uefi firmware. Kernel device tree is updated and therefore driver is, the combination of old device tree and new kernel is not working, This comes down to mainly on management of device trees. Here if we consider, firmware and kernel development is independent then driver should run with same level of functions with new kernel and old device tree. New functions/binding added in driver couldn't be used.
Every time a device tree changes in any of the above, that usually ends up in backwards incompatibility.
My idea to solve that would be to basically create a device tree that has self- contained overlays that only trigger when certain feature strings
are available.
This will be good, but can we predict what will be overlays ?
That way the base device tree could for example contain fixed clocks, but an overlay can get applied when the clock driver is enabled in the kernel configuration. That overlay would then enable the kernel to drive
clocks.
With assumption, firmware has nothing to fix for overlays. i.e clock-frequency (if given in overlay)
Further down, we could even extend dtc with annotations that indicate "this property should only be exposed when feature string X is available" to not force people to write overlays inside the device tree.
Update 1/ Updating whole device tree from OS [Capsule update can be used ]
I think the device tree should be part of firmware. If you need to update it, update your firmware (or a firmware specific method, not
specified by EBBR).
Edk2, supports DTB as part of firmware and separate data blob as well.. I think, this is work in progress by Bhupesh to update only DTB.
2/ Just modifying the device tree DTBO
Yes, dtbo support in the boot chain definitely makes sense.
My preferred way to handle DTBO in firmware will be
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fs
ou
rce.android.com%2Fdevices%2Farchitecture%2Fdto%2Fmultiple&data=02%7 C0
1
%7Cudit.kumar%40nxp.com%7Cd320a10fef8446e2aec308d5ac132cd3%7C686 e
a1d3b
c2b4c6fa92cd99c5c301635%7C0%7C0%7C636604120716683231&sdata=a8rseb
EngRA
dKP%2Fs3wFSfqMHYrOf4hn6JNfQpXdFgzU%3D&reserved=0 See picture Runtime DTO implementation for multiple DTs
To store this information in partition, options we have 1/ Run time variables
You mean EFI variables? We could certainly have a driver in firmware that reads certain EFI variables to apply dtbos from.
Yes, there will be a need of generic driver which reads variable and merge DTB and DTBO.
We probably don't need to provide a genetic DT driver in UEFI U-Boot, instead, we will have to mention how SoC/platform vendors publish DTB/DTBO in EBBR spec. For example, The EFI_CONFIGURATION_TABLE in EFI System table could be used to publish the pointer to DTB and DTBO. Declare two GUIDs in EBBR, one for DTB another for DTBO. OS boot loader is responsible to merge DTB/DTBO according DTB/DTBO discovered in EFI Configuration Table. To read DT from EFI variable into memory, memory map to DT in EEPROM or other mechanisms is platform implementation. No matter which approach, DT producer has to create configuration table in EFI system table follow the data structure defined in EBBR. Another way instead of using GUID in configuration table is to publish DTB/DTBO in EFI device path, this way is more UEFI oriented IMO. However, we have to defined corresponding device path node in UEFI spec for DT. Similar to using system configuration table. DT producer has to install EFI device path for either DTB or DTBO. Then OS boot loaders locate those EFI device paths of DTB and DTBO and merge it.
My thoughts based on UEFI perspective.
2/ Some driver in Linux writing to DTBO partition
What is a DTBO partition?
This is defined in android, if there is agreement we can define something like this or other.
3/ Some other way ??
In a lot of cases I think we will end up with enumerable extensions to systems that will want to have a dtbo applied by their respective driver. As outlined in the notes from last week, that driver could expose availability of such an extension to the OS (grub) which could then
apply an OS provided dtbo.
Could be, if grub does not need platform information for these DTBO.
I am not sure, if distro are updating device tree which is default shipped with
board ??
I would prefer they didn't :). Right now we leave people little chance, because device trees keep changing incompatibly. If we get rid of that problem, there will be very little incentive by distros to ship device
trees.
Thanks Udit
Arm.ebbr-discuss mailing list Arm.ebbr-discuss@arm.com
On 04/29/2018 11:36 PM, Chang, Abner (HPS SW/FW Technologist) wrote:
Some thoughts,
-----Original Message----- From: arm.ebbr-discuss-bounces@arm.com [mailto:arm.ebbr-discuss- bounces@arm.com] On Behalf Of Udit Kumar Sent: Saturday, April 28, 2018 4:07 PM To: Alexander Graf agraf@suse.de; William Mills wmills@ti.com Cc: boot-architecture@lists.linaro.org; nd@arm.com; Rod Dorris rod.dorris@nxp.com; arm.ebbr-discuss@arm.com Subject: Re: [Arm.ebbr-discuss] DT handling, [Ref Linux-Efi]
Hi Alex
-----Original Message----- From: Alexander Graf [mailto:agraf@suse.de] Sent: Friday, April 27, 2018 1:18 PM To: Udit Kumar udit.kumar@nxp.com; William Mills wmills@ti.com Cc: boot-architecture@lists.linaro.org; nd@arm.com; Rod Dorris rod.dorris@nxp.com; arm.ebbr-discuss@arm.com Subject: Re: DT handling, [Ref Linux-Efi]
On 27.04.18 08:24, Udit Kumar wrote:
Hi There is bit of discussion on linux-efi too , to handle DT update
I guess some members of this forum are active there too.
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw
ww .spinics.net%2Flists%2Flinux-
efi%2Fmsg13700.html&data=02%7C01%7Cudit
.k
umar%40nxp.com%7Cd320a10fef8446e2aec308d5ac132cd3%7C686ea1d3bc2 b
4c6fa9
2cd99c5c301635%7C0%7C0%7C636604120716683231&sdata=USXfARVLtgi2um i
%2BSw
LxLwJPqiDztVuNGpTGz09T0q0%3D&reserved=0
To summaries 1/ Ownership of DTB IMO should be firmware and we should retain this ownership in EBBR as well, Any objections/thoughts ?
I fully agree. On top of that we need to make clear that backward and forward compatibility are not optional.
This will be on kernel drivers, not to break the contract.
For that I think we may need to actually give people workable solutions to create device trees that are compatible with multiple levels of
kernel support.
The main areas I'm aware of that keep breaking are:
- fine grained interrupt controller support
- clock support
- power domain support
- pinctrl support
In-line with similar problems I am facing currently, new kernel doesn't boot with old uefi firmware. Kernel device tree is updated and therefore driver is, the combination of old device tree and new kernel is not working, This comes down to mainly on management of device trees. Here if we consider, firmware and kernel development is independent then driver should run with same level of functions with new kernel and old device tree. New functions/binding added in driver couldn't be used.
Every time a device tree changes in any of the above, that usually ends up in backwards incompatibility.
My idea to solve that would be to basically create a device tree that has self- contained overlays that only trigger when certain feature strings
are available.
This will be good, but can we predict what will be overlays ?
That way the base device tree could for example contain fixed clocks, but an overlay can get applied when the clock driver is enabled in the kernel configuration. That overlay would then enable the kernel to drive
clocks.
With assumption, firmware has nothing to fix for overlays. i.e clock-frequency (if given in overlay)
Further down, we could even extend dtc with annotations that indicate "this property should only be exposed when feature string X is available" to not force people to write overlays inside the device tree.
Update 1/ Updating whole device tree from OS [Capsule update can be used ]
I think the device tree should be part of firmware. If you need to update it, update your firmware (or a firmware specific method, not
specified by EBBR).
Edk2, supports DTB as part of firmware and separate data blob as well.. I think, this is work in progress by Bhupesh to update only DTB.
2/ Just modifying the device tree DTBO
Yes, dtbo support in the boot chain definitely makes sense.
My preferred way to handle DTBO in firmware will be
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fs
ou
rce.android.com%2Fdevices%2Farchitecture%2Fdto%2Fmultiple&data=02%7 C0
1
%7Cudit.kumar%40nxp.com%7Cd320a10fef8446e2aec308d5ac132cd3%7C686 e
a1d3b
c2b4c6fa92cd99c5c301635%7C0%7C0%7C636604120716683231&sdata=a8rseb
EngRA
dKP%2Fs3wFSfqMHYrOf4hn6JNfQpXdFgzU%3D&reserved=0 See picture Runtime DTO implementation for multiple DTs
To store this information in partition, options we have 1/ Run time variables
You mean EFI variables? We could certainly have a driver in firmware that reads certain EFI variables to apply dtbos from.
Yes, there will be a need of generic driver which reads variable and merge DTB and DTBO.
We probably don't need to provide a genetic DT driver in UEFI U-Boot, instead, we will have to mention how SoC/platform vendors publish DTB/DTBO in EBBR spec. For example, The EFI_CONFIGURATION_TABLE in EFI System table could be used to publish the pointer to DTB and DTBO. Declare two GUIDs in EBBR, one for DTB another for DTBO. OS boot loader is responsible to merge DTB/DTBO according DTB/DTBO discovered in EFI Configuration Table. To read DT from EFI variable into memory, memory map to DT in EEPROM or other mechanisms is platform implementation. No matter which approach, DT producer has to create configuration table in EFI system table follow the data structure defined in EBBR. Another way instead of using GUID in configuration table is to publish DTB/DTBO in EFI device path, this way is more UEFI oriented IMO. However, we have to defined corresponding device path node in UEFI spec for DT. Similar to using system configuration table. DT producer has to install EFI device path for either DTB or DTBO. Then OS boot loaders locate those EFI device paths of DTB and DTBO and merge it.
My thoughts based on UEFI perspective.
Point 1:
As brought up before, we need to figure out how much of this is in EBBR. If the firmware applies the overlays and the OS only updates the DTO/DTBO via capsule update, then it is not really part of the OS interface.
On the other hand, I do see value in the OS having some viability into this data for finer grain updates. If the firmware package for every variant of the BeagleBone has to be updated for each cape (daughter card) that comes out, that is a lot of churn.
However, currently having one DT that works with every kernel version that may run, I would call aspirational. I agree with the goal but is it achievable in 6 months? Should we not be looking at a solution for EBBR.0?
Likewise, we don't want every vendor using u-boot to do it a different way so even if not part of the Firmware / OS interface, there is value in making recommendations for structure. Should this be in EBBR or devicetree.org? I don't know but we need to discuss somewhere and it might as well be this group.
Point 2:
The Android binary format is not appropriate for general use. It is defined only for their model where there are exactly two DT to be combined, one that defines the SOC and one that defines the "board" aka "phone". The format defines only one axis of selection and thus even the SOC DTs and the board DTs cannot share a partition.
I agree with the principal of looking at prior art and defacto standards but we should not adopt something that is flawed.
I also think the Android practice of dedicating a whole partition to each file is very awkward. The firmware already has to be smart enough to read the EFI application from the EFI partition so having extra files in this partition should not a burden.
Bill
We probably don't need to provide a genetic DT driver in UEFI U-Boot, instead, we will have to mention how SoC/platform vendors publish DTB/DTBO in EBBR spec. For example, The EFI_CONFIGURATION_TABLE in EFI System table could be used to publish the pointer to DTB and DTBO. Declare two GUIDs in EBBR, one for DTB another for DTBO. OS boot loader is responsible to merge DTB/DTBO according DTB/DTBO discovered in EFI Configuration Table. To read DT from EFI variable into memory, memory map to DT in EEPROM or other mechanisms is platform implementation. No matter which approach, DT producer has to create configuration table in EFI system table follow the data structure defined in EBBR. Another way instead of using GUID in configuration table is to publish DTB/DTBO in EFI device path, this way is more UEFI oriented IMO. However, we have to defined corresponding device path node in UEFI spec for DT. Similar to using system configuration table. DT producer has to install EFI device path for either DTB or DTBO. Then OS boot loaders locate those EFI device paths of DTB and DTBO and merge it.
We are adding a requirement on OS boot loaders to merge it. IMO, merging should be done by firmware itself. In case, we want to host multiple distribution at same time, then this is likely to go with OS boot loaders
-----Original Message----- From: Udit Kumar [mailto:udit.kumar@nxp.com] Sent: Wednesday, May 02, 2018 12:26 PM To: Chang, Abner (HPS SW/FW Technologist) abner.chang@hpe.com; Alexander Graf agraf@suse.de; William Mills wmills@ti.com Cc: boot-architecture@lists.linaro.org; nd@arm.com; Rod Dorris rod.dorris@nxp.com; arm.ebbr-discuss@arm.com Subject: RE: DT handling, [Ref Linux-Efi]
We probably don't need to provide a genetic DT driver in UEFI U-Boot, instead, we will have to mention how SoC/platform vendors publish DTB/DTBO in EBBR spec. For example, The EFI_CONFIGURATION_TABLE in EFI System table could be used to publish the pointer to DTB and DTBO. Declare two GUIDs in EBBR, one for DTB another for DTBO. OS boot loader is responsible to merge DTB/DTBO according DTB/DTBO discovered in EFI Configuration Table. To read DT from EFI variable into memory, memory map to DT in EEPROM or other mechanisms is platform implementation. No matter which approach, DT producer has to create configuration table in EFI system table follow the data structure defined in EBBR. Another way instead of using GUID in configuration table is to publish DTB/DTBO in EFI device path, this way is more UEFI oriented IMO. However, we have to defined corresponding device path node in UEFI spec for DT. Similar to using system configuration table. DT producer has to install EFI device path for either DTB or DTBO. Then OS boot loaders locate those EFI device paths of DTB and DTBO and merge it.
We are adding a requirement on OS boot loaders to merge it. IMO, merging should be done by firmware itself. In case, we want to host multiple distribution at same time, then this is likely to go with OS boot loaders
That is fine to merge DT by firmware, we still can standardize how UBoot merges DT in EBBR. For example, SoC and other platform UBoot drivers produce their DT or DTO in their own drivers. UBoot provides a centralized EFI DT driver to collect DT/DTO from either EFI system configuration table or EFI device path. Then this centralized EFI DT driver produces the pointer to point to final DT in EFI system configuration table. OS boot loader cab just check EFI system configuration table to retrieve DT, something like this.
Abner
boot-architecture@lists.linaro.org
-
Alexander Graf -
Chang, Abner (HPS SW/FW Technologist) -
Rob Herring -
Rob Herring -
Udit Kumar -
William Mills