On Thu, May 3, 2018 at 9:29 AM, Alexander Graf agraf@suse.de wrote:
On 04/30/2018 08:36 PM, Rob Herring wrote:
On Fri, Apr 27, 2018 at 4:39 PM, Alexander Graf agraf@suse.de wrote:
Hi Rob,
On 27.04.18 18:40, Rob Herring wrote:
On Fri, Apr 27, 2018 at 2:47 AM, Alexander Graf agraf@suse.de wrote:
On 27.04.18 08:24, Udit Kumar wrote:
Hi There is bit of discussion on linux-efi too , to handle DT update
I guess some members of this forum are active there too.
https://www.spinics.net/lists/linux-efi/msg13700.html
To summaries 1/ Ownership of DTB IMO should be firmware and we should retain this ownership in EBBR as well, Any objections/thoughts ?
I fully agree. On top of that we need to make clear that backward and forward compatibility are not optional.
For that I think we may need to actually give people workable solutions to create device trees that are compatible with multiple levels of kernel support. The main areas I'm aware of that keep breaking are:
- fine grained interrupt controller support
Do you have an example of that? The only thing I can think of is people switching interrupts from the GIC to an always-on, low-power mode custom interrupt controller.
The last time I've seen that breakage was:
https://github.com/raspberrypi/linux/blob/rpi-4.14.y/arch/arm/boot/dts/bcm27...
which indeed does switch interrupts from the GIC to an interrupt muxer behind the GIC.
The problem is that once support for that lands upstream, you will have very little option but to break backwards compatibility today.
This one is unfortunate. It could have been handled better. An interrupt-map property in the aux ctrlr could have mapped the interrupts to GIC without any aux driver. Then when the aux driver lands, it just needs to remove the interrupt-map (on boot).
To do that you would've needed to know that you need the change in the first place ;).
Actually, I think we can still solve this. Add the interrupt-map now. Then when the aux driver lands it has to fixup the interrupt-map.
I think I actually hit the problem when testing my deferred probe patch. I saw a 30 sec delay in the console output when the pl011 driver probed and using the pl011 as the console (they really made a mess of the serial console on RPi 3).
Alternatively, I skimmed thru some discussions of the issue, but I'm not clear why the devices behind the aux controller can't all just treat their interrupts as shared. But that would be a simple change to the drivers' irq handlers, so I'm probably missing something. If that worked, then the DT would never need to change.
I guess whether this could have been handled better depends if folks knowingly ignored the issue or this was found after upstreaming support. The latter case may be unavoidable, but maybe we can make it
I think most of these cases are the latter.
rare enough we only need overlays in some exceptions. Whether we try to be stricter and do better up front or have some overlay based solution, either one is going to require folks be aware of the issues and effort to avoid them.
Yes, but with the overlay approach we can temper it up after the fact :).
I was assuming you did not want to be the one to find all the issues and fix them. You do want upstream to do a better job and avoid some of the issues to begin with, right?
I personally think what we need to do is have a flag day event. I think ideally that would be a move of the dts file out of the Linux tree into a common git repository. Once it's there, we'd need scripts that ensure backward compatibility all the way back to whatever was current at the flag day.
Well, we wouldn't move over platforms that don't have pinctrl, clocks, etc. because they aren't stable and many of your problems go away. You would also still have the same problem that the DT declared stable at the flag day is newer than the kernel you are on and may not work with it.
The problem also is how do you define when things are stable? Lots of folks ask for a way to declare bindings stable vs. unstable. My concern with that is things will never get moved to stable and anything unstable is free to change at will.
Can I have those compatibility checking scripts now please. I'll add them to the kernel because I'd like to know when we break things even if we don't enforce not breaking. Really, I don't see how we'd check compatibility in a script. What's compatible or not depends on the OS.
Projects like U-Boot (or maybe even other OSs) could then have a git submodule on that repo and be sure that they can rely on these device trees to be stable.
- clock support
Are there cases other than going from fixed, fake clocks to a real clock controller node. I'm inclined to stop allowing people to do that. A better way this could be done is just provide a clock controller driver with a bunch of fixed clocks. Then the switch from the dumb driver to the real driver is just a kernel change.
Going from fixed to in-kernel clock driver is one case I'm aware of, yes.
The other one I can think of right now is going from one clock interface to another. For example if you configure clocks using a firmware interface and that firmware interface moves to a newer, more shiny interface (say proprietary to SCMI). That switch may also regress users of the device tree.
If not only the clocks but the whole firmware interface changes, then I don't see how a stable DT matters at that point.
Firmware could expose both interfaces, but if the DT only mentions the "new" one, OSs that used to work will cease to work.
For the declared stable, post flag day DT, the clocks in DT will never change from one controller to another. Changing them doesn't sound stable to me.
I think Andre had other examples of where DT compatibility broke, so I'll let him chime in as well.
- power domain support
Example?
https://github.com/Xilinx/linux-xlnx/blob/master/arch/arm64/boot/dts/xilinx/...
Booting with the power-domains properties included in the device tree fails for me, because the driver framework considers them critical to load a device and just errors out with -EDEFER.
- pinctrl support
This would be the firmware initially does all the pin setup, then you move it to DT and drop the setup from the firmware? Otherwise I don't understand the problem in this case. We'd start with no pinctrl and then add it to the DT. Why wouldn't the kernel just ignore it?
Because it doesn't. The linux driver framework as it works today looks at the pinctrl parent for a specific device and if it can't find it loaded, throws -EDEFER for the driver getting loaded. I suppose the rationale behind that is that pinctrl drivers could be modules that get loaded after the driver you want to load.
Every time a device tree changes in any of the above, that usually ends up in backwards incompatibility.
TBC, you're talking about new dtb with old kernels. We've mainly cared
Yes. Both really.
about old dtbs and new kernels. So first we should agree the former is important too. I do, because simply you wouldn't want a BIOS update to make your PC stop booting your already installed OS.
Yes.
I'd like to solve this with policy and good practice before we try to apply technical solutions on top of a mess.
I'm not holding my hopes up, but we can try ;). Another full dimension of messiness are downstream, unapproved bindings that some times can't live in the same device tree as their upstream counterparts once upstream chose a different route.
For pinctrl and power domain, I suppose we could have a compatible whitelist in the kernel and instead of -EDEFER just consider the targets immutable if we hit none of the whitelisted strings.
Yes, I had similar thought as well, but was thinking a blacklist rather. We could modify the DT on boot to disable the listed nodes. Or various subsystems could have lists of compatibles to not return -EPROBE_DEFER. The one complexity with a blacklist is we'd need patches for stable that may not be in mainline unless we add the compatible to the blacklist and then immediately remove it when adding the real driver.
Patches to old kernels won't work for us, as the "installer" kernel is fixed at the time it gets put onto an iso. So while we can update the kernel later on, initial installation is expected to be possible with an old, unpatched kernel.
Okay, I'll keep that in mind. So any kernel fixes (like my patch) will only fix future installers.
I was considering whether for things like pinctrl, if we should just add a DT property to say the default configuration works. Default could be reset state or firmware initialized state (which could have been the firmware processing the DT pinctrl and configuring everything). Maybe EBBR should define that firmware does setup the pins (most platforms probably do already, just maybe not based on DT) and then should set the flag. I'm trying to be careful and define this as a reflection of the h/w, not a Linux property to disable deferred probe or not. The main problem I see is "works" is not a binary property. I guess it could be fine grained per device, but then that assumes people know if the default works or not. Of course, this would still need a kernel change and won't fix your installers now.
Rob