On 03.05.18 18:11, Rob Herring wrote:
On Thu, May 3, 2018 at 9:29 AM, Alexander Graf agraf@suse.de wrote:
On 04/30/2018 08:36 PM, Rob Herring wrote:
On Fri, Apr 27, 2018 at 4:39 PM, Alexander Graf agraf@suse.de wrote:
Hi Rob,
On 27.04.18 18:40, Rob Herring wrote:
On Fri, Apr 27, 2018 at 2:47 AM, Alexander Graf agraf@suse.de wrote:
On 27.04.18 08:24, Udit Kumar wrote: > > Hi > There is bit of discussion on linux-efi too , to handle DT update > > I guess some members of this forum are active there too. > > https://www.spinics.net/lists/linux-efi/msg13700.html > > To summaries > 1/ Ownership of DTB > IMO should be firmware and we should retain this > ownership in EBBR as well, Any objections/thoughts ?
I fully agree. On top of that we need to make clear that backward and forward compatibility are not optional.
For that I think we may need to actually give people workable solutions to create device trees that are compatible with multiple levels of kernel support. The main areas I'm aware of that keep breaking are:
- fine grained interrupt controller support
Do you have an example of that? The only thing I can think of is people switching interrupts from the GIC to an always-on, low-power mode custom interrupt controller.
The last time I've seen that breakage was:
https://github.com/raspberrypi/linux/blob/rpi-4.14.y/arch/arm/boot/dts/bcm27...
which indeed does switch interrupts from the GIC to an interrupt muxer behind the GIC.
The problem is that once support for that lands upstream, you will have very little option but to break backwards compatibility today.
This one is unfortunate. It could have been handled better. An interrupt-map property in the aux ctrlr could have mapped the interrupts to GIC without any aux driver. Then when the aux driver lands, it just needs to remove the interrupt-map (on boot).
To do that you would've needed to know that you need the change in the first place ;).
Actually, I think we can still solve this. Add the interrupt-map now. Then when the aux driver lands it has to fixup the interrupt-map> I think I actually hit the problem when testing my deferred probe patch. I saw a 30 sec delay in the console output when the pl011 driver probed and using the pl011 as the console (they really made a mess of the serial console on RPi 3).
Alternatively, I skimmed thru some discussions of the issue, but I'm not clear why the devices behind the aux controller can't all just treat their interrupts as shared. But that would be a simple change to the drivers' irq handlers, so I'm probably missing something. If that worked, then the DT would never need to change.
I guess whether this could have been handled better depends if folks knowingly ignored the issue or this was found after upstreaming support. The latter case may be unavoidable, but maybe we can make it
I think most of these cases are the latter.
rare enough we only need overlays in some exceptions. Whether we try to be stricter and do better up front or have some overlay based solution, either one is going to require folks be aware of the issues and effort to avoid them.
Yes, but with the overlay approach we can temper it up after the fact :).
I was assuming you did not want to be the one to find all the issues and fix them. You do want upstream to do a better job and avoid some of the issues to begin with, right?
Oh I don't want to be the one, but I don't believe in humans finding 100% of all errors. So I want to also have an answer for the 10% that fall through the cracks.
I personally think what we need to do is have a flag day event. I think ideally that would be a move of the dts file out of the Linux tree into a common git repository. Once it's there, we'd need scripts that ensure backward compatibility all the way back to whatever was current at the flag day.
Well, we wouldn't move over platforms that don't have pinctrl, clocks, etc. because they aren't stable and many of your problems go away. You
That's something we can easily define, true.
would also still have the same problem that the DT declared stable at the flag day is newer than the kernel you are on and may not work with it.
Yes, but that's normal for distros. SLES10 doesn't run all that great on Skylake either. People accept the fact that their new system may need a new OS. What they don't like is when we regress - so their hardware worked before and now doesn't anymore. Either by updating firmware or the OS.
The problem also is how do you define when things are stable? Lots of folks ask for a way to declare bindings stable vs. unstable. My concern with that is things will never get moved to stable and anything unstable is free to change at will.
We can always mandate the DT has to be in the stable tree to claim EBBR compatibility. Then distros could claim they only provide "good" support for EBBR platforms. And then people will have an incentive to make their DTs stable.
For SLES for example, we could tell SoC vendors we won't support them until their DT is stable. Today we don't have any handle on that. With a policy like the above it would be something easy for them to understand and follow.
Can I have those compatibility checking scripts now please. I'll add them to the kernel because I'd like to know when we break things even if we don't enforce not breaking. Really, I don't see how we'd check compatibility in a script. What's compatible or not depends on the OS.
Well, we have the list in this thread already. Changing a compatible of a device for example should at least spawn a checkpatch warning. If we don't manage to fix the clock and pinctrl dependency issue, we can create checkpatch errors when adding clock dependencies.
Projects like U-Boot (or maybe even other OSs) could then have a git submodule on that repo and be sure that they can rely on these device trees to be stable.
- clock support
Are there cases other than going from fixed, fake clocks to a real clock controller node. I'm inclined to stop allowing people to do that. A better way this could be done is just provide a clock controller driver with a bunch of fixed clocks. Then the switch from the dumb driver to the real driver is just a kernel change.
Going from fixed to in-kernel clock driver is one case I'm aware of, yes.
The other one I can think of right now is going from one clock interface to another. For example if you configure clocks using a firmware interface and that firmware interface moves to a newer, more shiny interface (say proprietary to SCMI). That switch may also regress users of the device tree.
If not only the clocks but the whole firmware interface changes, then I don't see how a stable DT matters at that point.
Firmware could expose both interfaces, but if the DT only mentions the "new" one, OSs that used to work will cease to work.
For the declared stable, post flag day DT, the clocks in DT will never change from one controller to another. Changing them doesn't sound stable to me.
I agree, but FWIW at least one SoC vendor is planning exactly that ;). It basically means they can only declare their DT stable after the transition or not change the fw interface at all I guess.
I think Andre had other examples of where DT compatibility broke, so I'll let him chime in as well.
- power domain support
Example?
https://github.com/Xilinx/linux-xlnx/blob/master/arch/arm64/boot/dts/xilinx/...
Booting with the power-domains properties included in the device tree fails for me, because the driver framework considers them critical to load a device and just errors out with -EDEFER.
- pinctrl support
This would be the firmware initially does all the pin setup, then you move it to DT and drop the setup from the firmware? Otherwise I don't understand the problem in this case. We'd start with no pinctrl and then add it to the DT. Why wouldn't the kernel just ignore it?
Because it doesn't. The linux driver framework as it works today looks at the pinctrl parent for a specific device and if it can't find it loaded, throws -EDEFER for the driver getting loaded. I suppose the rationale behind that is that pinctrl drivers could be modules that get loaded after the driver you want to load.
Every time a device tree changes in any of the above, that usually ends up in backwards incompatibility.
TBC, you're talking about new dtb with old kernels. We've mainly cared
Yes. Both really.
about old dtbs and new kernels. So first we should agree the former is important too. I do, because simply you wouldn't want a BIOS update to make your PC stop booting your already installed OS.
Yes.
I'd like to solve this with policy and good practice before we try to apply technical solutions on top of a mess.
I'm not holding my hopes up, but we can try ;). Another full dimension of messiness are downstream, unapproved bindings that some times can't live in the same device tree as their upstream counterparts once upstream chose a different route.
For pinctrl and power domain, I suppose we could have a compatible whitelist in the kernel and instead of -EDEFER just consider the targets immutable if we hit none of the whitelisted strings.
Yes, I had similar thought as well, but was thinking a blacklist rather. We could modify the DT on boot to disable the listed nodes. Or various subsystems could have lists of compatibles to not return -EPROBE_DEFER. The one complexity with a blacklist is we'd need patches for stable that may not be in mainline unless we add the compatible to the blacklist and then immediately remove it when adding the real driver.
Patches to old kernels won't work for us, as the "installer" kernel is fixed at the time it gets put onto an iso. So while we can update the kernel later on, initial installation is expected to be possible with an old, unpatched kernel.
Okay, I'll keep that in mind. So any kernel fixes (like my patch) will only fix future installers.
Yes, but that's ok. Few people expect new hardware to work flawlessly with old OSs.
What we shouldn't do is block the road for enabling boards or SoCs that would easily work with an older kernel though.
On x86 for example, while old SLES releases may not have all drivers needed to drive all devices of a new system, most parts are generic enough or handled via ACPI that you can make a system work by adding kernel modules.
I was considering whether for things like pinctrl, if we should just add a DT property to say the default configuration works. Default
That's tricky with overlays, no?
could be reset state or firmware initialized state (which could have been the firmware processing the DT pinctrl and configuring everything). Maybe EBBR should define that firmware does setup the pins (most platforms probably do already, just maybe not based on DT) and then should set the flag. I'm trying to be careful and define this as a reflection of the h/w, not a Linux property to disable deferred probe or not. The main problem I see is "works" is not a binary property. I guess it could be fine grained per device, but then that assumes people know if the default works or not. Of course, this would still need a kernel change and won't fix your installers now.
Yes, the problem is terribly complicated.
Honestly, if you want my opinion, I would move pinctrl and clock frameworks all the way up to EL3 and leave it to firmware to take care of them always. That interface would then need to be 100% stable.
That way we get back to the point above where enabling a new SoC becomes a matter of adding drivers to an older OS, because we don't rely on in-kernel drivers to drive simple things.
Alex