Hello Francois,
in our last meeting you suggested to investigate deeper what meaning modularization in device trees could mean.
One example I have seen is the definition of memory mappings for PCIe.
If for you example you look in the device tree of the Raspberry Pi 4 you can find that there is an entry:
pcie@7d500000 { ranges = < 0x2000000 0x00 0xf8000000 0x06 0x00 0x00 0x4000000>; };
The ranges property defines the mapping MEM 0x600000000..0x603ffffff -> 0xf8000000 and thereby the address and size of the memory windows used for communication.
On some boards e.g. the MacchitoBin you have multiple platform devices each with its own ranges definitions. Here are the values in May 2019:
pcie@f2600000 { ranges = < 0x81000000 0x0 0xf9000000 0x0 0xf9000000 0x0 0x10000 0x82000000 0x0 0xf6000000 0x0 0xf6000000 0x0 0xf00000>; } pcie@f2620000 { ranges = < 0x81000000 0x0 0xf9010000 0x0 0xf9010000 0x0 0x10000 0x82000000 0x0 0xf7000000 0x0 0xf7000000 0x0 0xf00000>; } pcie@f2640000 { ranges = < 0x81000000 0x0 0xf9020000 0x0 0xf9020000 0x0 0x10000 0x82000000 0x0 0xf8000000 0x0 0xf8000000 0x0 0xf00000>; }
On my MacchitoBin the GPU card was not usable because the memory window was too small to create the required BARs. Changing the window size and location solved the problem, commit d3446b266a8c ("arm64: dts: marvell: mcbin: enlarge PCI memory window").
The problem with the current way in qhich PCI devices are declared is that they do not indicate requirements and capabilities and let the software figure out a good mapping. But instead fixed mappings are hard-coded into the device-tree.
So for me this would be one element of DT modularization:
The description of a device should provide capabilities and requirements and avoid fixed software settings.
Best regards
Heinrich
Hi Heinrich,
I've integrated this in the use case section of the Techinical Report https://docs.google.com/document/d/1CLkhLRaz_zcCq44DLGmPZQFPbYHOC6nzPowaL0XmRk0/edit?usp=sharing (anyone with the link can comment).
BTW, I missed to publish an agenda for this week's call:
I'd like we review the Techinical Report https://docs.google.com/document/d/1CLkhLRaz_zcCq44DLGmPZQFPbYHOC6nzPowaL0XmRk0/edit?usp=sharing outline and possibly name some editors on sections.
Cheers
FF
On Mon, 8 Jun 2020 at 20:34, Heinrich Schuchardt xypron.glpk@gmx.de wrote:
Hello Francois,
in our last meeting you suggested to investigate deeper what meaning modularization in device trees could mean.
One example I have seen is the definition of memory mappings for PCIe.
If for you example you look in the device tree of the Raspberry Pi 4 you can find that there is an entry:
pcie@7d500000 { ranges = < 0x2000000 0x00 0xf8000000 0x06 0x00 0x00 0x4000000>; };
The ranges property defines the mapping MEM 0x600000000..0x603ffffff -> 0xf8000000 and thereby the address and size of the memory windows used for communication.
On some boards e.g. the MacchitoBin you have multiple platform devices each with its own ranges definitions. Here are the values in May 2019:
pcie@f2600000 { ranges = < 0x81000000 0x0 0xf9000000 0x0 0xf9000000 0x0 0x10000 0x82000000 0x0 0xf6000000 0x0 0xf6000000 0x0 0xf00000>; } pcie@f2620000 { ranges = < 0x81000000 0x0 0xf9010000 0x0 0xf9010000 0x0 0x10000 0x82000000 0x0 0xf7000000 0x0 0xf7000000 0x0 0xf00000>; } pcie@f2640000 { ranges = < 0x81000000 0x0 0xf9020000 0x0 0xf9020000 0x0 0x10000 0x82000000 0x0 0xf8000000 0x0 0xf8000000 0x0 0xf00000>; }
On my MacchitoBin the GPU card was not usable because the memory window was too small to create the required BARs. Changing the window size and location solved the problem, commit d3446b266a8c ("arm64: dts: marvell: mcbin: enlarge PCI memory window").
The problem with the current way in qhich PCI devices are declared is that they do not indicate requirements and capabilities and let the software figure out a good mapping. But instead fixed mappings are hard-coded into the device-tree.
So for me this would be one element of DT modularization:
The description of a device should provide capabilities and requirements and avoid fixed software settings.
Best regards
Heinrich
On Mon, Jun 8, 2020 at 8:35 PM Heinrich Schuchardt xypron.glpk@gmx.de wrote:
On my MacchitoBin the GPU card was not usable because the memory window was too small to create the required BARs. Changing the window size and location solved the problem, commit d3446b266a8c ("arm64: dts: marvell: mcbin: enlarge PCI memory window").
The problem with the current way in qhich PCI devices are declared is that they do not indicate requirements and capabilities and let the software figure out a good mapping. But instead fixed mappings are hard-coded into the device-tree.
There are historically two different approaches for dealing with the translation windows: On machines that run a proper firmware, you would typically have the firmware set up the mappings to whatever makes sense, and the ranges communicate what they are to the kernel.
On 32-bit ARM machines, we often could not rely on the mappings set up by the boot loader when they were either not fit for the workload whatever the kernel needed, or just left at whatever the hardware init values were and not set up at all. As a workaround, some of the ARM32 specific PCI host drivers just read the mappings from the DT and then program the registers with that. This is definitely what we do on the Marvell EBU series (kirkwood, orion, ...) that are very flexible with the PCI mappings but don't have any good mapping that works for all users equally (e.g. you might have to pick three out of config-space, io-space, prefetchable-mem and nonprefetchable-mem, but can't have all four).
It's possible that the mcbin just ended up doing the same thing because they were used to that from the 32-bit chips, when this really should have been something that gets set up by the boot loader and put into the DT.
Arnd
boot-architecture@lists.linaro.org
-
Arnd Bergmann -
François Ozog -
Heinrich Schuchardt