Hi all,
As some of you may be aware, I have been working around the lack of a clear upstreaming strategy for platform support by keeping such code in a dedicated repository I set up at Linaro for that purpose: https://git.linaro.org/uefi/OpenPlatformPkg.git
During discussions at the last Seattle plugfest we finally agreed on the (theoretical) details of how to use the edk2-platforms repository. After that I promised to migrate OpenPlatformPkg across to the edk2-platforms and edk2-non-osi structure, with the explicit end goal from my side that this should become the master branch for each.
And now, before the release of HURD 1.0, I have.
Current limitations (that I can remember): - A few references to OpenPlatformPkg remain, in ways that do not appear to break any of the platform builds. Most likely this affects dead code, but in case it's been accidentally orphaned, I thought it best to - I have simply nuked all references to Ebl (used in _addition_ to the UEFI shell, which was never the intent) and the efi-toolkit ramdisk driver. - The Marvell Yukon driver that I sent out for review last week has not migrated anywhere, and so has been temporarily disabled Mike suggested I should - USB support on the LeMaker Cello board depends on the patch "OptionRomPkg: add firmware loader driver for Renesas PD72020x" sent out by Ard on 18th of April. - I have dropped some of the binary-only modules from OpenPlatformPkg, and contacted the platform owners with requests for modifications. - The git history is quite messy and will be cleaned up, but I wanted to keep the transition quite visible in the RFC. - I haven't filled anything into the Maintainers.txt files - I am in favour of moving to a fully machine-readable format with wildcards as Laszlo has proposed in the past, and think this would be an excellent point to have that discussion (which can be had separately for edk2-platforms and edk2-non-osi from edk2). - Few of the platforms complete the FV generation stage, and I've inserted a couple of silly hacks to get them to get as far as they do. I think that either I am missing some points of how PACKAGES_PATH is intended to work, or I'm simply hitting corner cases no one has come across before. I could really use some help debugging these issues. (examples below).
The below depends on the 3-part series I sent out today for importing DwEmmcDxe and EfiTimeBaseLib from OpenPlatformPkg. But apart from that, I have uploaded branches called devel-OpenPlatformPkg to:
https://github.com/tianocore/edk2-platforms/tree/devel-OpenPlatformPkg https://github.com/tianocore/edk2-non-osi/tree/devel-OpenPlatformPkg
These branches _will_ be rebased occasionally until they get to a point where they can move out of devel- stage (and hopefully onto master).
Build issue description ======================= So, one of the hopefully easier ones is what I'm seeing when trying to build the Juno platform:
$ PACKAGES_PATH="/work/maint/edk2-platforms:/work/maint/edk2-non-osi" GCC5_AARCH64_PREFIX=aarch64-linux-gnu- build -a AARCH64 -t GCC5 -p Platform/ARM/Juno/ArmJuno.dsc -b RELEASE -n 9
results in:
<<< GenFds.py... : error F003: Output file for RAW section could not be found for Platform/ARM/Juno/AcpiTables/AcpiTables.inf
###
build.py... : error 7000: Failed to execute command GenFds -f /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.fdf --conf=/work/maint/edk2/Conf -o /work/maint/edk2/Build/ArmJuno/RELEASE_GCC5 -t GCC5 -b RELEASE -p /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.dsc -a AARCH64 -D "EFI_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" -D "EDK_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" -D "TOOL_CHAIN_TAG=GCC5" -D "TOOLCHAIN=GCC5" -D "TARGET=RELEASE" -D "FAMILY=GCC" -D "WORKSPACE=/work/maint/edk2" -D "EDK_TOOLS_PATH=/work/maint/edk2/BaseTools" -D "ARCH=AARCH64" -D "ECP_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" [/work/maint/edk2]
- Failed -
And when I copy and paste the above command manually, I get:
<<< GenFds.py... /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.dsc(34): error 000E: File/directory not found in workspace /work/maint/edk2-platforms/Platform/ARM/Juno/Platform/ARM/VExpress/ArmVExpress.dsc.inc /work/maint/edk2/Platform/ARM/VExpress/ArmVExpress.dsc.inc
So, to an uniformed observer, it seems the portion !include Platform/ARM/VExpress/ArmVExpress.dsc.inc from ArmJuno.dsc gets expanded to "directory ArmJuno.dsc is in" + "Platform/ARM/VExpress/ArmVExpress.dsc.inc" whereas I was hoping for it to try to find a match for "Platform/ARM/VExpress/ArmVExpress.dsc.inc" along PACKAGES_PATH (and find one in edk2-platforms).
I also have the impression that something similar is happening in ArmJuno.fdf for the line INF RuleOverride=ACPITABLE Platform/ARM/Juno/AcpiTables/AcpiTables.inf generating the error message from the original build command.
But I'm not quite sure how to debug these issues (short of fully figuring out the innards of MultipleWorkspace.py, MetaFileParser.py and a few others). Any ideas of where to look, or even what is going on? Would these cases be expected to work?
/ Leif
HI Leif It is great that you are adding more platform to edk2-platforms.
We (Intel) also have plan to add more boards there. In general, we are very close on having silicon and platform folder.
I have a quick look. One minor suggestions here: Take Arm folder as example. (I assume Juno is one package, and VExpress is the other package.) Can we name Juno to be JunoPkg, and VExpress to be VExpressPkg ? We do not add "Pkg" to a folder. And we usually add "Pkg" suffix to a package.
In general, I think it is a very good start. I may review the content in more detail and provide more feedback.
Thank you Yao Jiewen
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Leif Lindholm Sent: Thursday, May 4, 2017 6:56 AM To: edk2-devel@lists.01.org Cc: Ryan Harkin ryan.harkin@linaro.org; Ard Biesheuvel ard.biesheuvel@linaro.org; Chenhui Sun chenhui.sun@linaro.org; Andrew Fish afish@apple.com; Alan Ott alan@softiron.co.uk; Richardson, Brian brian.richardson@intel.com; Duran, Leo leo.duran@amd.com; haojian.zhuang@linaro.org; Linaro UEFI linaro-uefi@lists.linaro.org; Kinney, Michael D michael.d.kinney@intel.com; Heyi Guo heyi.guo@linaro.org Subject: [edk2] [RFC] migration of OpenPlatformPkg to tianocore
Hi all,
As some of you may be aware, I have been working around the lack of a clear upstreaming strategy for platform support by keeping such code in a dedicated repository I set up at Linaro for that purpose: https://git.linaro.org/uefi/OpenPlatformPkg.git
During discussions at the last Seattle plugfest we finally agreed on the (theoretical) details of how to use the edk2-platforms repository. After that I promised to migrate OpenPlatformPkg across to the edk2-platforms and edk2-non-osi structure, with the explicit end goal from my side that this should become the master branch for each.
And now, before the release of HURD 1.0, I have.
Current limitations (that I can remember): - A few references to OpenPlatformPkg remain, in ways that do not appear to break any of the platform builds. Most likely this affects dead code, but in case it's been accidentally orphaned, I thought it best to - I have simply nuked all references to Ebl (used in _addition_ to the UEFI shell, which was never the intent) and the efi-toolkit ramdisk driver. - The Marvell Yukon driver that I sent out for review last week has not migrated anywhere, and so has been temporarily disabled Mike suggested I should - USB support on the LeMaker Cello board depends on the patch "OptionRomPkg: add firmware loader driver for Renesas PD72020x" sent out by Ard on 18th of April. - I have dropped some of the binary-only modules from OpenPlatformPkg, and contacted the platform owners with requests for modifications. - The git history is quite messy and will be cleaned up, but I wanted to keep the transition quite visible in the RFC. - I haven't filled anything into the Maintainers.txt files - I am in favour of moving to a fully machine-readable format with wildcards as Laszlo has proposed in the past, and think this would be an excellent point to have that discussion (which can be had separately for edk2-platforms and edk2-non-osi from edk2). - Few of the platforms complete the FV generation stage, and I've inserted a couple of silly hacks to get them to get as far as they do. I think that either I am missing some points of how PACKAGES_PATH is intended to work, or I'm simply hitting corner cases no one has come across before. I could really use some help debugging these issues. (examples below).
The below depends on the 3-part series I sent out today for importing DwEmmcDxe and EfiTimeBaseLib from OpenPlatformPkg. But apart from that, I have uploaded branches called devel-OpenPlatformPkg to:
https://github.com/tianocore/edk2-platforms/tree/devel-OpenPlatformPkg https://github.com/tianocore/edk2-non-osi/tree/devel-OpenPlatformPkg
These branches _will_ be rebased occasionally until they get to a point where they can move out of devel- stage (and hopefully onto master).
Build issue description ======================= So, one of the hopefully easier ones is what I'm seeing when trying to build the Juno platform:
$ PACKAGES_PATH="/work/maint/edk2-platforms:/work/maint/edk2-non-osi" GCC5_AARCH64_PREFIX=aarch64-linux-gnu- build -a AARCH64 -t GCC5 -p Platform/ARM/Juno/ArmJuno.dsc -b RELEASE -n 9
results in:
<<< GenFds.py... : error F003: Output file for RAW section could not be found for Platform/ARM/Juno/AcpiTables/AcpiTables.inf
###
build.py... : error 7000: Failed to execute command GenFds -f /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.fdf --conf=/work/maint/edk2/Conf -o /work/maint/edk2/Build/ArmJuno/RELEASE_GCC5 -t GCC5 -b RELEASE -p /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.dsc -a AARCH64 -D "EFI_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" -D "EDK_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" -D "TOOL_CHAIN_TAG=GCC5" -D "TOOLCHAIN=GCC5" -D "TARGET=RELEASE" -D "FAMILY=GCC" -D "WORKSPACE=/work/maint/edk2" -D "EDK_TOOLS_PATH=/work/maint/edk2/BaseTools" -D "ARCH=AARCH64" -D "ECP_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" [/work/maint/edk2]
- Failed -
And when I copy and paste the above command manually, I get:
<<< GenFds.py... /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.dsc(34): error 000E: File/directory not found in workspace /work/maint/edk2-platforms/Platform/ARM/Juno/Platform/ARM/VExpress/ArmVExpress.dsc.inc /work/maint/edk2/Platform/ARM/VExpress/ArmVExpress.dsc.inc
So, to an uniformed observer, it seems the portion !include Platform/ARM/VExpress/ArmVExpress.dsc.inc from ArmJuno.dsc gets expanded to "directory ArmJuno.dsc is in" + "Platform/ARM/VExpress/ArmVExpress.dsc.inc" whereas I was hoping for it to try to find a match for "Platform/ARM/VExpress/ArmVExpress.dsc.inc" along PACKAGES_PATH (and find one in edk2-platforms).
I also have the impression that something similar is happening in ArmJuno.fdf for the line INF RuleOverride=ACPITABLE Platform/ARM/Juno/AcpiTables/AcpiTables.inf generating the error message from the original build command.
But I'm not quite sure how to debug these issues (short of fully figuring out the innards of MultipleWorkspace.py, MetaFileParser.py and a few others). Any ideas of where to look, or even what is going on? Would these cases be expected to work?
/ Leif _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.orgmailto:edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
Some comments for the build failure.
I think we might have different understanding on PACKAGES_PATH.
We are using below structure: ========================= Platform\ Intel\ PlatformAPkg PlatformBPkg Silicon\ Intel\ SiliconAPkg SiliconBPkg
We set PACKAGES_PATH to "Platform\Intel:Silicon\Intel". As such Build.DSC can include PlatformAPkg/DriverA/DriverA.inf. =========================
My suggestion for your case is below: We can use ========================= Platform\ Arm\ JunoPkg VExpressPkg
We set PACKAGES_PATH to "Platform\Arm". ArmJunoPkg.dsc can just use "VExpressPkg\ArmVExpress.dsc.inc" and "JunoPkg\Library\JunoPciHostBridgeLib\ JunoPciHostBridgeLib.dsc" =========================
The benefit of adding "Pkg" is that people can have a clear picture on where the path starts from. :)
Thank you Yao Jiewen
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Yao, Jiewen Sent: Friday, May 5, 2017 9:46 PM To: Leif Lindholm leif.lindholm@linaro.org; edk2-devel@lists.01.org Cc: Richardson, Brian brian.richardson@intel.com; Ard Biesheuvel ard.biesheuvel@linaro.org; Chenhui Sun chenhui.sun@linaro.org; Andrew Fish afish@apple.com; Alan Ott alan@softiron.co.uk; Ryan Harkin ryan.harkin@linaro.org; Duran, Leo leo.duran@amd.com; haojian.zhuang@linaro.org; Linaro UEFI linaro-uefi@lists.linaro.org; Kinney, Michael D michael.d.kinney@intel.com; Heyi Guo heyi.guo@linaro.org Subject: Re: [edk2] [RFC] migration of OpenPlatformPkg to tianocore
HI Leif It is great that you are adding more platform to edk2-platforms.
We (Intel) also have plan to add more boards there. In general, we are very close on having silicon and platform folder.
I have a quick look. One minor suggestions here: Take Arm folder as example. (I assume Juno is one package, and VExpress is the other package.) Can we name Juno to be JunoPkg, and VExpress to be VExpressPkg ? We do not add "Pkg" to a folder. And we usually add "Pkg" suffix to a package.
In general, I think it is a very good start. I may review the content in more detail and provide more feedback.
Thank you Yao Jiewen
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Leif Lindholm Sent: Thursday, May 4, 2017 6:56 AM To: edk2-devel@lists.01.orgmailto:edk2-devel@lists.01.org Cc: Ryan Harkin <ryan.harkin@linaro.orgmailto:ryan.harkin@linaro.org>; Ard Biesheuvel <ard.biesheuvel@linaro.orgmailto:ard.biesheuvel@linaro.org>; Chenhui Sun <chenhui.sun@linaro.orgmailto:chenhui.sun@linaro.org>; Andrew Fish <afish@apple.commailto:afish@apple.com>; Alan Ott <alan@softiron.co.ukmailto:alan@softiron.co.uk>; Richardson, Brian <brian.richardson@intel.commailto:brian.richardson@intel.com>; Duran, Leo <leo.duran@amd.commailto:leo.duran@amd.com>; haojian.zhuang@linaro.orgmailto:haojian.zhuang@linaro.org; Linaro UEFI <linaro-uefi@lists.linaro.orgmailto:linaro-uefi@lists.linaro.org>; Kinney, Michael D <michael.d.kinney@intel.commailto:michael.d.kinney@intel.com>; Heyi Guo <heyi.guo@linaro.orgmailto:heyi.guo@linaro.org> Subject: [edk2] [RFC] migration of OpenPlatformPkg to tianocore
Hi all,
As some of you may be aware, I have been working around the lack of a clear upstreaming strategy for platform support by keeping such code in a dedicated repository I set up at Linaro for that purpose: https://git.linaro.org/uefi/OpenPlatformPkg.git
During discussions at the last Seattle plugfest we finally agreed on the (theoretical) details of how to use the edk2-platforms repository. After that I promised to migrate OpenPlatformPkg across to the edk2-platforms and edk2-non-osi structure, with the explicit end goal from my side that this should become the master branch for each.
And now, before the release of HURD 1.0, I have.
Current limitations (that I can remember): - A few references to OpenPlatformPkg remain, in ways that do not appear to break any of the platform builds. Most likely this affects dead code, but in case it's been accidentally orphaned, I thought it best to - I have simply nuked all references to Ebl (used in _addition_ to the UEFI shell, which was never the intent) and the efi-toolkit ramdisk driver. - The Marvell Yukon driver that I sent out for review last week has not migrated anywhere, and so has been temporarily disabled Mike suggested I should - USB support on the LeMaker Cello board depends on the patch "OptionRomPkg: add firmware loader driver for Renesas PD72020x" sent out by Ard on 18th of April. - I have dropped some of the binary-only modules from OpenPlatformPkg, and contacted the platform owners with requests for modifications. - The git history is quite messy and will be cleaned up, but I wanted to keep the transition quite visible in the RFC. - I haven't filled anything into the Maintainers.txt files - I am in favour of moving to a fully machine-readable format with wildcards as Laszlo has proposed in the past, and think this would be an excellent point to have that discussion (which can be had separately for edk2-platforms and edk2-non-osi from edk2). - Few of the platforms complete the FV generation stage, and I've inserted a couple of silly hacks to get them to get as far as they do. I think that either I am missing some points of how PACKAGES_PATH is intended to work, or I'm simply hitting corner cases no one has come across before. I could really use some help debugging these issues. (examples below).
The below depends on the 3-part series I sent out today for importing DwEmmcDxe and EfiTimeBaseLib from OpenPlatformPkg. But apart from that, I have uploaded branches called devel-OpenPlatformPkg to:
https://github.com/tianocore/edk2-platforms/tree/devel-OpenPlatformPkg https://github.com/tianocore/edk2-non-osi/tree/devel-OpenPlatformPkg
These branches _will_ be rebased occasionally until they get to a point where they can move out of devel- stage (and hopefully onto master).
Build issue description ======================= So, one of the hopefully easier ones is what I'm seeing when trying to build the Juno platform:
$ PACKAGES_PATH="/work/maint/edk2-platforms:/work/maint/edk2-non-osi" GCC5_AARCH64_PREFIX=aarch64-linux-gnu- build -a AARCH64 -t GCC5 -p Platform/ARM/Juno/ArmJuno.dsc -b RELEASE -n 9
results in:
<<< GenFds.py... : error F003: Output file for RAW section could not be found for Platform/ARM/Juno/AcpiTables/AcpiTables.inf
###
build.py... : error 7000: Failed to execute command GenFds -f /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.fdf --conf=/work/maint/edk2/Conf -o /work/maint/edk2/Build/ArmJuno/RELEASE_GCC5 -t GCC5 -b RELEASE -p /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.dsc -a AARCH64 -D "EFI_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" -D "EDK_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" -D "TOOL_CHAIN_TAG=GCC5" -D "TOOLCHAIN=GCC5" -D "TARGET=RELEASE" -D "FAMILY=GCC" -D "WORKSPACE=/work/maint/edk2" -D "EDK_TOOLS_PATH=/work/maint/edk2/BaseTools" -D "ARCH=AARCH64" -D "ECP_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" [/work/maint/edk2]
- Failed -
And when I copy and paste the above command manually, I get:
<<< GenFds.py... /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.dsc(34): error 000E: File/directory not found in workspace /work/maint/edk2-platforms/Platform/ARM/Juno/Platform/ARM/VExpress/ArmVExpress.dsc.inc /work/maint/edk2/Platform/ARM/VExpress/ArmVExpress.dsc.inc
So, to an uniformed observer, it seems the portion !include Platform/ARM/VExpress/ArmVExpress.dsc.inc from ArmJuno.dsc gets expanded to "directory ArmJuno.dsc is in" + "Platform/ARM/VExpress/ArmVExpress.dsc.inc" whereas I was hoping for it to try to find a match for "Platform/ARM/VExpress/ArmVExpress.dsc.inc" along PACKAGES_PATH (and find one in edk2-platforms).
I also have the impression that something similar is happening in ArmJuno.fdf for the line INF RuleOverride=ACPITABLE Platform/ARM/Juno/AcpiTables/AcpiTables.inf generating the error message from the original build command.
But I'm not quite sure how to debug these issues (short of fully figuring out the innards of MultipleWorkspace.py, MetaFileParser.py and a few others). Any ideas of where to look, or even what is going on? Would these cases be expected to work?
/ Leif _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.orgmailto:edk2-devel@lists.01.org%3cmailto:edk2-devel@lists.01.org> https://lists.01.org/mailman/listinfo/edk2-devel _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.orgmailto:edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
Hi Jiewen,
Apologies for ridicilously slow response - I caught a bad cold and am only now getting back on track with this.
Many thanks for having a look, and your comments.
On Fri, May 05, 2017 at 02:03:43PM +0000, Yao, Jiewen wrote:
Some comments for the build failure.
I think we might have different understanding on PACKAGES_PATH.
We are using below structure:
Platform\ Intel\ PlatformAPkg PlatformBPkg Silicon\ Intel\ SiliconAPkg SiliconBPkg
We set PACKAGES_PATH to "Platform\Intel:Silicon\Intel". As such Build.DSC can include PlatformAPkg/DriverA/DriverA.inf. =========================
My suggestion for your case is below: We can use
Platform\ Arm\ JunoPkg VExpressPkg
Yes, so this is probably a good point. I tried to model this repository on Mike's proposal for updated directory structure, and is was not entirely clear to me whether the *Pkg naming convention was intended to be retained at lower levels when it disappeared at the top level.
/* BEGIN sidetrack */ My understanding from before was that the "package" convention stemmed from a situation where each top-level directory was treated as an "IP silo". Since that aspect disappeared with the proposed layout changes, and the edk2-* repositories can now be seen much more (if not entirely) as cohesive structures, I had simply assumed the Pkg suffix convention would disappear. I do not actually have a strong opinion on the matter. /* END sidetrack */
In the specific cases of JunoPkg and VExpressPkg, these are not technically "packages" in this tree, since their .dec files are still in edk2. However, these should move, and if the Pkg naming convention remains, they should indeed be renamed.
We set PACKAGES_PATH to "Platform\Arm". ArmJunoPkg.dsc can just use "VExpressPkg\ArmVExpress.dsc.inc" and "JunoPkg\Library\JunoPciHostBridgeLib\ JunoPciHostBridgeLib.dsc" =========================
The benefit of adding "Pkg" is that people can have a clear picture on where the path starts from. :)
OK, so I think we have a situation where I have misunderstood the intended purpose of the PACKAGES_PATH mechanism. But, I would like to discuss the possibility of extending it to cover the use-case that I thought it provided - or potentially adding new functionality to provide the same.
Since my goal is to have many platforms (for scope, let's call it hundreds) in the edk2-platforms master branch, needing to specify for each platform all directories that contains packages used by that platform becomes quite tedious. Especially when that stretches across multiple packages in multiple repositories.
As an example, take the SoftIron 1000 platform. If the PACKAGES_PATH needs to contain all directories holding packages used outside of edk2, I would need to add: - edk2-non-osi/Platform/SoftIron/ - edk2-non-osi/Silicon/AMD/Styx/ - edk2-platforms/Platform/SoftIron/ - edk2-platforms/Silicon/AMD/Styx/
And for each other platform, I would need to specify a similar (unique for that platform) PACKAGES_PATH. And I can see more complex splits than that appearing.
Now, if everything I want to build resides across the master branch of edk2, the master branch of edk2-platforms and the master branch of edk2-non-osi, then I just need a way for the build command to search in all three repositories. At that point, the only platform-specific information I need in order to build is the path to the .dsc.
So, what I was hoping for was a way to simply extend the path resolution in .dsc/.fdf files to multiple repositories. I had hoped PACKAGES_PATH could do this, but since this was never its design goal I understand why it does not. Do you think the PACKAGES_PATH functionality could/should be extended to support this, or do I need a new feature (WORKSPACES_PATH?)?
Best Regards,
Leif
Thank you Yao Jiewen
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Yao, Jiewen Sent: Friday, May 5, 2017 9:46 PM To: Leif Lindholm leif.lindholm@linaro.org; edk2-devel@lists.01.org Cc: Richardson, Brian brian.richardson@intel.com; Ard Biesheuvel ard.biesheuvel@linaro.org; Chenhui Sun chenhui.sun@linaro.org; Andrew Fish afish@apple.com; Alan Ott alan@softiron.co.uk; Ryan Harkin ryan.harkin@linaro.org; Duran, Leo leo.duran@amd.com; haojian.zhuang@linaro.org; Linaro UEFI linaro-uefi@lists.linaro.org; Kinney, Michael D michael.d.kinney@intel.com; Heyi Guo heyi.guo@linaro.org Subject: Re: [edk2] [RFC] migration of OpenPlatformPkg to tianocore
HI Leif It is great that you are adding more platform to edk2-platforms.
We (Intel) also have plan to add more boards there. In general, we are very close on having silicon and platform folder.
I have a quick look. One minor suggestions here: Take Arm folder as example. (I assume Juno is one package, and VExpress is the other package.) Can we name Juno to be JunoPkg, and VExpress to be VExpressPkg ? We do not add "Pkg" to a folder. And we usually add "Pkg" suffix to a package.
In general, I think it is a very good start. I may review the content in more detail and provide more feedback.
Thank you Yao Jiewen
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Leif Lindholm Sent: Thursday, May 4, 2017 6:56 AM To: edk2-devel@lists.01.orgmailto:edk2-devel@lists.01.org Cc: Ryan Harkin <ryan.harkin@linaro.orgmailto:ryan.harkin@linaro.org>; Ard Biesheuvel <ard.biesheuvel@linaro.orgmailto:ard.biesheuvel@linaro.org>; Chenhui Sun <chenhui.sun@linaro.orgmailto:chenhui.sun@linaro.org>; Andrew Fish <afish@apple.commailto:afish@apple.com>; Alan Ott <alan@softiron.co.ukmailto:alan@softiron.co.uk>; Richardson, Brian <brian.richardson@intel.commailto:brian.richardson@intel.com>; Duran, Leo <leo.duran@amd.commailto:leo.duran@amd.com>; haojian.zhuang@linaro.orgmailto:haojian.zhuang@linaro.org; Linaro UEFI <linaro-uefi@lists.linaro.orgmailto:linaro-uefi@lists.linaro.org>; Kinney, Michael D <michael.d.kinney@intel.commailto:michael.d.kinney@intel.com>; Heyi Guo <heyi.guo@linaro.orgmailto:heyi.guo@linaro.org> Subject: [edk2] [RFC] migration of OpenPlatformPkg to tianocore
Hi all,
As some of you may be aware, I have been working around the lack of a clear upstreaming strategy for platform support by keeping such code in a dedicated repository I set up at Linaro for that purpose: https://git.linaro.org/uefi/OpenPlatformPkg.git
During discussions at the last Seattle plugfest we finally agreed on the (theoretical) details of how to use the edk2-platforms repository. After that I promised to migrate OpenPlatformPkg across to the edk2-platforms and edk2-non-osi structure, with the explicit end goal from my side that this should become the master branch for each.
And now, before the release of HURD 1.0, I have.
Current limitations (that I can remember):
- A few references to OpenPlatformPkg remain, in ways that do not appear to break any of the platform builds. Most likely this affects dead code, but in case it's been accidentally orphaned, I thought it best to
- I have simply nuked all references to Ebl (used in _addition_ to the UEFI shell, which was never the intent) and the efi-toolkit ramdisk driver.
- The Marvell Yukon driver that I sent out for review last week has not migrated anywhere, and so has been temporarily disabled Mike suggested I should
- USB support on the LeMaker Cello board depends on the patch "OptionRomPkg: add firmware loader driver for Renesas PD72020x" sent out by Ard on 18th of April.
- I have dropped some of the binary-only modules from OpenPlatformPkg, and contacted the platform owners with requests for modifications.
- The git history is quite messy and will be cleaned up, but I wanted to keep the transition quite visible in the RFC.
- I haven't filled anything into the Maintainers.txt files - I am in favour of moving to a fully machine-readable format with wildcards as Laszlo has proposed in the past, and think this would be an excellent point to have that discussion (which can be had separately for edk2-platforms and edk2-non-osi from edk2).
- Few of the platforms complete the FV generation stage, and I've inserted a couple of silly hacks to get them to get as far as they do. I think that either I am missing some points of how PACKAGES_PATH is intended to work, or I'm simply hitting corner cases no one has come across before. I could really use some help debugging these issues. (examples below).
The below depends on the 3-part series I sent out today for importing DwEmmcDxe and EfiTimeBaseLib from OpenPlatformPkg. But apart from that, I have uploaded branches called devel-OpenPlatformPkg to:
https://github.com/tianocore/edk2-platforms/tree/devel-OpenPlatformPkg https://github.com/tianocore/edk2-non-osi/tree/devel-OpenPlatformPkg
These branches _will_ be rebased occasionally until they get to a point where they can move out of devel- stage (and hopefully onto master).
Build issue description
So, one of the hopefully easier ones is what I'm seeing when trying to build the Juno platform:
$ PACKAGES_PATH="/work/maint/edk2-platforms:/work/maint/edk2-non-osi" GCC5_AARCH64_PREFIX=aarch64-linux-gnu- build -a AARCH64 -t GCC5 -p Platform/ARM/Juno/ArmJuno.dsc -b RELEASE -n 9
results in:
<<< GenFds.py... : error F003: Output file for RAW section could not be found for Platform/ARM/Juno/AcpiTables/AcpiTables.inf
###
build.py... : error 7000: Failed to execute command GenFds -f /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.fdf --conf=/work/maint/edk2/Conf -o /work/maint/edk2/Build/ArmJuno/RELEASE_GCC5 -t GCC5 -b RELEASE -p /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.dsc -a AARCH64 -D "EFI_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" -D "EDK_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" -D "TOOL_CHAIN_TAG=GCC5" -D "TOOLCHAIN=GCC5" -D "TARGET=RELEASE" -D "FAMILY=GCC" -D "WORKSPACE=/work/maint/edk2" -D "EDK_TOOLS_PATH=/work/maint/edk2/BaseTools" -D "ARCH=AARCH64" -D "ECP_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" [/work/maint/edk2]
- Failed -
And when I copy and paste the above command manually, I get:
<<< GenFds.py... /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.dsc(34): error 000E: File/directory not found in workspace /work/maint/edk2-platforms/Platform/ARM/Juno/Platform/ARM/VExpress/ArmVExpress.dsc.inc /work/maint/edk2/Platform/ARM/VExpress/ArmVExpress.dsc.inc
So, to an uniformed observer, it seems the portion !include Platform/ARM/VExpress/ArmVExpress.dsc.inc from ArmJuno.dsc gets expanded to "directory ArmJuno.dsc is in" + "Platform/ARM/VExpress/ArmVExpress.dsc.inc" whereas I was hoping for it to try to find a match for "Platform/ARM/VExpress/ArmVExpress.dsc.inc" along PACKAGES_PATH (and find one in edk2-platforms).
I also have the impression that something similar is happening in ArmJuno.fdf for the line INF RuleOverride=ACPITABLE Platform/ARM/Juno/AcpiTables/AcpiTables.inf generating the error message from the original build command.
But I'm not quite sure how to debug these issues (short of fully figuring out the innards of MultipleWorkspace.py, MetaFileParser.py and a few others). Any ideas of where to look, or even what is going on? Would these cases be expected to work?
/ Leif _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.orgmailto:edk2-devel@lists.01.org%3cmailto:edk2-devel@lists.01.org> https://lists.01.org/mailman/listinfo/edk2-devel _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.orgmailto:edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
Hi Leif,
For directory name convention, I do recommend if a directory contains a package DEC file, then the directory should end in "Pkg".
I would not recommend a change to the behavior of PACKAGES_PATH at this time. Let's attempt to work through the detailed proposal with current PACKAGES_PATH behavior and see if additional features or tools are really required.
One of the key learnings I had in working in the directory structure proposal for the edk2 repo is to minimize the number of directory levels added so the PACKAGE_PATH setting does not get too long.
For the edk2-platforms layout, we can choose to have the default setting for PACKAGES_PATH to contain paths to all packages in edk2-platforms repo. This would allow all platforms to be built with a single PACKAGES_PATH setting. This is what we would also do if we moved forward with adding directory layout to edk2 repo. This does imply that the PACKAGES_PATH could be fairly long (depending on depth of directories added to these repos).
If a developer wants to pull a subset of the content from the edk2 related repositories using the git sparse checkout feature to limit the content to exactly what they need for a specific platform, then it would make sense to reduce the number of paths in PACKAGES_PATH to match the spare checkout.
We may need some tool(s) to help manage multiple repos and PACKAGES_PATH. For example, you could have a tool that searches a repo for .dec files and update PACKAGES_PATH to make all packages in that repo available to the build.
IMHO, adding directory structure to the repos is to help put packages into Different categories (e.g. core, silicon, platform, vendor specific). The side effect of adding these categories in the directory structure is increased complexity in the build configuration and build tools.
Best regards,
Mike
-----Original Message----- From: Leif Lindholm [mailto:leif.lindholm@linaro.org] Sent: Thursday, June 1, 2017 8:33 AM To: Yao, Jiewen jiewen.yao@intel.com Cc: edk2-devel@lists.01.org; Richardson, Brian brian.richardson@intel.com; Ard Biesheuvel ard.biesheuvel@linaro.org; Chenhui Sun chenhui.sun@linaro.org; Andrew Fish afish@apple.com; Alan Ott alan@softiron.co.uk; Ryan Harkin ryan.harkin@linaro.org; Duran, Leo leo.duran@amd.com; haojian.zhuang@linaro.org; Linaro UEFI linaro-uefi@lists.linaro.org; Kinney, Michael D michael.d.kinney@intel.com; Heyi Guo heyi.guo@linaro.org Subject: Re: [edk2] [RFC] migration of OpenPlatformPkg to tianocore
Hi Jiewen,
Apologies for ridicilously slow response - I caught a bad cold and am only now getting back on track with this.
Many thanks for having a look, and your comments.
On Fri, May 05, 2017 at 02:03:43PM +0000, Yao, Jiewen wrote:
Some comments for the build failure.
I think we might have different understanding on PACKAGES_PATH.
We are using below structure:
Platform\ Intel\ PlatformAPkg PlatformBPkg Silicon\ Intel\ SiliconAPkg SiliconBPkg
We set PACKAGES_PATH to "Platform\Intel:Silicon\Intel". As such Build.DSC can include PlatformAPkg/DriverA/DriverA.inf. =========================
My suggestion for your case is below: We can use
Platform\ Arm\ JunoPkg VExpressPkg
Yes, so this is probably a good point. I tried to model this repository on Mike's proposal for updated directory structure, and is was not entirely clear to me whether the *Pkg naming convention was intended to be retained at lower levels when it disappeared at the top level.
/* BEGIN sidetrack */ My understanding from before was that the "package" convention stemmed from a situation where each top-level directory was treated as an "IP silo". Since that aspect disappeared with the proposed layout changes, and the edk2-* repositories can now be seen much more (if not entirely) as cohesive structures, I had simply assumed the Pkg suffix convention would disappear. I do not actually have a strong opinion on the matter. /* END sidetrack */
In the specific cases of JunoPkg and VExpressPkg, these are not technically "packages" in this tree, since their .dec files are still in edk2. However, these should move, and if the Pkg naming convention remains, they should indeed be renamed.
We set PACKAGES_PATH to "Platform\Arm". ArmJunoPkg.dsc can just use "VExpressPkg\ArmVExpress.dsc.inc" and "JunoPkg\Library\JunoPciHostBridgeLib\ JunoPciHostBridgeLib.dsc" =========================
The benefit of adding "Pkg" is that people can have a clear picture on where the path starts from. :)
OK, so I think we have a situation where I have misunderstood the intended purpose of the PACKAGES_PATH mechanism. But, I would like to discuss the possibility of extending it to cover the use-case that I thought it provided - or potentially adding new functionality to provide the same.
Since my goal is to have many platforms (for scope, let's call it hundreds) in the edk2-platforms master branch, needing to specify for each platform all directories that contains packages used by that platform becomes quite tedious. Especially when that stretches across multiple packages in multiple repositories.
As an example, take the SoftIron 1000 platform. If the PACKAGES_PATH needs to contain all directories holding packages used outside of edk2, I would need to add: - edk2-non-osi/Platform/SoftIron/ - edk2-non-osi/Silicon/AMD/Styx/ - edk2-platforms/Platform/SoftIron/ - edk2-platforms/Silicon/AMD/Styx/
And for each other platform, I would need to specify a similar (unique for that platform) PACKAGES_PATH. And I can see more complex splits than that appearing.
Now, if everything I want to build resides across the master branch of edk2, the master branch of edk2-platforms and the master branch of edk2-non-osi, then I just need a way for the build command to search in all three repositories. At that point, the only platform-specific information I need in order to build is the path to the .dsc.
So, what I was hoping for was a way to simply extend the path resolution in .dsc/.fdf files to multiple repositories. I had hoped PACKAGES_PATH could do this, but since this was never its design goal I understand why it does not. Do you think the PACKAGES_PATH functionality could/should be extended to support this, or do I need a new feature (WORKSPACES_PATH?)?
Best Regards,
Leif
Thank you Yao Jiewen
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Yao, Jiewen Sent: Friday, May 5, 2017 9:46 PM To: Leif Lindholm leif.lindholm@linaro.org; edk2-devel@lists.01.org Cc: Richardson, Brian brian.richardson@intel.com; Ard Biesheuvel ard.biesheuvel@linaro.org; Chenhui Sun chenhui.sun@linaro.org; Andrew Fish afish@apple.com; Alan Ott alan@softiron.co.uk; Ryan Harkin ryan.harkin@linaro.org; Duran, Leo leo.duran@amd.com; haojian.zhuang@linaro.org; Linaro UEFI linaro-uefi@lists.linaro.org; Kinney, Michael D michael.d.kinney@intel.com; Heyi Guo heyi.guo@linaro.org Subject: Re: [edk2] [RFC] migration of OpenPlatformPkg to tianocore
HI Leif It is great that you are adding more platform to edk2-platforms.
We (Intel) also have plan to add more boards there. In general, we are very close on having silicon and platform folder.
I have a quick look. One minor suggestions here: Take Arm folder as example. (I assume Juno is one package, and VExpress is the other package.) Can we name Juno to be JunoPkg, and VExpress to be VExpressPkg ? We do not add "Pkg" to a folder. And we usually add "Pkg" suffix to a package.
In general, I think it is a very good start. I may review the content in more detail and provide more feedback.
Thank you Yao Jiewen
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Leif Lindholm Sent: Thursday, May 4, 2017 6:56 AM To: edk2-devel@lists.01.orgmailto:edk2-devel@lists.01.org Cc: Ryan Harkin <ryan.harkin@linaro.orgmailto:ryan.harkin@linaro.org>; Ard Biesheuvel <ard.biesheuvel@linaro.orgmailto:ard.biesheuvel@linaro.org>; Chenhui Sun <chenhui.sun@linaro.orgmailto:chenhui.sun@linaro.org>; Andrew Fish <afish@apple.commailto:afish@apple.com>; Alan Ott <alan@softiron.co.ukmailto:alan@softiron.co.uk>; Richardson, Brian <brian.richardson@intel.commailto:brian.richardson@intel.com>; Duran, Leo <leo.duran@amd.commailto:leo.duran@amd.com>; haojian.zhuang@linaro.orgmailto:haojian.zhuang@linaro.org; Linaro UEFI <linaro-uefi@lists.linaro.orgmailto:linaro-uefi@lists.linaro.org>; Kinney, Michael D <michael.d.kinney@intel.commailto:michael.d.kinney@intel.com>; Heyi Guo <heyi.guo@linaro.orgmailto:heyi.guo@linaro.org> Subject: [edk2] [RFC] migration of OpenPlatformPkg to tianocore
Hi all,
As some of you may be aware, I have been working around the lack of a clear upstreaming strategy for platform support by keeping such code in a dedicated repository I set up at Linaro for that purpose: https://git.linaro.org/uefi/OpenPlatformPkg.git
During discussions at the last Seattle plugfest we finally agreed on the (theoretical) details of how to use the edk2-platforms repository. After that I promised to migrate OpenPlatformPkg across to the edk2-platforms and edk2-non-osi structure, with the explicit end goal from my side that this should become the master branch for each.
And now, before the release of HURD 1.0, I have.
Current limitations (that I can remember):
- A few references to OpenPlatformPkg remain, in ways that do not appear to break any of the platform builds. Most likely this affects dead code, but in case it's been accidentally orphaned, I thought it best to
- I have simply nuked all references to Ebl (used in _addition_ to the UEFI shell, which was never the intent) and the efi-toolkit ramdisk driver.
- The Marvell Yukon driver that I sent out for review last week has not migrated anywhere, and so has been temporarily disabled Mike suggested I should
- USB support on the LeMaker Cello board depends on the patch "OptionRomPkg: add firmware loader driver for Renesas PD72020x" sent out by Ard on 18th of April.
- I have dropped some of the binary-only modules from OpenPlatformPkg, and contacted the platform owners with requests for modifications.
- The git history is quite messy and will be cleaned up, but I wanted to keep the transition quite visible in the RFC.
- I haven't filled anything into the Maintainers.txt files - I am in favour of moving to a fully machine-readable format with wildcards as Laszlo has proposed in the past, and think this would be an excellent point to have that discussion (which can be had separately for edk2-platforms and edk2-non-osi from edk2).
- Few of the platforms complete the FV generation stage, and I've inserted a couple of silly hacks to get them to get as far as they do. I think that either I am missing some points of how PACKAGES_PATH is intended to work, or I'm simply hitting corner cases no one has come across before. I could really use some help debugging these issues. (examples below).
The below depends on the 3-part series I sent out today for importing DwEmmcDxe and EfiTimeBaseLib from OpenPlatformPkg. But apart from that, I have uploaded branches called devel-OpenPlatformPkg to:
https://github.com/tianocore/edk2-platforms/tree/devel-OpenPlatformPkg https://github.com/tianocore/edk2-non-osi/tree/devel-OpenPlatformPkg
These branches _will_ be rebased occasionally until they get to a point where they can move out of devel- stage (and hopefully onto master).
Build issue description
So, one of the hopefully easier ones is what I'm seeing when trying to build the Juno platform:
$ PACKAGES_PATH="/work/maint/edk2-platforms:/work/maint/edk2-non-osi" GCC5_AARCH64_PREFIX=aarch64-linux-gnu- build -a AARCH64 -t GCC5 -p Platform/ARM/Juno/ArmJuno.dsc -b RELEASE -n 9
results in:
<<< GenFds.py... : error F003: Output file for RAW section could not be found for Platform/ARM/Juno/AcpiTables/AcpiTables.inf
###
build.py... : error 7000: Failed to execute command GenFds -f /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.fdf --conf=/work/maint/edk2/Conf -o /work/maint/edk2/Build/ArmJuno/RELEASE_GCC5 -t GCC5 -b RELEASE -p /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.dsc -a AARCH64 -D "EFI_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" -D "EDK_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" -D "TOOL_CHAIN_TAG=GCC5" -D "TOOLCHAIN=GCC5" -D "TARGET=RELEASE" -D "FAMILY=GCC" -D "WORKSPACE=/work/maint/edk2" -D "EDK_TOOLS_PATH=/work/maint/edk2/BaseTools" -D "ARCH=AARCH64" -D "ECP_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" [/work/maint/edk2]
- Failed -
And when I copy and paste the above command manually, I get:
<<< GenFds.py... /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.dsc(34): error 000E: File/directory not found in workspace /work/maint/edk2-platforms/Platform/ARM/Juno/Platform/ARM/VExpress/ArmVExpress.dsc.inc /work/maint/edk2/Platform/ARM/VExpress/ArmVExpress.dsc.inc
So, to an uniformed observer, it seems the portion !include Platform/ARM/VExpress/ArmVExpress.dsc.inc from ArmJuno.dsc gets expanded to "directory ArmJuno.dsc is in" + "Platform/ARM/VExpress/ArmVExpress.dsc.inc" whereas I was hoping for it to try to find a match for "Platform/ARM/VExpress/ArmVExpress.dsc.inc" along PACKAGES_PATH (and find one in edk2-platforms).
I also have the impression that something similar is happening in ArmJuno.fdf for the line INF RuleOverride=ACPITABLE Platform/ARM/Juno/AcpiTables/AcpiTables.inf generating the error message from the original build command.
But I'm not quite sure how to debug these issues (short of fully figuring out the innards of MultipleWorkspace.py, MetaFileParser.py and a few others). Any ideas of where to look, or even what is going on? Would these cases be expected to work?
/ Leif _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.orgmailto:edk2-devel@lists.01.org%3cmailto:edk2-devel@lists.01.org> https://lists.01.org/mailman/listinfo/edk2-devel _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.orgmailto:edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
Thanks Mike,
On Thu, Jun 01, 2017 at 04:49:25PM +0000, Kinney, Michael D wrote:
For directory name convention, I do recommend if a directory contains a package DEC file, then the directory should end in "Pkg".
Yes, that's fine. I'll do that.
I would not recommend a change to the behavior of PACKAGES_PATH at this time. Let's attempt to work through the detailed proposal with current PACKAGES_PATH behavior and see if additional features or tools are really required.
I'm OK with rejigging all of this such that we use PACKAGES_PATH in its current functionality for now. But this is something that will become prohibitively annoying at tens at platforms.
One of the key learnings I had in working in the directory structure proposal for the edk2 repo is to minimize the number of directory levels added so the PACKAGE_PATH setting does not get too long.
Right, but my concern is less with operating system limits on environment variables than having yet another place to modify when adding a new platform, or adding a new module to an existing one.
For the edk2-platforms layout, we can choose to have the default setting for PACKAGES_PATH to contain paths to all packages in edk2-platforms repo. This would allow all platforms to be built with a single PACKAGES_PATH setting. This is what we would also do if we moved forward with adding directory layout to edk2 repo. This does imply that the PACKAGES_PATH could be fairly long (depending on depth of directories added to these repos).
If a developer wants to pull a subset of the content from the edk2 related repositories using the git sparse checkout feature to limit the content to exactly what they need for a specific platform, then it would make sense to reduce the number of paths in PACKAGES_PATH to match the spare checkout.
We may need some tool(s) to help manage multiple repos and PACKAGES_PATH. For example, you could have a tool that searches a repo for .dec files and update PACKAGES_PATH to make all packages in that repo available to the build.
IMHO, adding directory structure to the repos is to help put packages into Different categories (e.g. core, silicon, platform, vendor specific). The side effect of adding these categories in the directory structure is increased complexity in the build configuration and build tools.
I do not dispute this. This is why I think in the end something like WORKSPACES_PATH is going to be a be a much more scalable solution. But let's revisit that a few months down the line.
Best Regards,
Leif
Best regards,
Mike
-----Original Message----- From: Leif Lindholm [mailto:leif.lindholm@linaro.org] Sent: Thursday, June 1, 2017 8:33 AM To: Yao, Jiewen jiewen.yao@intel.com Cc: edk2-devel@lists.01.org; Richardson, Brian brian.richardson@intel.com; Ard Biesheuvel ard.biesheuvel@linaro.org; Chenhui Sun chenhui.sun@linaro.org; Andrew Fish afish@apple.com; Alan Ott alan@softiron.co.uk; Ryan Harkin ryan.harkin@linaro.org; Duran, Leo leo.duran@amd.com; haojian.zhuang@linaro.org; Linaro UEFI linaro-uefi@lists.linaro.org; Kinney, Michael D michael.d.kinney@intel.com; Heyi Guo heyi.guo@linaro.org Subject: Re: [edk2] [RFC] migration of OpenPlatformPkg to tianocore
Hi Jiewen,
Apologies for ridicilously slow response - I caught a bad cold and am only now getting back on track with this.
Many thanks for having a look, and your comments.
On Fri, May 05, 2017 at 02:03:43PM +0000, Yao, Jiewen wrote:
Some comments for the build failure.
I think we might have different understanding on PACKAGES_PATH.
We are using below structure:
Platform\ Intel\ PlatformAPkg PlatformBPkg Silicon\ Intel\ SiliconAPkg SiliconBPkg
We set PACKAGES_PATH to "Platform\Intel:Silicon\Intel". As such Build.DSC can include PlatformAPkg/DriverA/DriverA.inf. =========================
My suggestion for your case is below: We can use
Platform\ Arm\ JunoPkg VExpressPkg
Yes, so this is probably a good point. I tried to model this repository on Mike's proposal for updated directory structure, and is was not entirely clear to me whether the *Pkg naming convention was intended to be retained at lower levels when it disappeared at the top level.
/* BEGIN sidetrack */ My understanding from before was that the "package" convention stemmed from a situation where each top-level directory was treated as an "IP silo". Since that aspect disappeared with the proposed layout changes, and the edk2-* repositories can now be seen much more (if not entirely) as cohesive structures, I had simply assumed the Pkg suffix convention would disappear. I do not actually have a strong opinion on the matter. /* END sidetrack */
In the specific cases of JunoPkg and VExpressPkg, these are not technically "packages" in this tree, since their .dec files are still in edk2. However, these should move, and if the Pkg naming convention remains, they should indeed be renamed.
We set PACKAGES_PATH to "Platform\Arm". ArmJunoPkg.dsc can just use "VExpressPkg\ArmVExpress.dsc.inc" and "JunoPkg\Library\JunoPciHostBridgeLib\ JunoPciHostBridgeLib.dsc" =========================
The benefit of adding "Pkg" is that people can have a clear picture on where the path starts from. :)
OK, so I think we have a situation where I have misunderstood the intended purpose of the PACKAGES_PATH mechanism. But, I would like to discuss the possibility of extending it to cover the use-case that I thought it provided - or potentially adding new functionality to provide the same.
Since my goal is to have many platforms (for scope, let's call it hundreds) in the edk2-platforms master branch, needing to specify for each platform all directories that contains packages used by that platform becomes quite tedious. Especially when that stretches across multiple packages in multiple repositories.
As an example, take the SoftIron 1000 platform. If the PACKAGES_PATH needs to contain all directories holding packages used outside of edk2, I would need to add:
- edk2-non-osi/Platform/SoftIron/
- edk2-non-osi/Silicon/AMD/Styx/
- edk2-platforms/Platform/SoftIron/
- edk2-platforms/Silicon/AMD/Styx/
And for each other platform, I would need to specify a similar (unique for that platform) PACKAGES_PATH. And I can see more complex splits than that appearing.
Now, if everything I want to build resides across the master branch of edk2, the master branch of edk2-platforms and the master branch of edk2-non-osi, then I just need a way for the build command to search in all three repositories. At that point, the only platform-specific information I need in order to build is the path to the .dsc.
So, what I was hoping for was a way to simply extend the path resolution in .dsc/.fdf files to multiple repositories. I had hoped PACKAGES_PATH could do this, but since this was never its design goal I understand why it does not. Do you think the PACKAGES_PATH functionality could/should be extended to support this, or do I need a new feature (WORKSPACES_PATH?)?
Best Regards,
Leif
Thank you Yao Jiewen
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Yao, Jiewen Sent: Friday, May 5, 2017 9:46 PM To: Leif Lindholm leif.lindholm@linaro.org; edk2-devel@lists.01.org Cc: Richardson, Brian brian.richardson@intel.com; Ard Biesheuvel ard.biesheuvel@linaro.org; Chenhui Sun chenhui.sun@linaro.org; Andrew Fish afish@apple.com; Alan Ott alan@softiron.co.uk; Ryan Harkin ryan.harkin@linaro.org; Duran, Leo leo.duran@amd.com; haojian.zhuang@linaro.org; Linaro UEFI linaro-uefi@lists.linaro.org; Kinney, Michael D michael.d.kinney@intel.com; Heyi Guo heyi.guo@linaro.org Subject: Re: [edk2] [RFC] migration of OpenPlatformPkg to tianocore
HI Leif It is great that you are adding more platform to edk2-platforms.
We (Intel) also have plan to add more boards there. In general, we are very close on having silicon and platform folder.
I have a quick look. One minor suggestions here: Take Arm folder as example. (I assume Juno is one package, and VExpress is the other package.) Can we name Juno to be JunoPkg, and VExpress to be VExpressPkg ? We do not add "Pkg" to a folder. And we usually add "Pkg" suffix to a package.
In general, I think it is a very good start. I may review the content in more detail and provide more feedback.
Thank you Yao Jiewen
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Leif Lindholm Sent: Thursday, May 4, 2017 6:56 AM To: edk2-devel@lists.01.orgmailto:edk2-devel@lists.01.org Cc: Ryan Harkin <ryan.harkin@linaro.orgmailto:ryan.harkin@linaro.org>; Ard Biesheuvel <ard.biesheuvel@linaro.orgmailto:ard.biesheuvel@linaro.org>; Chenhui Sun <chenhui.sun@linaro.orgmailto:chenhui.sun@linaro.org>; Andrew Fish <afish@apple.commailto:afish@apple.com>; Alan Ott <alan@softiron.co.ukmailto:alan@softiron.co.uk>; Richardson, Brian <brian.richardson@intel.commailto:brian.richardson@intel.com>; Duran, Leo <leo.duran@amd.commailto:leo.duran@amd.com>; haojian.zhuang@linaro.orgmailto:haojian.zhuang@linaro.org; Linaro UEFI <linaro-uefi@lists.linaro.orgmailto:linaro-uefi@lists.linaro.org>; Kinney, Michael D <michael.d.kinney@intel.commailto:michael.d.kinney@intel.com>; Heyi Guo <heyi.guo@linaro.orgmailto:heyi.guo@linaro.org> Subject: [edk2] [RFC] migration of OpenPlatformPkg to tianocore
Hi all,
As some of you may be aware, I have been working around the lack of a clear upstreaming strategy for platform support by keeping such code in a dedicated repository I set up at Linaro for that purpose: https://git.linaro.org/uefi/OpenPlatformPkg.git
During discussions at the last Seattle plugfest we finally agreed on the (theoretical) details of how to use the edk2-platforms repository. After that I promised to migrate OpenPlatformPkg across to the edk2-platforms and edk2-non-osi structure, with the explicit end goal from my side that this should become the master branch for each.
And now, before the release of HURD 1.0, I have.
Current limitations (that I can remember):
- A few references to OpenPlatformPkg remain, in ways that do not appear to break any of the platform builds. Most likely this affects dead code, but in case it's been accidentally orphaned, I thought it best to
- I have simply nuked all references to Ebl (used in _addition_ to the UEFI shell, which was never the intent) and the efi-toolkit ramdisk driver.
- The Marvell Yukon driver that I sent out for review last week has not migrated anywhere, and so has been temporarily disabled Mike suggested I should
- USB support on the LeMaker Cello board depends on the patch "OptionRomPkg: add firmware loader driver for Renesas PD72020x" sent out by Ard on 18th of April.
- I have dropped some of the binary-only modules from OpenPlatformPkg, and contacted the platform owners with requests for modifications.
- The git history is quite messy and will be cleaned up, but I wanted to keep the transition quite visible in the RFC.
- I haven't filled anything into the Maintainers.txt files - I am in favour of moving to a fully machine-readable format with wildcards as Laszlo has proposed in the past, and think this would be an excellent point to have that discussion (which can be had separately for edk2-platforms and edk2-non-osi from edk2).
- Few of the platforms complete the FV generation stage, and I've inserted a couple of silly hacks to get them to get as far as they do. I think that either I am missing some points of how PACKAGES_PATH is intended to work, or I'm simply hitting corner cases no one has come across before. I could really use some help debugging these issues. (examples below).
The below depends on the 3-part series I sent out today for importing DwEmmcDxe and EfiTimeBaseLib from OpenPlatformPkg. But apart from that, I have uploaded branches called devel-OpenPlatformPkg to:
https://github.com/tianocore/edk2-platforms/tree/devel-OpenPlatformPkg https://github.com/tianocore/edk2-non-osi/tree/devel-OpenPlatformPkg
These branches _will_ be rebased occasionally until they get to a point where they can move out of devel- stage (and hopefully onto master).
Build issue description
So, one of the hopefully easier ones is what I'm seeing when trying to build the Juno platform:
$ PACKAGES_PATH="/work/maint/edk2-platforms:/work/maint/edk2-non-osi" GCC5_AARCH64_PREFIX=aarch64-linux-gnu- build -a AARCH64 -t GCC5 -p Platform/ARM/Juno/ArmJuno.dsc -b RELEASE -n 9
results in:
<<< GenFds.py... : error F003: Output file for RAW section could not be found for Platform/ARM/Juno/AcpiTables/AcpiTables.inf
###
build.py... : error 7000: Failed to execute command GenFds -f /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.fdf --conf=/work/maint/edk2/Conf -o /work/maint/edk2/Build/ArmJuno/RELEASE_GCC5 -t GCC5 -b RELEASE -p /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.dsc -a AARCH64 -D "EFI_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" -D "EDK_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" -D "TOOL_CHAIN_TAG=GCC5" -D "TOOLCHAIN=GCC5" -D "TARGET=RELEASE" -D "FAMILY=GCC" -D "WORKSPACE=/work/maint/edk2" -D "EDK_TOOLS_PATH=/work/maint/edk2/BaseTools" -D "ARCH=AARCH64" -D "ECP_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" [/work/maint/edk2]
- Failed -
And when I copy and paste the above command manually, I get:
<<< GenFds.py... /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.dsc(34): error 000E: File/directory not found in workspace /work/maint/edk2-platforms/Platform/ARM/Juno/Platform/ARM/VExpress/ArmVExpress.dsc.inc /work/maint/edk2/Platform/ARM/VExpress/ArmVExpress.dsc.inc
So, to an uniformed observer, it seems the portion !include Platform/ARM/VExpress/ArmVExpress.dsc.inc from ArmJuno.dsc gets expanded to "directory ArmJuno.dsc is in" + "Platform/ARM/VExpress/ArmVExpress.dsc.inc" whereas I was hoping for it to try to find a match for "Platform/ARM/VExpress/ArmVExpress.dsc.inc" along PACKAGES_PATH (and find one in edk2-platforms).
I also have the impression that something similar is happening in ArmJuno.fdf for the line INF RuleOverride=ACPITABLE Platform/ARM/Juno/AcpiTables/AcpiTables.inf generating the error message from the original build command.
But I'm not quite sure how to debug these issues (short of fully figuring out the innards of MultipleWorkspace.py, MetaFileParser.py and a few others). Any ideas of where to look, or even what is going on? Would these cases be expected to work?
/ Leif _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.org<mailto:edk2-devel@lists.01.orgmailto:edk2-devel@lists.01.org%3cmailto:edk2-devel@lists.01.org> https://lists.01.org/mailman/listinfo/edk2-devel _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.orgmailto:edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
Hi all,
OK, so I have now updated both edk2-non-osi and edk2-platforms (devel-OpenPlatformPkg branches) with package renaming, and updating .dsc/.fdf files to .
It appears the problems I'm facing are mainly caused by the GenFds stage not seeing a view consistent with the actual compilation stage.
To demonstrate, I build the Juno platform: $ . edksetup.sh $ make -C BaseTools $ PACKAGES_PATH=/work/git/edk2-platforms/Platform/ARM \ GCC5_AARCH64_PREFIX=aarch64-linux-gnu- build -n 9 -a AARCH64 \ -t GCC5 -p JunoPkg/ArmJuno.dsc -b RELEASE
This build fails with: --- Fd File Name:BL33_AP_UEFI
Generate Region at Offset 0x0 Region Size = 0xF8000 Region Name = FV
Generating FVMAIN_COMPACT FV #### Generating FVMAIN FV
GenFds.py... : error F003: Output file for RAW section could not be found for JunoPkg/AcpiTables/AcpiTables.inf
build.py... : error 7000: Failed to execute command GenFds -f /work/git/edk2-platforms/Platform/ARM/JunoPkg/ArmJuno.fdf --conf=/work/git/edk2/Conf -o /work/git/edk2/Build/ArmJuno/RELEASE_GCC5 -t GCC5 -b RELEASE -p /work/git/edk2-platforms/Platform/ARM/JunoPkg/ArmJuno.dsc -a AARCH64 -D "EFI_SOURCE=/work/git/edk2/EdkCompatibilityPkg" -D "EDK_SOURCE=/work/git/edk2/EdkCompatibilityPkg" -D "TOOL_CHAIN_TAG=GCC5" -D "TOOLCHAIN=GCC5" -D "TARGET=RELEASE" -D "FAMILY=GCC" -D "WORKSPACE=/work/git/edk2" -D "EDK_TOOLS_PATH=/work/git/edk2/BaseTools" -D "ARCH=AARCH64" -D "ECP_SOURCE=/work/git/edk2/EdkCompatibilityPkg" [/work/git/edk2]
- Failed - --- , but if I add the symlink ln -s edk2-platforms/Platform/ARM/JunoPkg/ Build/ArmJuno/RELEASE_GCC5/AARCH64/ it completes successfully on the next pass.
Same when building VExpressPkg/ArmVExpress-FVP-AArch64.dsc ln -s edk2-platforms/Platform/ARM/VExpressPkg \ Build/ArmVExpress-FVP-AArch64/RELEASE_GCC5/AARCH64/ resolves the build error.
Similarly, when I build the Overdrive/Overdrive.dsc, using: $ PACKAGES_PATH=/work/git/edk2-platforms/Platform/AMD:/work/git/edk2-platforms/Silicon/AMD:/work/git/edk2-non-osi/Silicon/AMD/Styx:/work/git/edk2-non-osi/Platform/AMD \ GCC5_AARCH64_PREFIX=aarch64-linux-gnu- build -n 9 -a AARCH64 -t GCC5 -p OverdriveBoard/OverdriveBoard.dsc -b RELEASE this fails with --- Fd File Name:STYX_ROM
Generate Region at Offset 0x0 Region Size = 0x200000 Region File Name = /work/git/edk2/../edk2-non-osi/Platform/AMD/OverdriveBoard/PreUefiFirmware.bin
Generate Region at Offset 0x200000 Region Size = 0x260000 Region Name = FV
Generating STYX_EFI FV ################Return Value = 2 GenFw: ERROR 0001: Error opening file
/work/git/edk2/Build/Overdrive/RELEASE_GCC5/AARCH64/StyxPkg/Drivers/PlatInitPei/PlatInitPei/OUTPUT/PlatInitPei.efi
GenFds.py... ### ['GenFw', '-t', '-o', ### '/work/git/edk2/Build/Overdrive/RELEASE_GCC5/FV/Ffs/769694a4-2572-4f29-a5bb-33d7df7be001PlatInitPei/769694a4-2572-4f29-a5bb-33d7df7be001Te.raw', ### '/work/git/edk2/Build/Overdrive/RELEASE_GCC5/AARCH64/StyxPkg/Drivers/PlatInitPei/PlatInitPei/OUTPUT/PlatInitPei.efi'] : error 7000: Failed to generate firmware image
build.py... : error 7000: Failed to execute command GenFds -f /work/git/edk2-platforms/Platform/AMD/OverdriveBoard/OverdriveBoard.fdf --conf=/work/git/edk2/Conf -o /work/git/edk2/Build/Overdrive/RELEASE_GCC5 -t GCC5 -b RELEASE -p /work/git/edk2-platforms/Platform/AMD/OverdriveBoard/OverdriveBoard.dsc -a AARCH64 -D "EFI_SOURCE=/work/git/edk2/EdkCompatibilityPkg" -D "EDK_SOURCE=/work/git/edk2/EdkCompatibilityPkg" -D "TOOL_CHAIN_TAG=GCC5" -D "TOOLCHAIN=GCC5" -D "TARGET=RELEASE" -D "FAMILY=GCC" -D "WORKSPACE=/work/git/edk2" -D "FIRMWARE_VER=b941c34" -D "EDK_TOOLS_PATH=/work/git/edk2/BaseTools" -D "ARCH=AARCH64" -D "ECP_SOURCE=/work/git/edk2/EdkCompatibilityPkg" [/work/git/edk2]
- Failed - ---
Again, resolved by ln -s edk2-platforms/Silicon/AMD/StyxPkg/ Build/Overdrive/RELEASE_GCC5/AARCH64/
Am I doing something wrong here? Are there any magic runes that could help out?
/ Leif
On Fri, May 05, 2017 at 01:45:51PM +0000, Yao, Jiewen wrote:
HI Leif It is great that you are adding more platform to edk2-platforms.
We (Intel) also have plan to add more boards there. In general, we are very close on having silicon and platform folder.
I have a quick look. One minor suggestions here: Take Arm folder as example. (I assume Juno is one package, and VExpress is the other package.) Can we name Juno to be JunoPkg, and VExpress to be VExpressPkg ? We do not add "Pkg" to a folder. And we usually add "Pkg" suffix to a package.
In general, I think it is a very good start. I may review the content in more detail and provide more feedback.
Thank you Yao Jiewen
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Leif Lindholm Sent: Thursday, May 4, 2017 6:56 AM To: edk2-devel@lists.01.org Cc: Ryan Harkin ryan.harkin@linaro.org; Ard Biesheuvel ard.biesheuvel@linaro.org; Chenhui Sun chenhui.sun@linaro.org; Andrew Fish afish@apple.com; Alan Ott alan@softiron.co.uk; Richardson, Brian brian.richardson@intel.com; Duran, Leo leo.duran@amd.com; haojian.zhuang@linaro.org; Linaro UEFI linaro-uefi@lists.linaro.org; Kinney, Michael D michael.d.kinney@intel.com; Heyi Guo heyi.guo@linaro.org Subject: [edk2] [RFC] migration of OpenPlatformPkg to tianocore
Hi all,
As some of you may be aware, I have been working around the lack of a clear upstreaming strategy for platform support by keeping such code in a dedicated repository I set up at Linaro for that purpose: https://git.linaro.org/uefi/OpenPlatformPkg.git
During discussions at the last Seattle plugfest we finally agreed on the (theoretical) details of how to use the edk2-platforms repository. After that I promised to migrate OpenPlatformPkg across to the edk2-platforms and edk2-non-osi structure, with the explicit end goal from my side that this should become the master branch for each.
And now, before the release of HURD 1.0, I have.
Current limitations (that I can remember):
- A few references to OpenPlatformPkg remain, in ways that do not appear to break any of the platform builds. Most likely this affects dead code, but in case it's been accidentally orphaned, I thought it best to
- I have simply nuked all references to Ebl (used in _addition_ to the UEFI shell, which was never the intent) and the efi-toolkit ramdisk driver.
- The Marvell Yukon driver that I sent out for review last week has not migrated anywhere, and so has been temporarily disabled Mike suggested I should
- USB support on the LeMaker Cello board depends on the patch "OptionRomPkg: add firmware loader driver for Renesas PD72020x" sent out by Ard on 18th of April.
- I have dropped some of the binary-only modules from OpenPlatformPkg, and contacted the platform owners with requests for modifications.
- The git history is quite messy and will be cleaned up, but I wanted to keep the transition quite visible in the RFC.
- I haven't filled anything into the Maintainers.txt files - I am in favour of moving to a fully machine-readable format with wildcards as Laszlo has proposed in the past, and think this would be an excellent point to have that discussion (which can be had separately for edk2-platforms and edk2-non-osi from edk2).
- Few of the platforms complete the FV generation stage, and I've inserted a couple of silly hacks to get them to get as far as they do. I think that either I am missing some points of how PACKAGES_PATH is intended to work, or I'm simply hitting corner cases no one has come across before. I could really use some help debugging these issues. (examples below).
The below depends on the 3-part series I sent out today for importing DwEmmcDxe and EfiTimeBaseLib from OpenPlatformPkg. But apart from that, I have uploaded branches called devel-OpenPlatformPkg to:
https://github.com/tianocore/edk2-platforms/tree/devel-OpenPlatformPkg https://github.com/tianocore/edk2-non-osi/tree/devel-OpenPlatformPkg
These branches _will_ be rebased occasionally until they get to a point where they can move out of devel- stage (and hopefully onto master).
Build issue description
So, one of the hopefully easier ones is what I'm seeing when trying to build the Juno platform:
$ PACKAGES_PATH="/work/maint/edk2-platforms:/work/maint/edk2-non-osi" GCC5_AARCH64_PREFIX=aarch64-linux-gnu- build -a AARCH64 -t GCC5 -p Platform/ARM/Juno/ArmJuno.dsc -b RELEASE -n 9
results in:
<<< GenFds.py... : error F003: Output file for RAW section could not be found for Platform/ARM/Juno/AcpiTables/AcpiTables.inf
###
build.py... : error 7000: Failed to execute command GenFds -f /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.fdf --conf=/work/maint/edk2/Conf -o /work/maint/edk2/Build/ArmJuno/RELEASE_GCC5 -t GCC5 -b RELEASE -p /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.dsc -a AARCH64 -D "EFI_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" -D "EDK_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" -D "TOOL_CHAIN_TAG=GCC5" -D "TOOLCHAIN=GCC5" -D "TARGET=RELEASE" -D "FAMILY=GCC" -D "WORKSPACE=/work/maint/edk2" -D "EDK_TOOLS_PATH=/work/maint/edk2/BaseTools" -D "ARCH=AARCH64" -D "ECP_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" [/work/maint/edk2]
- Failed -
And when I copy and paste the above command manually, I get:
<<< GenFds.py... /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.dsc(34): error 000E: File/directory not found in workspace /work/maint/edk2-platforms/Platform/ARM/Juno/Platform/ARM/VExpress/ArmVExpress.dsc.inc /work/maint/edk2/Platform/ARM/VExpress/ArmVExpress.dsc.inc
So, to an uniformed observer, it seems the portion !include Platform/ARM/VExpress/ArmVExpress.dsc.inc from ArmJuno.dsc gets expanded to "directory ArmJuno.dsc is in" + "Platform/ARM/VExpress/ArmVExpress.dsc.inc" whereas I was hoping for it to try to find a match for "Platform/ARM/VExpress/ArmVExpress.dsc.inc" along PACKAGES_PATH (and find one in edk2-platforms).
I also have the impression that something similar is happening in ArmJuno.fdf for the line INF RuleOverride=ACPITABLE Platform/ARM/Juno/AcpiTables/AcpiTables.inf generating the error message from the original build command.
But I'm not quite sure how to debug these issues (short of fully figuring out the innards of MultipleWorkspace.py, MetaFileParser.py and a few others). Any ideas of where to look, or even what is going on? Would these cases be expected to work?
/ Leif _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.orgmailto:edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
Hi Leif,
I pulled the latest versions of the repos and I was able to complete a build with no errors and without using symbolic links.
The change I made is to set WORKSPACE to the directory immediately above the repos and set PACKAGES_PATH to directories that contain the packages required for the platform to build.
This is the same technique used to build QuarkPlatformPkg that uses content from both the edk2 and the edk2-non-osi repos. See the Readme.md in the QuarkPlatformPkg with instructions for setting up the build env for Linux and Windows for an example.
https://github.com/tianocore/edk2/tree/master/QuarkPlatformPkg
I wrote a small shell script that I put in the directory above the repos to setup the build environment for the JunoPkg platform.
export GCC5_AARCH64_PREFIX=aarch64-linux-gnu-
export WORKSPACE=$PWD
export PACKAGES_PATH=\ $WORKSPACE/edk2:\ $WORKSPACE/edk2-platforms/Platform/ARM:\ $WORKSPACE/edk2-non-osi
cd edk2 make -C BaseTools . edksetup.sh BaseTools
build -a AARCH64 -t GCC5 -p JunoPkg/ArmJuno.dsc
=== end of build log === Generate Region at Offset 0x0 Region Size = 0xF8000 Region Name = FV
Generating FVMAIN_COMPACT FV #### Generating FVMAIN FV #################################### ######################################## ######################################## ######################################## ################################ Generate Region at Offset 0x0 Region Size = 0xF8000 Region Name = FV
GUID cross reference file can be found at /home/mdkinney/GitHub/tianocore/Build/ArmJuno/DEBUG_GCC5/FV/Guid.xref
FV Space Information FVMAIN_COMPACT [94%Full] 1015808 total, 956200 used, 59608 free FVMAIN [99%Full] 8237696 total, 8237688 used, 8 free
- Done - Build end time: 18:49:57, Jun.13 2017 Build total time: 00:04:22 ========================
If you want to support building any of the platforms, then PACKAGES_PATH can be set as follows:
export PACKAGES_PATH=\ $WORKSPACE/edk2:\ $WORKSPACE/edk2-platforms/Silicon/Hisilicon:\ $WORKSPACE/edk2-platforms/Silicon/AMD:\ $WORKSPACE/edk2-platforms/Platform/Hisilicon:\ $WORKSPACE/edk2-platforms/Platform/Marvell:\ $WORKSPACE/edk2-platforms/Platform/ARM:\ $WORKSPACE/edk2-non-osi/Silicon/Intel:\ $WORKSPACE/edk2-non-osi/Silicon/AMD/Styx:
Best regards,
Mike
-----Original Message----- From: Leif Lindholm [mailto:leif.lindholm@linaro.org] Sent: Wednesday, June 7, 2017 7:58 AM To: Yao, Jiewen jiewen.yao@intel.com Cc: edk2-devel@lists.01.org; Ryan Harkin ryan.harkin@linaro.org; Ard Biesheuvel ard.biesheuvel@linaro.org; Chenhui Sun chenhui.sun@linaro.org; Andrew Fish afish@apple.com; Alan Ott alan@softiron.co.uk; Richardson, Brian brian.richardson@intel.com; Duran, Leo leo.duran@amd.com; haojian.zhuang@linaro.org; Linaro UEFI linaro-uefi@lists.linaro.org; Kinney, Michael D michael.d.kinney@intel.com; Heyi Guo heyi.guo@linaro.org Subject: Re: [edk2] [RFC] migration of OpenPlatformPkg to tianocore
Hi all,
OK, so I have now updated both edk2-non-osi and edk2-platforms (devel-OpenPlatformPkg branches) with package renaming, and updating .dsc/.fdf files to .
It appears the problems I'm facing are mainly caused by the GenFds stage not seeing a view consistent with the actual compilation stage.
To demonstrate, I build the Juno platform: $ . edksetup.sh $ make -C BaseTools $ PACKAGES_PATH=/work/git/edk2-platforms/Platform/ARM \ GCC5_AARCH64_PREFIX=aarch64-linux-gnu- build -n 9 -a AARCH64 \ -t GCC5 -p JunoPkg/ArmJuno.dsc -b RELEASE
This build fails with: --- Fd File Name:BL33_AP_UEFI
Generate Region at Offset 0x0 Region Size = 0xF8000 Region Name = FV
Generating FVMAIN_COMPACT FV #### Generating FVMAIN FV
GenFds.py... : error F003: Output file for RAW section could not be found for JunoPkg/AcpiTables/AcpiTables.inf
build.py... : error 7000: Failed to execute command GenFds -f /work/git/edk2-platforms/Platform/ARM/JunoPkg/ArmJuno.fdf --conf=/work/git/edk2/Conf -o /work/git/edk2/Build/ArmJuno/RELEASE_GCC5 -t GCC5 -b RELEASE -p /work/git/edk2-platforms/Platform/ARM/JunoPkg/ArmJuno.dsc -a AARCH64 -D "EFI_SOURCE=/work/git/edk2/EdkCompatibilityPkg" -D "EDK_SOURCE=/work/git/edk2/EdkCompatibilityPkg" -D "TOOL_CHAIN_TAG=GCC5" -D "TOOLCHAIN=GCC5" -D "TARGET=RELEASE" -D "FAMILY=GCC" -D "WORKSPACE=/work/git/edk2" -D "EDK_TOOLS_PATH=/work/git/edk2/BaseTools" -D "ARCH=AARCH64" -D "ECP_SOURCE=/work/git/edk2/EdkCompatibilityPkg" [/work/git/edk2]
- Failed - --- , but if I add the symlink ln -s edk2-platforms/Platform/ARM/JunoPkg/ Build/ArmJuno/RELEASE_GCC5/AARCH64/ it completes successfully on the next pass.
Same when building VExpressPkg/ArmVExpress-FVP-AArch64.dsc ln -s edk2-platforms/Platform/ARM/VExpressPkg \ Build/ArmVExpress-FVP-AArch64/RELEASE_GCC5/AARCH64/ resolves the build error.
Similarly, when I build the Overdrive/Overdrive.dsc, using: $ PACKAGES_PATH=/work/git/edk2-platforms/Platform/AMD:/work/git/edk2-platforms/Silicon/AMD:/work/git/edk2-non-osi/Silicon/AMD/Styx:/work/git/edk2-non-osi/Platform/AMD \ GCC5_AARCH64_PREFIX=aarch64-linux-gnu- build -n 9 -a AARCH64 -t GCC5 -p OverdriveBoard/OverdriveBoard.dsc -b RELEASE this fails with --- Fd File Name:STYX_ROM
Generate Region at Offset 0x0 Region Size = 0x200000 Region File Name = /work/git/edk2/../edk2-non-osi/Platform/AMD/OverdriveBoard/PreUefiFirmware.bin
Generate Region at Offset 0x200000 Region Size = 0x260000 Region Name = FV
Generating STYX_EFI FV ################Return Value = 2 GenFw: ERROR 0001: Error opening file
/work/git/edk2/Build/Overdrive/RELEASE_GCC5/AARCH64/StyxPkg/Drivers/PlatInitPei/PlatInitPei/OUTPUT/PlatInitPei.efi
GenFds.py... ### ['GenFw', '-t', '-o', ### '/work/git/edk2/Build/Overdrive/RELEASE_GCC5/FV/Ffs/769694a4-2572-4f29-a5bb-33d7df7be001PlatInitPei/769694a4-2572-4f29-a5bb-33d7df7be001Te.raw', ### '/work/git/edk2/Build/Overdrive/RELEASE_GCC5/AARCH64/StyxPkg/Drivers/PlatInitPei/PlatInitPei/OUTPUT/PlatInitPei.efi'] : error 7000: Failed to generate firmware image
build.py... : error 7000: Failed to execute command GenFds -f /work/git/edk2-platforms/Platform/AMD/OverdriveBoard/OverdriveBoard.fdf --conf=/work/git/edk2/Conf -o /work/git/edk2/Build/Overdrive/RELEASE_GCC5 -t GCC5 -b RELEASE -p /work/git/edk2-platforms/Platform/AMD/OverdriveBoard/OverdriveBoard.dsc -a AARCH64 -D "EFI_SOURCE=/work/git/edk2/EdkCompatibilityPkg" -D "EDK_SOURCE=/work/git/edk2/EdkCompatibilityPkg" -D "TOOL_CHAIN_TAG=GCC5" -D "TOOLCHAIN=GCC5" -D "TARGET=RELEASE" -D "FAMILY=GCC" -D "WORKSPACE=/work/git/edk2" -D "FIRMWARE_VER=b941c34" -D "EDK_TOOLS_PATH=/work/git/edk2/BaseTools" -D "ARCH=AARCH64" -D "ECP_SOURCE=/work/git/edk2/EdkCompatibilityPkg" [/work/git/edk2]
- Failed - ---
Again, resolved by ln -s edk2-platforms/Silicon/AMD/StyxPkg/ Build/Overdrive/RELEASE_GCC5/AARCH64/
Am I doing something wrong here? Are there any magic runes that could help out?
/ Leif
On Fri, May 05, 2017 at 01:45:51PM +0000, Yao, Jiewen wrote:
HI Leif It is great that you are adding more platform to edk2-platforms.
We (Intel) also have plan to add more boards there. In general, we are very close on having silicon and platform folder.
I have a quick look. One minor suggestions here: Take Arm folder as example. (I assume Juno is one package, and VExpress is the other package.) Can we name Juno to be JunoPkg, and VExpress to be VExpressPkg ? We do not add "Pkg" to a folder. And we usually add "Pkg" suffix to a package.
In general, I think it is a very good start. I may review the content in more detail and provide more feedback.
Thank you Yao Jiewen
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Leif Lindholm Sent: Thursday, May 4, 2017 6:56 AM To: edk2-devel@lists.01.org Cc: Ryan Harkin ryan.harkin@linaro.org; Ard Biesheuvel ard.biesheuvel@linaro.org; Chenhui Sun chenhui.sun@linaro.org; Andrew Fish afish@apple.com; Alan Ott alan@softiron.co.uk; Richardson, Brian brian.richardson@intel.com; Duran, Leo leo.duran@amd.com; haojian.zhuang@linaro.org; Linaro UEFI linaro-uefi@lists.linaro.org; Kinney, Michael D michael.d.kinney@intel.com; Heyi Guo heyi.guo@linaro.org Subject: [edk2] [RFC] migration of OpenPlatformPkg to tianocore
Hi all,
As some of you may be aware, I have been working around the lack of a clear upstreaming strategy for platform support by keeping such code in a dedicated repository I set up at Linaro for that purpose: https://git.linaro.org/uefi/OpenPlatformPkg.git
During discussions at the last Seattle plugfest we finally agreed on the (theoretical) details of how to use the edk2-platforms repository. After that I promised to migrate OpenPlatformPkg across to the edk2-platforms and edk2-non-osi structure, with the explicit end goal from my side that this should become the master branch for each.
And now, before the release of HURD 1.0, I have.
Current limitations (that I can remember):
- A few references to OpenPlatformPkg remain, in ways that do not appear to break any of the platform builds. Most likely this affects dead code, but in case it's been accidentally orphaned, I thought it best to
- I have simply nuked all references to Ebl (used in _addition_ to the UEFI shell, which was never the intent) and the efi-toolkit ramdisk driver.
- The Marvell Yukon driver that I sent out for review last week has not migrated anywhere, and so has been temporarily disabled Mike suggested I should
- USB support on the LeMaker Cello board depends on the patch "OptionRomPkg: add firmware loader driver for Renesas PD72020x" sent out by Ard on 18th of April.
- I have dropped some of the binary-only modules from OpenPlatformPkg, and contacted the platform owners with requests for modifications.
- The git history is quite messy and will be cleaned up, but I wanted to keep the transition quite visible in the RFC.
- I haven't filled anything into the Maintainers.txt files - I am in favour of moving to a fully machine-readable format with wildcards as Laszlo has proposed in the past, and think this would be an excellent point to have that discussion (which can be had separately for edk2-platforms and edk2-non-osi from edk2).
- Few of the platforms complete the FV generation stage, and I've inserted a couple of silly hacks to get them to get as far as they do. I think that either I am missing some points of how PACKAGES_PATH is intended to work, or I'm simply hitting corner cases no one has come across before. I could really use some help debugging these issues. (examples below).
The below depends on the 3-part series I sent out today for importing DwEmmcDxe and EfiTimeBaseLib from OpenPlatformPkg. But apart from that, I have uploaded branches called devel-OpenPlatformPkg to:
https://github.com/tianocore/edk2-platforms/tree/devel-OpenPlatformPkg https://github.com/tianocore/edk2-non-osi/tree/devel-OpenPlatformPkg
These branches _will_ be rebased occasionally until they get to a point where they can move out of devel- stage (and hopefully onto master).
Build issue description
So, one of the hopefully easier ones is what I'm seeing when trying to build the Juno platform:
$ PACKAGES_PATH="/work/maint/edk2-platforms:/work/maint/edk2-non-osi" GCC5_AARCH64_PREFIX=aarch64-linux-gnu- build -a AARCH64 -t GCC5 -p Platform/ARM/Juno/ArmJuno.dsc -b RELEASE -n 9
results in:
<<< GenFds.py... : error F003: Output file for RAW section could not be found for Platform/ARM/Juno/AcpiTables/AcpiTables.inf
###
build.py... : error 7000: Failed to execute command GenFds -f /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.fdf --conf=/work/maint/edk2/Conf -o /work/maint/edk2/Build/ArmJuno/RELEASE_GCC5 -t GCC5 -b RELEASE -p /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.dsc -a AARCH64 -D "EFI_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" -D "EDK_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" -D "TOOL_CHAIN_TAG=GCC5" -D "TOOLCHAIN=GCC5" -D "TARGET=RELEASE" -D "FAMILY=GCC" -D "WORKSPACE=/work/maint/edk2" -D "EDK_TOOLS_PATH=/work/maint/edk2/BaseTools" -D "ARCH=AARCH64" -D "ECP_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" [/work/maint/edk2]
- Failed -
And when I copy and paste the above command manually, I get:
<<< GenFds.py... /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.dsc(34): error 000E: File/directory not found in workspace /work/maint/edk2-platforms/Platform/ARM/Juno/Platform/ARM/VExpress/ArmVExpress.dsc.inc /work/maint/edk2/Platform/ARM/VExpress/ArmVExpress.dsc.inc
So, to an uniformed observer, it seems the portion !include Platform/ARM/VExpress/ArmVExpress.dsc.inc from ArmJuno.dsc gets expanded to "directory ArmJuno.dsc is in" + "Platform/ARM/VExpress/ArmVExpress.dsc.inc" whereas I was hoping for it to try to find a match for "Platform/ARM/VExpress/ArmVExpress.dsc.inc" along PACKAGES_PATH (and find one in edk2-platforms).
I also have the impression that something similar is happening in ArmJuno.fdf for the line INF RuleOverride=ACPITABLE Platform/ARM/Juno/AcpiTables/AcpiTables.inf generating the error message from the original build command.
But I'm not quite sure how to debug these issues (short of fully figuring out the innards of MultipleWorkspace.py, MetaFileParser.py and a few others). Any ideas of where to look, or even what is going on? Would these cases be expected to work?
/ Leif _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.orgmailto:edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
Hi Mike,
Yes, this way I am able to build all of my packages! Many thanks!
This also works with the simplified PACKAGES_PATH usage model I had initially been trying for (just point to edk2-platforms and edk2-non-osi top level directories).
I have pushed updated and slightly cleaned up versions to devel-OpenPlatformPkg branches on edk2-platforms and edk2-non-osi.
I'm working on updated helper scripts in my uefi-tools repository and some documentation for this, and will post a new thread when I have this ready.
/ Leif
On Tue, Jun 13, 2017 at 11:13:08PM +0000, Kinney, Michael D wrote:
Hi Leif,
I pulled the latest versions of the repos and I was able to complete a build with no errors and without using symbolic links.
The change I made is to set WORKSPACE to the directory immediately above the repos and set PACKAGES_PATH to directories that contain the packages required for the platform to build.
This is the same technique used to build QuarkPlatformPkg that uses content from both the edk2 and the edk2-non-osi repos. See the Readme.md in the QuarkPlatformPkg with instructions for setting up the build env for Linux and Windows for an example.
https://github.com/tianocore/edk2/tree/master/QuarkPlatformPkg
I wrote a small shell script that I put in the directory above the repos to setup the build environment for the JunoPkg platform.
export GCC5_AARCH64_PREFIX=aarch64-linux-gnu-
export WORKSPACE=$PWD
export PACKAGES_PATH=\ $WORKSPACE/edk2:\ $WORKSPACE/edk2-platforms/Platform/ARM:\ $WORKSPACE/edk2-non-osi
cd edk2 make -C BaseTools . edksetup.sh BaseTools
build -a AARCH64 -t GCC5 -p JunoPkg/ArmJuno.dsc
=== end of build log === Generate Region at Offset 0x0 Region Size = 0xF8000 Region Name = FV
Generating FVMAIN_COMPACT FV #### Generating FVMAIN FV #################################### ######################################## ######################################## ######################################## ################################ Generate Region at Offset 0x0 Region Size = 0xF8000 Region Name = FV
GUID cross reference file can be found at /home/mdkinney/GitHub/tianocore/Build/ArmJuno/DEBUG_GCC5/FV/Guid.xref
FV Space Information FVMAIN_COMPACT [94%Full] 1015808 total, 956200 used, 59608 free FVMAIN [99%Full] 8237696 total, 8237688 used, 8 free
- Done -
Build end time: 18:49:57, Jun.13 2017 Build total time: 00:04:22 ========================
If you want to support building any of the platforms, then PACKAGES_PATH can be set as follows:
export PACKAGES_PATH=\ $WORKSPACE/edk2:\ $WORKSPACE/edk2-platforms/Silicon/Hisilicon:\ $WORKSPACE/edk2-platforms/Silicon/AMD:\ $WORKSPACE/edk2-platforms/Platform/Hisilicon:\ $WORKSPACE/edk2-platforms/Platform/Marvell:\ $WORKSPACE/edk2-platforms/Platform/ARM:\ $WORKSPACE/edk2-non-osi/Silicon/Intel:\ $WORKSPACE/edk2-non-osi/Silicon/AMD/Styx:
Best regards,
Mike
-----Original Message----- From: Leif Lindholm [mailto:leif.lindholm@linaro.org] Sent: Wednesday, June 7, 2017 7:58 AM To: Yao, Jiewen jiewen.yao@intel.com Cc: edk2-devel@lists.01.org; Ryan Harkin ryan.harkin@linaro.org; Ard Biesheuvel ard.biesheuvel@linaro.org; Chenhui Sun chenhui.sun@linaro.org; Andrew Fish afish@apple.com; Alan Ott alan@softiron.co.uk; Richardson, Brian brian.richardson@intel.com; Duran, Leo leo.duran@amd.com; haojian.zhuang@linaro.org; Linaro UEFI linaro-uefi@lists.linaro.org; Kinney, Michael D michael.d.kinney@intel.com; Heyi Guo heyi.guo@linaro.org Subject: Re: [edk2] [RFC] migration of OpenPlatformPkg to tianocore
Hi all,
OK, so I have now updated both edk2-non-osi and edk2-platforms (devel-OpenPlatformPkg branches) with package renaming, and updating .dsc/.fdf files to .
It appears the problems I'm facing are mainly caused by the GenFds stage not seeing a view consistent with the actual compilation stage.
To demonstrate, I build the Juno platform: $ . edksetup.sh $ make -C BaseTools $ PACKAGES_PATH=/work/git/edk2-platforms/Platform/ARM \ GCC5_AARCH64_PREFIX=aarch64-linux-gnu- build -n 9 -a AARCH64 \ -t GCC5 -p JunoPkg/ArmJuno.dsc -b RELEASE
This build fails with:
Fd File Name:BL33_AP_UEFI
Generate Region at Offset 0x0 Region Size = 0xF8000 Region Name = FV
Generating FVMAIN_COMPACT FV #### Generating FVMAIN FV
GenFds.py... : error F003: Output file for RAW section could not be found for JunoPkg/AcpiTables/AcpiTables.inf
build.py... : error 7000: Failed to execute command GenFds -f /work/git/edk2-platforms/Platform/ARM/JunoPkg/ArmJuno.fdf --conf=/work/git/edk2/Conf -o /work/git/edk2/Build/ArmJuno/RELEASE_GCC5 -t GCC5 -b RELEASE -p /work/git/edk2-platforms/Platform/ARM/JunoPkg/ArmJuno.dsc -a AARCH64 -D "EFI_SOURCE=/work/git/edk2/EdkCompatibilityPkg" -D "EDK_SOURCE=/work/git/edk2/EdkCompatibilityPkg" -D "TOOL_CHAIN_TAG=GCC5" -D "TOOLCHAIN=GCC5" -D "TARGET=RELEASE" -D "FAMILY=GCC" -D "WORKSPACE=/work/git/edk2" -D "EDK_TOOLS_PATH=/work/git/edk2/BaseTools" -D "ARCH=AARCH64" -D "ECP_SOURCE=/work/git/edk2/EdkCompatibilityPkg" [/work/git/edk2]
- Failed -
, but if I add the symlink ln -s edk2-platforms/Platform/ARM/JunoPkg/ Build/ArmJuno/RELEASE_GCC5/AARCH64/ it completes successfully on the next pass.
Same when building VExpressPkg/ArmVExpress-FVP-AArch64.dsc ln -s edk2-platforms/Platform/ARM/VExpressPkg \ Build/ArmVExpress-FVP-AArch64/RELEASE_GCC5/AARCH64/ resolves the build error.
Similarly, when I build the Overdrive/Overdrive.dsc, using: $ PACKAGES_PATH=/work/git/edk2-platforms/Platform/AMD:/work/git/edk2-platforms/Silicon/AMD:/work/git/edk2-non-osi/Silicon/AMD/Styx:/work/git/edk2-non-osi/Platform/AMD \ GCC5_AARCH64_PREFIX=aarch64-linux-gnu- build -n 9 -a AARCH64 -t GCC5 -p OverdriveBoard/OverdriveBoard.dsc -b RELEASE this fails with
Fd File Name:STYX_ROM
Generate Region at Offset 0x0 Region Size = 0x200000 Region File Name = /work/git/edk2/../edk2-non-osi/Platform/AMD/OverdriveBoard/PreUefiFirmware.bin
Generate Region at Offset 0x200000 Region Size = 0x260000 Region Name = FV
Generating STYX_EFI FV ################Return Value = 2 GenFw: ERROR 0001: Error opening file
/work/git/edk2/Build/Overdrive/RELEASE_GCC5/AARCH64/StyxPkg/Drivers/PlatInitPei/PlatInitPei/OUTPUT/PlatInitPei.efi
GenFds.py... ### ['GenFw', '-t', '-o', ### '/work/git/edk2/Build/Overdrive/RELEASE_GCC5/FV/Ffs/769694a4-2572-4f29-a5bb-33d7df7be001PlatInitPei/769694a4-2572-4f29-a5bb-33d7df7be001Te.raw', ### '/work/git/edk2/Build/Overdrive/RELEASE_GCC5/AARCH64/StyxPkg/Drivers/PlatInitPei/PlatInitPei/OUTPUT/PlatInitPei.efi'] : error 7000: Failed to generate firmware image
build.py... : error 7000: Failed to execute command GenFds -f /work/git/edk2-platforms/Platform/AMD/OverdriveBoard/OverdriveBoard.fdf --conf=/work/git/edk2/Conf -o /work/git/edk2/Build/Overdrive/RELEASE_GCC5 -t GCC5 -b RELEASE -p /work/git/edk2-platforms/Platform/AMD/OverdriveBoard/OverdriveBoard.dsc -a AARCH64 -D "EFI_SOURCE=/work/git/edk2/EdkCompatibilityPkg" -D "EDK_SOURCE=/work/git/edk2/EdkCompatibilityPkg" -D "TOOL_CHAIN_TAG=GCC5" -D "TOOLCHAIN=GCC5" -D "TARGET=RELEASE" -D "FAMILY=GCC" -D "WORKSPACE=/work/git/edk2" -D "FIRMWARE_VER=b941c34" -D "EDK_TOOLS_PATH=/work/git/edk2/BaseTools" -D "ARCH=AARCH64" -D "ECP_SOURCE=/work/git/edk2/EdkCompatibilityPkg" [/work/git/edk2]
- Failed -
Again, resolved by ln -s edk2-platforms/Silicon/AMD/StyxPkg/ Build/Overdrive/RELEASE_GCC5/AARCH64/
Am I doing something wrong here? Are there any magic runes that could help out?
/ Leif
On Fri, May 05, 2017 at 01:45:51PM +0000, Yao, Jiewen wrote:
HI Leif It is great that you are adding more platform to edk2-platforms.
We (Intel) also have plan to add more boards there. In general, we are very close on having silicon and platform folder.
I have a quick look. One minor suggestions here: Take Arm folder as example. (I assume Juno is one package, and VExpress is the other package.) Can we name Juno to be JunoPkg, and VExpress to be VExpressPkg ? We do not add "Pkg" to a folder. And we usually add "Pkg" suffix to a package.
In general, I think it is a very good start. I may review the content in more detail and provide more feedback.
Thank you Yao Jiewen
From: edk2-devel [mailto:edk2-devel-bounces@lists.01.org] On Behalf Of Leif Lindholm Sent: Thursday, May 4, 2017 6:56 AM To: edk2-devel@lists.01.org Cc: Ryan Harkin ryan.harkin@linaro.org; Ard Biesheuvel ard.biesheuvel@linaro.org; Chenhui Sun chenhui.sun@linaro.org; Andrew Fish afish@apple.com; Alan Ott alan@softiron.co.uk; Richardson, Brian brian.richardson@intel.com; Duran, Leo leo.duran@amd.com; haojian.zhuang@linaro.org; Linaro UEFI linaro-uefi@lists.linaro.org; Kinney, Michael D michael.d.kinney@intel.com; Heyi Guo heyi.guo@linaro.org Subject: [edk2] [RFC] migration of OpenPlatformPkg to tianocore
Hi all,
As some of you may be aware, I have been working around the lack of a clear upstreaming strategy for platform support by keeping such code in a dedicated repository I set up at Linaro for that purpose: https://git.linaro.org/uefi/OpenPlatformPkg.git
During discussions at the last Seattle plugfest we finally agreed on the (theoretical) details of how to use the edk2-platforms repository. After that I promised to migrate OpenPlatformPkg across to the edk2-platforms and edk2-non-osi structure, with the explicit end goal from my side that this should become the master branch for each.
And now, before the release of HURD 1.0, I have.
Current limitations (that I can remember):
- A few references to OpenPlatformPkg remain, in ways that do not appear to break any of the platform builds. Most likely this affects dead code, but in case it's been accidentally orphaned, I thought it best to
- I have simply nuked all references to Ebl (used in _addition_ to the UEFI shell, which was never the intent) and the efi-toolkit ramdisk driver.
- The Marvell Yukon driver that I sent out for review last week has not migrated anywhere, and so has been temporarily disabled Mike suggested I should
- USB support on the LeMaker Cello board depends on the patch "OptionRomPkg: add firmware loader driver for Renesas PD72020x" sent out by Ard on 18th of April.
- I have dropped some of the binary-only modules from OpenPlatformPkg, and contacted the platform owners with requests for modifications.
- The git history is quite messy and will be cleaned up, but I wanted to keep the transition quite visible in the RFC.
- I haven't filled anything into the Maintainers.txt files - I am in favour of moving to a fully machine-readable format with wildcards as Laszlo has proposed in the past, and think this would be an excellent point to have that discussion (which can be had separately for edk2-platforms and edk2-non-osi from edk2).
- Few of the platforms complete the FV generation stage, and I've inserted a couple of silly hacks to get them to get as far as they do. I think that either I am missing some points of how PACKAGES_PATH is intended to work, or I'm simply hitting corner cases no one has come across before. I could really use some help debugging these issues. (examples below).
The below depends on the 3-part series I sent out today for importing DwEmmcDxe and EfiTimeBaseLib from OpenPlatformPkg. But apart from that, I have uploaded branches called devel-OpenPlatformPkg to:
https://github.com/tianocore/edk2-platforms/tree/devel-OpenPlatformPkg https://github.com/tianocore/edk2-non-osi/tree/devel-OpenPlatformPkg
These branches _will_ be rebased occasionally until they get to a point where they can move out of devel- stage (and hopefully onto master).
Build issue description
So, one of the hopefully easier ones is what I'm seeing when trying to build the Juno platform:
$ PACKAGES_PATH="/work/maint/edk2-platforms:/work/maint/edk2-non-osi" GCC5_AARCH64_PREFIX=aarch64-linux-gnu- build -a AARCH64 -t GCC5 -p Platform/ARM/Juno/ArmJuno.dsc -b RELEASE -n 9
results in:
<<< GenFds.py... : error F003: Output file for RAW section could not be found for Platform/ARM/Juno/AcpiTables/AcpiTables.inf
###
build.py... : error 7000: Failed to execute command GenFds -f /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.fdf --conf=/work/maint/edk2/Conf -o /work/maint/edk2/Build/ArmJuno/RELEASE_GCC5 -t GCC5 -b RELEASE -p /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.dsc -a AARCH64 -D "EFI_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" -D "EDK_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" -D "TOOL_CHAIN_TAG=GCC5" -D "TOOLCHAIN=GCC5" -D "TARGET=RELEASE" -D "FAMILY=GCC" -D "WORKSPACE=/work/maint/edk2" -D "EDK_TOOLS_PATH=/work/maint/edk2/BaseTools" -D "ARCH=AARCH64" -D "ECP_SOURCE=/work/maint/edk2/EdkCompatibilityPkg" [/work/maint/edk2]
- Failed -
And when I copy and paste the above command manually, I get:
<<< GenFds.py... /work/maint/edk2-platforms/Platform/ARM/Juno/ArmJuno.dsc(34): error 000E: File/directory not found in workspace /work/maint/edk2-platforms/Platform/ARM/Juno/Platform/ARM/VExpress/ArmVExpress.dsc.inc /work/maint/edk2/Platform/ARM/VExpress/ArmVExpress.dsc.inc
So, to an uniformed observer, it seems the portion !include Platform/ARM/VExpress/ArmVExpress.dsc.inc from ArmJuno.dsc gets expanded to "directory ArmJuno.dsc is in" + "Platform/ARM/VExpress/ArmVExpress.dsc.inc" whereas I was hoping for it to try to find a match for "Platform/ARM/VExpress/ArmVExpress.dsc.inc" along PACKAGES_PATH (and find one in edk2-platforms).
I also have the impression that something similar is happening in ArmJuno.fdf for the line INF RuleOverride=ACPITABLE Platform/ARM/Juno/AcpiTables/AcpiTables.inf generating the error message from the original build command.
But I'm not quite sure how to debug these issues (short of fully figuring out the innards of MultipleWorkspace.py, MetaFileParser.py and a few others). Any ideas of where to look, or even what is going on? Would these cases be expected to work?
/ Leif _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.orgmailto:edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
Hi all,
I have now pushed updated versions of devel-OpenPlatformPkg branches to edk2-platforms and edk2-non-osi. I now also have a Readme.md at the top-level of edk2-platforms describing
All the silly hacks are gone, and all platforms now build.
I would really appreciate if people could have a look (and a play).
Regards,
Leif
On Wed, May 03, 2017 at 11:55:39PM +0100, Leif Lindholm wrote:
Hi all,
As some of you may be aware, I have been working around the lack of a clear upstreaming strategy for platform support by keeping such code in a dedicated repository I set up at Linaro for that purpose: https://git.linaro.org/uefi/OpenPlatformPkg.git
During discussions at the last Seattle plugfest we finally agreed on the (theoretical) details of how to use the edk2-platforms repository. After that I promised to migrate OpenPlatformPkg across to the edk2-platforms and edk2-non-osi structure, with the explicit end goal from my side that this should become the master branch for each.
And now, before the release of HURD 1.0, I have.
Current limitations (that I can remember):
- A few references to OpenPlatformPkg remain, in ways that do not appear to break any of the platform builds. Most likely this affects dead code, but in case it's been accidentally orphaned, I thought it best to
- I have simply nuked all references to Ebl (used in _addition_ to the UEFI shell, which was never the intent) and the efi-toolkit ramdisk driver.
- The Marvell Yukon driver that I sent out for review last week has not migrated anywhere, and so has been temporarily disabled Mike suggested I should
- USB support on the LeMaker Cello board depends on the patch "OptionRomPkg: add firmware loader driver for Renesas PD72020x" sent out by Ard on 18th of April.
- I have dropped some of the binary-only modules from OpenPlatformPkg, and contacted the platform owners with requests for modifications.
- The git history is quite messy and will be cleaned up, but I wanted to keep the transition quite visible in the RFC.
- I haven't filled anything into the Maintainers.txt files - I am in favour of moving to a fully machine-readable format with wildcards as Laszlo has proposed in the past, and think this would be an excellent point to have that discussion (which can be had separately for edk2-platforms and edk2-non-osi from edk2).
- Few of the platforms complete the FV generation stage, and I've inserted a couple of silly hacks to get them to get as far as they do. I think that either I am missing some points of how PACKAGES_PATH is intended to work, or I'm simply hitting corner cases no one has come across before. I could really use some help debugging these issues. (examples below).
The below depends on the 3-part series I sent out today for importing DwEmmcDxe and EfiTimeBaseLib from OpenPlatformPkg. But apart from that, I have uploaded branches called devel-OpenPlatformPkg to:
https://github.com/tianocore/edk2-platforms/tree/devel-OpenPlatformPkg https://github.com/tianocore/edk2-non-osi/tree/devel-OpenPlatformPkg
These branches _will_ be rebased occasionally until they get to a point where they can move out of devel- stage (and hopefully onto master).
On 21 June 2017 at 17:44, Leif Lindholm leif.lindholm@linaro.org wrote:
Hi all,
I have now pushed updated versions of devel-OpenPlatformPkg branches to edk2-platforms and edk2-non-osi. I now also have a Readme.md at the top-level of edk2-platforms describing
All the silly hacks are gone, and all platforms now build.
I would really appreciate if people could have a look (and a play).
Cello does not build atm in the new arrangement:
build.py... : error 000E: File/directory not found in workspace OpenPlatformPkg/Drivers/Xhci/RenesasFirmwarePD720202/RenesasFirmwarePD720202.inf is not found in packages path: /home/ard/build/edk2/../edk2-platforms /home/ard/build/edk2/../edk2-non-osi
On 22 June 2017 at 11:39, Ard Biesheuvel ard.biesheuvel@linaro.org wrote:
On 21 June 2017 at 17:44, Leif Lindholm leif.lindholm@linaro.org wrote:
Hi all,
I have now pushed updated versions of devel-OpenPlatformPkg branches to edk2-platforms and edk2-non-osi. I now also have a Readme.md at the top-level of edk2-platforms describing
All the silly hacks are gone, and all platforms now build.
I would really appreciate if people could have a look (and a play).
Cello does not build atm in the new arrangement:
build.py... : error 000E: File/directory not found in workspace OpenPlatformPkg/Drivers/Xhci/RenesasFirmwarePD720202/RenesasFirmwarePD720202.inf is not found in packages path: /home/ard/build/edk2/../edk2-platforms /home/ard/build/edk2/../edk2-non-osi
I also think the new PACKAGES_PATH breaks the inclusion of Dsdt.hex:
/home/ard/build/edk2-platforms/Silicon/AMD/Styx/AcpiTables/Dsdt.c:22:20: fatal error: Dsdt.hex: No such file or directory #include <Dsdt.hex> ^ compilation terminated.
which resides at
Build/Cello/DEBUG_GCC5/AARCH64/edk2-platforms/Silicon/AMD/Styx/AcpiTables/AcpiAml/OUTPUT/Dsdt.hex
but the include is defined as
GCC:*_*_AARCH64_PLATFORM_FLAGS = -I$(BIN_DIR)/Silicon/AMD/Styx/AcpiTables/AcpiAml/OUTPUT
On Thu, Jun 22, 2017 at 11:46:16AM +0000, Ard Biesheuvel wrote:
On 22 June 2017 at 11:39, Ard Biesheuvel ard.biesheuvel@linaro.org wrote:
On 21 June 2017 at 17:44, Leif Lindholm leif.lindholm@linaro.org wrote:
Hi all,
I have now pushed updated versions of devel-OpenPlatformPkg branches to edk2-platforms and edk2-non-osi. I now also have a Readme.md at the top-level of edk2-platforms describing
All the silly hacks are gone, and all platforms now build.
I would really appreciate if people could have a look (and a play).
Cello does not build atm in the new arrangement:
build.py... : error 000E: File/directory not found in workspace OpenPlatformPkg/Drivers/Xhci/RenesasFirmwarePD720202/RenesasFirmwarePD720202.inf is not found in packages path: /home/ard/build/edk2/../edk2-platforms /home/ard/build/edk2/../edk2-non-osi
I also think the new PACKAGES_PATH breaks the inclusion of Dsdt.hex:
/home/ard/build/edk2-platforms/Silicon/AMD/Styx/AcpiTables/Dsdt.c:22:20: fatal error: Dsdt.hex: No such file or directory #include <Dsdt.hex> ^ compilation terminated.
which resides at
Build/Cello/DEBUG_GCC5/AARCH64/edk2-platforms/Silicon/AMD/Styx/AcpiTables/AcpiAml/OUTPUT/Dsdt.hex
but the include is defined as
GCC:*_*_AARCH64_PLATFORM_FLAGS = -I$(BIN_DIR)/Silicon/AMD/Styx/AcpiTables/AcpiAml/OUTPUT
You're not reading the documentation ;) (Have a glance at Readme.md.)
Yes, this is the behaviour I had before this latest reshuffle. Mike's proposed (and functional) workaround is to not have "edk2" as your WORKSPACE, but a common superdirectory of edk2, edk2-platforms and edk2-non-osi.
Hopefully at some point we can get rid of this restriction, but for now it's sort of necessary.
/ Leif
On 22 June 2017 at 12:49, Leif Lindholm leif.lindholm@linaro.org wrote:
On Thu, Jun 22, 2017 at 11:46:16AM +0000, Ard Biesheuvel wrote:
On 22 June 2017 at 11:39, Ard Biesheuvel ard.biesheuvel@linaro.org wrote:
On 21 June 2017 at 17:44, Leif Lindholm leif.lindholm@linaro.org wrote:
Hi all,
I have now pushed updated versions of devel-OpenPlatformPkg branches to edk2-platforms and edk2-non-osi. I now also have a Readme.md at the top-level of edk2-platforms describing
All the silly hacks are gone, and all platforms now build.
I would really appreciate if people could have a look (and a play).
Cello does not build atm in the new arrangement:
build.py... : error 000E: File/directory not found in workspace OpenPlatformPkg/Drivers/Xhci/RenesasFirmwarePD720202/RenesasFirmwarePD720202.inf is not found in packages path: /home/ard/build/edk2/../edk2-platforms /home/ard/build/edk2/../edk2-non-osi
I also think the new PACKAGES_PATH breaks the inclusion of Dsdt.hex:
/home/ard/build/edk2-platforms/Silicon/AMD/Styx/AcpiTables/Dsdt.c:22:20: fatal error: Dsdt.hex: No such file or directory #include <Dsdt.hex> ^ compilation terminated.
which resides at
Build/Cello/DEBUG_GCC5/AARCH64/edk2-platforms/Silicon/AMD/Styx/AcpiTables/AcpiAml/OUTPUT/Dsdt.hex
but the include is defined as
GCC:*_*_AARCH64_PLATFORM_FLAGS = -I$(BIN_DIR)/Silicon/AMD/Styx/AcpiTables/AcpiAml/OUTPUT
You're not reading the documentation ;) (Have a glance at Readme.md.)
Yes, this is the behaviour I had before this latest reshuffle. Mike's proposed (and functional) workaround is to not have "edk2" as your WORKSPACE, but a common superdirectory of edk2, edk2-platforms and edk2-non-osi.
Hopefully at some point we can get rid of this restriction, but for now it's sort of necessary.
Thanks for clearing that up. And as it turns out, WORKSPACE is not required to be a parent directory to the core EDK2 checkout or any of the alternative sources listed in PACKAGES_PATH, which is actually quite convenient.
On Thu, Jun 22, 2017 at 03:57:10PM +0000, Ard Biesheuvel wrote:
Build/Cello/DEBUG_GCC5/AARCH64/edk2-platforms/Silicon/AMD/Styx/AcpiTables/AcpiAml/OUTPUT/Dsdt.hex
but the include is defined as
GCC:*_*_AARCH64_PLATFORM_FLAGS = -I$(BIN_DIR)/Silicon/AMD/Styx/AcpiTables/AcpiAml/OUTPUT
You're not reading the documentation ;) (Have a glance at Readme.md.)
Yes, this is the behaviour I had before this latest reshuffle. Mike's proposed (and functional) workaround is to not have "edk2" as your WORKSPACE, but a common superdirectory of edk2, edk2-platforms and edk2-non-osi.
Hopefully at some point we can get rid of this restriction, but for now it's sort of necessary.
Thanks for clearing that up. And as it turns out, WORKSPACE is not required to be a parent directory to the core EDK2 checkout or any of the alternative sources listed in PACKAGES_PATH, which is actually quite convenient.
*cough* yes, apparently that works when you actually set WORKSPACE in the same terminal as you then run edksetup.sh from...
/ Leif
On Thu, Jun 22, 2017 at 11:39:53AM +0000, Ard Biesheuvel wrote:
On 21 June 2017 at 17:44, Leif Lindholm leif.lindholm@linaro.org wrote:
Hi all,
I have now pushed updated versions of devel-OpenPlatformPkg branches to edk2-platforms and edk2-non-osi. I now also have a Readme.md at the top-level of edk2-platforms describing
All the silly hacks are gone, and all platforms now build.
I would really appreciate if people could have a look (and a play).
Cello does not build atm in the new arrangement:
build.py... : error 000E: File/directory not found in workspace OpenPlatformPkg/Drivers/Xhci/RenesasFirmwarePD720202/RenesasFirmwarePD720202.inf is not found in packages path: /home/ard/build/edk2/../edk2-platforms /home/ard/build/edk2/../edk2-non-osi
Yes, it will fail when you build with RENESAS_XHCI_FW_DIR set, until such a point as that driver appears somewhere. (And since it doesn't live anywhere right now, I didn't have anything to replace the path with.)
Ray - could you respond to Ard's reply from 19 April please? Subject: "[PATCH] OptionRomPkg: add firmware loader driver for Renesas PD72020x"
I would prefer to keep generic device drivers out of EDK2 platforms, but they need somewhere else to live.
Regards,
Leif
-
Ard Biesheuvel -
Kinney, Michael D -
Leif Lindholm -
Yao, Jiewen