Re: mix of ACPICA regression and EFISTUB regression (Was: kernel >= v6.2 no longer boots on Apple's Virtualization.framework (x86_64); likely to be related to ACPICA)
[Resending as a plain text email]
Turned out that this is a mixture of an ACPICA issue and an EFISTUB issue.
Kernel v6.2 can boot by reverting the *both* of the following two commits: - 5c62d5aab8752e5ee7bfbe75ed6060db1c787f98 "ACPICA: Events: Support fixed PCIe wake event" - e346bebbd36b1576a3335331fed61bb48c6d8823 "efi: libstub: Always enable initrd command line loader and bump version"
Kernel v6.3 can boot by just reverting e346bebb, as 5c62d5a has been already reverted in 8e41e0a575664d26bb87e012c39435c4c3914ed9. The situation is the same for v6.4-rc3 too.
Note that in my test I let Virtualization.framework directly load bzImage without GRUB (akin to `qemu-system-x86_64 -kernel bzImage`). Apparently, reverting e346bebb is not necessary for loading bzImage via GRUB.
Also, the reporter can't provide dmesg log (forget to attach serial console?).
Uploaded v6.1 dmesg in the bugzilla. v6.2 dmesg can't be provided, as it hangs before printing something in console=hvc0. (IIUC, console=ttyS0 (RS-232C) is not implemented in Virtualization.framework.)
2023年5月25日(木) 21:46 Bagas Sanjaya bagasdotme@gmail.com:
Hi,
I notice a regression report on Bugzilla [1]. Quoting from it:
Linux kernel >= v6.2 no longer boots on Apple's Virtualization.framework (x86_64).
It is reported that the issue is not reproducible on ARM64: https://github.com/lima-vm/lima/issues/1577#issuecomment-1561577694
## Reproduction
Checkout the kernel repo, and run `make defconfig bzImage`.
Create an initrd (see the attached `initrd-example.txt`)
Transfer the bzImage and initrd to an Intel Mac.
On Mac, download `RunningLinuxInAVirtualMachine.zip` from https://developer.apple.com/documentation/virtualization/running_linux_in_a_... , and build the `LinuxVirtualMachine` binary with Xcode. Building this binary with Xcode requires logging in to Apple. If you do not like logging in, a third party equivalent such as https://github.com/Code-Hex/vz/blob/v3.0.6/example/linux/main.go can be used.
Run `LinuxVirtualMachine /tmp/bzImage /tmp/initrd.img`. v6.1 successfully boots into the busybox shell. v6.2 just hangs before printing something in the console.
## Tested versions
v6.1: OK ... v6.1.0-rc2-00002-g60f2096b59bc (included in v6.2-rc1): OK v6.1.0-rc2-00003-g5c62d5aab875 (included in v6.2-rc1): NG <-- This commit caused a regression ... v6.2-rc1: NG ... v6.2: NG ... v6.3.0-rc7-00181-g8e41e0a57566 (included in v6.3): NG <-- Reverts 5c62d5aab875 but still NG ... v6.3: NG v6.4-rc3: NGTested on MacBookPro 2020 (Intel(R) Core(TM) i7-1068NG7 CPU @ 2.30GHz) running macOS 13.4.
The issue seems a regression in [5c62d5aab8752e5ee7bfbe75ed6060db1c787f98](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i...) "ACPICA: Events: Support fixed PCIe wake event".
This commit was introduced in v6.2-rc1, and apparently reverted in v6.3 ([8e41e0a575664d26bb87e012c39435c4c3914ed9](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i...)). However, v6.3 and the latest v6.4-rc3 still don't boot.
See bugzilla for the full thread.
Interestingly, this regression still occurs despite the culprit is reverted in 8e41e0a575664d ("Revert "ACPICA: Events: Support fixed PCIe wake event""), so this (obviously) isn't wake-on-lan regression, but rather early boot one.
Also, the reporter can't provide dmesg log (forget to attach serial console?).
Anyway, I'm adding it to regzbot:
#regzbot introduced: 5c62d5aab8752e https://bugzilla.kernel.org/show_bug.cgi?id=217485 #regzbot title: Linux v6.2+ (x86_64) no longer boots on Apple's Virtualization framework (ACPICA issue)
Thanks.
-- An old man doll... just what I always wanted! - Clara
On Sat, 27 May 2023 at 20:00, Akihiro Suda suda.kyoto@gmail.com wrote:
[Resending as a plain text email]
Turned out that this is a mixture of an ACPICA issue and an EFISTUB issue.
Kernel v6.2 can boot by reverting the *both* of the following two commits:
- 5c62d5aab8752e5ee7bfbe75ed6060db1c787f98 "ACPICA: Events: Support
fixed PCIe wake event"
- e346bebbd36b1576a3335331fed61bb48c6d8823 "efi: libstub: Always
enable initrd command line loader and bump version"
Kernel v6.3 can boot by just reverting e346bebb, as 5c62d5a has been already reverted in 8e41e0a575664d26bb87e012c39435c4c3914ed9. The situation is the same for v6.4-rc3 too.
Note that in my test I let Virtualization.framework directly load bzImage without GRUB (akin to `qemu-system-x86_64 -kernel bzImage`). Apparently, reverting e346bebb is not necessary for loading bzImage via GRUB.
Are you using OVMF? Which versions of qemu and OVMF are you using?
Are you using OVMF? Which versions of qemu and OVMF are you using?
I'm using Apple's Virtualization.framework, not QEMU.
It doesn't use UEFI when it directly loads bzImage. ( dmesg: https://bugzilla.kernel.org/attachment.cgi?id=304323 )
Despite that, it still expects LINUX_EFISTUB_MINOR_VERSION (include/linux/pe.h) referred from arch/x86/boot/header.S to be 0x0. I confirmed that the kernel can boot by just setting LINUX_EFISTUB_MINOR_VERSION to 0x0.
Would it be possible to revert the LINUX_EFISTUB_MINOR_VERSION value (not the actual code) to 0x0? Or will it break something else?
Anyway, I'll try to make a request to Apple to remove the LINUX_EFISTUB_MINOR_VERSION check.
2023年5月28日(日) 3:04 Ard Biesheuvel ardb@kernel.org:
On Sat, 27 May 2023 at 20:00, Akihiro Suda suda.kyoto@gmail.com wrote:
[Resending as a plain text email]
Turned out that this is a mixture of an ACPICA issue and an EFISTUB issue.
Kernel v6.2 can boot by reverting the *both* of the following two commits:
- 5c62d5aab8752e5ee7bfbe75ed6060db1c787f98 "ACPICA: Events: Support
fixed PCIe wake event"
- e346bebbd36b1576a3335331fed61bb48c6d8823 "efi: libstub: Always
enable initrd command line loader and bump version"
Kernel v6.3 can boot by just reverting e346bebb, as 5c62d5a has been already reverted in 8e41e0a575664d26bb87e012c39435c4c3914ed9. The situation is the same for v6.4-rc3 too.
Note that in my test I let Virtualization.framework directly load bzImage without GRUB (akin to `qemu-system-x86_64 -kernel bzImage`). Apparently, reverting e346bebb is not necessary for loading bzImage via GRUB.
Are you using OVMF? Which versions of qemu and OVMF are you using?
On Sat, 27 May 2023 at 20:34, Akihiro Suda suda.kyoto@gmail.com wrote:
Are you using OVMF? Which versions of qemu and OVMF are you using?
I'm using Apple's Virtualization.framework, not QEMU.
It doesn't use UEFI when it directly loads bzImage. ( dmesg: https://bugzilla.kernel.org/attachment.cgi?id=304323 )
Despite that, it still expects LINUX_EFISTUB_MINOR_VERSION (include/linux/pe.h) referred from arch/x86/boot/header.S to be 0x0. I confirmed that the kernel can boot by just setting LINUX_EFISTUB_MINOR_VERSION to 0x0.
Thanks for checking that, that is very helpful/
Would it be possible to revert the LINUX_EFISTUB_MINOR_VERSION value (not the actual code) to 0x0? Or will it break something else?
Anyway, I'll try to make a request to Apple to remove the LINUX_EFISTUB_MINOR_VERSION check.
Yes, that makes the most sense. If the existing virtual machine BIOS has a hardcoded check that the EFI stub version is 1.0 even if it does not boot via EFI to begin with, I don't see how we can reasonably treat this as a regression that needs fixing on the Linux side.
The version bump to PE image version v1.1 sets a baseline across all Linux archifectures that can boot via EFI that initrd loading is supported via the command line as well as via the LoadFile2 protocol. Reverting that would substantially reduce the value of having this identification embedded into the image.
On Sat, May 27, 2023 at 11:42 AM Ard Biesheuvel ardb@kernel.org wrote:
Yes, that makes the most sense. If the existing virtual machine BIOS has a hardcoded check that the EFI stub version is 1.0 even if it does not boot via EFI to begin with, I don't see how we can reasonably treat this as a regression that needs fixing on the Linux side.
Well, we consider firmware issues to be the same as any hardware issue. If firmware has a bug that requires us to do things certain ways, that's really no different from hardware that requires some insane init sequence.
So why not just say that LINUX_EFISTUB_MINOR_VERSION should be 0, and just add the comment that versioning doesn't work?
I'm not sure why this was tied into always enabling the initrd command line loader.
Numbered version checks are a fundamentally broken and stupid concept anyway. Don't do them. Just leave it at zero, and maybe some day there is a sane model that actually has a bitfield of capabilities and requirements.
Linus
On Sat, 27 May 2023 at 21:40, Linus Torvalds torvalds@linux-foundation.org wrote:
On Sat, May 27, 2023 at 11:42 AM Ard Biesheuvel ardb@kernel.org wrote:
Yes, that makes the most sense. If the existing virtual machine BIOS has a hardcoded check that the EFI stub version is 1.0 even if it does not boot via EFI to begin with, I don't see how we can reasonably treat this as a regression that needs fixing on the Linux side.
Well, we consider firmware issues to be the same as any hardware issue. If firmware has a bug that requires us to do things certain ways, that's really no different from hardware that requires some insane init sequence.
So why not just say that LINUX_EFISTUB_MINOR_VERSION should be 0, and just add the comment that versioning doesn't work?
Fair enough. Or we could try bumping it from v1.1 to v2.0 (or v3.0 if we make it a bit mask).
Akihiro, would you mind checking if changing the major/minor to any of these values results in the same problem?
Unfortunately, the only data point we have is that a non-EFI bootloader (which is unlikely to carry a PE/COFF loader) needs the byte at that specific offset to be 0x0, and we really have no idea why, or whether we could hit this in other ways (i.e., by changing the PE/COFF header to comply with new MS requirements for secure boot, which is another thing that is in progress)
I'm not sure why this was tied into always enabling the initrd command line loader.
For x86, it doesn't actually make a difference, but on other architectures, the command line initrd= loader could be disabled, but that possibility was removed. The idea was that by bumping the version to v1.1 at the same time, generic EFI loaders would be able to identify this capability without arch specific conditionals in the logic.
Currently, GRUB and systemd-stub check this version field, but only for v1.0 or higher. Upstream GRUB switched to this generic version of the EFI loader just this week, but does not actually use initrd= at all for EFI boot (on any architecture).
Numbered version checks are a fundamentally broken and stupid concept anyway. Don't do them. Just leave it at zero, and maybe some day there is a sane model that actually has a bitfield of capabilities and requirements.
Yeah, maybe you're right. Currently, only a single feature is tied to LINUX_EFISTUB_MAJOR_VERSION==1 (LoadFile2 support for initrd loading), and this PE/COFF version field has no meaning to UEFI firmware itself, so we could simply treat these fields as bit masks if we wanted to (and setting the initrd command line loader bit for x86 would be redundant anyway)
But not being able to freely set such a bit because some rarely used non-EFI BIOS implementation imposes requirements on the contents of the EFI specific image header is rather disappointing.
Fair enough. Or we could try bumping it from v1.1 to v2.0 (or v3.0 if we make it a bit mask).
Akihiro, would you mind checking if changing the major/minor to any of these values results in the same problem?
Surprisingly, v2.0 and v3.0 boot, although v1.1, v2.1, v2.2, v3.1, etc. do not boot.
Looks like Apple's vmlinuz loader only requires LINUX_EFISTUB_MINOR_VERSION to be 0x0 and does not care about LINUX_EFISTUB_MAJOR_VERSION.
2023年5月28日(日) 6:48 Ard Biesheuvel ardb@kernel.org:
On Sat, 27 May 2023 at 21:40, Linus Torvalds torvalds@linux-foundation.org wrote:
On Sat, May 27, 2023 at 11:42 AM Ard Biesheuvel ardb@kernel.org wrote:
Yes, that makes the most sense. If the existing virtual machine BIOS has a hardcoded check that the EFI stub version is 1.0 even if it does not boot via EFI to begin with, I don't see how we can reasonably treat this as a regression that needs fixing on the Linux side.
Well, we consider firmware issues to be the same as any hardware issue. If firmware has a bug that requires us to do things certain ways, that's really no different from hardware that requires some insane init sequence.
So why not just say that LINUX_EFISTUB_MINOR_VERSION should be 0, and just add the comment that versioning doesn't work?
Fair enough. Or we could try bumping it from v1.1 to v2.0 (or v3.0 if we make it a bit mask).
Akihiro, would you mind checking if changing the major/minor to any of these values results in the same problem?
Unfortunately, the only data point we have is that a non-EFI bootloader (which is unlikely to carry a PE/COFF loader) needs the byte at that specific offset to be 0x0, and we really have no idea why, or whether we could hit this in other ways (i.e., by changing the PE/COFF header to comply with new MS requirements for secure boot, which is another thing that is in progress)
I'm not sure why this was tied into always enabling the initrd command line loader.
For x86, it doesn't actually make a difference, but on other architectures, the command line initrd= loader could be disabled, but that possibility was removed. The idea was that by bumping the version to v1.1 at the same time, generic EFI loaders would be able to identify this capability without arch specific conditionals in the logic.
Currently, GRUB and systemd-stub check this version field, but only for v1.0 or higher. Upstream GRUB switched to this generic version of the EFI loader just this week, but does not actually use initrd= at all for EFI boot (on any architecture).
Numbered version checks are a fundamentally broken and stupid concept anyway. Don't do them. Just leave it at zero, and maybe some day there is a sane model that actually has a bitfield of capabilities and requirements.
Yeah, maybe you're right. Currently, only a single feature is tied to LINUX_EFISTUB_MAJOR_VERSION==1 (LoadFile2 support for initrd loading), and this PE/COFF version field has no meaning to UEFI firmware itself, so we could simply treat these fields as bit masks if we wanted to (and setting the initrd command line loader bit for x86 would be redundant anyway)
But not being able to freely set such a bit because some rarely used non-EFI BIOS implementation imposes requirements on the contents of the EFI specific image header is rather disappointing.
linux-stable-mirror@lists.linaro.org
-
Akihiro Suda -
Ard Biesheuvel -
Linus Torvalds