I'm adding some EBBR text around the CPU state at boot and I've lost track of what is being done for multicore bringup. What is the current state-of-the-art for multicore boot protocol when PSCI isn't available?
SBBR allows for the MP Boot Protocol; with the intend of phasing it out:
https://acpica.org/sites/acpica/files/MP%20Startup%20for%20ARM%20platforms.d...
The kernel documents the spin table method:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/tree...
What are the cool kids currently doing here?
g. IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hi,
On 18/05/18 12:04, Grant Likely wrote:
I'm adding some EBBR text around the CPU state at boot and I've lost track of what is being done for multicore bringup. What is the current state-of-the-art for multicore boot protocol when PSCI isn't available?
It is my understanding that for arm64 PSCI is the only accepted SMP bringup method. Now the first ARM64 hardware platform didn't have EL3, but wanted to have KVM running, so there was this decision to allow this *one* alternative method to enable this platform. If I understand Will and Catalin correctly, there is a strong push for *no* other methods being added to the kernel.
So for ARM64 the answer should be: PSCI or spin table. Full stop.
For ARM it's quite a different story, though. "New" platforms tend to implement PSCI, but there are quite some platforms out there with their own methods. Not sure we care about them?
Actually, what is the purpose of the EBBR elaborating on SMP bringup in the first place?
Cheers, Andre.
SBBR allows for the MP Boot Protocol; with the intend of phasing it out:
https://acpica.org/sites/acpica/files/MP%20Startup%20for%20ARM%20platforms.d...
The kernel documents the spin table method:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/tree...
What are the cool kids currently doing here?
g. _______________________________________________ Arm.ebbr-discuss mailing list Arm.ebbr-discuss@arm.com
On 18/05/2018 12:13, Andre Przywara wrote:
Hi,
On 18/05/18 12:04, Grant Likely wrote:
I'm adding some EBBR text around the CPU state at boot and I've lost track of what is being done for multicore bringup. What is the current state-of-the-art for multicore boot protocol when PSCI isn't available?
It is my understanding that for arm64 PSCI is the only accepted SMP bringup method. Now the first ARM64 hardware platform didn't have EL3, but wanted to have KVM running, so there was this decision to allow this *one* alternative method to enable this platform. If I understand Will and Catalin correctly, there is a strong push for *no* other methods being added to the kernel.
So for ARM64 the answer should be: PSCI or spin table. Full stop.
AFAIU, MP Boot (ACPI Parking Protocol?) and Spin table are two separate methods. Correct? So that would be three methods we've got in-kernel at the moment.
For ARM it's quite a different story, though. "New" platforms tend to implement PSCI, but there are quite some platforms out there with their own methods. Not sure we care about them?
If EL3 is present, I think PSCI is the only interface EBBR should allow. Proprietary methods are the antithesis of what EBBR is intended to do.
Actually, what is the purpose of the EBBR elaborating on SMP bringup in the first place?
Primary reason is because EBBR specifies PSCI when EL3 exists. I don't want EBBR to go into details, but rather to point at the relevant specs. The question is what to do when EL3 isn't present.
"We don't care about those" is an acceptable answer.
I'm also think it is appropriate to compare the scope of EBBR with SBBR. ie. An SBBR compliant platform should be by definition EBBR compliant (SBBR is more restrictive), but the converse is not true. However, to claim that requires allowing for Microsoft's MP Boot Protocol specified in SBBR; or explicitly carving out platforms that don't implement PSCI as being incompatible with EBBR.
g.
Cheers, Andre.
SBBR allows for the MP Boot Protocol; with the intend of phasing it out:
https://acpica.org/sites/acpica/files/MP%20Startup%20for%20ARM%20platforms.d...
The kernel documents the spin table method:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/tree...
What are the cool kids currently doing here?
g. _______________________________________________ Arm.ebbr-discuss mailing list Arm.ebbr-discuss@arm.com
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
On 18 May 2018 at 13:37, Grant Likely grant.likely@arm.com wrote:
On 18/05/2018 12:13, Andre Przywara wrote:
Hi,
On 18/05/18 12:04, Grant Likely wrote:
I'm adding some EBBR text around the CPU state at boot and I've lost track of what is being done for multicore bringup. What is the current state-of-the-art for multicore boot protocol when PSCI isn't available?
It is my understanding that for arm64 PSCI is the only accepted SMP bringup method. Now the first ARM64 hardware platform didn't have EL3, but wanted to have KVM running, so there was this decision to allow this *one* alternative method to enable this platform. If I understand Will and Catalin correctly, there is a strong push for *no* other methods being added to the kernel.
So for ARM64 the answer should be: PSCI or spin table. Full stop.
AFAIU, MP Boot (ACPI Parking Protocol?) and Spin table are two separate methods. Correct? So that would be three methods we've got in-kernel at the moment.
Indeed.
For ARM it's quite a different story, though. "New" platforms tend to implement PSCI, but there are quite some platforms out there with their own methods. Not sure we care about them?
If EL3 is present, I think PSCI is the only interface EBBR should allow. Proprietary methods are the antithesis of what EBBR is intended to do.
ARMv8-A permits implementation without EL2 or EL3 IIUC, so from a spec pov, it makes sense to mandate spin table or ACPI parking protocol for that use case.
Actually, what is the purpose of the EBBR elaborating on SMP bringup in the first place?
Primary reason is because EBBR specifies PSCI when EL3 exists. I don't want EBBR to go into details, but rather to point at the relevant specs. The question is what to do when EL3 isn't present.
"We don't care about those" is an acceptable answer.
I'm also think it is appropriate to compare the scope of EBBR with SBBR. ie. An SBBR compliant platform should be by definition EBBR compliant (SBBR is more restrictive), but the converse is not true. However, to claim that requires allowing for Microsoft's MP Boot Protocol specified in SBBR; or explicitly carving out platforms that don't implement PSCI as being incompatible with EBBR.
g.
Cheers, Andre.
SBBR allows for the MP Boot Protocol; with the intend of phasing it out:
https://acpica.org/sites/acpica/files/MP%20Startup%20for%20ARM%20platforms.d...
The kernel documents the spin table method:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/tree...
What are the cool kids currently doing here?
g. _______________________________________________ Arm.ebbr-discuss mailing list Arm.ebbr-discuss@arm.com
Arm.ebbr-discuss mailing list Arm.ebbr-discuss@arm.com
On 18/05/2018 12:40, Ard Biesheuvel wrote:
On 18 May 2018 at 13:37, Grant Likely grant.likely@arm.com wrote:
On 18/05/2018 12:13, Andre Przywara wrote:
Hi,
On 18/05/18 12:04, Grant Likely wrote:
I'm adding some EBBR text around the CPU state at boot and I've lost track of what is being done for multicore bringup. What is the current state-of-the-art for multicore boot protocol when PSCI isn't available?
It is my understanding that for arm64 PSCI is the only accepted SMP bringup method. Now the first ARM64 hardware platform didn't have EL3, but wanted to have KVM running, so there was this decision to allow this *one* alternative method to enable this platform. If I understand Will and Catalin correctly, there is a strong push for *no* other methods being added to the kernel.
So for ARM64 the answer should be: PSCI or spin table. Full stop.
AFAIU, MP Boot (ACPI Parking Protocol?) and Spin table are two separate methods. Correct? So that would be three methods we've got in-kernel at the moment.
Indeed.
For ARM it's quite a different story, though. "New" platforms tend to implement PSCI, but there are quite some platforms out there with their own methods. Not sure we care about them?
If EL3 is present, I think PSCI is the only interface EBBR should allow. Proprietary methods are the antithesis of what EBBR is intended to do.
ARMv8-A permits implementation without EL2 or EL3 IIUC, so from a spec pov, it makes sense to mandate spin table or ACPI parking protocol for that use case.
One missing bit; I don't believe there are DT bindings for the ACPI parking protocol, and similarly no ACPI binding for spin tables. Therefore a platform without EL3 must choose to do only DT or only ACPI. I'm rapidly approaching pendanticry here because in practice this is unlikely to be a problem. The vast majority of EBBR platforms will be DT based, and the platforms that want to have an ACPI option will probably target full SBSA/SBBR compliance -- and therefore have EL3.
So, what I'll propose is allow for any of the three (PSCI, ACPI Parking Protocol, or spin table) with the caveats: * PSCI is required when EL3 exists * DT w/o EL3 requires spin-table * ACPI w/o EL3 requires ACPI Parking Protocol (and is stupid)
We'll see if anyone screams about PSCI being required. If so, I'll relax it to allow spin-table w/ proprietary EL3.
g. IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
On 18 May 2018 at 13:49, Grant Likely grant.likely@arm.com wrote:
On 18/05/2018 12:40, Ard Biesheuvel wrote:
On 18 May 2018 at 13:37, Grant Likely grant.likely@arm.com wrote:
On 18/05/2018 12:13, Andre Przywara wrote:
Hi,
On 18/05/18 12:04, Grant Likely wrote:
I'm adding some EBBR text around the CPU state at boot and I've lost track of what is being done for multicore bringup. What is the current state-of-the-art for multicore boot protocol when PSCI isn't available?
It is my understanding that for arm64 PSCI is the only accepted SMP bringup method. Now the first ARM64 hardware platform didn't have EL3, but wanted to have KVM running, so there was this decision to allow this *one* alternative method to enable this platform. If I understand Will and Catalin correctly, there is a strong push for *no* other methods being added to the kernel.
So for ARM64 the answer should be: PSCI or spin table. Full stop.
AFAIU, MP Boot (ACPI Parking Protocol?) and Spin table are two separate methods. Correct? So that would be three methods we've got in-kernel at the moment.
Indeed.
For ARM it's quite a different story, though. "New" platforms tend to implement PSCI, but there are quite some platforms out there with their own methods. Not sure we care about them?
If EL3 is present, I think PSCI is the only interface EBBR should allow. Proprietary methods are the antithesis of what EBBR is intended to do.
ARMv8-A permits implementation without EL2 or EL3 IIUC, so from a spec pov, it makes sense to mandate spin table or ACPI parking protocol for that use case.
One missing bit; I don't believe there are DT bindings for the ACPI parking protocol, and similarly no ACPI binding for spin tables. Therefore a platform without EL3 must choose to do only DT or only ACPI.
I'd argue that a platform must choose between DT and ACPI anyway, since we don't want to leave it up to the OS to decide which platform description is the most accurate one. We have standard EDK2 components now to make this configurable from the UEFI menus. This means the firmware can switch between spintable and parking protocol at the same time.
I'm rapidly approaching pendanticry here because in practice this is unlikely to be a problem. The vast majority of EBBR platforms will be DT based, and the platforms that want to have an ACPI option will probably target full SBSA/SBBR compliance -- and therefore have EL3.
So, what I'll propose is allow for any of the three (PSCI, ACPI Parking Protocol, or spin table) with the caveats:
- PSCI is required when EL3 exists
- DT w/o EL3 requires spin-table
- ACPI w/o EL3 requires ACPI Parking Protocol (and is stupid)
PSCI can also be implemented at EL2, and we should not [implicitly] ban this. You will lose access to KVM if you use EL2 for firmware, but this could be an appropriate thing to do depending on the use case.
We'll see if anyone screams about PSCI being required. If so, I'll relax it to allow spin-table w/ proprietary EL3.
g. IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
On 18/05/2018 13:05, Ard Biesheuvel wrote:
On 18 May 2018 at 13:49, Grant Likely grant.likely@arm.com wrote:
On 18/05/2018 12:40, Ard Biesheuvel wrote:
On 18 May 2018 at 13:37, Grant Likely grant.likely@arm.com wrote:
On 18/05/2018 12:13, Andre Przywara wrote:
Hi,
On 18/05/18 12:04, Grant Likely wrote:
I'm adding some EBBR text around the CPU state at boot and I've lost track of what is being done for multicore bringup. What is the current state-of-the-art for multicore boot protocol when PSCI isn't available?
It is my understanding that for arm64 PSCI is the only accepted SMP bringup method. Now the first ARM64 hardware platform didn't have EL3, but wanted to have KVM running, so there was this decision to allow this *one* alternative method to enable this platform. If I understand Will and Catalin correctly, there is a strong push for *no* other methods being added to the kernel.
So for ARM64 the answer should be: PSCI or spin table. Full stop.
AFAIU, MP Boot (ACPI Parking Protocol?) and Spin table are two separate methods. Correct? So that would be three methods we've got in-kernel at the moment.
Indeed.
For ARM it's quite a different story, though. "New" platforms tend to implement PSCI, but there are quite some platforms out there with their own methods. Not sure we care about them?
If EL3 is present, I think PSCI is the only interface EBBR should allow. Proprietary methods are the antithesis of what EBBR is intended to do.
ARMv8-A permits implementation without EL2 or EL3 IIUC, so from a spec pov, it makes sense to mandate spin table or ACPI parking protocol for that use case.
One missing bit; I don't believe there are DT bindings for the ACPI parking protocol, and similarly no ACPI binding for spin tables. Therefore a platform without EL3 must choose to do only DT or only ACPI.
I'd argue that a platform must choose between DT and ACPI anyway, since we don't want to leave it up to the OS to decide which platform description is the most accurate one. We have standard EDK2 components now to make this configurable from the UEFI menus. This means the firmware can switch between spintable and parking protocol at the same time.
Yes, that is what I mean. From the firmware:OS interface perspective only one or the other should be provided, but being able to select which one before OS Loader time is just fine.
I'm rapidly approaching pendanticry here because in practice this is unlikely to be a problem. The vast majority of EBBR platforms will be DT based, and the platforms that want to have an ACPI option will probably target full SBSA/SBBR compliance -- and therefore have EL3.
So, what I'll propose is allow for any of the three (PSCI, ACPI Parking Protocol, or spin table) with the caveats:
- PSCI is required when EL3 exists
- DT w/o EL3 requires spin-table
- ACPI w/o EL3 requires ACPI Parking Protocol (and is stupid)
PSCI can also be implemented at EL2, and we should not [implicitly] ban this. You will lose access to KVM if you use EL2 for firmware, but this could be an appropriate thing to do depending on the use case.
Good point. I'll adjust the language I'm using.
g. IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
boot-architecture@lists.linaro.org
-
Andre Przywara -
Ard Biesheuvel -
Grant Likely