Hi Masami,

Here are the instructions that I've been working on:

https://github.com/ARM-software/ebbr/wiki/EBBR-Testing-Guide https://github.com/ARM-software/ebbr/wiki/U-Boot-Configuration-Guide

I hope that helps

g.

On 12/05/2021 02:02, Masami Hiramatsu wrote:

Hello Grant and Vincent,

I'm interested in passing SCT on my board with the latest U-Boot. Would you know which CONFIG options are required for the SCT? I guess some CONFIG_EFI* options are related, but some of them may not be needed. Is there any good documentation about configuring U-Boot?

Thank you,

2021年5月10日(月) 22:36 Grant Likely grant.likely@arm.com:

...

For the EBBR meeting today:

Attached is the test result report from running a recent version of U-Boot on an iMX8 platform and parsed with the SCT_Parser tool.

g.

-------- Forwarded Message -------- Subject: Re: EBBR Testing topic for today? Date: Mon, 10 May 2021 14:22:56 +0100 From: Vincent Stehle Vincent.Stehle@arm.com To: Grant Likely Grant.Likely@arm.com

Hi Grant,

Here are some more SCT results: a run from this morning on Compulab, with latest firmware from Paul (+ 3 patches) and latest sct parser config.

INFO main: 0 dropped(s), 0 failure(s), 66 ignored(s), 10753 pass(s), 60 waived(s), 0 warning(s)

(The attached summary excludes the pass.)

Best regards, Vincent.

---

boot-architecture mailing list boot-architecture@lists.linaro.org https://lists.linaro.org/mailman/listinfo/boot-architecture

Hi Grant,

I have a comment on the U-Boot configuration: https://github.com/ARM-software/ebbr/wiki/U-Boot-Configuration-Guide

The U-Boot Sniff Test requires bootefi hello and selftest, --- u-boot=> bootefi hello ${fdtcontroladdr} u-boot=> bootefi selftest ${fdtcontroladdr} ---

So, U-Boot needs to enable below options too.

CONFIG_CMD_BOOTEFI_HELLO_COMPILE=y CONFIG_CMD_BOOTEFI_HELLO=y CONFIG_CMD_BOOTEFI_SELFTEST=y

And I have a question about SCT's EBBR result (according to the EBBR-Testing-Guide). Recently my team ran the SCT on our board and got some failures. One of them was related to the Auth variable,

/opt/sct/SctPkg/TestCase/UEFI/EFI/RuntimeServices/VariableServices/BlackBoxTest/AuthVariableServicesBBTestFunction.c 635 Status - Invalid Parameter RT.SetVariable - Create one Time Base Auth Variable, the expect return status should be EFI_SUCCESS FAILURE

But I think U-Boot Auth Variable is not enabled unless UEFI secure boot is enabled. So other test cases are waived. But this specific test case failed.

guid: 008E18A5-C345-48AE-9134-61A692E30B87 set guid: 9338D0EC-807B-4750-986A-8F2A91BB3616

Thank you,

2021年5月12日(水) 19:46 Masami Hiramatsu masami.hiramatsu@linaro.org:

Hi Grant,

Thanks, this is what I need. I'll check them out!

2021年5月12日(水) 18:56 Grant Likely grant.likely@arm.com:

...

Hi Masami,

Here are the instructions that I've been working on:

https://github.com/ARM-software/ebbr/wiki/EBBR-Testing-Guide https://github.com/ARM-software/ebbr/wiki/U-Boot-Configuration-Guide

I hope that helps

g.

On 12/05/2021 02:02, Masami Hiramatsu wrote:

...

Hello Grant and Vincent,

I'm interested in passing SCT on my board with the latest U-Boot. Would you know which CONFIG options are required for the SCT? I guess some CONFIG_EFI* options are related, but some of them may not be needed. Is there any good documentation about configuring U-Boot?

Thank you,

2021年5月10日(月) 22:36 Grant Likely grant.likely@arm.com:

...

For the EBBR meeting today:

Attached is the test result report from running a recent version of U-Boot on an iMX8 platform and parsed with the SCT_Parser tool.

g.

-------- Forwarded Message -------- Subject: Re: EBBR Testing topic for today? Date: Mon, 10 May 2021 14:22:56 +0100 From: Vincent Stehle Vincent.Stehle@arm.com To: Grant Likely Grant.Likely@arm.com

Hi Grant,

Here are some more SCT results: a run from this morning on Compulab, with latest firmware from Paul (+ 3 patches) and latest sct parser config.

INFO main: 0 dropped(s), 0 failure(s), 66 ignored(s), 10753 pass(s), 60 waived(s), 0 warning(s)

(The attached summary excludes the pass.)

Best regards, Vincent.

---

boot-architecture mailing list boot-architecture@lists.linaro.org https://lists.linaro.org/mailman/listinfo/boot-architecture

-- Masami Hiramatsu

-- Masami Hiramatsu

Hi,

2021年5月20日(木) 23:55 Grant Likely grant.likely@arm.com:

Thanks Masami,

Comments below...

On 20/05/2021 11:09, Masami Hiramatsu wrote:

...

Hi Grant,

I have a comment on the U-Boot configuration: https://github.com/ARM-software/ebbr/wiki/U-Boot-Configuration-Guide

The U-Boot Sniff Test requires bootefi hello and selftest,

u-boot=> bootefi hello ${fdtcontroladdr} u-boot=> bootefi selftest ${fdtcontroladdr}

---

So, U-Boot needs to enable below options too.

CONFIG_CMD_BOOTEFI_HELLO_COMPILE=y CONFIG_CMD_BOOTEFI_HELLO=y CONFIG_CMD_BOOTEFI_SELFTEST=y

While these commands are useful, they should not be required to be built in. I've added those options to the bottom of the configuration guide and moved the test commands out of the sniff test and into the configuration guide for those who want to run those tests.

Thanks for the update!

...

And I have a question about SCT's EBBR result (according to the EBBR-Testing-Guide). Recently my team ran the SCT on our board and got some failures. One of them was related to the Auth variable,

/opt/sct/SctPkg/TestCase/UEFI/EFI/RuntimeServices/VariableServices/BlackBoxTest/AuthVariableServicesBBTestFunction.c 635 Status - Invalid Parameter RT.SetVariable - Create one Time Base Auth Variable, the expect return status should be EFI_SUCCESS FAILURE

But I think U-Boot Auth Variable is not enabled unless UEFI secure boot is enabled. So other test cases are waived. But this specific test case failed.

guid: 008E18A5-C345-48AE-9134-61A692E30B87 set guid: 9338D0EC-807B-4750-986A-8F2A91BB3616

Easy answer here is to enable secure boot in U-Boot to get rid of these failures. I've added that option to the wiki

CONFIG_EFI_SECURE_BOOT=y

Hmm, but there are some other Auth Variable tests are correctly waived on the result. So I think it should be waived too. And as you said below, since SystemReady IR doesn't require the secure boot, I think the secure boot should not be enabled for EBBR test.

Vincent, this is something to discuss. Secure Boot is supported, but not required by SystemReady IR. Should we ignore failures when secure boot is disabled in the parsing script?

Thank you,

g.

...

Thank you,

2021年5月12日(水) 19:46 Masami Hiramatsu masami.hiramatsu@linaro.org:

...

Hi Grant,

Thanks, this is what I need. I'll check them out!

2021年5月12日(水) 18:56 Grant Likely grant.likely@arm.com:

...

Hi Masami,

Here are the instructions that I've been working on:

https://github.com/ARM-software/ebbr/wiki/EBBR-Testing-Guide https://github.com/ARM-software/ebbr/wiki/U-Boot-Configuration-Guide

I hope that helps

g.

On 12/05/2021 02:02, Masami Hiramatsu wrote:

...

Hello Grant and Vincent,

I'm interested in passing SCT on my board with the latest U-Boot. Would you know which CONFIG options are required for the SCT? I guess some CONFIG_EFI* options are related, but some of them may not be needed. Is there any good documentation about configuring U-Boot?

Thank you,

2021年5月10日(月) 22:36 Grant Likely grant.likely@arm.com:

...

For the EBBR meeting today:

Attached is the test result report from running a recent version of U-Boot on an iMX8 platform and parsed with the SCT_Parser tool.

g.

-------- Forwarded Message -------- Subject: Re: EBBR Testing topic for today? Date: Mon, 10 May 2021 14:22:56 +0100 From: Vincent Stehle Vincent.Stehle@arm.com To: Grant Likely Grant.Likely@arm.com

Hi Grant,

Here are some more SCT results: a run from this morning on Compulab, with latest firmware from Paul (+ 3 patches) and latest sct parser config.

INFO main: 0 dropped(s), 0 failure(s), 66 ignored(s), 10753 pass(s), 60 waived(s), 0 warning(s)

(The attached summary excludes the pass.)

Best regards, Vincent.

---

boot-architecture mailing list boot-architecture@lists.linaro.org https://lists.linaro.org/mailman/listinfo/boot-architecture

-- Masami Hiramatsu

-- Masami Hiramatsu

-- Masami Hiramatsu

2021年5月21日(金) 11:28 AKASHI Takahiro takahiro.akashi@linaro.org:

On Fri, May 21, 2021 at 09:00:11AM +0900, Masami Hiramatsu wrote:

...

Hi,

2021年5月20日(木) 23:55 Grant Likely grant.likely@arm.com:

...

Thanks Masami,

Comments below...

On 20/05/2021 11:09, Masami Hiramatsu wrote:

...

Hi Grant,

I have a comment on the U-Boot configuration: https://github.com/ARM-software/ebbr/wiki/U-Boot-Configuration-Guide

Another comment: Please don't use "efidebug capsule update" command. The only way that the current implementation of capsule update with a capsule file supports (or CONFIG_EFI_CAPSULE_ON_DISK) is "to reboot the system".

Agreed. What I used to run the capsule update is

1. Put the capsule file under (ESP*)/EFI/UpdateCapsule/ 2. Set up BootXXXX and BootOrder(or BootNext) so that it boot from the image in (ESP*) 3. Reboot 4. Run "bootefi bootmgr" or simply "printenv -e".

Note that if you put the (ESP*) on the USB flash, you have to scan the USB BEFORE step 4.

Thanks,

-Takahiro Akashi

...

...

...

The U-Boot Sniff Test requires bootefi hello and selftest,

u-boot=> bootefi hello ${fdtcontroladdr} u-boot=> bootefi selftest ${fdtcontroladdr}

---

So, U-Boot needs to enable below options too.

CONFIG_CMD_BOOTEFI_HELLO_COMPILE=y CONFIG_CMD_BOOTEFI_HELLO=y CONFIG_CMD_BOOTEFI_SELFTEST=y

While these commands are useful, they should not be required to be built in. I've added those options to the bottom of the configuration guide and moved the test commands out of the sniff test and into the configuration guide for those who want to run those tests.

Thanks for the update!

...

...

And I have a question about SCT's EBBR result (according to the EBBR-Testing-Guide). Recently my team ran the SCT on our board and got some failures. One of them was related to the Auth variable,

/opt/sct/SctPkg/TestCase/UEFI/EFI/RuntimeServices/VariableServices/BlackBoxTest/AuthVariableServicesBBTestFunction.c 635 Status - Invalid Parameter RT.SetVariable - Create one Time Base Auth Variable, the expect return status should be EFI_SUCCESS FAILURE

But I think U-Boot Auth Variable is not enabled unless UEFI secure boot is enabled. So other test cases are waived. But this specific test case failed.

guid: 008E18A5-C345-48AE-9134-61A692E30B87 set guid: 9338D0EC-807B-4750-986A-8F2A91BB3616

Easy answer here is to enable secure boot in U-Boot to get rid of these failures. I've added that option to the wiki

CONFIG_EFI_SECURE_BOOT=y

Hmm, but there are some other Auth Variable tests are correctly waived on the result. So I think it should be waived too. And as you said below, since SystemReady IR doesn't require the secure boot, I think the secure boot should not be enabled for EBBR test.

...

Vincent, this is something to discuss. Secure Boot is supported, but not required by SystemReady IR. Should we ignore failures when secure boot is disabled in the parsing script?

Thank you,

...

g.

...

Thank you,

2021年5月12日(水) 19:46 Masami Hiramatsu masami.hiramatsu@linaro.org:

...

Hi Grant,

Thanks, this is what I need. I'll check them out!

2021年5月12日(水) 18:56 Grant Likely grant.likely@arm.com:

...

Hi Masami,

Here are the instructions that I've been working on:

https://github.com/ARM-software/ebbr/wiki/EBBR-Testing-Guide https://github.com/ARM-software/ebbr/wiki/U-Boot-Configuration-Guide

I hope that helps

g.

On 12/05/2021 02:02, Masami Hiramatsu wrote: > Hello Grant and Vincent, > > I'm interested in passing SCT on my board with the latest U-Boot. > Would you know which CONFIG options are required for the SCT? > I guess some CONFIG_EFI* options are related, but some of them may not > be needed. > Is there any good documentation about configuring U-Boot? > > Thank you, > > 2021年5月10日(月) 22:36 Grant Likely grant.likely@arm.com: >> >> For the EBBR meeting today: >> >> Attached is the test result report from running a recent version of >> U-Boot on an iMX8 platform and parsed with the SCT_Parser tool. >> >> g. >> >> >> -------- Forwarded Message -------- >> Subject: Re: EBBR Testing topic for today? >> Date: Mon, 10 May 2021 14:22:56 +0100 >> From: Vincent Stehle Vincent.Stehle@arm.com >> To: Grant Likely Grant.Likely@arm.com >> >> >> >> Hi Grant, >> >> Here are some more SCT results: a run from this morning on Compulab, >> with latest firmware from Paul (+ 3 patches) and latest sct parser config. >> >> INFO main: 0 dropped(s), 0 failure(s), 66 ignored(s), 10753 pass(s), 60 >> waived(s), 0 warning(s) >> >> (The attached summary excludes the pass.) >> >> Best regards, >> Vincent. >> >> >> >> >> _______________________________________________ >> boot-architecture mailing list >> boot-architecture@lists.linaro.org >> https://lists.linaro.org/mailman/listinfo/boot-architecture > > >

-- Masami Hiramatsu

-- Masami Hiramatsu

-- Masami Hiramatsu _______________________________________________ boot-architecture mailing list boot-architecture@lists.linaro.org https://lists.linaro.org/mailman/listinfo/boot-architecture

On Fri, May 21, 2021 at 09:27:15AM +0100, Grant Likely wrote:

On 21/05/2021 03:28, AKASHI Takahiro wrote:

...

On Fri, May 21, 2021 at 09:00:11AM +0900, Masami Hiramatsu wrote:

...

Hi,

2021年5月20日(木) 23:55 Grant Likely grant.likely@arm.com:

...

Thanks Masami,

Comments below...

On 20/05/2021 11:09, Masami Hiramatsu wrote:

...

Hi Grant,

I have a comment on the U-Boot configuration: https://github.com/ARM-software/ebbr/wiki/U-Boot-Configuration-Guide

Another comment: Please don't use "efidebug capsule update" command. The only way that the current implementation of capsule update with a capsule file supports (or CONFIG_EFI_CAPSULE_ON_DISK) is "to reboot the system".

-Takahiro Akashi

On that note, have you done any testing with the Tianocoree CapsuleApp.efi? We've been experimenting with it, but haven't got it successfully working yet.

No. The test scenario that I ran in developing is located in U-Boot:test/py/tests/test_efi_capsule/(test_capsule_firmware.py)

-Takahiro Akashi

g.

On 5/31/21 7:51 PM, Vincent Stehlé wrote:

On Thu, May 20, 2021 at 03:55:21PM +0100, Grant Likely wrote: ..

...

...

And I have a question about SCT's EBBR result (according to the EBBR-Testing-Guide). Recently my team ran the SCT on our board and got some failures. One of them was related to the Auth variable,

/opt/sct/SctPkg/TestCase/UEFI/EFI/RuntimeServices/VariableServices/BlackBoxTest/AuthVariableServicesBBTestFunction.c 635 Status - Invalid Parameter RT.SetVariable - Create one Time Base Auth Variable, the expect return status should be EFI_SUCCESS FAILURE

But I think U-Boot Auth Variable is not enabled unless UEFI secure boot is enabled. So other test cases are waived. But this specific test case failed.

guid: 008E18A5-C345-48AE-9134-61A692E30B87 set guid: 9338D0EC-807B-4750-986A-8F2A91BB3616

Easy answer here is to enable secure boot in U-Boot to get rid of these failures. I've added that option to the wiki

CONFIG_EFI_SECURE_BOOT=y

Vincent, this is something to discuss. Secure Boot is supported, but not required by SystemReady IR. Should we ignore failures when secure boot is disabled in the parsing script?

Hi Grant and Masami-san,

I think we should ignore all authenticated EFI variables related failures, as those are not required by EBBR.

See this sct parser EBBR.yaml config file[1], where I have added some rules to ignore authenticated EFI variables related failures.

We have in the EBBR a chapter "UEFI Secure Boot (Optional)". If we have bugs in authenticated EFI variables, we may not have secure boot.

What does the SystemReady IR certification require concerning secure boot?

Best regards

Heinrich

Masami-san, could you please try to re-parse your tests results with the updated config file? If the specific test you mentioned still results in FAILURE, would you please send us your tests results Summary.ekl? This would allow us to add the proper rule to the parser config file.

Thanks!

Best regards,

Vincent Stehlé System Architect - Arm

Hi Vincent,

2021年6月1日(火) 2:51 Vincent Stehlé vincent.stehle@arm.com:

On Thu, May 20, 2021 at 03:55:21PM +0100, Grant Likely wrote: ..

...

...

And I have a question about SCT's EBBR result (according to the EBBR-Testing-Guide). Recently my team ran the SCT on our board and got some failures. One of them was related to the Auth variable,

/opt/sct/SctPkg/TestCase/UEFI/EFI/RuntimeServices/VariableServices/BlackBoxTest/AuthVariableServicesBBTestFunction.c 635 Status - Invalid Parameter RT.SetVariable - Create one Time Base Auth Variable, the expect return status should be EFI_SUCCESS FAILURE

But I think U-Boot Auth Variable is not enabled unless UEFI secure boot is enabled. So other test cases are waived. But this specific test case failed.

guid: 008E18A5-C345-48AE-9134-61A692E30B87 set guid: 9338D0EC-807B-4750-986A-8F2A91BB3616

Easy answer here is to enable secure boot in U-Boot to get rid of these failures. I've added that option to the wiki

CONFIG_EFI_SECURE_BOOT=y

Vincent, this is something to discuss. Secure Boot is supported, but not required by SystemReady IR. Should we ignore failures when secure boot is disabled in the parsing script?

Hi Grant and Masami-san,

I think we should ignore all authenticated EFI variables related failures, as those are not required by EBBR.

See this sct parser EBBR.yaml config file[1], where I have added some rules to ignore authenticated EFI variables related failures.

Thanks for the update!

Masami-san, could you please try to re-parse your tests results with the updated config file? If the specific test you mentioned still results in FAILURE, would you please send us your tests results Summary.ekl? This would allow us to add the proper rule to the parser config file.

Hmm, I've retried SCT (ACS included version) it but the specific test case has been failed.

GUID:008E18A5-C345-48AE-9134-61A692E30B87 Set GUID: 9338D0EC-807B-4750-986A-8F2A91BB3616 log: /home/edhcha01/RELEASE_BUILD/arm-systemready/IR/scripts/edk2-test/uefi-sct/SctPkg/TestCase/UEFI/EFI/RuntimeServices/VariableServices/BlackBoxTest/AuthVariableServicesBBTestFunction.c 635 Status - Invalid Parameter

I found your commit commit 2f3bf6340a28c93984653bd751aa71437cd1193d Author: Vincent Stehlé vincent.stehle@arm.com Date: Mon May 31 18:59:17 2021 +0200

EBBR.yaml: ignore authenticated variables failures

which ignores the auth variables, and it seems that the Set GUID is correct but GUID is not correct.

I also attached the Summary.ekl.

Thank you,

Hi Vincent,

Thanks for the fix! I confirmed it is marked as ignored now :)

Regards,

2021年6月1日(火) 18:08 Vincent Stehlé vincent.stehle@arm.com:

+Jeffrey

On Tue, Jun 01, 2021 at 04:59:32PM +0900, Masami Hiramatsu wrote: ..

...

Hmm, I've retried SCT (ACS included version) it but the specific test case has been failed.

GUID:008E18A5-C345-48AE-9134-61A692E30B87 Set GUID: 9338D0EC-807B-4750-986A-8F2A91BB3616 log: /home/edhcha01/RELEASE_BUILD/arm-systemready/IR/scripts/edk2-test/uefi-sct/SctPkg/TestCase/UEFI/EFI/RuntimeServices/VariableServices/BlackBoxTest/AuthVariableServicesBBTestFunction.c 635 Status - Invalid Parameter

I found your commit commit 2f3bf6340a28c93984653bd751aa71437cd1193d Author: Vincent Stehlé vincent.stehle@arm.com Date: Mon May 31 18:59:17 2021 +0200

EBBR.yaml: ignore authenticated variables failures

which ignores the auth variables, and it seems that the Set GUID is correct but GUID is not correct.

I also attached the Summary.ekl.

Thank you Masami-san.

I have added an entry matching this specific failure in [1]. This should make it into Jeffrey's repo after some time and, ultimately, into the EBBR.yaml version used by the ACS.

Best regards,

Vincent Stehlé System Architect - Arm

-- Masami Hiramatsu