I am upstreaming TF-A support for the Renesas R-car gen3 platforms [1].
In order to prevent the boot cpu from being switched off (hotpluged) I am having to work around the fact that OPTEE_OS does not support MIGRATE_INFO_TYPE or MIGRATE_INFO_UP_CPU. Its SPD at TF-A level [2] returns that the TrustedOS if fully MP aware and has not special requirements for migration.
The work around I implemented (patch below and [3]) will query the platform whenever the opteed SPD indicates that it has no migration restrictions (which currently is always)
Is there any work ongoing on OPTEE_OS to support MIGRATE_INFO and update the TF-A SPD or am I missing some fundamental point?
thanks Jorge
[1] https://github.com/ARM-software/arm-trusted-firmware/pull/1582 [2] https://github.com/ARM-software/arm-trusted-firmware/blob/master/services/sp... [3] https://github.com/ARM-software/arm-trusted-firmware/pull/1582/commits/9a7fe...
psci: implement platform hook to migrate_info
--- include/lib/psci/psci.h | 2 ++ lib/psci/psci_main.c | 35 ++++++++++++++++++++++++++++++++--- 2 files changed, 34 insertions(+), 3 deletions(-)
diff --git a/include/lib/psci/psci.h b/include/lib/psci/psci.h index b27e481..e705a81 100644 --- a/include/lib/psci/psci.h +++ b/include/lib/psci/psci.h @@ -322,12 +322,14 @@ typedef struct plat_psci_ops { int (*write_mem_protect)(int val); int (*system_reset2)(int is_vendor, int reset_type, u_register_t cookie); + int (*migrate_info)(u_register_t *mpidr); } plat_psci_ops_t;
/******************************************************************************* * Function & Data prototypes ******************************************************************************/ unsigned int psci_version(void); +int psci_migrate_info(u_register_t *mpidr); int psci_cpu_on(u_register_t target_cpu, uintptr_t entrypoint, u_register_t context_id); diff --git a/lib/psci/psci_main.c b/lib/psci/psci_main.c index fd822bc..ce5419a 100644 --- a/lib/psci/psci_main.c +++ b/lib/psci/psci_main.c @@ -44,6 +44,23 @@ int psci_cpu_on(u_register_t target_cpu, return psci_cpu_on_start(target_cpu, &ep); }
+int psci_platform_migrate_info(u_register_t *mpidr) +{ + int rc; + + if (psci_plat_pm_ops->migrate_info == NULL) { + /* no migrate restrictions */ + return PSCI_TOS_NOT_PRESENT_MP; + } + + rc = psci_plat_pm_ops->migrate_info(mpidr); + + assert((rc == PSCI_TOS_UP_MIG_CAP) || (rc == PSCI_TOS_NOT_UP_MIG_CAP) || + (rc == PSCI_TOS_NOT_PRESENT_MP) || (rc == PSCI_E_NOT_SUPPORTED)); + + return rc; +} + unsigned int psci_version(void) { return PSCI_MAJOR_VER | PSCI_MINOR_VER; @@ -276,8 +293,14 @@ int psci_migrate(u_register_t target_cpu) int psci_migrate_info_type(void) { u_register_t resident_cpu_mpidr; + int rc; + + rc = psci_spd_migrate_info(&resident_cpu_mpidr); + if ((rc != PSCI_TOS_NOT_UP_MIG_CAP) && (rc != PSCI_TOS_UP_MIG_CAP)) + return psci_platform_migrate_info(NULL); + + return rc;
- return psci_spd_migrate_info(&resident_cpu_mpidr); }
u_register_t psci_migrate_info_up_cpu(void) @@ -290,8 +313,14 @@ u_register_t psci_migrate_info_up_cpu(void) * psci_spd_migrate_info() returns. */ rc = psci_spd_migrate_info(&resident_cpu_mpidr); - if ((rc != PSCI_TOS_NOT_UP_MIG_CAP) && (rc != PSCI_TOS_UP_MIG_CAP)) - return (u_register_t)(register_t) PSCI_E_INVALID_PARAMS; + if ((rc != PSCI_TOS_NOT_UP_MIG_CAP) && (rc != PSCI_TOS_UP_MIG_CAP)) { + rc = psci_platform_migrate_info(&resident_cpu_mpidr); + if ((rc != PSCI_TOS_NOT_UP_MIG_CAP) && + (rc != PSCI_TOS_UP_MIG_CAP)) + /* neither the SDP nor the platform have any migration + restrictions: any core can be hotpluged */ + return (u_register_t)(register_t) PSCI_E_INVALID_PARAMS; + }
return resident_cpu_mpidr; }
On Wed, Oct 3, 2018 at 2:27 PM, Jorge Ramirez-Ortiz jramirez@baylibre.com wrote:
I am upstreaming TF-A support for the Renesas R-car gen3 platforms [1].
In order to prevent the boot cpu from being switched off (hotpluged) I am having to work around the fact that OPTEE_OS does not support MIGRATE_INFO_TYPE or MIGRATE_INFO_UP_CPU.
Why should OP-TEE care about that?
Its SPD at TF-A level [2] returns that the TrustedOS if fully MP aware and has not special requirements for migration.
Isn't that true for upstream OP-TEE?
The work around I implemented (patch below and [3]) will query the platform whenever the opteed SPD indicates that it has no migration restrictions (which currently is always)
Is there any work ongoing on OPTEE_OS to support MIGRATE_INFO and update the TF-A SPD or am I missing some fundamental point?
No, there's no work ongoing in this area. Please elaborate on why this is needed.
Thanks, Jens
thanks Jorge
[1] https://github.com/ARM-software/arm-trusted-firmware/pull/1582 [2] https://github.com/ARM-software/arm-trusted-firmware/blob/master/services/sp... [3] https://github.com/ARM-software/arm-trusted-firmware/pull/1582/commits/9a7fe...
psci: implement platform hook to migrate_info
include/lib/psci/psci.h | 2 ++ lib/psci/psci_main.c | 35 ++++++++++++++++++++++++++++++++--- 2 files changed, 34 insertions(+), 3 deletions(-)
diff --git a/include/lib/psci/psci.h b/include/lib/psci/psci.h index b27e481..e705a81 100644 --- a/include/lib/psci/psci.h +++ b/include/lib/psci/psci.h @@ -322,12 +322,14 @@ typedef struct plat_psci_ops { int (*write_mem_protect)(int val); int (*system_reset2)(int is_vendor, int reset_type, u_register_t cookie);
int (*migrate_info)(u_register_t *mpidr);} plat_psci_ops_t;
/*******************************************************************************
- Function & Data prototypes
******************************************************************************/ unsigned int psci_version(void); +int psci_migrate_info(u_register_t *mpidr); int psci_cpu_on(u_register_t target_cpu, uintptr_t entrypoint, u_register_t context_id); diff --git a/lib/psci/psci_main.c b/lib/psci/psci_main.c index fd822bc..ce5419a 100644 --- a/lib/psci/psci_main.c +++ b/lib/psci/psci_main.c @@ -44,6 +44,23 @@ int psci_cpu_on(u_register_t target_cpu, return psci_cpu_on_start(target_cpu, &ep); }
+int psci_platform_migrate_info(u_register_t *mpidr) +{
int rc;if (psci_plat_pm_ops->migrate_info == NULL) {/* no migrate restrictions */return PSCI_TOS_NOT_PRESENT_MP;}rc = psci_plat_pm_ops->migrate_info(mpidr);assert((rc == PSCI_TOS_UP_MIG_CAP) || (rc ==PSCI_TOS_NOT_UP_MIG_CAP) ||
(rc == PSCI_TOS_NOT_PRESENT_MP) || (rc ==PSCI_E_NOT_SUPPORTED));
return rc;+}
unsigned int psci_version(void) { return PSCI_MAJOR_VER | PSCI_MINOR_VER; @@ -276,8 +293,14 @@ int psci_migrate(u_register_t target_cpu) int psci_migrate_info_type(void) { u_register_t resident_cpu_mpidr;
int rc;rc = psci_spd_migrate_info(&resident_cpu_mpidr);if ((rc != PSCI_TOS_NOT_UP_MIG_CAP) && (rc != PSCI_TOS_UP_MIG_CAP))return psci_platform_migrate_info(NULL);return rc;
return psci_spd_migrate_info(&resident_cpu_mpidr);}
u_register_t psci_migrate_info_up_cpu(void) @@ -290,8 +313,14 @@ u_register_t psci_migrate_info_up_cpu(void) * psci_spd_migrate_info() returns. */ rc = psci_spd_migrate_info(&resident_cpu_mpidr);
if ((rc != PSCI_TOS_NOT_UP_MIG_CAP) && (rc != PSCI_TOS_UP_MIG_CAP))return (u_register_t)(register_t) PSCI_E_INVALID_PARAMS;
if ((rc != PSCI_TOS_NOT_UP_MIG_CAP) && (rc != PSCI_TOS_UP_MIG_CAP)){
rc = psci_platform_migrate_info(&resident_cpu_mpidr);if ((rc != PSCI_TOS_NOT_UP_MIG_CAP) &&(rc != PSCI_TOS_UP_MIG_CAP))/* neither the SDP nor the platform have anymigration
restrictions: any core can be hotpluged */return (u_register_t)(register_t)PSCI_E_INVALID_PARAMS;
} return resident_cpu_mpidr;}
2.7.4
Tee-dev mailing list Tee-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/tee-dev
On 10/03/2018 03:17 PM, Jens Wiklander wrote:
On Wed, Oct 3, 2018 at 2:27 PM, Jorge Ramirez-Ortiz jramirez@baylibre.com wrote:
I am upstreaming TF-A support for the Renesas R-car gen3 platforms [1].
In order to prevent the boot cpu from being switched off (hotpluged) I am having to work around the fact that OPTEE_OS does not support MIGRATE_INFO_TYPE or MIGRATE_INFO_UP_CPU.
Why should OP-TEE care about that?
the boot cpu boots OP-TEE; when OP-TEE has migration requirements the boot cpu must not be hot-plugged. So OP-TEE in away should care about implementing those interfaces (unless every OP-TEE ever created is guaranteed to have no migration restrictions).
Its SPD at TF-A level [2] returns that the TrustedOS if fully MP aware and has not special requirements for migration.
Isn't that true for upstream OP-TEE?
? but what happens if the user is working with a non-upstream OP-TEE? The way TF-A has been written hard-codes the requirement of an OP-TEE with no migration restrictions.
IMO the code at this level should be generic and always query. why dont you see it this way?
The work around I implemented (patch below and [3]) will query the platform whenever the opteed SPD indicates that it has no migration restrictions (which currently is always)
Is there any work ongoing on OPTEE_OS to support MIGRATE_INFO and update the TF-A SPD or am I missing some fundamental point?
No, there's no work ongoing in this area. Please elaborate on why this is needed.
see the above comments (should I elaborate more? not sure what else to add :))
Hi Jorge,
On 03.10.18 16:31, Jorge Ramirez-Ortiz wrote:
On 10/03/2018 03:17 PM, Jens Wiklander wrote:
On Wed, Oct 3, 2018 at 2:27 PM, Jorge Ramirez-Ortiz jramirez@baylibre.com wrote:
I am upstreaming TF-A support for the Renesas R-car gen3 platforms [1].
In order to prevent the boot cpu from being switched off (hotpluged) I am having to work around the fact that OPTEE_OS does not support MIGRATE_INFO_TYPE or MIGRATE_INFO_UP_CPU.
Why should OP-TEE care about that?
the boot cpu boots OP-TEE; when OP-TEE has migration requirements the boot cpu must not be hot-plugged. So OP-TEE in away should care about implementing those interfaces (unless every OP-TEE ever created is guaranteed to have no migration restrictions).
Aren't you confusing OP-TEE with TEE?
OP-TEE is written in a way, that it don't cares on which CPU it is running. AFAIR, it can even boot on non-CPU0 (but I can be mistaken there).
So, I can't imagine, how there can be any migration restrictions for OP-TEE at all.
On 10/03/2018 03:48 PM, Volodymyr Babchuk wrote:
Hi Jorge,
On 03.10.18 16:31, Jorge Ramirez-Ortiz wrote:
On 10/03/2018 03:17 PM, Jens Wiklander wrote:
On Wed, Oct 3, 2018 at 2:27 PM, Jorge Ramirez-Ortiz jramirez@baylibre.com wrote:
I am upstreaming TF-A support for the Renesas R-car gen3 platforms [1].
In order to prevent the boot cpu from being switched off (hotpluged) I am having to work around the fact that OPTEE_OS does not support MIGRATE_INFO_TYPE or MIGRATE_INFO_UP_CPU.
Why should OP-TEE care about that?
the boot cpu boots OP-TEE; when OP-TEE has migration requirements the boot cpu must not be hot-plugged. So OP-TEE in away should care about implementing those interfaces (unless every OP-TEE ever created is guaranteed to have no migration restrictions).
Aren't you confusing OP-TEE with TEE?
I dont claim to know a lot about this but I dont think so, isnt OP-TEE the one implementing to the PSCI API? https://github.com/OP-TEE/optee_os/blob/master/core/arch/arm/sm/psci.c#L70
I'd assume that the TrustedOS should just query the TEE?
OP-TEE is written in a way, that it don't cares on which CPU it is running. AFAIR, it can even boot on non-CPU0 (but I can be mistaken there).
So, I can't imagine, how there can be any migration restrictions for OP-TEE at all.
Jorge,
On 03.10.18 16:56, Jorge Ramirez-Ortiz wrote:
On 10/03/2018 03:48 PM, Volodymyr Babchuk wrote:
Hi Jorge,
On 03.10.18 16:31, Jorge Ramirez-Ortiz wrote:
On 10/03/2018 03:17 PM, Jens Wiklander wrote:
On Wed, Oct 3, 2018 at 2:27 PM, Jorge Ramirez-Ortiz jramirez@baylibre.com wrote:
I am upstreaming TF-A support for the Renesas R-car gen3 platforms [1].
In order to prevent the boot cpu from being switched off (hotpluged) I am having to work around the fact that OPTEE_OS does not support MIGRATE_INFO_TYPE or MIGRATE_INFO_UP_CPU.
Why should OP-TEE care about that?
the boot cpu boots OP-TEE; when OP-TEE has migration requirements the boot cpu must not be hot-plugged. So OP-TEE in away should care about implementing those interfaces (unless every OP-TEE ever created is guaranteed to have no migration restrictions).
Aren't you confusing OP-TEE with TEE?
I dont claim to know a lot about this but I dont think so, isnt OP-TEE the one implementing to the PSCI API? https://github.com/OP-TEE/optee_os/blob/master/core/arch/arm/sm/psci.c#L70
That piece of code is a secure monitor for ARMv7 systems, where ARM TF is absent. OP-TEE implements very simple secure monitor with PSCI support for such systems. But it is preferred to have full-scale Secure Monitor like ARM-TF. In the latter case, OP-TEE does not bothers about PSCI at all.
On ARMv7, where firmware is absent, it is possible to built OP-TEE with embedded Secure Monitor and in this case OP-TEE will act both as Trusted Execution Environment (TEE) and as firmware (for example, by providing PSCI services). If I remember right, this is due to ARMv7 limitation: Secure Monitor mode has no separate MMU context and it shares MMU state with Secure SVC mode. Thus, it is quite hard to create separate firmware (like ARM TF).
Luckily, this was fixed on ARMv8 and we now have separate firmware (ARM-TF on most systems) and SecureOS/TEE (like OP-TEE).
I'd assume that the TrustedOS should just query the TEE?
ARM TF have SPD module written specially for OP-TEE, so ARM-TF already knows a lot about how OP-TEE works. So, I see no sense, why ARM-TF should query OP-TEE about things that are not subjected to change.
On 10/03/2018 05:52 PM, Volodymyr Babchuk wrote:
Jorge,
On 03.10.18 16:56, Jorge Ramirez-Ortiz wrote:
On 10/03/2018 03:48 PM, Volodymyr Babchuk wrote:
Hi Jorge,
On 03.10.18 16:31, Jorge Ramirez-Ortiz wrote:
On 10/03/2018 03:17 PM, Jens Wiklander wrote:
On Wed, Oct 3, 2018 at 2:27 PM, Jorge Ramirez-Ortiz jramirez@baylibre.com wrote:
I am upstreaming TF-A support for the Renesas R-car gen3 platforms [1].
In order to prevent the boot cpu from being switched off (hotpluged) I am having to work around the fact that OPTEE_OS does not support MIGRATE_INFO_TYPE or MIGRATE_INFO_UP_CPU.
Why should OP-TEE care about that?
the boot cpu boots OP-TEE; when OP-TEE has migration requirements the boot cpu must not be hot-plugged. So OP-TEE in away should care about implementing those interfaces (unless every OP-TEE ever created is guaranteed to have no migration restrictions).
Aren't you confusing OP-TEE with TEE?
I dont claim to know a lot about this but I dont think so, isnt OP-TEE the one implementing to the PSCI API? https://github.com/OP-TEE/optee_os/blob/master/core/arch/arm/sm/psci.c#L70
That piece of code is a secure monitor for ARMv7 systems, where ARM TF is absent. OP-TEE implements very simple secure monitor with PSCI support for such systems. But it is preferred to have full-scale Secure Monitor like ARM-TF. In the latter case, OP-TEE does not bothers about PSCI at all.
ahhhh I see. perfect I understand now. Thanks for explaining.
On ARMv7, where firmware is absent, it is possible to built OP-TEE with embedded Secure Monitor and in this case OP-TEE will act both as Trusted Execution Environment (TEE) and as firmware (for example, by providing PSCI services). If I remember right, this is due to ARMv7 limitation: Secure Monitor mode has no separate MMU context and it shares MMU state with Secure SVC mode. Thus, it is quite hard to create separate firmware (like ARM TF).
Luckily, this was fixed on ARMv8 and we now have separate firmware (ARM-TF on most systems) and SecureOS/TEE (like OP-TEE).
ok
I'd assume that the TrustedOS should just query the TEE?
ARM TF have SPD module written specially for OP-TEE, so ARM-TF already knows a lot about how OP-TEE works.
right and this is what I was referring to. So similar to what TF-A does today when calling cpu_on/off/suspend/resume via the optee SPD I thought it should also query about migrate (instead of assuming that the tee has no restrictions).
From what you are all saying, I am working against my SoC restriction and not a TEE one; even more since nobody has required this interface before. So I think I am all set. thanks a lot for the explanations!
So, I see no sense, why ARM-TF should query OP-TEE about things that are not subjected to change.
On Wed, Oct 3, 2018 at 3:31 PM, Jorge Ramirez-Ortiz jramirez@baylibre.com wrote:
On 10/03/2018 03:17 PM, Jens Wiklander wrote:
On Wed, Oct 3, 2018 at 2:27 PM, Jorge Ramirez-Ortiz jramirez@baylibre.com wrote:
I am upstreaming TF-A support for the Renesas R-car gen3 platforms [1].
In order to prevent the boot cpu from being switched off (hotpluged) I am having to work around the fact that OPTEE_OS does not support MIGRATE_INFO_TYPE or MIGRATE_INFO_UP_CPU.
Why should OP-TEE care about that?
the boot cpu boots OP-TEE; when OP-TEE has migration requirements the boot cpu must not be hot-plugged.
This sounds a bit odd, what in OP-TEE makes the boot cpu so special?
Isn't it rather the SoC that has this requirement? If so perhaps this restriction belongs in the PCSI implementation instead?
So OP-TEE in away should care about implementing those interfaces (unless every OP-TEE ever created is guaranteed to have no migration restrictions).
Trying to predict what every fork of OP-TEE will ever do is not easy so I'm sticking to upstream OP-TEE.
Its SPD at TF-A level [2] returns that the TrustedOS if fully MP aware and has not special requirements for migration.
Isn't that true for upstream OP-TEE?
? but what happens if the user is working with a non-upstream OP-TEE? The way TF-A has been written hard-codes the requirement of an OP-TEE with no migration restrictions.
We try to not make it harder that necessary for the forks, but if there's a conflicts of interests it's upstream that matters.
IMO the code at this level should be generic and always query. why dont you see it this way?
So far I'm not convinced that it's needed, but I'll try to keep an open mind.
The work around I implemented (patch below and [3]) will query the platform whenever the opteed SPD indicates that it has no migration restrictions (which currently is always)
Is there any work ongoing on OPTEE_OS to support MIGRATE_INFO and update the TF-A SPD or am I missing some fundamental point?
No, there's no work ongoing in this area. Please elaborate on why this is needed.
see the above comments (should I elaborate more? not sure what else to add :))
Yes, I'd like to know why this isn't done in the PSCI implementation in TF-A.
Thanks, Jens
On 10/03/2018 03:56 PM, Jens Wiklander wrote:
On Wed, Oct 3, 2018 at 3:31 PM, Jorge Ramirez-Ortiz jramirez@baylibre.com wrote:
On 10/03/2018 03:17 PM, Jens Wiklander wrote:
On Wed, Oct 3, 2018 at 2:27 PM, Jorge Ramirez-Ortiz jramirez@baylibre.com wrote:
I am upstreaming TF-A support for the Renesas R-car gen3 platforms [1].
In order to prevent the boot cpu from being switched off (hotpluged) I am having to work around the fact that OPTEE_OS does not support MIGRATE_INFO_TYPE or MIGRATE_INFO_UP_CPU.
Why should OP-TEE care about that?
the boot cpu boots OP-TEE; when OP-TEE has migration requirements the boot cpu must not be hot-plugged.
This sounds a bit odd, what in OP-TEE makes the boot cpu so special?
Isn't it rather the SoC that has this requirement? If so perhaps this restriction belongs in the PCSI implementation instead?
yes I think this must be it (I suspected this to be the case but after this thread I am convinced that this must be the situation); the rcar-gen3 switches off everything when suspending so when resuming it must always start TF-A with a well known core (the boot cpu always follows the cold boot path before resuming the other cores via the warm boot path)
So OP-TEE in away should care about implementing those interfaces (unless every OP-TEE ever created is guaranteed to have no migration restrictions).
Trying to predict what every fork of OP-TEE will ever do is not easy so I'm sticking to upstream OP-TEE.
Its SPD at TF-A level [2] returns that the TrustedOS if fully MP aware and has not special requirements for migration.
Isn't that true for upstream OP-TEE?
? but what happens if the user is working with a non-upstream OP-TEE? The way TF-A has been written hard-codes the requirement of an OP-TEE with no migration restrictions.
We try to not make it harder that necessary for the forks, but if there's a conflicts of interests it's upstream that matters.
IMO the code at this level should be generic and always query. why dont you see it this way?
So far I'm not convinced that it's needed, but I'll try to keep an open mind.
maybe it is only ATF that needs changing to allow the platform to be queried about migration requirements ok makes sense. thanks !
-
Jens Wiklander -
Jorge Ramirez-Ortiz -
Volodymyr Babchuk