Hi All,
While enabling trusty os with xen, I took same approach as OP-TEE, with OP-TEE running in secure world. But I am also thinking this might introduce potential issue is that secure world OS communicate with DomU. If there are some misbehavior in secure world OS, it might let XEN hypervisor not work proper.
In my setup, trusty os sometimes panic in secure world, xen will not able to control the panic core anymore.
So I am thinking whether we need to emulating secure world in a XEN VM which is the VM running DomU. Just like what ACRN did to run trusty os.
Any comments?
Thanks, Peng.
Hi Peng,
On Mon, 15 Jun 2020 at 05:07, Peng Fan peng.fan@nxp.com wrote:
Hi All,
While enabling trusty os with xen, I took same approach as OP-TEE, with OP-TEE running in secure world. But I am also thinking this might introduce potential issue is that secure world OS communicate with DomU. If there are some misbehavior in secure world OS, it might let XEN hypervisor not work proper.
In my setup, trusty os sometimes panic in secure world, xen will not able to control the panic core anymore.
So I am thinking whether we need to emulating secure world in a XEN VM which is the VM running DomU. Just like what ACRN did to run trusty os.
Well, it depends on whom you are trusting more. Both XEN and TEE are minimal OS implementations with aim at security. I'm speaking about generic TEE OS, not about particular OS like OP-TEE or Trusty. Problem is that, if TEE is running inside VM, it will be susceptible to a hypervisor misbehaviour. You need to understand that Xen and privileged domain (dom0, mostly) can access memory of any guest. At least, in default configuration. There are means to harden this setup. But anyways, Xen can't be stopped from reading TEE's secrets.
If this is okay for your needs, then you can run TEE as a VM of course.
So, this is heavilly depends on your security threats model. There can't be universal solution. Also, I'm proposing to check Google's requirements for Trusty environment. Do they allow it to run outside of TrustZone? For example, GPD TEE System Architecture document clearly says that TEE should be separated from REE by hardware mechanisms that are not controlled by REE (section 2.2.1). I believe, that should be a similar document for Trusty.
Hi,
Subject: Re: [Tee-dev] TEE with XEN
Hi Peng,
On Mon, 15 Jun 2020 at 05:07, Peng Fan peng.fan@nxp.com wrote:
Hi All,
While enabling trusty os with xen, I took same approach as OP-TEE, with OP-TEE running in secure world. But I am also thinking this might introduce potential issue is that secure world OS communicate with DomU. If there are some misbehavior in secure world OS, it might let XEN hypervisor not work proper.
In my setup, trusty os sometimes panic in secure world, xen will not able to control the panic core anymore.
So I am thinking whether we need to emulating secure world in a XEN VM which is the VM running DomU. Just like what ACRN did to run trusty os.
Well, it depends on whom you are trusting more. Both XEN and TEE are minimal OS implementations with aim at security.
XEN is targeting safety. TEE is targeting security.
I'm speaking about generic
TEE OS, not about particular OS like OP-TEE or Trusty. Problem is that, if TEE is running inside VM, it will be susceptible to a hypervisor misbehaviour. You need to understand that Xen and privileged domain (dom0, mostly) can access memory of any guest. At least, in default configuration. There are means to harden this setup. But anyways, Xen can't be stopped from reading TEE's secrets.
Yes. Understand.
If this is okay for your needs, then you can run TEE as a VM of course.
So, this is heavilly depends on your security threats model. There can't be universal solution. Also, I'm proposing to check Google's requirements for Trusty environment.
Let me try to ask Google guys to see any feedback.
Thanks, Peng.
Do they allow it to run outside of TrustZone? For example, GPD TEE System Architecture document clearly says that TEE should be separated from REE by hardware mechanisms that are not controlled by REE (section 2.2.1). I believe, that should be a similar document for Trusty.
-- WBR Volodymyr Babchuk aka lorc [+380976646013] mailto: vlad.babchuk@gmail.com
+Bertrand and Stefano
On 16/06/2020 02:24, Volodymyr Babchuk wrote:
Hi Peng,
On Mon, 15 Jun 2020 at 05:07, Peng Fan peng.fan@nxp.com wrote:
Hi All,
While enabling trusty os with xen, I took same approach as OP-TEE, with OP-TEE running in secure world. But I am also thinking this might introduce potential issue is that secure world OS communicate with DomU. If there are some misbehavior in secure world OS, it might let XEN hypervisor not work proper.
In my setup, trusty os sometimes panic in secure world, xen will not able to control the panic core anymore.
May I ask in which case Trusty is panicking?
So I am thinking whether we need to emulating secure world in a XEN VM which is the VM running DomU. Just like what ACRN did to run trusty os.
Well, it depends on whom you are trusting more. Both XEN and TEE are minimal OS implementations with aim at security. I'm speaking about generic TEE OS, not about particular OS like OP-TEE or Trusty. Problem is that, if TEE is running inside VM, it will be susceptible to a hypervisor misbehaviour. You need to understand that Xen and privileged domain (dom0, mostly) can access memory of any guest. At least, in default configuration. There are means to harden this setup. But anyways, Xen can't be stopped from reading TEE's secrets.
IIRC, we discussed this approach for OP-TEE in the past. There was other potential pitfalls with it. For instance, you wouldn't be able to directly access any secure device from that guest (it is running in non-secure world).
There are also issues in term of latency as you may have the following model:
domU -> Xen -> domU TEE -> (Xen -> host TEE -> Xen -> domU TEE) -> Xen -> domU.
The bit in () is if you require to call the host TEE.
One possibility would be to use Secure-EL2 for your Trusty OS. But I don't know whether your platform supports it.
Depending on whether you can modify Trusty OS, alternative would be to make itvirtualization aware as OP-TEE did. The core would need to be resilient and the panic only affect a given client.
Cheers,
On Thu, 18 Jun 2020, Julien Grall wrote:
+Bertrand and Stefano
Thanks for CC'ing me
On 16/06/2020 02:24, Volodymyr Babchuk wrote:
Hi Peng,
On Mon, 15 Jun 2020 at 05:07, Peng Fan peng.fan@nxp.com wrote:
Hi All,
While enabling trusty os with xen, I took same approach as OP-TEE, with OP-TEE running in secure world. But I am also thinking this might introduce potential issue is that secure world OS communicate with DomU. If there are some misbehavior in secure world OS, it might let XEN hypervisor not work proper.
In my setup, trusty os sometimes panic in secure world, xen will not able to control the panic core anymore.
May I ask in which case Trusty is panicking?
So I am thinking whether we need to emulating secure world in a XEN VM which is the VM running DomU. Just like what ACRN did to run trusty os.
Well, it depends on whom you are trusting more. Both XEN and TEE are minimal OS implementations with aim at security. I'm speaking about generic TEE OS, not about particular OS like OP-TEE or Trusty. Problem is that, if TEE is running inside VM, it will be susceptible to a hypervisor misbehaviour. You need to understand that Xen and privileged domain (dom0, mostly) can access memory of any guest. At least, in default configuration. There are means to harden this setup. But anyways, Xen can't be stopped from reading TEE's secrets.
IIRC, we discussed this approach for OP-TEE in the past. There was other potential pitfalls with it. For instance, you wouldn't be able to directly access any secure device from that guest (it is running in non-secure world).
Given that Secure World has access to Normal World but not vice versa, it is more natural to run Trusty in one of the Secure execution levels. The expectation is that Trusty has higher privilege, thus it is more "trusted" than anything running in Normal World, including Xen.
Of course no client should be able to crash Trusty, so the reality sometimes can be very different from the theory :-)
If I were you, I would consider running the TEE "inside" the VM only if the service that the TEE provides is strongly coupled with the VM and doesn't actually need a high level of privilege to function (i.e. doesn't access secure devices or at least not often.)
Depending on whether you can modify Trusty OS, alternative would be to make itvirtualization aware as OP-TEE did. The core would need to be resilient and the panic only affect a given client.
This would most probably be the best compromise.
On 18 Jun 2020, at 23:17, Stefano Stabellini sstabellini@kernel.org wrote:
On Thu, 18 Jun 2020, Julien Grall wrote:
+Bertrand and Stefano
Thanks for CC'ing me
On 16/06/2020 02:24, Volodymyr Babchuk wrote:
Hi Peng,
On Mon, 15 Jun 2020 at 05:07, Peng Fan peng.fan@nxp.com wrote:
Hi All,
While enabling trusty os with xen, I took same approach as OP-TEE, with OP-TEE running in secure world. But I am also thinking this might introduce potential issue is that secure world OS communicate with DomU. If there are some misbehavior in secure world OS, it might let XEN hypervisor not work proper.
In my setup, trusty os sometimes panic in secure world, xen will not able to control the panic core anymore.
May I ask in which case Trusty is panicking?
So I am thinking whether we need to emulating secure world in a XEN VM which is the VM running DomU. Just like what ACRN did to run trusty os.
Well, it depends on whom you are trusting more. Both XEN and TEE are minimal OS implementations with aim at security. I'm speaking about generic TEE OS, not about particular OS like OP-TEE or Trusty. Problem is that, if TEE is running inside VM, it will be susceptible to a hypervisor misbehaviour. You need to understand that Xen and privileged domain (dom0, mostly) can access memory of any guest. At least, in default configuration. There are means to harden this setup. But anyways, Xen can't be stopped from reading TEE's secrets.
IIRC, we discussed this approach for OP-TEE in the past. There was other potential pitfalls with it. For instance, you wouldn't be able to directly access any secure device from that guest (it is running in non-secure world).
Given that Secure World has access to Normal World but not vice versa, it is more natural to run Trusty in one of the Secure execution levels. The expectation is that Trusty has higher privilege, thus it is more "trusted" than anything running in Normal World, including Xen.
Of course no client should be able to crash Trusty, so the reality sometimes can be very different from the theory :-)
If I were you, I would consider running the TEE "inside" the VM only if the service that the TEE provides is strongly coupled with the VM and doesn't actually need a high level of privilege to function (i.e. doesn't access secure devices or at least not often.)
I could see some scenarios where this would make sense if you trust one VM more then an other. But this falls into a model of “virtualizing” optee using a VM where from the system point of view any usage of this VM from the functionality could not be more trusted then Xen And the VM providing the service.
Depending on whether you can modify Trusty OS, alternative would be to make itvirtualization aware as OP-TEE did. The core would need to be resilient and the panic only affect a given client.
This would most probably be the best compromise.
Agree.
Bertrand
Hi,
On 18 Jun 2020, at 19:05, Julien Grall julien@xen.org wrote:
+Bertrand and Stefano
On 16/06/2020 02:24, Volodymyr Babchuk wrote:
Hi Peng, On Mon, 15 Jun 2020 at 05:07, Peng Fan peng.fan@nxp.com wrote:
Hi All,
While enabling trusty os with xen, I took same approach as OP-TEE, with OP-TEE running in secure world. But I am also thinking this might introduce potential issue is that secure world OS communicate with DomU. If there are some misbehavior in secure world OS, it might let XEN hypervisor not work proper.
In my setup, trusty os sometimes panic in secure world, xen will not able to control the panic core anymore.
May I ask in which case Trusty is panicking?
In any case, optee should protect itself against this and it should be considered that optee is more priviledged then Xen. So if optee is crashing we cannot expect that Xen can recover or fix it.
I would more consider this as a bug that optee needs to be robust against.
So I am thinking whether we need to emulating secure world in a XEN VM which is the VM running DomU. Just like what ACRN did to run trusty os.
Well, it depends on whom you are trusting more. Both XEN and TEE are minimal OS implementations with aim at security. I'm speaking about generic TEE OS, not about particular OS like OP-TEE or Trusty. Problem is that, if TEE is running inside VM, it will be susceptible to a hypervisor misbehaviour. You need to understand that Xen and privileged domain (dom0, mostly) can access memory of any guest. At least, in default configuration. There are means to harden this setup. But anyways, Xen can't be stopped from reading TEE's secrets.
IIRC, we discussed this approach for OP-TEE in the past. There was other potential pitfalls with it. For instance, you wouldn't be able to directly access any secure device from that guest (it is running in non-secure world).
There are also issues in term of latency as you may have the following model:
domU -> Xen -> domU TEE -> (Xen -> host TEE -> Xen -> domU TEE) -> Xen -> domU.
The bit in () is if you require to call the host TEE.
One possibility would be to use Secure-EL2 for your Trusty OS. But I don't know whether your platform supports it.
Depending on whether you can modify Trusty OS, alternative would be to make itvirtualization aware as OP-TEE did. The core would need to be resilient and the panic only affect a given client.
I do not have right a clear idea of what is the status of optee and xen but in theory I would see 2 possible ways to handle this: - without optee modification, something in a guest (Dom0 or an other priviledged one) needs to have access to optee and emulate optee access for others. - with optee modifications, optee needs to have a concept of client and Xen would need to passthrough optee requests but being responsible of adding a “client” identifier. Maybe also informing Optee when a new client is created/removed.
The second scenario could then be somehow splitted in the previous one from Julien if some parts would need to be emulated somewhere in some kind of combination of the 2 models.
In any case i would always consider that anything running on optee (or in general in the secure world) is more trusted then Xen.
Regards Bertrand
Hi Bertrand,
Subject: Re: [Tee-dev] TEE with XEN
Hi,
On 18 Jun 2020, at 19:05, Julien Grall julien@xen.org wrote:
+Bertrand and Stefano
On 16/06/2020 02:24, Volodymyr Babchuk wrote:
Hi Peng, On Mon, 15 Jun 2020 at 05:07, Peng Fan peng.fan@nxp.com wrote:
Hi All,
While enabling trusty os with xen, I took same approach as OP-TEE, with OP-TEE running in secure world. But I am also thinking this might introduce potential issue is that secure world OS communicate with
DomU.
If there are some misbehavior in secure world OS, it might let XEN hypervisor not work proper.
In my setup, trusty os sometimes panic in secure world, xen will not able to control the panic core anymore.
May I ask in which case Trusty is panicking?
In any case, optee should protect itself against this and it should be considered that optee is more priviledged then Xen. So if optee is crashing we cannot expect that Xen can recover or fix it.
I would more consider this as a bug that optee needs to be robust against.
ok. I am not using OP-TEE, currently I use google trusty OS.
I have two OS, Dom0 linux + DomU android auto.
DomU android auto needs trusty OS, Dom0 Linux not need that.
I not wanna trusty OS for DomU could bring any detect to Dom0 or xen.
One more case is if dom0 linux needs OP-TEE, DomU needs google trusty, how we handle this in one SoC?
So I am thinking whether we need to emulating secure world in a XEN VM which is the VM running DomU. Just like what ACRN did to run trusty os.
Well, it depends on whom you are trusting more. Both XEN and TEE are minimal OS implementations with aim at security. I'm speaking about generic TEE OS, not about particular OS like OP-TEE or Trusty. Problem is that, if TEE is running inside VM, it will be susceptible to a hypervisor misbehaviour. You need to understand that Xen and privileged domain (dom0, mostly) can access memory of any guest. At least, in default configuration. There are means to harden this setup. But anyways, Xen can't be stopped from reading TEE's secrets.
IIRC, we discussed this approach for OP-TEE in the past. There was other
potential pitfalls with it. For instance, you wouldn't be able to directly access any secure device from that guest (it is running in non-secure world).
There are also issues in term of latency as you may have the following
model:
domU -> Xen -> domU TEE -> (Xen -> host TEE -> Xen -> domU TEE) -> Xen ->
domU.
The bit in () is if you require to call the host TEE.
One possibility would be to use Secure-EL2 for your Trusty OS. But I don't
know whether your platform supports it.
Depending on whether you can modify Trusty OS, alternative would be to
make itvirtualization aware as OP-TEE did. The core would need to be resilient and the panic only affect a given client.
I do not have right a clear idea of what is the status of optee and xen but in theory I would see 2 possible ways to handle this:
- without optee modification, something in a guest (Dom0 or an other
priviledged one) needs to have access to optee and emulate optee access for others.
- with optee modifications, optee needs to have a concept of client and Xen
would need to passthrough optee requests but being responsible of adding a “client” identifier. Maybe also informing Optee when a new client is created/removed.
The second scenario could then be somehow splitted in the previous one from Julien if some parts would need to be emulated somewhere in some kind of combination of the 2 models.
In any case i would always consider that anything running on optee (or in general in the secure world) is more trusted then Xen.
Ok, this means optee runs on all cores in secure world, but this would not work when we need to support multiple OSes with their own TEE.
Regards, Peng.
Regards Bertrand
On 19 Jun 2020, at 09:52, Peng Fan peng.fan@nxp.com wrote:
Hi Bertrand,
Subject: Re: [Tee-dev] TEE with XEN
Hi,
On 18 Jun 2020, at 19:05, Julien Grall julien@xen.org wrote:
+Bertrand and Stefano
On 16/06/2020 02:24, Volodymyr Babchuk wrote:
Hi Peng, On Mon, 15 Jun 2020 at 05:07, Peng Fan peng.fan@nxp.com wrote:
Hi All,
While enabling trusty os with xen, I took same approach as OP-TEE, with OP-TEE running in secure world. But I am also thinking this might introduce potential issue is that secure world OS communicate with
DomU.
If there are some misbehavior in secure world OS, it might let XEN hypervisor not work proper.
In my setup, trusty os sometimes panic in secure world, xen will not able to control the panic core anymore.
May I ask in which case Trusty is panicking?
In any case, optee should protect itself against this and it should be considered that optee is more priviledged then Xen. So if optee is crashing we cannot expect that Xen can recover or fix it.
I would more consider this as a bug that optee needs to be robust against.
ok. I am not using OP-TEE, currently I use google trusty OS.
Sorry i should have been more generic. Please read this as “Anything running in secure world”, being optee or trusty.
I have two OS, Dom0 linux + DomU android auto.
DomU android auto needs trusty OS, Dom0 Linux not need that.
But i would guess your Dom0 is more “critical” then your DomU. In this case you must make sure that any resource given to your DomU cannot affect your Dom0. For example: if the DomU is starting a very heavy operation in blocked in trusty, any interrupt for non-secure could be blocked, thus affecting the ability of your Dom0.
I not wanna trusty OS for DomU could bring any detect to Dom0 or xen.
One more case is if dom0 linux needs OP-TEE, DomU needs google trusty, how we handle this in one SoC?
You have a shared resource in this case, someone more or as trusted as the clients needs to decide how the resource can be shared. In this case could be Dom0 or Xen or Trusty or op-Tee (if i should make an order).
So I am thinking whether we need to emulating secure world in a XEN VM which is the VM running DomU. Just like what ACRN did to run trusty os.
Well, it depends on whom you are trusting more. Both XEN and TEE are minimal OS implementations with aim at security. I'm speaking about generic TEE OS, not about particular OS like OP-TEE or Trusty. Problem is that, if TEE is running inside VM, it will be susceptible to a hypervisor misbehaviour. You need to understand that Xen and privileged domain (dom0, mostly) can access memory of any guest. At least, in default configuration. There are means to harden this setup. But anyways, Xen can't be stopped from reading TEE's secrets.
IIRC, we discussed this approach for OP-TEE in the past. There was other
potential pitfalls with it. For instance, you wouldn't be able to directly access any secure device from that guest (it is running in non-secure world).
There are also issues in term of latency as you may have the following
model:
domU -> Xen -> domU TEE -> (Xen -> host TEE -> Xen -> domU TEE) -> Xen ->
domU.
The bit in () is if you require to call the host TEE.
One possibility would be to use Secure-EL2 for your Trusty OS. But I don't
know whether your platform supports it.
Depending on whether you can modify Trusty OS, alternative would be to
make itvirtualization aware as OP-TEE did. The core would need to be resilient and the panic only affect a given client.
I do not have right a clear idea of what is the status of optee and xen but in theory I would see 2 possible ways to handle this:
- without optee modification, something in a guest (Dom0 or an other
priviledged one) needs to have access to optee and emulate optee access for others.
- with optee modifications, optee needs to have a concept of client and Xen
would need to passthrough optee requests but being responsible of adding a “client” identifier. Maybe also informing Optee when a new client is created/removed.
The second scenario could then be somehow splitted in the previous one from Julien if some parts would need to be emulated somewhere in some kind of combination of the 2 models.
In any case i would always consider that anything running on optee (or in general in the secure world) is more trusted then Xen.
Ok, this means optee runs on all cores in secure world, but this would not work when we need to support multiple OSes with their own TEE.
I would think you have one TEE running on all cores (or runnable in this case). So the Tee needs to handle several contexts or someone needs to virtualize it.
Regards Bertrand
Subject: Re: [Tee-dev] TEE with XEN
On 19 Jun 2020, at 09:52, Peng Fan peng.fan@nxp.com wrote:
Hi Bertrand,
Subject: Re: [Tee-dev] TEE with XEN
Hi,
On 18 Jun 2020, at 19:05, Julien Grall julien@xen.org wrote:
+Bertrand and Stefano
On 16/06/2020 02:24, Volodymyr Babchuk wrote:
Hi Peng, On Mon, 15 Jun 2020 at 05:07, Peng Fan peng.fan@nxp.com wrote:
Hi All,
While enabling trusty os with xen, I took same approach as OP-TEE, with OP-TEE running in secure world. But I am also thinking this might introduce potential issue is that secure world OS communicate with
DomU.
If there are some misbehavior in secure world OS, it might let XEN hypervisor not work proper.
In my setup, trusty os sometimes panic in secure world, xen will not able to control the panic core anymore.
May I ask in which case Trusty is panicking?
In any case, optee should protect itself against this and it should be considered that optee is more priviledged then Xen. So if optee is crashing we cannot expect that Xen can recover or fix it.
I would more consider this as a bug that optee needs to be robust against.
ok. I am not using OP-TEE, currently I use google trusty OS.
Sorry i should have been more generic. Please read this as “Anything running in secure world”, being optee or trusty.
I have two OS, Dom0 linux + DomU android auto.
DomU android auto needs trusty OS, Dom0 Linux not need that.
But i would guess your Dom0 is more “critical” then your DomU. In this case you must make sure that any resource given to your DomU cannot affect your Dom0. For example: if the DomU is starting a very heavy operation in blocked in trusty, any interrupt for non-secure could be blocked, thus affecting the ability of your Dom0.
I not wanna trusty OS for DomU could bring any detect to Dom0 or xen.
One more case is if dom0 linux needs OP-TEE, DomU needs google trusty, how we handle this in one SoC?
You have a shared resource in this case, someone more or as trusted as the clients needs to decide how the resource can be shared. In this case could be Dom0 or Xen or Trusty or op-Tee (if i should make an order).
So I am thinking whether we need to emulating secure world in a XEN VM which is the VM running DomU. Just like what ACRN did to run trusty os.
Well, it depends on whom you are trusting more. Both XEN and TEE are minimal OS implementations with aim at security. I'm speaking about generic TEE OS, not about particular OS like OP-TEE or Trusty. Problem is that, if TEE is running inside VM, it will be susceptible to a hypervisor misbehaviour. You need to understand that Xen and privileged domain (dom0, mostly) can access memory of
any guest.
At least, in default configuration. There are means to harden this setup. But anyways, Xen can't be stopped from reading TEE's secrets.
IIRC, we discussed this approach for OP-TEE in the past. There was other
potential pitfalls with it. For instance, you wouldn't be able to directly access any secure device from that guest (it is running in
non-secure world).
There are also issues in term of latency as you may have the following
model:
domU -> Xen -> domU TEE -> (Xen -> host TEE -> Xen -> domU TEE) -> Xen ->
domU.
The bit in () is if you require to call the host TEE.
One possibility would be to use Secure-EL2 for your Trusty OS. But I don't
know whether your platform supports it.
Depending on whether you can modify Trusty OS, alternative would be to
make itvirtualization aware as OP-TEE did. The core would need to be resilient and the panic only affect a given client.
I do not have right a clear idea of what is the status of optee and xen but in theory I would see 2 possible ways to handle this:
- without optee modification, something in a guest (Dom0 or an other
priviledged one) needs to have access to optee and emulate optee access for others.
- with optee modifications, optee needs to have a concept of client
and Xen would need to passthrough optee requests but being responsible of adding a “client” identifier. Maybe also informing Optee when a new client is created/removed.
The second scenario could then be somehow splitted in the previous one from Julien if some parts would need to be emulated somewhere in some kind of combination of the 2 models.
In any case i would always consider that anything running on optee (or in general in the secure world) is more trusted then Xen.
Ok, this means optee runs on all cores in secure world, but this would not work when we need to support multiple OSes with their own TEE.
I would think you have one TEE running on all cores (or runnable in this case). So the Tee needs to handle several contexts or someone needs to virtualize it.
This back to my original question, should I virtualize TEE in a XEN dedicated VM? or I need to emulate secure world to let one VM could have secure and non-secure world?
Thanks, Peng.
Regards Bertrand
Hi Peng,
On Fri, 19 Jun 2020 at 12:06, Peng Fan peng.fan@nxp.com wrote:
Subject: Re: [Tee-dev] TEE with XEN
On 19 Jun 2020, at 09:52, Peng Fan peng.fan@nxp.com wrote:
Hi Bertrand,
Subject: Re: [Tee-dev] TEE with XEN
Hi,
On 18 Jun 2020, at 19:05, Julien Grall julien@xen.org wrote:
+Bertrand and Stefano
On 16/06/2020 02:24, Volodymyr Babchuk wrote:
Hi Peng, On Mon, 15 Jun 2020 at 05:07, Peng Fan peng.fan@nxp.com wrote: > > Hi All, > > While enabling trusty os with xen, I took same approach as OP-TEE, > with OP-TEE running in secure world. But I am also thinking this > might introduce potential issue is that secure world OS > communicate with
DomU.
> If there are some misbehavior in secure world OS, it might let XEN > hypervisor not work proper. > > In my setup, trusty os sometimes panic in secure world, xen will > not able to control the panic core anymore.
May I ask in which case Trusty is panicking?
In any case, optee should protect itself against this and it should be considered that optee is more priviledged then Xen. So if optee is crashing we cannot expect that Xen can recover or fix it.
I would more consider this as a bug that optee needs to be robust against.
ok. I am not using OP-TEE, currently I use google trusty OS.
Sorry i should have been more generic. Please read this as “Anything running in secure world”, being optee or trusty.
I have two OS, Dom0 linux + DomU android auto.
DomU android auto needs trusty OS, Dom0 Linux not need that.
But i would guess your Dom0 is more “critical” then your DomU. In this case you must make sure that any resource given to your DomU cannot affect your Dom0. For example: if the DomU is starting a very heavy operation in blocked in trusty, any interrupt for non-secure could be blocked, thus affecting the ability of your Dom0.
I not wanna trusty OS for DomU could bring any detect to Dom0 or xen.
One more case is if dom0 linux needs OP-TEE, DomU needs google trusty, how we handle this in one SoC?
You have a shared resource in this case, someone more or as trusted as the clients needs to decide how the resource can be shared. In this case could be Dom0 or Xen or Trusty or op-Tee (if i should make an order).
> > So I am thinking whether we need to emulating secure world in a > XEN VM which is the VM running DomU. Just like what ACRN did to > run trusty os. Well, it depends on whom you are trusting more. Both XEN and TEE are minimal OS implementations with aim at security. I'm speaking about generic TEE OS, not about particular OS like OP-TEE or Trusty. Problem is that, if TEE is running inside VM, it will be susceptible to a hypervisor misbehaviour. You need to understand that Xen and privileged domain (dom0, mostly) can access memory of
any guest.
At least, in default configuration. There are means to harden this setup. But anyways, Xen can't be stopped from reading TEE's secrets.
IIRC, we discussed this approach for OP-TEE in the past. There was other
potential pitfalls with it. For instance, you wouldn't be able to directly access any secure device from that guest (it is running in
non-secure world).
There are also issues in term of latency as you may have the following
model:
domU -> Xen -> domU TEE -> (Xen -> host TEE -> Xen -> domU TEE) -> Xen ->
domU.
The bit in () is if you require to call the host TEE.
One possibility would be to use Secure-EL2 for your Trusty OS. But I don't
know whether your platform supports it.
Depending on whether you can modify Trusty OS, alternative would be to
make itvirtualization aware as OP-TEE did. The core would need to be resilient and the panic only affect a given client.
I do not have right a clear idea of what is the status of optee and xen but in theory I would see 2 possible ways to handle this:
- without optee modification, something in a guest (Dom0 or an other
priviledged one) needs to have access to optee and emulate optee access for others.
- with optee modifications, optee needs to have a concept of client
and Xen would need to passthrough optee requests but being responsible of adding a “client” identifier. Maybe also informing Optee when a new client is created/removed.
The second scenario could then be somehow splitted in the previous one from Julien if some parts would need to be emulated somewhere in some kind of combination of the 2 models.
In any case i would always consider that anything running on optee (or in general in the secure world) is more trusted then Xen.
Ok, this means optee runs on all cores in secure world, but this would not work when we need to support multiple OSes with their own TEE.
I would think you have one TEE running on all cores (or runnable in this case). So the Tee needs to handle several contexts or someone needs to virtualize it.
This back to my original question, should I virtualize TEE in a XEN dedicated VM? or I need to emulate secure world to let one VM could have secure and non-secure world?
Well, I think that the best approach is that we did in the OP-TEE: make Trusty virtualization-aware, so it can handle multiple VMs.
Things are more funny if you want to use multiple different TEEs (like OP-TEE and Trusty) at the same time. If this is your case, then the best approach is to implement something like para-virtualization in the Secure World. But this would require quite big efforts, of course.
Subject: Re: [Tee-dev] TEE with XEN
Hi Peng,
On Fri, 19 Jun 2020 at 12:06, Peng Fan peng.fan@nxp.com wrote:
Subject: Re: [Tee-dev] TEE with XEN
On 19 Jun 2020, at 09:52, Peng Fan peng.fan@nxp.com wrote:
Hi Bertrand,
Subject: Re: [Tee-dev] TEE with XEN
Hi,
On 18 Jun 2020, at 19:05, Julien Grall julien@xen.org wrote:
+Bertrand and Stefano
On 16/06/2020 02:24, Volodymyr Babchuk wrote: > Hi Peng, > On Mon, 15 Jun 2020 at 05:07, Peng Fan peng.fan@nxp.com
wrote:
>> >> Hi All, >> >> While enabling trusty os with xen, I took same approach as >> OP-TEE, with OP-TEE running in secure world. But I am also >> thinking this might introduce potential issue is that secure >> world OS communicate with
DomU.
>> If there are some misbehavior in secure world OS, it might let >> XEN hypervisor not work proper. >> >> In my setup, trusty os sometimes panic in secure world, xen >> will not able to control the panic core anymore.
May I ask in which case Trusty is panicking?
In any case, optee should protect itself against this and it should be considered that optee is more priviledged then Xen. So if optee is crashing we cannot expect that Xen can recover or fix it.
I would more consider this as a bug that optee needs to be robust
against.
ok. I am not using OP-TEE, currently I use google trusty OS.
Sorry i should have been more generic. Please read this as “Anything running in secure world”, being optee or
trusty.
I have two OS, Dom0 linux + DomU android auto.
DomU android auto needs trusty OS, Dom0 Linux not need that.
But i would guess your Dom0 is more “critical” then your DomU. In this case you must make sure that any resource given to your DomU cannot affect your Dom0. For example: if the DomU is starting a very heavy operation in blocked in trusty, any interrupt for non-secure could be blocked, thus affecting the ability of your Dom0.
I not wanna trusty OS for DomU could bring any detect to Dom0 or xen.
One more case is if dom0 linux needs OP-TEE, DomU needs google trusty, how we handle this in one SoC?
You have a shared resource in this case, someone more or as trusted as the clients needs to decide how the resource can be shared. In this case could be Dom0 or Xen or Trusty or op-Tee (if i should make an order).
>> >> So I am thinking whether we need to emulating secure world in >> a XEN VM which is the VM running DomU. Just like what ACRN did >> to run trusty os. > Well, it depends on whom you are trusting more. Both XEN and > TEE are minimal OS implementations with aim at security. I'm > speaking about generic TEE OS, not about particular OS like OP-TEE
or Trusty.
> Problem is that, if TEE is running inside VM, it will be > susceptible to a hypervisor misbehaviour. You need to > understand that Xen and privileged domain (dom0, mostly) can > access memory of
any guest.
> At least, in default configuration. There are means to harden > this setup. But anyways, Xen can't be stopped from reading TEE's
secrets.
IIRC, we discussed this approach for OP-TEE in the past. There was other
potential pitfalls with it. For instance, you wouldn't be able to directly access any secure device from that guest (it is running in
non-secure world).
There are also issues in term of latency as you may have the following
model:
domU -> Xen -> domU TEE -> (Xen -> host TEE -> Xen -> domU TEE) -> Xen ->
domU.
The bit in () is if you require to call the host TEE.
One possibility would be to use Secure-EL2 for your Trusty OS. But I don't
know whether your platform supports it.
Depending on whether you can modify Trusty OS, alternative would be to
make itvirtualization aware as OP-TEE did. The core would need to be resilient and the panic only affect a given client.
I do not have right a clear idea of what is the status of optee and xen but in theory I would see 2 possible ways to handle this:
- without optee modification, something in a guest (Dom0 or an
other priviledged one) needs to have access to optee and emulate optee access for others.
- with optee modifications, optee needs to have a concept of
client and Xen would need to passthrough optee requests but being responsible of adding a “client” identifier. Maybe also informing Optee when a new client is created/removed.
The second scenario could then be somehow splitted in the previous one from Julien if some parts would need to be emulated somewhere in some kind of combination of the 2 models.
In any case i would always consider that anything running on optee (or in general in the secure world) is more trusted then Xen.
Ok, this means optee runs on all cores in secure world, but this would not work when we need to support multiple OSes with their own
TEE.
I would think you have one TEE running on all cores (or runnable in this
case).
So the Tee needs to handle several contexts or someone needs to
virtualize it.
This back to my original question, should I virtualize TEE in a XEN dedicated
VM?
or I need to emulate secure world to let one VM could have secure and non-secure world?
Well, I think that the best approach is that we did in the OP-TEE: make Trusty virtualization-aware, so it can handle multiple VMs.
Trusty virtualization-aware, you mean android Linux trusty driver communicate with OP-TEE or else?
Things are more funny if you want to use multiple different TEEs (like OP-TEE and Trusty) at the same time. If this is your case, then the best approach is to implement something like para-virtualization in the Secure World. But this would require quite big efforts, of course.
This is just a brainstorm idea, I not have the case right now. I just wanna avoid trusty os that only work with android auto vm not break dom0 and xen.
Thanks, Peng.
-- WBR Volodymyr Babchuk aka lorc [+380976646013] mailto: vlad.babchuk@gmail.com
On Fri, 19 Jun 2020 at 12:50, Peng Fan peng.fan@nxp.com wrote:
Subject: Re: [Tee-dev] TEE with XEN
Hi Peng,
On Fri, 19 Jun 2020 at 12:06, Peng Fan peng.fan@nxp.com wrote:
Subject: Re: [Tee-dev] TEE with XEN
On 19 Jun 2020, at 09:52, Peng Fan peng.fan@nxp.com wrote:
Hi Bertrand,
Subject: Re: [Tee-dev] TEE with XEN
Hi,
> On 18 Jun 2020, at 19:05, Julien Grall julien@xen.org wrote: > > +Bertrand and Stefano > > On 16/06/2020 02:24, Volodymyr Babchuk wrote: >> Hi Peng, >> On Mon, 15 Jun 2020 at 05:07, Peng Fan peng.fan@nxp.com
wrote:
>>> >>> Hi All, >>> >>> While enabling trusty os with xen, I took same approach as >>> OP-TEE, with OP-TEE running in secure world. But I am also >>> thinking this might introduce potential issue is that secure >>> world OS communicate with DomU. >>> If there are some misbehavior in secure world OS, it might let >>> XEN hypervisor not work proper. >>> >>> In my setup, trusty os sometimes panic in secure world, xen >>> will not able to control the panic core anymore. > > May I ask in which case Trusty is panicking?
In any case, optee should protect itself against this and it should be considered that optee is more priviledged then Xen. So if optee is crashing we cannot expect that Xen can recover or fix it.
I would more consider this as a bug that optee needs to be robust
against.
ok. I am not using OP-TEE, currently I use google trusty OS.
Sorry i should have been more generic. Please read this as “Anything running in secure world”, being optee or
trusty.
I have two OS, Dom0 linux + DomU android auto.
DomU android auto needs trusty OS, Dom0 Linux not need that.
But i would guess your Dom0 is more “critical” then your DomU. In this case you must make sure that any resource given to your DomU cannot affect your Dom0. For example: if the DomU is starting a very heavy operation in blocked in trusty, any interrupt for non-secure could be blocked, thus affecting the ability of your Dom0.
I not wanna trusty OS for DomU could bring any detect to Dom0 or xen.
One more case is if dom0 linux needs OP-TEE, DomU needs google trusty, how we handle this in one SoC?
You have a shared resource in this case, someone more or as trusted as the clients needs to decide how the resource can be shared. In this case could be Dom0 or Xen or Trusty or op-Tee (if i should make an order).
> >>> >>> So I am thinking whether we need to emulating secure world in >>> a XEN VM which is the VM running DomU. Just like what ACRN did >>> to run trusty os. >> Well, it depends on whom you are trusting more. Both XEN and >> TEE are minimal OS implementations with aim at security. I'm >> speaking about generic TEE OS, not about particular OS like OP-TEE
or Trusty.
>> Problem is that, if TEE is running inside VM, it will be >> susceptible to a hypervisor misbehaviour. You need to >> understand that Xen and privileged domain (dom0, mostly) can >> access memory of
any guest.
>> At least, in default configuration. There are means to harden >> this setup. But anyways, Xen can't be stopped from reading TEE's
secrets.
> > IIRC, we discussed this approach for OP-TEE in the past. There > was other potential pitfalls with it. For instance, you wouldn't be able to directly access any secure device from that guest (it is running in
non-secure world).
> > There are also issues in term of latency as you may have the > following model: > > domU -> Xen -> domU TEE -> (Xen -> host TEE -> Xen -> domU TEE) > -> Xen -> domU. > > The bit in () is if you require to call the host TEE. > > One possibility would be to use Secure-EL2 for your Trusty OS. > But I don't know whether your platform supports it. > > Depending on whether you can modify Trusty OS, alternative would > be to make itvirtualization aware as OP-TEE did. The core would need to be resilient and the panic only affect a given client.
I do not have right a clear idea of what is the status of optee and xen but in theory I would see 2 possible ways to handle this:
- without optee modification, something in a guest (Dom0 or an
other priviledged one) needs to have access to optee and emulate optee access for others.
- with optee modifications, optee needs to have a concept of
client and Xen would need to passthrough optee requests but being responsible of adding a “client” identifier. Maybe also informing Optee when a new client is created/removed.
The second scenario could then be somehow splitted in the previous one from Julien if some parts would need to be emulated somewhere in some kind of combination of the 2 models.
In any case i would always consider that anything running on optee (or in general in the secure world) is more trusted then Xen.
Ok, this means optee runs on all cores in secure world, but this would not work when we need to support multiple OSes with their own
TEE.
I would think you have one TEE running on all cores (or runnable in this
case).
So the Tee needs to handle several contexts or someone needs to
virtualize it.
This back to my original question, should I virtualize TEE in a XEN dedicated
VM?
or I need to emulate secure world to let one VM could have secure and non-secure world?
Well, I think that the best approach is that we did in the OP-TEE: make Trusty virtualization-aware, so it can handle multiple VMs.
Trusty virtualization-aware, you mean android Linux trusty driver communicate with OP-TEE or else?
Okay, OP-TEE is implemented right as Bertran has suggested: OP-TEE can work with multiple VMs at the same time. It has special calls to create and destroy VM contextes.
So, when a new VM/guest is spawned, Xen says "Hey, OP-TEE, there is a new VM with ID NNN". OP-TEE then initializes internal state for the new VM. At any moment Xen can say "Oops, that VM with ID NNN is dead". OP-TEE will immediately destroy the internal state for that VM.
Also, Xen needs to perform certain actions every time VM calls OP-TEE: translate addresses, tell OP-TEE id of that VM, lock guest pages... And no changes to the OP-TEE client in Linux are needed. It thinks that it communicates directly with the OP-TEE. OP-TEE is running in the Secure World as usual.
The same can be done for any other TEE. Trusty, for example. Downside is that you can't run two different TEEs in the Secure world.
Actually, ARM added virtualization support in Secure mode, but AFAIK, i.MX8 does not support it.
To sum it up: 1. If you want to run only Trusty on your system, and you want to run it in Secure World, you need to make Trusty virtualization-aware, and write mediator in Xen. This is what I did for OP-TEE.
2. If you want to run Trusty AND OP-TEE in the Secure World, then you need to do p1. and then implement some paravirtualization support in Secure Monitor, Trusty and OP-TEE
3. If your use case does not require high security, you can try to run Trusty as a VM. But, I am almost certain that this will not pass Google's requirements. So no Google Pay, no E-SIM, no DRM for this setup. But, you should clarify this with Google. I'm not a Google representative :)
On 19 Jun 2020, at 10:12, Volodymyr Babchuk vlad.babchuk@gmail.com wrote:
Hi Peng,
On Fri, 19 Jun 2020 at 12:06, Peng Fan peng.fan@nxp.com wrote:
Subject: Re: [Tee-dev] TEE with XEN
On 19 Jun 2020, at 09:52, Peng Fan peng.fan@nxp.com wrote:
Hi Bertrand,
Subject: Re: [Tee-dev] TEE with XEN
Hi,
On 18 Jun 2020, at 19:05, Julien Grall julien@xen.org wrote:
+Bertrand and Stefano
On 16/06/2020 02:24, Volodymyr Babchuk wrote: > Hi Peng, > On Mon, 15 Jun 2020 at 05:07, Peng Fan peng.fan@nxp.com wrote: >> >> Hi All, >> >> While enabling trusty os with xen, I took same approach as OP-TEE, >> with OP-TEE running in secure world. But I am also thinking this >> might introduce potential issue is that secure world OS >> communicate with
DomU.
>> If there are some misbehavior in secure world OS, it might let XEN >> hypervisor not work proper. >> >> In my setup, trusty os sometimes panic in secure world, xen will >> not able to control the panic core anymore.
May I ask in which case Trusty is panicking?
In any case, optee should protect itself against this and it should be considered that optee is more priviledged then Xen. So if optee is crashing we cannot expect that Xen can recover or fix it.
I would more consider this as a bug that optee needs to be robust against.
ok. I am not using OP-TEE, currently I use google trusty OS.
Sorry i should have been more generic. Please read this as “Anything running in secure world”, being optee or trusty.
I have two OS, Dom0 linux + DomU android auto.
DomU android auto needs trusty OS, Dom0 Linux not need that.
But i would guess your Dom0 is more “critical” then your DomU. In this case you must make sure that any resource given to your DomU cannot affect your Dom0. For example: if the DomU is starting a very heavy operation in blocked in trusty, any interrupt for non-secure could be blocked, thus affecting the ability of your Dom0.
I not wanna trusty OS for DomU could bring any detect to Dom0 or xen.
One more case is if dom0 linux needs OP-TEE, DomU needs google trusty, how we handle this in one SoC?
You have a shared resource in this case, someone more or as trusted as the clients needs to decide how the resource can be shared. In this case could be Dom0 or Xen or Trusty or op-Tee (if i should make an order).
>> >> So I am thinking whether we need to emulating secure world in a >> XEN VM which is the VM running DomU. Just like what ACRN did to >> run trusty os. > Well, it depends on whom you are trusting more. Both XEN and TEE > are minimal OS implementations with aim at security. I'm speaking > about generic TEE OS, not about particular OS like OP-TEE or Trusty. > Problem is that, if TEE is running inside VM, it will be > susceptible to a hypervisor misbehaviour. You need to understand > that Xen and privileged domain (dom0, mostly) can access memory of
any guest.
> At least, in default configuration. There are means to harden this > setup. But anyways, Xen can't be stopped from reading TEE's secrets.
IIRC, we discussed this approach for OP-TEE in the past. There was other
potential pitfalls with it. For instance, you wouldn't be able to directly access any secure device from that guest (it is running in
non-secure world).
There are also issues in term of latency as you may have the following
model:
domU -> Xen -> domU TEE -> (Xen -> host TEE -> Xen -> domU TEE) -> Xen ->
domU.
The bit in () is if you require to call the host TEE.
One possibility would be to use Secure-EL2 for your Trusty OS. But I don't
know whether your platform supports it.
Depending on whether you can modify Trusty OS, alternative would be to
make itvirtualization aware as OP-TEE did. The core would need to be resilient and the panic only affect a given client.
I do not have right a clear idea of what is the status of optee and xen but in theory I would see 2 possible ways to handle this:
- without optee modification, something in a guest (Dom0 or an other
priviledged one) needs to have access to optee and emulate optee access for others.
- with optee modifications, optee needs to have a concept of client
and Xen would need to passthrough optee requests but being responsible of adding a “client” identifier. Maybe also informing Optee when a new client is created/removed.
The second scenario could then be somehow splitted in the previous one from Julien if some parts would need to be emulated somewhere in some kind of combination of the 2 models.
In any case i would always consider that anything running on optee (or in general in the secure world) is more trusted then Xen.
Ok, this means optee runs on all cores in secure world, but this would not work when we need to support multiple OSes with their own TEE.
I would think you have one TEE running on all cores (or runnable in this case). So the Tee needs to handle several contexts or someone needs to virtualize it.
This back to my original question, should I virtualize TEE in a XEN dedicated VM? or I need to emulate secure world to let one VM could have secure and non-secure world?
Well, I think that the best approach is that we did in the OP-TEE: make Trusty virtualization-aware, so it can handle multiple VMs.
Things are more funny if you want to use multiple different TEEs (like OP-TEE and Trusty) at the same time. If this is your case, then the best approach is to implement something like para-virtualization in the Secure World. But this would require quite big efforts, of course.
I agree this is the best approach to make secure world handle several clients. But it might be the most complex one though.
Using a VM for it might be easier but you need to check that it achieving the level of trust that you need. You could have a VM acting as a muxer/checker and passing down requests to the secure world
The definitive answer really depends on the amount of effort, the level of security and your general system needs.
Bertrand
-
Bertrand Marquis -
Julien Grall -
Peng Fan -
Stefano Stabellini -
Volodymyr Babchuk