Hi, I am trying to run a multinode test job https://staging.validation.linaro.org/scheduler/job/175378/definition in LAVA v2. It failed https://staging.validation.linaro.org/scheduler/job/175378#L555. After a short discussion with Chase, I realized if I want to run multinode test job I have to use "lava-multinode" protocol. However, I still need "lava-lxc" to deploy the test image for hikey. I am wondering if it is possible to use two protocols, lava-lxc for deployment test image and lava-multinode to run the test, in the same test job? If yes, could you please provide some examples?
Thanks, Arthur
On 7 June 2017 at 06:25, Arthur She arthur.she@linaro.org wrote:
Hi, I am trying to run a multinode test job in LAVA v2.
That doesn't look like a MultiNode test job. What are you trying to achieve?
LXC support does not require MultiNode but LXC can be used in a MultiNode situation, should you require to have one test job involving a HiKey and another test job on mustang or panda or something and then synchronise the two devices.
Neither of those links are MultiNode jobs. Make sure you are not confusing the LXC protocol with MultiNode.
The MultiNode protocol cannot be used to pass information from the HiKey to the LXC which is managing the adb/fastboot calls to that same HiKey. Use adb to pull or push data to the device. (For non-AOSP boots, use tools available within the LXC.)
It failed. After a short discussion with Chase, I realized if I want to run multinode test job I have to use "lava-multinode" protocol. However, I still need "lava-lxc" to deploy the test image for hikey. I am wondering if it is possible to use two protocols, lava-lxc for deployment test image and lava-multinode to run the test, in the same test job?
Yes.
https://git.linaro.org/lava-team/refactoring.git/tree/lxc-multinode.yaml
However, it does not sound like you actually need MultiNode.
What are you trying to do with the HiKey?
Hi Neil, Thanks for your reply. I am trying to compose a test definition to run robot framework test. The test definition is not yet finished, I'd just want to have a test before I go further. It's something like Android CTS test, run the test in the host and the host will interact with the target, hikey, through network. Basically, what I need is 1. the IP address of the client 2. run some program in the client before the host starts the test in order to accept the commands from it. 3. before the host finished the test, the client shouldn't be released by LAVA. Which shouldn't be a problem, I think. I am not sure if I have to use multi node test to accomplish it. Chase gave me some CTS examples. However, I haven't had any ideas to get the client IP address with that approach.
Do you have any suggestions?
Thanks a lot. Arthur
2017-06-06 23:59 GMT-07:00 Neil Williams neil.williams@linaro.org:
On 7 June 2017 at 06:25, Arthur She arthur.she@linaro.org wrote:
Hi, I am trying to run a multinode test job in LAVA v2.
That doesn't look like a MultiNode test job. What are you trying to achieve?
LXC support does not require MultiNode but LXC can be used in a MultiNode situation, should you require to have one test job involving a HiKey and another test job on mustang or panda or something and then synchronise the two devices.
Neither of those links are MultiNode jobs. Make sure you are not confusing the LXC protocol with MultiNode.
The MultiNode protocol cannot be used to pass information from the HiKey to the LXC which is managing the adb/fastboot calls to that same HiKey. Use adb to pull or push data to the device. (For non-AOSP boots, use tools available within the LXC.)
It failed. After a short discussion with Chase, I realized if I want to
run
multinode test job I have to use "lava-multinode" protocol. However, I still need "lava-lxc" to deploy the test image for hikey. I am wondering if it is possible to use two protocols, lava-lxc for deployment test image and lava-multinode to run the test, in the same
test
job?
Yes.
https://git.linaro.org/lava-team/refactoring.git/tree/lxc-multinode.yaml
However, it does not sound like you actually need MultiNode.
What are you trying to do with the HiKey?
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
On 7 June 2017 at 18:12, Arthur She arthur.she@linaro.org wrote:
Hi Neil, Thanks for your reply. I am trying to compose a test definition to run robot framework test. The test definition is not yet finished, I'd just want to have a test before I go further. It's something like Android CTS test, run the test in the host and the host will interact with the target, hikey, through network.
That needs a static IPv4 address to be assigned by the lab team. Unless there is an adb command which can retrieve the IPv4 address of the HiKey.
Basically, what I need is 1. the IP address of the client 2. run some program in the client before the host starts the test in order to accept the commands from it.
That is NOT MultiNode. MultiNode would require the HiKey to be running a POSIX shell which can run the test shell helper scripts to interact with the MultiNode protocol. Even then, the LXC is not a MultiNode node, so would not be able to retrieve information from the MultiNode protocol.
Think of the LXC as a transparent container - it acts as the dispatcher but has the ability to run arbitrary binaries like adb. In the HiKey test job and in many others, the LXC is not a device, it is the dispatcher itself.
- before the host finished the test, the client shouldn't be released by
LAVA. Which shouldn't be a problem, I think. I am not sure if I have to use multi node test to accomplish it.
No, it only uses LXC.
Chase gave me some CTS examples. However, I haven't had any ideas to get the client IP address with that approach.
If you cannot obtain it with adb, it will have to be static and assigned by the lab team. Open an issue in CT&T https://projects.linaro.org/servicedesk/customer/portal/1
Do you have any suggestions?
Thanks a lot. Arthur
2017-06-06 23:59 GMT-07:00 Neil Williams neil.williams@linaro.org:
On 7 June 2017 at 06:25, Arthur She arthur.she@linaro.org wrote:
Hi, I am trying to run a multinode test job in LAVA v2.
That doesn't look like a MultiNode test job. What are you trying to achieve?
LXC support does not require MultiNode but LXC can be used in a MultiNode situation, should you require to have one test job involving a HiKey and another test job on mustang or panda or something and then synchronise the two devices.
Neither of those links are MultiNode jobs. Make sure you are not confusing the LXC protocol with MultiNode.
The MultiNode protocol cannot be used to pass information from the HiKey to the LXC which is managing the adb/fastboot calls to that same HiKey. Use adb to pull or push data to the device. (For non-AOSP boots, use tools available within the LXC.)
It failed. After a short discussion with Chase, I realized if I want to run multinode test job I have to use "lava-multinode" protocol. However, I still need "lava-lxc" to deploy the test image for hikey. I am wondering if it is possible to use two protocols, lava-lxc for deployment test image and lava-multinode to run the test, in the same test job?
Yes.
https://git.linaro.org/lava-team/refactoring.git/tree/lxc-multinode.yaml
However, it does not sound like you actually need MultiNode.
What are you trying to do with the HiKey?
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
On 7 June 2017 at 18:19, Neil Williams neil.williams@linaro.org wrote:
On 7 June 2017 at 18:12, Arthur She arthur.she@linaro.org wrote:
Hi Neil, Thanks for your reply. I am trying to compose a test definition to run robot framework test. The test definition is not yet finished, I'd just want to have a test before I go further. It's something like Android CTS test, run the test in the host and the host will interact with the target, hikey, through network.
That needs a static IPv4 address to be assigned by the lab team. Unless there is an adb command which can retrieve the IPv4 address of the HiKey.
This is not android test, so adb is out of the equation.
milosz
Basically, what I need is 1. the IP address of the client 2. run some program in the client before the host starts the test in order to accept the commands from it.
That is NOT MultiNode. MultiNode would require the HiKey to be running a POSIX shell which can run the test shell helper scripts to interact with the MultiNode protocol. Even then, the LXC is not a MultiNode node, so would not be able to retrieve information from the MultiNode protocol.
Think of the LXC as a transparent container - it acts as the dispatcher but has the ability to run arbitrary binaries like adb. In the HiKey test job and in many others, the LXC is not a device, it is the dispatcher itself.
- before the host finished the test, the client shouldn't be released by
LAVA. Which shouldn't be a problem, I think. I am not sure if I have to use multi node test to accomplish it.
No, it only uses LXC.
Chase gave me some CTS examples. However, I haven't had any ideas to get the client IP address with that approach.
If you cannot obtain it with adb, it will have to be static and assigned by the lab team. Open an issue in CT&T https://projects.linaro.org/servicedesk/customer/portal/1
Do you have any suggestions?
Thanks a lot. Arthur
2017-06-06 23:59 GMT-07:00 Neil Williams neil.williams@linaro.org:
On 7 June 2017 at 06:25, Arthur She arthur.she@linaro.org wrote:
Hi, I am trying to run a multinode test job in LAVA v2.
That doesn't look like a MultiNode test job. What are you trying to achieve?
LXC support does not require MultiNode but LXC can be used in a MultiNode situation, should you require to have one test job involving a HiKey and another test job on mustang or panda or something and then synchronise the two devices.
Neither of those links are MultiNode jobs. Make sure you are not confusing the LXC protocol with MultiNode.
The MultiNode protocol cannot be used to pass information from the HiKey to the LXC which is managing the adb/fastboot calls to that same HiKey. Use adb to pull or push data to the device. (For non-AOSP boots, use tools available within the LXC.)
It failed. After a short discussion with Chase, I realized if I want to run multinode test job I have to use "lava-multinode" protocol. However, I still need "lava-lxc" to deploy the test image for hikey. I am wondering if it is possible to use two protocols, lava-lxc for deployment test image and lava-multinode to run the test, in the same test job?
Yes.
https://git.linaro.org/lava-team/refactoring.git/tree/lxc-multinode.yaml
However, it does not sound like you actually need MultiNode.
What are you trying to do with the HiKey?
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/ _______________________________________________ Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
On 7 June 2017 at 18:21, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
On 7 June 2017 at 18:19, Neil Williams neil.williams@linaro.org wrote:
On 7 June 2017 at 18:12, Arthur She arthur.she@linaro.org wrote:
Hi Neil, Thanks for your reply. I am trying to compose a test definition to run robot framework test. The test definition is not yet finished, I'd just want to have a test before I go further. It's something like Android CTS test, run the test in the host and the host will interact with the target, hikey, through network.
Be very careful with terminology here. There is no host or target, that leads to thinking of this as MultiNode. There is a device and there is the dispatcher which happens to use an LXC to isolate programs like fastboot.
That needs a static IPv4 address to be assigned by the lab team. Unless there is an adb command which can retrieve the IPv4 address of the HiKey.
This is not android test, so adb is out of the equation.
OK, thanks Milosz.
So to try and clear things up:
0: LAVA needs the LXC to use fastboot to deploy the files to the HiKey - the same LXC can be used to run a test shell to use the network.
1: The LXC is transparent to the HiKey and if programs in the LXC need to talk to the HiKey over the network, MultiNode is not going to be able to help because there is only one node involved - the HiKey.
2: If there was a tool which could talk to the HiKey over USB and retrieve data, that could be used (which is how adb does it) but that would need a similar tool / service running on the HiKey.
3: In the absence of this tool, it is equivalent to trying to get the dispatcher to communicate with the device over the network - the device needs a fixed IPv4 address which can be set in the device configuration so that the dispatcher / LXC can use it.
4. In your local tests, assume that this address can be obtained by running a shell script (this would be how LAVA provides the information).
5. Please be very clear about what operating system is being used on the device in discussions about this method and that LXC is used for the deployment stage. Avoid comparisons with CTS and AOSP test jobs.
6. Keep MultiNode for situations where there are two DUTs - remember that the LXC needed to deploy files is actually part of the dispatcher, not a device.
Basically, what I need is 1. the IP address of the client 2. run some program in the client before the host starts the test in order to accept the commands from it.
Please clarify here what you mean by host and client. Avoid thinking of these as MultiNode. There is a DUT and there is an LXC acting as the dispatcher. As each can run a POSIX shell, the terms need to be consistent.
Think of the LXC as a transparent container - it acts as the dispatcher but has the ability to run arbitrary binaries like adb. In the HiKey test job and in many others, the LXC is not a device, it is the dispatcher itself.
- before the host finished the test, the client shouldn't be released by
LAVA. Which shouldn't be a problem, I think.
Again, this hints of MultiNode - the LXC for a HiKey persists until the test job itself completes. The device is powered off and the LXC is destroyed - which happens first is irrelevant because nothing from a test shell is allowed to happen between the two events. There is no synchronisation to do here, no waiting for one or the other.
I am not sure if I have to use multi node test to accomplish it.
No, it only uses LXC.
Chase gave me some CTS examples. However, I haven't had any ideas to get the client IP address with that approach.
If you cannot obtain it with adb, it will have to be static and assigned by the lab team. Open an issue in CT&T https://projects.linaro.org/servicedesk/customer/portal/1
That still applies.
Do you have any suggestions?
Thanks a lot. Arthur
2017-06-06 23:59 GMT-07:00 Neil Williams neil.williams@linaro.org:
On 7 June 2017 at 06:25, Arthur She arthur.she@linaro.org wrote:
Hi, I am trying to run a multinode test job in LAVA v2.
That doesn't look like a MultiNode test job. What are you trying to achieve?
LXC support does not require MultiNode but LXC can be used in a MultiNode situation, should you require to have one test job involving a HiKey and another test job on mustang or panda or something and then synchronise the two devices.
Neither of those links are MultiNode jobs. Make sure you are not confusing the LXC protocol with MultiNode.
The MultiNode protocol cannot be used to pass information from the HiKey to the LXC which is managing the adb/fastboot calls to that same HiKey. Use adb to pull or push data to the device. (For non-AOSP boots, use tools available within the LXC.)
It failed. After a short discussion with Chase, I realized if I want to run multinode test job I have to use "lava-multinode" protocol. However, I still need "lava-lxc" to deploy the test image for hikey. I am wondering if it is possible to use two protocols, lava-lxc for deployment test image and lava-multinode to run the test, in the same test job?
Yes.
https://git.linaro.org/lava-team/refactoring.git/tree/lxc-multinode.yaml
However, it does not sound like you actually need MultiNode.
What are you trying to do with the HiKey?
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/ _______________________________________________ Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
Thanks Neil.
2017-06-07 12:35 GMT-07:00 Neil Williams neil.williams@linaro.org:
On 7 June 2017 at 18:21, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
On 7 June 2017 at 18:19, Neil Williams neil.williams@linaro.org wrote:
On 7 June 2017 at 18:12, Arthur She arthur.she@linaro.org wrote:
Hi Neil, Thanks for your reply. I am trying to compose a test definition to run robot framework test. The test definition is not yet finished, I'd just want to have a test
before
I go further. It's something like Android CTS test, run the test in the host and the
host
will interact with the target, hikey, through network.
Be very careful with terminology here. There is no host or target, that leads to thinking of this as MultiNode. There is a device and there is the dispatcher which happens to use an LXC to isolate programs like fastboot.
That needs a static IPv4 address to be assigned by the lab team. Unless there is an adb command which can retrieve the IPv4 address of the HiKey.
This is not android test, so adb is out of the equation.
OK, thanks Milosz.
So to try and clear things up:
0: LAVA needs the LXC to use fastboot to deploy the files to the HiKey
- the same LXC can be used to run a test shell to use the network.
Good
1: The LXC is transparent to the HiKey and if programs in the LXC need to talk to the HiKey over the network, MultiNode is not going to be able to help because there is only one node involved - the HiKey.
I see.
2: If there was a tool which could talk to the HiKey over USB and retrieve data, that could be used (which is how adb does it) but that would need a similar tool / service running on the HiKey.
We don't have such kind of tool in our Hikey OE image.
3: In the absence of this tool, it is equivalent to trying to get the dispatcher to communicate with the device over the network - the device needs a fixed IPv4 address which can be set in the device configuration so that the dispatcher / LXC can use it.
If we set a static IP address for Hikey. Would it cause IP conflict? I mean will this static IP address conflicts with other devices in the LAB or when running the same test job for multiple times simultaneously ?
- In your local tests, assume that this address can be obtained by
running a shell script (this would be how LAVA provides the information).
- Please be very clear about what operating system is being used on
the device in discussions about this method and that LXC is used for the deployment stage. Avoid comparisons with CTS and AOSP test jobs.
- Keep MultiNode for situations where there are two DUTs - remember
that the LXC needed to deploy files is actually part of the dispatcher, not a device.
Basically, what I need is 1. the IP address of the client 2. run some program in the client before the host starts the test in order to
accept the
commands from it.
Please clarify here what you mean by host and client. Avoid thinking of these as MultiNode. There is a DUT and there is an LXC acting as the dispatcher. As each can run a POSIX shell, the terms need to be consistent.
Think of the LXC as a transparent container - it acts as the dispatcher but has the ability to run arbitrary binaries like adb. In the HiKey test job and in many others, the LXC is not a device, it is the dispatcher itself.
Thanks for the explanation. I think I got a feel for it.
- before the host finished the test, the client shouldn't be released
by
LAVA. Which shouldn't be a problem, I think.
Again, this hints of MultiNode - the LXC for a HiKey persists until the test job itself completes. The device is powered off and the LXC is destroyed - which happens first is irrelevant because nothing from a test shell is allowed to happen between the two events. There is no synchronisation to do here, no waiting for one or the other.
I am not sure if I have to use multi node test to accomplish it.
No, it only uses LXC.
Chase gave me some CTS examples. However, I haven't had any ideas to
get the
client IP address with that approach.
If you cannot obtain it with adb, it will have to be static and assigned by the lab team. Open an issue in CT&T https://projects.linaro.org/servicedesk/customer/portal/1
That still applies.
Do you have any suggestions?
Thanks a lot. Arthur
2017-06-06 23:59 GMT-07:00 Neil Williams neil.williams@linaro.org:
On 7 June 2017 at 06:25, Arthur She arthur.she@linaro.org wrote:
Hi, I am trying to run a multinode test job in LAVA v2.
That doesn't look like a MultiNode test job. What are you trying to achieve?
LXC support does not require MultiNode but LXC can be used in a MultiNode situation, should you require to have one test job involving a HiKey and another test job on mustang or panda or something and then synchronise the two devices.
Neither of those links are MultiNode jobs. Make sure you are not confusing the LXC protocol with MultiNode.
The MultiNode protocol cannot be used to pass information from the HiKey to the LXC which is managing the adb/fastboot calls to that same HiKey. Use adb to pull or push data to the device. (For non-AOSP boots, use tools available within the LXC.)
It failed. After a short discussion with Chase, I realized if I
want to
run multinode test job I have to use "lava-multinode" protocol. However, I still need "lava-lxc" to deploy the test image for hikey. I am wondering if it is possible to use two protocols, lava-lxc for deployment test image and lava-multinode to run the test, in the
same
test job?
Yes.
https://git.linaro.org/lava-team/refactoring.git/tree/lxc-mu
ltinode.yaml
However, it does not sound like you actually need MultiNode.
What are you trying to do with the HiKey?
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/ _______________________________________________ Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
On 8 June 2017 at 06:10, Arthur She arthur.she@linaro.org wrote:
Thanks Neil.
2017-06-07 12:35 GMT-07:00 Neil Williams neil.williams@linaro.org:
On 7 June 2017 at 18:21, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
On 7 June 2017 at 18:19, Neil Williams neil.williams@linaro.org wrote:
On 7 June 2017 at 18:12, Arthur She arthur.she@linaro.org wrote:
Hi Neil, Thanks for your reply. I am trying to compose a test definition to run robot framework test. The test definition is not yet finished, I'd just want to have a test before I go further. It's something like Android CTS test, run the test in the host and the host will interact with the target, hikey, through network.
Be very careful with terminology here. There is no host or target, that leads to thinking of this as MultiNode. There is a device and there is the dispatcher which happens to use an LXC to isolate programs like fastboot.
That needs a static IPv4 address to be assigned by the lab team. Unless there is an adb command which can retrieve the IPv4 address of the HiKey.
This is not android test, so adb is out of the equation.
OK, thanks Milosz.
So to try and clear things up:
0: LAVA needs the LXC to use fastboot to deploy the files to the HiKey
- the same LXC can be used to run a test shell to use the network.
Good
1: The LXC is transparent to the HiKey and if programs in the LXC need to talk to the HiKey over the network, MultiNode is not going to be able to help because there is only one node involved - the HiKey.
I see.
2: If there was a tool which could talk to the HiKey over USB and retrieve data, that could be used (which is how adb does it) but that would need a similar tool / service running on the HiKey.
We don't have such kind of tool in our Hikey OE image.
3: In the absence of this tool, it is equivalent to trying to get the dispatcher to communicate with the device over the network - the device needs a fixed IPv4 address which can be set in the device configuration so that the dispatcher / LXC can use it.
If we set a static IP address for Hikey. Would it cause IP conflict? I mean will this static IP address conflicts with other devices in the LAB or when running the same test job for multiple times simultaneously ?
It will be assigned from the MAC address of the network adaptor used by each HiKey, so as long as that does not change, the IP will not conflict. However, the lab team do want to minimise the number of fixed IP addresses used for devices, so this support may only be available on designed HiKeys and the test jobs would need to use device tags.
Create the request on CTT as per the link given in the previous message, included below.
- In your local tests, assume that this address can be obtained by
running a shell script (this would be how LAVA provides the information).
- Please be very clear about what operating system is being used on
the device in discussions about this method and that LXC is used for the deployment stage. Avoid comparisons with CTS and AOSP test jobs.
- Keep MultiNode for situations where there are two DUTs - remember
that the LXC needed to deploy files is actually part of the dispatcher, not a device.
Basically, what I need is 1. the IP address of the client 2. run some program in the client before the host starts the test in order to accept the commands from it.
Please clarify here what you mean by host and client. Avoid thinking of these as MultiNode. There is a DUT and there is an LXC acting as the dispatcher. As each can run a POSIX shell, the terms need to be consistent.
Think of the LXC as a transparent container - it acts as the dispatcher but has the ability to run arbitrary binaries like adb. In the HiKey test job and in many others, the LXC is not a device, it is the dispatcher itself.
Thanks for the explanation. I think I got a feel for it.
- before the host finished the test, the client shouldn't be released
by LAVA. Which shouldn't be a problem, I think.
Again, this hints of MultiNode - the LXC for a HiKey persists until the test job itself completes. The device is powered off and the LXC is destroyed - which happens first is irrelevant because nothing from a test shell is allowed to happen between the two events. There is no synchronisation to do here, no waiting for one or the other.
I am not sure if I have to use multi node test to accomplish it.
No, it only uses LXC.
Chase gave me some CTS examples. However, I haven't had any ideas to get the client IP address with that approach.
If you cannot obtain it with adb, it will have to be static and assigned by the lab team. Open an issue in CT&T https://projects.linaro.org/servicedesk/customer/portal/1
That still applies.
Make a request for a number of HiKeys (not all) to have a fixed IPv4 address based on the MAC address of the network adaptor for the purposes of your particular use case.
Do you have any suggestions?
Thanks a lot. Arthur
2017-06-06 23:59 GMT-07:00 Neil Williams neil.williams@linaro.org:
On 7 June 2017 at 06:25, Arthur She arthur.she@linaro.org wrote: > Hi, > I am trying to run a multinode test job in LAVA v2.
That doesn't look like a MultiNode test job. What are you trying to achieve?
LXC support does not require MultiNode but LXC can be used in a MultiNode situation, should you require to have one test job involving a HiKey and another test job on mustang or panda or something and then synchronise the two devices.
Neither of those links are MultiNode jobs. Make sure you are not confusing the LXC protocol with MultiNode.
The MultiNode protocol cannot be used to pass information from the HiKey to the LXC which is managing the adb/fastboot calls to that same HiKey. Use adb to pull or push data to the device. (For non-AOSP boots, use tools available within the LXC.)
> It failed. After a short discussion with Chase, I realized if I > want to > run > multinode test job I have to use "lava-multinode" protocol. > However, I still need "lava-lxc" to deploy the test image for > hikey. > I am wondering if it is possible to use two protocols, lava-lxc for > deployment test image and lava-multinode to run the test, in the > same > test > job?
Yes.
https://git.linaro.org/lava-team/refactoring.git/tree/lxc-multinode.yaml
However, it does not sound like you actually need MultiNode.
What are you trying to do with the HiKey?
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/ _______________________________________________ Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
Thanks Neil for your suggestion.
2017-06-07 23:43 GMT-07:00 Neil Williams neil.williams@linaro.org:
On 8 June 2017 at 06:10, Arthur She arthur.she@linaro.org wrote:
Thanks Neil.
2017-06-07 12:35 GMT-07:00 Neil Williams neil.williams@linaro.org:
On 7 June 2017 at 18:21, Milosz Wasilewski <
milosz.wasilewski@linaro.org>
wrote:
On 7 June 2017 at 18:19, Neil Williams neil.williams@linaro.org
wrote:
On 7 June 2017 at 18:12, Arthur She arthur.she@linaro.org wrote:
Hi Neil, Thanks for your reply. I am trying to compose a test definition to run robot framework
test.
The test definition is not yet finished, I'd just want to have a
test
before I go further. It's something like Android CTS test, run the test in the host and
the
host will interact with the target, hikey, through network.
Be very careful with terminology here. There is no host or target, that leads to thinking of this as MultiNode. There is a device and there is the dispatcher which happens to use an LXC to isolate programs like fastboot.
That needs a static IPv4 address to be assigned by the lab team. Unless there is an adb command which can retrieve the IPv4 address of the HiKey.
This is not android test, so adb is out of the equation.
OK, thanks Milosz.
So to try and clear things up:
0: LAVA needs the LXC to use fastboot to deploy the files to the HiKey
- the same LXC can be used to run a test shell to use the network.
Good
1: The LXC is transparent to the HiKey and if programs in the LXC need to talk to the HiKey over the network, MultiNode is not going to be able to help because there is only one node involved - the HiKey.
I see.
2: If there was a tool which could talk to the HiKey over USB and retrieve data, that could be used (which is how adb does it) but that would need a similar tool / service running on the HiKey.
We don't have such kind of tool in our Hikey OE image.
3: In the absence of this tool, it is equivalent to trying to get the dispatcher to communicate with the device over the network - the device needs a fixed IPv4 address which can be set in the device configuration so that the dispatcher / LXC can use it.
If we set a static IP address for Hikey. Would it cause IP conflict? I mean will this static IP address conflicts with other devices in the
LAB
or when running the same test job for multiple times simultaneously ?
It will be assigned from the MAC address of the network adaptor used by each HiKey, so as long as that does not change, the IP will not conflict. However, the lab team do want to minimise the number of fixed IP addresses used for devices, so this support may only be available on designed HiKeys and the test jobs would need to use device tags.
That means we have to hard code "device : IP address" mapping in somewhere. Either in the test job then pass it to the test definition or in the test definition itself. Because this test definition would be run in validation.l.o and staging.validation.l.o, I'd like to have it flexible enough. I copied the idea from your test job https://staging.validation.linaro.org/scheduler/job/171251.1 and submitted a multinode test job https://staging.validation.linaro.org/scheduler/job/175545 which pass client IP address through multinode API. It seems it is working fine. Would that be a good idea?
Create the request on CTT as per the link given in the previous message, included below.
- In your local tests, assume that this address can be obtained by
running a shell script (this would be how LAVA provides the information).
- Please be very clear about what operating system is being used on
the device in discussions about this method and that LXC is used for the deployment stage. Avoid comparisons with CTS and AOSP test jobs.
- Keep MultiNode for situations where there are two DUTs - remember
that the LXC needed to deploy files is actually part of the dispatcher, not a device.
Basically, what I need is 1. the IP address of the client 2. run
some
program in the client before the host starts the test in order to accept the commands from it.
Please clarify here what you mean by host and client. Avoid thinking of these as MultiNode. There is a DUT and there is an LXC acting as the dispatcher. As each can run a POSIX shell, the terms need to be consistent.
Think of the LXC as a transparent container - it acts as the dispatcher but has the ability to run arbitrary binaries like adb. In the HiKey test job and in many others, the LXC is not a device, it is the dispatcher itself.
Thanks for the explanation. I think I got a feel for it.
- before the host finished the test, the client shouldn't be
released
by LAVA. Which shouldn't be a problem, I think.
Again, this hints of MultiNode - the LXC for a HiKey persists until the test job itself completes. The device is powered off and the LXC is destroyed - which happens first is irrelevant because nothing from a test shell is allowed to happen between the two events. There is no synchronisation to do here, no waiting for one or the other.
I am not sure if I have to use multi node test to accomplish it.
No, it only uses LXC.
Chase gave me some CTS examples. However, I haven't had any ideas to get the client IP address with that approach.
If you cannot obtain it with adb, it will have to be static and assigned by the lab team. Open an issue in CT&T https://projects.linaro.org/servicedesk/customer/portal/1
That still applies.
Make a request for a number of HiKeys (not all) to have a fixed IPv4 address based on the MAC address of the network adaptor for the purposes of your particular use case.
Do you have any suggestions?
Thanks a lot. Arthur
2017-06-06 23:59 GMT-07:00 Neil Williams <neil.williams@linaro.org
:
> > On 7 June 2017 at 06:25, Arthur She arthur.she@linaro.org wrote: > > Hi, > > I am trying to run a multinode test job in LAVA v2. > > That doesn't look like a MultiNode test job. What are you trying to > achieve? > > LXC support does not require MultiNode but LXC can be used in a > MultiNode situation, should you require to have one test job > involving > a HiKey and another test job on mustang or panda or something and > then > synchronise the two devices. > > Neither of those links are MultiNode jobs. Make sure you are not > confusing the LXC protocol with MultiNode. > > The MultiNode protocol cannot be used to pass information from the > HiKey to the LXC which is managing the adb/fastboot calls to that > same > HiKey. Use adb to pull or push data to the device. (For non-AOSP > boots, use tools available within the LXC.) > > > It failed. After a short discussion with Chase, I realized if I > > want to > > run > > multinode test job I have to use "lava-multinode" protocol. > > However, I still need "lava-lxc" to deploy the test image for > > hikey. > > I am wondering if it is possible to use two protocols, lava-lxc
for
> > deployment test image and lava-multinode to run the test, in the > > same > > test > > job? > > Yes. > > > https://git.linaro.org/lava-team/refactoring.git/tree/lxc-
multinode.yaml
> > However, it does not sound like you actually need MultiNode. > > What are you trying to do with the HiKey? > > > -- > > Neil Williams > ============= > neil.williams@linaro.org > http://www.linux.codehelp.co.uk/
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/ _______________________________________________ Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
On 9 June 2017 at 17:00, Arthur She arthur.she@linaro.org wrote:
Thanks Neil for your suggestion.
[cut]
It will be assigned from the MAC address of the network adaptor used by each HiKey, so as long as that does not change, the IP will not conflict. However, the lab team do want to minimise the number of fixed IP addresses used for devices, so this support may only be available on designed HiKeys and the test jobs would need to use device tags.
That means we have to hard code "device : IP address" mapping in somewhere. Either in the test job then pass it to the test definition or in the test definition itself. Because this test definition would be run in validation.l.o and staging.validation.l.o, I'd like to have it flexible enough. I copied the idea from your test job and submitted a multinode test job which pass client IP address through multinode API. It seems it is working fine. Would that be a good idea?
Arthur, I'm wondering why do you need to run the tests via ssh? Is this the same test we discussed at Connect (video capture with robotframework)? If so, the tests run entirely on the target. So there would be no need for the host part.
milosz
On 9 June 2017 at 17:00, Arthur She arthur.she@linaro.org wrote:
Thanks Neil for your suggestion.
2017-06-07 23:43 GMT-07:00 Neil Williams neil.williams@linaro.org:
On 8 June 2017 at 06:10, Arthur She arthur.she@linaro.org wrote:
Thanks Neil.
2017-06-07 12:35 GMT-07:00 Neil Williams neil.williams@linaro.org:
On 7 June 2017 at 18:21, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
On 7 June 2017 at 18:19, Neil Williams neil.williams@linaro.org wrote:
On 7 June 2017 at 18:12, Arthur She arthur.she@linaro.org wrote: > Hi Neil, > Thanks for your reply. > I am trying to compose a test definition to run robot framework > test. > The test definition is not yet finished, I'd just want to have a > test > before > I go further. > It's something like Android CTS test, run the test in the host and > the > host > will interact with the target, hikey, through network.
Be very careful with terminology here. There is no host or target, that leads to thinking of this as MultiNode. There is a device and there is the dispatcher which happens to use an LXC to isolate programs like fastboot.
That needs a static IPv4 address to be assigned by the lab team. Unless there is an adb command which can retrieve the IPv4 address of the HiKey.
This is not android test, so adb is out of the equation.
OK, thanks Milosz.
So to try and clear things up:
0: LAVA needs the LXC to use fastboot to deploy the files to the HiKey
- the same LXC can be used to run a test shell to use the network.
Good
1: The LXC is transparent to the HiKey and if programs in the LXC need to talk to the HiKey over the network, MultiNode is not going to be able to help because there is only one node involved - the HiKey.
I see.
2: If there was a tool which could talk to the HiKey over USB and retrieve data, that could be used (which is how adb does it) but that would need a similar tool / service running on the HiKey.
We don't have such kind of tool in our Hikey OE image.
3: In the absence of this tool, it is equivalent to trying to get the dispatcher to communicate with the device over the network - the device needs a fixed IPv4 address which can be set in the device configuration so that the dispatcher / LXC can use it.
If we set a static IP address for Hikey. Would it cause IP conflict? I mean will this static IP address conflicts with other devices in the LAB or when running the same test job for multiple times simultaneously ?
It will be assigned from the MAC address of the network adaptor used by each HiKey, so as long as that does not change, the IP will not conflict. However, the lab team do want to minimise the number of fixed IP addresses used for devices, so this support may only be available on designed HiKeys and the test jobs would need to use device tags.
That means we have to hard code "device : IP address" mapping in somewhere.
No, not the test writer - the admins. It becomes part of the device configuration and the test writer has no control over that value, the test writer can simply obtain it.
Either in the test job then pass it to the test definition or in the test definition itself. Because this test definition would be run in validation.l.o and staging.validation.l.o, I'd like to have it flexible enough. I copied the idea from your test job and submitted a multinode test job which pass client IP address through multinode API. It seems it is working fine. Would that be a good idea?
No. The LXC is not visible to MultiNode.
Create the request on CTT as per the link given in the previous message, included below.
- In your local tests, assume that this address can be obtained by
running a shell script (this would be how LAVA provides the information).
- Please be very clear about what operating system is being used on
the device in discussions about this method and that LXC is used for the deployment stage. Avoid comparisons with CTS and AOSP test jobs.
- Keep MultiNode for situations where there are two DUTs - remember
that the LXC needed to deploy files is actually part of the dispatcher, not a device.
> Basically, what I need is 1. the IP address of the client 2. run > some > program in the client before the host starts the test in order to > accept the > commands from it.
Please clarify here what you mean by host and client. Avoid thinking of these as MultiNode. There is a DUT and there is an LXC acting as the dispatcher. As each can run a POSIX shell, the terms need to be consistent.
Think of the LXC as a transparent container - it acts as the dispatcher but has the ability to run arbitrary binaries like adb. In the HiKey test job and in many others, the LXC is not a device, it is the dispatcher itself.
Thanks for the explanation. I think I got a feel for it.
> 3. before the host finished the test, the client shouldn't be > released > by > LAVA. Which shouldn't be a problem, I think.
Again, this hints of MultiNode - the LXC for a HiKey persists until the test job itself completes. The device is powered off and the LXC is destroyed - which happens first is irrelevant because nothing from a test shell is allowed to happen between the two events. There is no synchronisation to do here, no waiting for one or the other.
> I am not sure if I have to use multi node test to accomplish it.
No, it only uses LXC.
> Chase gave me some CTS examples. However, I haven't had any ideas > to > get the > client IP address with that approach.
If you cannot obtain it with adb, it will have to be static and assigned by the lab team. Open an issue in CT&T https://projects.linaro.org/servicedesk/customer/portal/1
That still applies.
Make a request for a number of HiKeys (not all) to have a fixed IPv4 address based on the MAC address of the network adaptor for the purposes of your particular use case.
> Do you have any suggestions? > > Thanks a lot. > Arthur > > > 2017-06-06 23:59 GMT-07:00 Neil Williams > neil.williams@linaro.org: >> >> On 7 June 2017 at 06:25, Arthur She arthur.she@linaro.org wrote: >> > Hi, >> > I am trying to run a multinode test job in LAVA v2. >> >> That doesn't look like a MultiNode test job. What are you trying >> to >> achieve? >> >> LXC support does not require MultiNode but LXC can be used in a >> MultiNode situation, should you require to have one test job >> involving >> a HiKey and another test job on mustang or panda or something and >> then >> synchronise the two devices. >> >> Neither of those links are MultiNode jobs. Make sure you are not >> confusing the LXC protocol with MultiNode. >> >> The MultiNode protocol cannot be used to pass information from the >> HiKey to the LXC which is managing the adb/fastboot calls to that >> same >> HiKey. Use adb to pull or push data to the device. (For non-AOSP >> boots, use tools available within the LXC.) >> >> > It failed. After a short discussion with Chase, I realized if I >> > want to >> > run >> > multinode test job I have to use "lava-multinode" protocol. >> > However, I still need "lava-lxc" to deploy the test image for >> > hikey. >> > I am wondering if it is possible to use two protocols, lava-lxc >> > for >> > deployment test image and lava-multinode to run the test, in the >> > same >> > test >> > job? >> >> Yes. >> >> >> >> https://git.linaro.org/lava-team/refactoring.git/tree/lxc-multinode.yaml >> >> However, it does not sound like you actually need MultiNode. >> >> What are you trying to do with the HiKey? >> >> >> -- >> >> Neil Williams >> ============= >> neil.williams@linaro.org >> http://www.linux.codehelp.co.uk/ > >
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/ _______________________________________________ Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
On 9 June 2017 at 17:17, Milosz Wasilewski milosz.wasilewski@linaro.org wrote:
On 9 June 2017 at 17:00, Arthur She arthur.she@linaro.org wrote:
Thanks Neil for your suggestion.
[cut]
It will be assigned from the MAC address of the network adaptor used by each HiKey, so as long as that does not change, the IP will not conflict. However, the lab team do want to minimise the number of fixed IP addresses used for devices, so this support may only be available on designed HiKeys and the test jobs would need to use device tags.
That means we have to hard code "device : IP address" mapping in somewhere. Either in the test job then pass it to the test definition or in the test definition itself. Because this test definition would be run in validation.l.o and staging.validation.l.o, I'd like to have it flexible enough. I copied the idea from your test job and submitted a multinode test job which pass client IP address through multinode API. It seems it is working fine. Would that be a good idea?
Arthur, I'm wondering why do you need to run the tests via ssh? Is this the same test we discussed at Connect (video capture with robotframework)? If so, the tests run entirely on the target. So there would be no need for the host part.
Take care when converting V1 testjobs to V2. V1 is full of host|client which is entirely down to the need for a full VM and MultiNode. V2 replaces the VM with LXC and drops MultiNode, so wherever the old job talked about host or client, that is irrelevant for the V2 job and will only confuse things.
Milosz, As I mentioned this test definition is not yet finished. And it will be used for running all robot framework tests. Some of them will be run through ssh, for example OP TEE xtest, some of them are not. For the video playback verification, host side will communicate with target side through WebDriver protocol. i.e. robotframework communicates with chromedriver which will be run in the hikey. I know Naresh had made a ui-browser-test https://git.linaro.org/people/naresh.kamboju/test-definitions-robot.git/tree/automated/linux/ui-browser-test which can be run in a single target. And Daniel had run the test https://validation.linaro.org/scheduler/job/1507227 successfully with a special built of OE. However, in order to run robotframework in hikey we have to put some extra software packages in the image, plus OpenCV if we want to run video playback verification. Beside that, I am thinking about to improve the algorithm which we used for the judgement of video playback (This video https://www.youtube.com/watch?v=IbLNm3LsMaw gave me some inspiration.). That might consume a lot of storage and computing power, so I think run it on the host side should be a good idea.
Thanks, Arthur
2017-06-09 9:17 GMT-07:00 Milosz Wasilewski milosz.wasilewski@linaro.org:
On 9 June 2017 at 17:00, Arthur She arthur.she@linaro.org wrote:
Thanks Neil for your suggestion.
[cut]
It will be assigned from the MAC address of the network adaptor used by each HiKey, so as long as that does not change, the IP will not conflict. However, the lab team do want to minimise the number of fixed IP addresses used for devices, so this support may only be available on designed HiKeys and the test jobs would need to use device tags.
That means we have to hard code "device : IP address" mapping in
somewhere.
Either in the test job then pass it to the test definition or in the test definition itself. Because this test definition would be run in validation.l.o and staging.validation.l.o, I'd like to have it flexible enough. I copied the idea from your test job and submitted a multinode test job which pass client IP address through multinode API. It seems it is
working
fine. Would that be a good idea?
Arthur, I'm wondering why do you need to run the tests via ssh? Is this the same test we discussed at Connect (video capture with robotframework)? If so, the tests run entirely on the target. So there would be no need for the host part.
milosz
[cut]
That means we have to hard code "device : IP address" mapping in somewhere.
No, not the test writer - the admins. It becomes part of the device configuration and the test writer has no control over that value, the test writer can simply obtain it.
So, how could the transparent LXC get the IP address of the DUT, hikey?
Either in the test job then pass it to the test definition or in the test definition itself. Because this test definition would be run in validation.l.o and staging.validation.l.o, I'd like to have it flexible enough. I copied the idea from your test job and submitted a multinode test job which pass client IP address through multinode API. It seems it is
working
fine. Would that be a good idea?
No. The LXC is not visible to MultiNode.
I am confused. Please check the job file https://staging.validation.linaro.org/scheduler/job/175598/multinode_definition. In this test job, I allocated two devices, LXC and hikey. Hikey sent its IP address through multinode API, lava-send https://staging.validation.linaro.org/scheduler/job/175598.0#L547 to LXC, and the LXC ping hikey successfully with that one https://staging.validation.linaro.org/scheduler/job/175598.1#L362.
Create the request on CTT as per the link given in the previous message, included below.
- In your local tests, assume that this address can be obtained by
running a shell script (this would be how LAVA provides the information).
- Please be very clear about what operating system is being used on
the device in discussions about this method and that LXC is used for the deployment stage. Avoid comparisons with CTS and AOSP test jobs.
- Keep MultiNode for situations where there are two DUTs - remember
that the LXC needed to deploy files is actually part of the dispatcher, not a device.
>> Basically, what I need is 1. the IP address of the client 2. run >> some >> program in the client before the host starts the test in order to >> accept the >> commands from it.
Please clarify here what you mean by host and client. Avoid thinking of these as MultiNode. There is a DUT and there is an LXC acting as the dispatcher. As each can run a POSIX shell, the terms need to be consistent.
> Think of the LXC as a transparent container - it acts as the > dispatcher but has the ability to run arbitrary binaries like adb. > In > the HiKey test job and in many others, the LXC is not a device, it > is > the dispatcher itself.
Thanks for the explanation. I think I got a feel for it.
> >> 3. before the host finished the test, the client shouldn't be >> released >> by >> LAVA. Which shouldn't be a problem, I think.
Again, this hints of MultiNode - the LXC for a HiKey persists until the test job itself completes. The device is powered off and the LXC is destroyed - which happens first is irrelevant because nothing from a test shell is allowed to happen between the two events. There is no synchronisation to do here, no waiting for one or the other.
>> I am not sure if I have to use multi node test to accomplish it. > > No, it only uses LXC. > >> Chase gave me some CTS examples. However, I haven't had any ideas >> to >> get the >> client IP address with that approach. > > If you cannot obtain it with adb, it will have to be static and > assigned by the lab team. Open an issue in CT&T > https://projects.linaro.org/servicedesk/customer/portal/1
That still applies.
Make a request for a number of HiKeys (not all) to have a fixed IPv4 address based on the MAC address of the network adaptor for the purposes of your particular use case.
> >> Do you have any suggestions? >> >> Thanks a lot. >> Arthur >> >> >> 2017-06-06 23:59 GMT-07:00 Neil Williams >> neil.williams@linaro.org: >>> >>> On 7 June 2017 at 06:25, Arthur She arthur.she@linaro.org
wrote:
>>> > Hi, >>> > I am trying to run a multinode test job in LAVA v2. >>> >>> That doesn't look like a MultiNode test job. What are you trying >>> to >>> achieve? >>> >>> LXC support does not require MultiNode but LXC can be used in a >>> MultiNode situation, should you require to have one test job >>> involving >>> a HiKey and another test job on mustang or panda or something
and
>>> then >>> synchronise the two devices. >>> >>> Neither of those links are MultiNode jobs. Make sure you are not >>> confusing the LXC protocol with MultiNode. >>> >>> The MultiNode protocol cannot be used to pass information from
the
>>> HiKey to the LXC which is managing the adb/fastboot calls to
that
>>> same >>> HiKey. Use adb to pull or push data to the device. (For non-AOSP >>> boots, use tools available within the LXC.) >>> >>> > It failed. After a short discussion with Chase, I realized if
I
>>> > want to >>> > run >>> > multinode test job I have to use "lava-multinode" protocol. >>> > However, I still need "lava-lxc" to deploy the test image for >>> > hikey. >>> > I am wondering if it is possible to use two protocols,
lava-lxc
>>> > for >>> > deployment test image and lava-multinode to run the test, in
the
>>> > same >>> > test >>> > job? >>> >>> Yes. >>> >>> >>> >>> https://git.linaro.org/lava-team/refactoring.git/tree/lxc-
multinode.yaml
>>> >>> However, it does not sound like you actually need MultiNode. >>> >>> What are you trying to do with the HiKey? >>> >>> >>> -- >>> >>> Neil Williams >>> ============= >>> neil.williams@linaro.org >>> http://www.linux.codehelp.co.uk/ >> >> > > > > -- > > Neil Williams > ============= > neil.williams@linaro.org > http://www.linux.codehelp.co.uk/ > _______________________________________________ > Lava-users mailing list > Lava-users@lists.linaro.org > https://lists.linaro.org/mailman/listinfo/lava-users
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
On 12 June 2017 at 00:39, Arthur She arthur.she@linaro.org wrote:
[cut]
That means we have to hard code "device : IP address" mapping in somewhere.
No, not the test writer - the admins. It becomes part of the device configuration and the test writer has no control over that value, the test writer can simply obtain it.
So, how could the transparent LXC get the IP address of the DUT, hikey?
https://git.linaro.org/lava/lava-dispatcher.git/tree/lava_dispatcher/pipelin...
Once the lab have configured the lab DHCP to always assign a particular IPv4 address to the MAC address of the USB network adaptor for a specific HiKey, that IPv4 address is added to the device configuration of that HiKey by the lab team. The LAVA overlay for the LXC then contains that information and the test writer calls lava-target-ip to echo it.
Either in the test job then pass it to the test definition or in the test definition itself. Because this test definition would be run in validation.l.o and staging.validation.l.o, I'd like to have it flexible enough. I copied the idea from your test job and submitted a multinode test job which pass client IP address through multinode API. It seems it is working fine. Would that be a good idea?
No. The LXC is not visible to MultiNode.
I am confused. Please check the job file. In this test job, I allocated two devices, LXC and hikey. Hikey sent its IP address through multinode API, lava-send to LXC, and the LXC ping hikey successfully with that one.
So you now have two LXC's when only one is needed (once the LAB team do their part). It will take slightly longer to run this test and it suffers the same problems as the V1 test jobs using KVM because now a dedicated LXC device has to be reserved alongside the HiKey.
The test job has two LXCs because one is explicitly in the MultiNode group as a standalone device of the device-type LXC and another transparent one is attached to the HiKey. You also have the complexity of a MultiNode test job with the need to extract the IPv4 address from the HiKey, transmit that through the MultiNode group and make the LXC wait until that value is retrieved. With the static IPv4 address, the one transparent LXC has access to that IPv4 address immediately and there is one less LXC to manage.
Your arrangement will work well during testing but please file the request to have a static IPv4 address for nominated HiKey devices so that when this becomes part of a CI loop, we don't create more work for the admins and more delays for test writers by using MultiNode unnecessarily. (If this test job was to be submitted in batches, it would quickly reserve all available LXC devices. That's manageable if only one test job is doing things that way but it does not scale.)
Create the request on CTT as per the link given in the previous message, included below.
- In your local tests, assume that this address can be obtained by
running a shell script (this would be how LAVA provides the information).
- Please be very clear about what operating system is being used on
the device in discussions about this method and that LXC is used for the deployment stage. Avoid comparisons with CTS and AOSP test jobs.
- Keep MultiNode for situations where there are two DUTs - remember
that the LXC needed to deploy files is actually part of the dispatcher, not a device.
>>> Basically, what I need is 1. the IP address of the client 2. run >>> some >>> program in the client before the host starts the test in order >>> to >>> accept the >>> commands from it.
Please clarify here what you mean by host and client. Avoid thinking of these as MultiNode. There is a DUT and there is an LXC acting as the dispatcher. As each can run a POSIX shell, the terms need to be consistent.
>> Think of the LXC as a transparent container - it acts as the >> dispatcher but has the ability to run arbitrary binaries like >> adb. >> In >> the HiKey test job and in many others, the LXC is not a device, >> it >> is >> the dispatcher itself.
Thanks for the explanation. I think I got a feel for it.
>> >>> 3. before the host finished the test, the client shouldn't be >>> released >>> by >>> LAVA. Which shouldn't be a problem, I think.
Again, this hints of MultiNode - the LXC for a HiKey persists until the test job itself completes. The device is powered off and the LXC is destroyed - which happens first is irrelevant because nothing from a test shell is allowed to happen between the two events. There is no synchronisation to do here, no waiting for one or the other.
>>> I am not sure if I have to use multi node test to accomplish it. >> >> No, it only uses LXC. >> >>> Chase gave me some CTS examples. However, I haven't had any >>> ideas >>> to >>> get the >>> client IP address with that approach. >> >> If you cannot obtain it with adb, it will have to be static and >> assigned by the lab team. Open an issue in CT&T >> https://projects.linaro.org/servicedesk/customer/portal/1
That still applies.
Make a request for a number of HiKeys (not all) to have a fixed IPv4 address based on the MAC address of the network adaptor for the purposes of your particular use case.
>> >>> Do you have any suggestions? >>> >>> Thanks a lot. >>> Arthur >>> >>> >>> 2017-06-06 23:59 GMT-07:00 Neil Williams >>> neil.williams@linaro.org: >>>> >>>> On 7 June 2017 at 06:25, Arthur She arthur.she@linaro.org >>>> wrote: >>>> > Hi, >>>> > I am trying to run a multinode test job in LAVA v2. >>>> >>>> That doesn't look like a MultiNode test job. What are you >>>> trying >>>> to >>>> achieve? >>>> >>>> LXC support does not require MultiNode but LXC can be used in a >>>> MultiNode situation, should you require to have one test job >>>> involving >>>> a HiKey and another test job on mustang or panda or something >>>> and >>>> then >>>> synchronise the two devices. >>>> >>>> Neither of those links are MultiNode jobs. Make sure you are >>>> not >>>> confusing the LXC protocol with MultiNode. >>>> >>>> The MultiNode protocol cannot be used to pass information from >>>> the >>>> HiKey to the LXC which is managing the adb/fastboot calls to >>>> that >>>> same >>>> HiKey. Use adb to pull or push data to the device. (For >>>> non-AOSP >>>> boots, use tools available within the LXC.) >>>> >>>> > It failed. After a short discussion with Chase, I realized if >>>> > I >>>> > want to >>>> > run >>>> > multinode test job I have to use "lava-multinode" protocol. >>>> > However, I still need "lava-lxc" to deploy the test image for >>>> > hikey. >>>> > I am wondering if it is possible to use two protocols, >>>> > lava-lxc >>>> > for >>>> > deployment test image and lava-multinode to run the test, in >>>> > the >>>> > same >>>> > test >>>> > job? >>>> >>>> Yes. >>>> >>>> >>>> >>>> >>>> https://git.linaro.org/lava-team/refactoring.git/tree/lxc-multinode.yaml >>>> >>>> However, it does not sound like you actually need MultiNode. >>>> >>>> What are you trying to do with the HiKey? >>>> >>>> >>>> -- >>>> >>>> Neil Williams >>>> ============= >>>> neil.williams@linaro.org >>>> http://www.linux.codehelp.co.uk/ >>> >>> >> >> >> >> -- >> >> Neil Williams >> ============= >> neil.williams@linaro.org >> http://www.linux.codehelp.co.uk/ >> _______________________________________________ >> Lava-users mailing list >> Lava-users@lists.linaro.org >> https://lists.linaro.org/mailman/listinfo/lava-users
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
On 12 June 2017 at 01:12, Arthur She arthur.she@linaro.org wrote:
Milosz, As I mentioned this test definition is not yet finished. And it will be used for running all robot framework tests. Some of them will be run through ssh, for example OP TEE xtest, some of them are not. For the video playback verification, host side will communicate with target side through WebDriver protocol. i.e. robotframework communicates with chromedriver which will be run in the hikey. I know Naresh had made a ui-browser-test which can be run in a single target. And Daniel had run the test successfully with a special built of OE. However, in order to run robotframework in hikey we have to put some extra software packages in the image, plus OpenCV if we want to run video playback verification. Beside that, I am thinking about to improve the algorithm which we used for the judgement of video playback (This video gave me some inspiration.).
Here is another approach that would allow you to test as well the whole display pipeline:
https://chromium.googlesource.com/chromiumos/third_party/autotest/+/master/s...
HTH,
Tomeu
That might consume a lot of storage and computing power, so I think run it on the host side should be a good idea.
Thanks, Arthur
2017-06-09 9:17 GMT-07:00 Milosz Wasilewski milosz.wasilewski@linaro.org:
On 9 June 2017 at 17:00, Arthur She arthur.she@linaro.org wrote:
Thanks Neil for your suggestion.
[cut]
It will be assigned from the MAC address of the network adaptor used by each HiKey, so as long as that does not change, the IP will not conflict. However, the lab team do want to minimise the number of fixed IP addresses used for devices, so this support may only be available on designed HiKeys and the test jobs would need to use device tags.
That means we have to hard code "device : IP address" mapping in somewhere. Either in the test job then pass it to the test definition or in the test definition itself. Because this test definition would be run in validation.l.o and staging.validation.l.o, I'd like to have it flexible enough. I copied the idea from your test job and submitted a multinode test job which pass client IP address through multinode API. It seems it is working fine. Would that be a good idea?
Arthur, I'm wondering why do you need to run the tests via ssh? Is this the same test we discussed at Connect (video capture with robotframework)? If so, the tests run entirely on the target. So there would be no need for the host part.
milosz
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
2017-06-12 0:49 GMT-07:00 Neil Williams neil.williams@linaro.org:
On 12 June 2017 at 00:39, Arthur She arthur.she@linaro.org wrote:
[cut]
That means we have to hard code "device : IP address" mapping in somewhere.
No, not the test writer - the admins. It becomes part of the device configuration and the test writer has no control over that value, the test writer can simply obtain it.
So, how could the transparent LXC get the IP address of the DUT, hikey?
https://git.linaro.org/lava/lava-dispatcher.git/tree/lava_di spatcher/pipeline/lava_test_shell/lava-target-ip
Once the lab have configured the lab DHCP to always assign a particular IPv4 address to the MAC address of the USB network adaptor for a specific HiKey, that IPv4 address is added to the device configuration of that HiKey by the lab team. The LAVA overlay for the LXC then contains that information and the test writer calls lava-target-ip to echo it.
It looks great.
Either in the test job then pass it to the test definition or in the test definition itself. Because this test definition would be run in validation.l.o and staging.validation.l.o, I'd like to have it flexible enough. I copied the idea from your test job and submitted a multinode test
job
which pass client IP address through multinode API. It seems it is working fine. Would that be a good idea?
No. The LXC is not visible to MultiNode.
I am confused. Please check the job file. In this test job, I allocated
two
devices, LXC and hikey. Hikey sent its IP address through multinode API, lava-send to LXC, and
the
LXC ping hikey successfully with that one.
So you now have two LXC's when only one is needed (once the LAB team do their part). It will take slightly longer to run this test and it suffers the same problems as the V1 test jobs using KVM because now a dedicated LXC device has to be reserved alongside the HiKey.
The test job has two LXCs because one is explicitly in the MultiNode group as a standalone device of the device-type LXC and another transparent one is attached to the HiKey. You also have the complexity of a MultiNode test job with the need to extract the IPv4 address from the HiKey, transmit that through the MultiNode group and make the LXC wait until that value is retrieved. With the static IPv4 address, the one transparent LXC has access to that IPv4 address immediately and there is one less LXC to manage.
Your arrangement will work well during testing but please file the request to have a static IPv4 address for nominated HiKey devices so that when this becomes part of a CI loop, we don't create more work for the admins and more delays for test writers by using MultiNode unnecessarily. (If this test job was to be submitted in batches, it would quickly reserve all available LXC devices. That's manageable if only one test job is doing things that way but it does not scale.)
That makes sense. Thanks Neil.
Create the request on CTT as per the link given in the previous message, included below.
> > > 4. In your local tests, assume that this address can be obtained
by
> running a shell script (this would be how LAVA provides the > information). > > 5. Please be very clear about what operating system is being used
on
> the device in discussions about this method and that LXC is used
for
> the deployment stage. Avoid comparisons with CTS and AOSP test
jobs.
> > 6. Keep MultiNode for situations where there are two DUTs -
remember
> that the LXC needed to deploy files is actually part of the > dispatcher, not a device. > > > >>> Basically, what I need is 1. the IP address of the client 2.
run
> >>> some > >>> program in the client before the host starts the test in order > >>> to > >>> accept the > >>> commands from it. > > Please clarify here what you mean by host and client. Avoid
thinking
> of these as MultiNode. There is a DUT and there is an LXC acting
as
> the dispatcher. As each can run a POSIX shell, the terms need to
be
> consistent. > > >> Think of the LXC as a transparent container - it acts as the > >> dispatcher but has the ability to run arbitrary binaries like > >> adb. > >> In > >> the HiKey test job and in many others, the LXC is not a device, > >> it > >> is > >> the dispatcher itself.
Thanks for the explanation. I think I got a feel for it.
> > >> > >>> 3. before the host finished the test, the client shouldn't be > >>> released > >>> by > >>> LAVA. Which shouldn't be a problem, I think. > > Again, this hints of MultiNode - the LXC for a HiKey persists
until
> the test job itself completes. The device is powered off and the
LXC
> is destroyed - which happens first is irrelevant because nothing > from > a test shell is allowed to happen between the two events. There is > no > synchronisation to do here, no waiting for one or the other. > > >>> I am not sure if I have to use multi node test to accomplish
it.
> >> > >> No, it only uses LXC. > >> > >>> Chase gave me some CTS examples. However, I haven't had any > >>> ideas > >>> to > >>> get the > >>> client IP address with that approach. > >> > >> If you cannot obtain it with adb, it will have to be static and > >> assigned by the lab team. Open an issue in CT&T > >> https://projects.linaro.org/servicedesk/customer/portal/1 > > That still applies.
Make a request for a number of HiKeys (not all) to have a fixed IPv4 address based on the MAC address of the network adaptor for the purposes of your particular use case.
> > >> > >>> Do you have any suggestions? > >>> > >>> Thanks a lot. > >>> Arthur > >>> > >>> > >>> 2017-06-06 23:59 GMT-07:00 Neil Williams > >>> neil.williams@linaro.org: > >>>> > >>>> On 7 June 2017 at 06:25, Arthur She arthur.she@linaro.org > >>>> wrote: > >>>> > Hi, > >>>> > I am trying to run a multinode test job in LAVA v2. > >>>> > >>>> That doesn't look like a MultiNode test job. What are you > >>>> trying > >>>> to > >>>> achieve? > >>>> > >>>> LXC support does not require MultiNode but LXC can be used
in a
> >>>> MultiNode situation, should you require to have one test job > >>>> involving > >>>> a HiKey and another test job on mustang or panda or something > >>>> and > >>>> then > >>>> synchronise the two devices. > >>>> > >>>> Neither of those links are MultiNode jobs. Make sure you are > >>>> not > >>>> confusing the LXC protocol with MultiNode. > >>>> > >>>> The MultiNode protocol cannot be used to pass information
from
> >>>> the > >>>> HiKey to the LXC which is managing the adb/fastboot calls to > >>>> that > >>>> same > >>>> HiKey. Use adb to pull or push data to the device. (For > >>>> non-AOSP > >>>> boots, use tools available within the LXC.) > >>>> > >>>> > It failed. After a short discussion with Chase, I realized
if
> >>>> > I > >>>> > want to > >>>> > run > >>>> > multinode test job I have to use "lava-multinode" protocol. > >>>> > However, I still need "lava-lxc" to deploy the test image
for
> >>>> > hikey. > >>>> > I am wondering if it is possible to use two protocols, > >>>> > lava-lxc > >>>> > for > >>>> > deployment test image and lava-multinode to run the test,
in
> >>>> > the > >>>> > same > >>>> > test > >>>> > job? > >>>> > >>>> Yes. > >>>> > >>>> > >>>> > >>>> > >>>> https://git.linaro.org/lava-team/refactoring.git/tree/lxc-mu
ltinode.yaml
> >>>> > >>>> However, it does not sound like you actually need MultiNode. > >>>> > >>>> What are you trying to do with the HiKey? > >>>> > >>>> > >>>> -- > >>>> > >>>> Neil Williams > >>>> ============= > >>>> neil.williams@linaro.org > >>>> http://www.linux.codehelp.co.uk/ > >>> > >>> > >> > >> > >> > >> -- > >> > >> Neil Williams > >> ============= > >> neil.williams@linaro.org > >> http://www.linux.codehelp.co.uk/ > >> _______________________________________________ > >> Lava-users mailing list > >> Lava-users@lists.linaro.org > >> https://lists.linaro.org/mailman/listinfo/lava-users > > > > -- > > Neil Williams > ============= > neil.williams@linaro.org > http://www.linux.codehelp.co.uk/
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
-
Arthur She -
Milosz Wasilewski -
Neil Williams -
Tomeu Vizoso