发件人: Neil Williams [mailto:neil.williams@linaro.org] 发送时间: 2018年6月26日 16:25 收件人: Chenchun (coston) chenchun7@huawei.com 抄送: Lava Users Mailman list lava-users@lists.linaro.org; Lixuan (F) joe.lixuan@huawei.com; Fengguilin (Alan) fengguilin@huawei.com; yangjianxiang yangjianxiang@huawei.com; Yewenzhong (jackyewenzhong) jack.yewenzhong@huawei.com; Liyuan (Larry, Turing Solution) Larry.T@huawei.com; zengmeifang zengmeifang@huawei.com; pangbo (A) pangbo6@huawei.com; liuchunfeng (A) liuchunfeng2@huawei.com; Zhangyi ac zhangyi.ac@huawei.com 主题: Re: [Lava-users] Some problem I met in Setting up CI environment using Lava2
On Tue, 26 Jun 2018 at 09:08, Chenchun (coston) <chenchun7@huawei.commailto:chenchun7@huawei.com> wrote: Hi all
Problem 1:
I find Lava2 re-download Test case repository if more than two test suite needed test in a lava job. Personal think this is waste of time. How can I make lava download Test
case repository only once in a lava job ( ps : all of our test suite in some repository) .I want to know lava can do or cant not do, if can please tell me how to do THX.
Not currently supported. We already support shallow clones by default and you can use the history option to remove the .git directory if you are short of space.
https://staging.validation.linaro.org/static/docs/v2/test-repositories.html#...
We have looked at various options for this support - none were reliable across all devices & test jobs at scale.
Not due to short of space But spent so many time to re-download
Problem 2:
As all we know lava must install os before execute the test task in standard procedure. But in our practical application we find no need mechanical implementation of the
Process. In some case, we just need install os only ones and repeated test, even we install manually os and just use lava to do test. We want lava2 like M-lab install os and
execute test task uncoupling. we met the problems : can not judge the os installed in DUT(device under test ) whether is our SUT(system under test). All caused by very little
information can we read from DUT. I want to know how can I uncoupling install system and execute test ,user can choose only install os 、only execute test 、install os and execute
test 、install os ones and repeat execute test ...
That is entirely down to your own test shell definitions - follow best practice for portability and then run separate test actions.
Do not use simplistic testing with the problems of persistence - each test job needs to start with a clean setup, not whatever was left behind by a previous test job.
Please explain in more detail what you are actually trying to achieve.
The test writer can already choose to install (i.e. deploy) and then test - the test job specifies deploy, boot and test actions.
*If* the test writer knows that it is then safe to run other tests, those can be added into another job by extending the list of test definitions used by the action.
Not all device-types support rebooting the device between test actions in the same test job. This is a limitation of the hardware.
We want lava more smarter. do not re-deploy system if more than two lava job show the same type system(centos、ubuntu、debian)—just deploy system ones. We
have eliminated destructive cases
problem 3:
According to your experience, optimally how many DUT can mount under a lava worker and what is a constrain bottleneck?
That depends on a wide range of factors.
* What kind of test jobs are being run on the worker? ** TFTP is lighter load, fastboot is high load (1 CPU core per fastboot device + at least 1 core for the underlying OS) * What is the I/O performance of the worker? ** Many images need manipulation and that can produce high I/O load spikes * Infrastructure limits play a role as well - network switch load and LAN topology.
There is no definitive answer to your question.
Start small, scale slowly and test with known good functional tests at every change.
We will try fellow the rules: Start small, scale slowly and test with known good functional tests at every change.
Best
coston
_______________________________________________ Lava-users mailing list Lava-users@lists.linaro.orgmailto:Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
--
Neil Williams ============= neil.williams@linaro.orgmailto:neil.williams@linaro.org http://www.linux.codehelp.co.uk/
On Tue, 26 Jun 2018 at 11:00, Chenchun (coston) chenchun7@huawei.com wrote:
Stripping the CC list - the mailing list does not support such a long list, make sure people are subscribed to the list.
*发件人:* Neil Williams [mailto:neil.williams@linaro.org] *发送时间:* 2018年6月26日 16:25 *收件人:* Chenchun (coston) chenchun7@huawei.com *抄送:* Lava Users Mailman list lava-users@lists.linaro.org; Lixuan (F) < joe.lixuan@huawei.com>; Fengguilin (Alan) fengguilin@huawei.com; yangjianxiang yangjianxiang@huawei.com; Yewenzhong (jackyewenzhong) < jack.yewenzhong@huawei.com>; Liyuan (Larry, Turing Solution) < Larry.T@huawei.com>; zengmeifang zengmeifang@huawei.com; pangbo (A) < pangbo6@huawei.com>; liuchunfeng (A) liuchunfeng2@huawei.com; Zhangyi ac zhangyi.ac@huawei.com *主题:* Re: [Lava-users] Some problem I met in Setting up CI environment using Lava2
On Tue, 26 Jun 2018 at 09:08, Chenchun (coston) chenchun7@huawei.com wrote:
Hi all
Problem 1:
I find Lava2 re-download Test case repository if more than two test suite needed test in a lava job. Personal think this is waste of time. How can I make lava download Test
case repository only once in a lava job ( ps : all of our test suite in some repository) .I want to know lava can do or cant not do, if can please tell me how to do THX.
Not currently supported. We already support shallow clones by default and you can use the history option to remove the .git directory if you are short of space.
https://staging.validation.linaro.org/static/docs/v2/test-repositories.html#...
We have looked at various options for this support - none were reliable across all devices & test jobs at scale.
*Not due to short of space But spent so many time to re-download*
That needs to be handled in your own test shell definitions and test repository at this time.
Problem 2:
As all we know lava must install os before execute the test task in standard procedure. But in our practical application we find no need mechanical implementation of the
Process. In some case, we just need install os only ones and repeated test, even we install manually os and just use lava to do test. We want lava2 like M-lab install os and
execute test task uncoupling. we met the problems : can not judge the os installed in DUT(device under test ) whether is our SUT(system under test). All caused by very little
information can we read from DUT. I want to know how can I uncoupling install system and execute test ,user can choose only install os 、only execute test 、install os and execute
test 、install os ones and repeat execute test ...
That is entirely down to your own test shell definitions - follow best practice for portability and then run separate test actions.
Do not use simplistic testing with the problems of persistence - each test job needs to start with a clean setup, not whatever was left behind by a previous test job.
Please explain in more detail what you are actually trying to achieve.
The test writer can already choose to install (i.e. deploy) and then test
- the test job specifies deploy, boot and test actions.
*If* the test writer knows that it is then safe to run other tests, those can be added into another job by extending the list of test definitions used by the action.
Not all device-types support rebooting the device between test actions in the same test job. This is a limitation of the hardware.
*We want lava more smarter. do not re-deploy system if more than two lava job show the same type system(centos**、**ubuntu**、**debian)—just deploy system ones. We *
*have eliminated destructive cases*
You will need to handle that entirely within your test job submissions. If two tests require exactly the same deployment, then those tests must be in the same test job submission. There is no support for using what was left behind by the previous test job., that is dangerous and unreliable.
problem 3:
According to your experience, optimally how many DUT can mount under a lava worker and what is a constrain bottleneck?
That depends on a wide range of factors.
- What kind of test jobs are being run on the worker?
** TFTP is lighter load, fastboot is high load (1 CPU core per fastboot device + at least 1 core for the underlying OS)
- What is the I/O performance of the worker?
** Many images need manipulation and that can produce high I/O load spikes
- Infrastructure limits play a role as well - network switch load and LAN
topology.
There is no definitive answer to your question.
Start small, scale slowly and test with known good functional tests at every change.
*We will try fellow the rules: Start small, scale slowly and test with known good functional tests at every change.*
Best
coston
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
-
Chenchun (coston) -
Neil Williams