Hi All,
We use IOZone test to measure performance of our DUT Same command executed several time in a Lava session provide a diversity of results
iozone -az -i0 -i1 -I -e -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -f /mnt_emmc//tmp/iozone.tmp
Example1: kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 102400 4 5053 4420 12471 12137 102400 16 5070 5218 22101 22207 102400 512 11733 12630 40799 40862 102400 1024 11446 11494 39982 39976 102400 16384 13839 14235 42093 42094
Example 2: kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 102400 4 5088 5006 13496 13095 102400 16 5395 5549 17199 17220 102400 512 9203 10038 27819 27586 102400 1024 9382 8482 32514 32430 102400 16384 13569 13986 41992 42081
When same command is executed without LAVA with same DUT, results are more homogeneous (similar to exemple1) and permit to define clear targets
How to reduce interaction from Lava to measure performance of DUT ? Is it possible to disable some checks / interactions with DUT during execution of each test (around 3 / 4 minutes) ?
Thanks in advance for your answer Florence Rouger-Jung
On 26 January 2018 at 08:03, Florence ROUGER JUNG < florence.rouger-jung@st.com> wrote:
Hi All,
We use IOZone test to measure performance of our DUT
Same command executed several time in a Lava session provide a diversity of results
iozone -az -i0 -i1 -I -e -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -f /mnt_emmc//tmp/iozone.tmp
Example1:
kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
102400 4 5053 4420 12471 12137
102400 16 5070 5218 22101 22207
102400 512 11733 12630 40799 40862
102400 1024 11446 11494 39982 39976
102400 16384 13839 14235 42093 42094
Example 2:
kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
102400 4 5088 5006 13496 13095
102400 16 5395 5549 17199 17220
102400 512 9203 10038 27819 27586
102400 1024 9382 8482 32514 32430
102400 16384 13569 13986 41992 42081
When same command is executed without LAVA with same DUT, results are more homogeneous (similar to exemple1) and permit to define clear targets
I'm not familiar with IOZone, so any problem with those numbers needs to be fully explained - I can't see a pattern.
The command line suggests that you're expecting the data to be written directly to the DUT filesystem from a command running on the DUT, so LAVA is doing absolutely nothing during that operation.
How to reduce interaction from Lava to measure performance of DUT ?
During the execution of a Lava Test Shell there is no impact on the DUT from running it in LAVA - LAVA is simply watching for patterns on the serial connection and writing logs. LAVA is not interacting with the board in any way. During the operation of a test shell script, LAVA is in strictly read-only mode. Nothing is sent to the DUT once the test shell starts.
Is it possible to disable some checks / interactions with DUT during execution of each test (around 3 / 4 minutes) ?
Right now, that question doesn't make sense because there are no checks / interactions with the DUT during the execution of a test.
Hello,
Are you running the board in the same environment for both tests? For instance it might (I haven't tested) have some impacts if in one case you are booting using tftp/nfs and in second case booting from the emmc directly (flashed board).
2018-01-26 11:10 GMT+01:00 Neil Williams neil.williams@linaro.org:
On 26 January 2018 at 08:03, Florence ROUGER JUNG < florence.rouger-jung@st.com> wrote:
Hi All,
We use IOZone test to measure performance of our DUT
Same command executed several time in a Lava session provide a diversity of results
iozone -az -i0 -i1 -I -e -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -f /mnt_emmc//tmp/iozone.tmp
Example1:
kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
102400 4 5053 4420 12471 12137
102400 16 5070 5218 22101 22207
102400 512 11733 12630 40799 40862
102400 1024 11446 11494 39982 39976
102400 16384 13839 14235 42093 42094
Example 2:
kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
102400 4 5088 5006 13496 13095
102400 16 5395 5549 17199 17220
102400 512 9203 10038 27819 27586
102400 1024 9382 8482 32514 32430
102400 16384 13569 13986 41992 42081
When same command is executed without LAVA with same DUT, results are more homogeneous (similar to exemple1) and permit to define clear targets
I'm not familiar with IOZone, so any problem with those numbers needs to be fully explained - I can't see a pattern.
The command line suggests that you're expecting the data to be written directly to the DUT filesystem from a command running on the DUT, so LAVA is doing absolutely nothing during that operation.
How to reduce interaction from Lava to measure performance of DUT ?
During the execution of a Lava Test Shell there is no impact on the DUT from running it in LAVA - LAVA is simply watching for patterns on the serial connection and writing logs. LAVA is not interacting with the board in any way. During the operation of a test shell script, LAVA is in strictly read-only mode. Nothing is sent to the DUT once the test shell starts.
Is it possible to disable some checks / interactions with DUT during execution of each test (around 3 / 4 minutes) ?
Right now, that question doesn't make sense because there are no checks / interactions with the DUT during the execution of a test.
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
Hello,
I use same environment for both tests, flashed board on SDCard, boot from SDCard and then mounted EMMC to execute iozone command Florence
From: Remi Duraffort [mailto:remi.duraffort@linaro.org] Sent: vendredi 26 janvier 2018 12:05 To: Neil Williams neil.williams@linaro.org Cc: Florence ROUGER JUNG florence.rouger-jung@st.com; lava-users@lists.linaro.org Subject: Re: [Lava-users] IOZone test to measure performance of DUT
Hello,
Are you running the board in the same environment for both tests? For instance it might (I haven't tested) have some impacts if in one case you are booting using tftp/nfs and in second case booting from the emmc directly (flashed board).
2018-01-26 11:10 GMT+01:00 Neil Williams <neil.williams@linaro.orgmailto:neil.williams@linaro.org>: On 26 January 2018 at 08:03, Florence ROUGER JUNG <florence.rouger-jung@st.commailto:florence.rouger-jung@st.com> wrote: Hi All,
We use IOZone test to measure performance of our DUT Same command executed several time in a Lava session provide a diversity of results
iozone -az -i0 -i1 -I -e -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -f /mnt_emmc//tmp/iozone.tmp
Example1: kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 102400 4 5053 4420 12471 12137 102400 16 5070 5218 22101 22207 102400 512 11733 12630 40799 40862 102400 1024 11446 11494 39982 39976 102400 16384 13839 14235 42093 42094
Example 2: kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 102400 4 5088 5006 13496 13095 102400 16 5395 5549 17199 17220 102400 512 9203 10038 27819 27586 102400 1024 9382 8482 32514 32430 102400 16384 13569 13986 41992 42081
When same command is executed without LAVA with same DUT, results are more homogeneous (similar to exemple1) and permit to define clear targets
I'm not familiar with IOZone, so any problem with those numbers needs to be fully explained - I can't see a pattern.
The command line suggests that you're expecting the data to be written directly to the DUT filesystem from a command running on the DUT, so LAVA is doing absolutely nothing during that operation.
How to reduce interaction from Lava to measure performance of DUT ?
During the execution of a Lava Test Shell there is no impact on the DUT from running it in LAVA - LAVA is simply watching for patterns on the serial connection and writing logs. LAVA is not interacting with the board in any way. During the operation of a test shell script, LAVA is in strictly read-only mode. Nothing is sent to the DUT once the test shell starts.
Is it possible to disable some checks / interactions with DUT during execution of each test (around 3 / 4 minutes) ?
Right now, that question doesn't make sense because there are no checks / interactions with the DUT during the execution of a test.
--
Neil Williams ============= neil.williams@linaro.orgmailto:neil.williams@linaro.org http://www.linux.codehelp.co.uk/
_______________________________________________ Lava-users mailing list Lava-users@lists.linaro.orgmailto:Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
-- Rémi Duraffort LAVA Team
On Fri, Jan 26, 2018 at 11:12:54AM +0000, Florence ROUGER JUNG wrote:
Hello,
I use same environment for both tests, flashed board on SDCard, boot from SDCard and then mounted EMMC to execute iozone command
Florence
Hmmm, OK. Does iozone print a lot of output to the terminal as it runs, maybe?
Cheers,
That's right, at first I used following command and there was some output to the terminal iozone $OPT -f /mnt_emmc/tmp/iozone.tmp | tee result.log
But now I redirect output iozone $OPT -f /mnt_emmc/tmp/iozone.tmp > iozone.log 2>&1 performance are better but remain not homogeneous with Lava
Florence
-----Original Message----- From: Steve McIntyre [mailto:steve.mcintyre@linaro.org] Sent: vendredi 26 janvier 2018 14:43 To: Florence ROUGER JUNG florence.rouger-jung@st.com Cc: Remi Duraffort remi.duraffort@linaro.org; Neil Williams neil.williams@linaro.org; lava-users@lists.linaro.org Subject: Re: [Lava-users] IOZone test to measure performance of DUT
On Fri, Jan 26, 2018 at 11:12:54AM +0000, Florence ROUGER JUNG wrote:
Hello,
I use same environment for both tests, flashed board on SDCard, boot from SDCard and then mounted EMMC to execute iozone command
Florence
Hmmm, OK. Does iozone print a lot of output to the terminal as it runs, maybe?
Cheers,
On Fri, Jan 26, 2018 at 02:04:52PM +0000, Florence ROUGER JUNG wrote:
That's right, at first I used following command and there was some output to the terminal iozone $OPT -f /mnt_emmc/tmp/iozone.tmp | tee result.log
But now I redirect output iozone $OPT -f /mnt_emmc/tmp/iozone.tmp > iozone.log 2>&1 performance are better but remain not homogeneous with Lava
OK, that does surprise me. I could imagine that running a test with lots of output to the serial port could slow things down, but if you're redirecting everything then it should be making no difference at all. What kind of variance are you seeing?
Cheers,
See following attachments which contains some results of 4 commands executed in same Lava job or without Lava
Each test takes around 3/4 minutes. Is it some periodically action from LAVA to check if DUT is always available (ping/ssh ... ) which can explain these differences of results ?
Florence
-----Original Message----- From: Steve McIntyre [mailto:steve.mcintyre@linaro.org] Sent: vendredi 26 janvier 2018 15:14 To: Florence ROUGER JUNG florence.rouger-jung@st.com Cc: Remi Duraffort remi.duraffort@linaro.org; Neil Williams neil.williams@linaro.org; lava-users@lists.linaro.org Subject: Re: [Lava-users] IOZone test to measure performance of DUT
On Fri, Jan 26, 2018 at 02:04:52PM +0000, Florence ROUGER JUNG wrote:
That's right, at first I used following command and there was some output to the terminal iozone $OPT -f /mnt_emmc/tmp/iozone.tmp | tee result.log
But now I redirect output iozone $OPT -f /mnt_emmc/tmp/iozone.tmp > iozone.log 2>&1 performance are better but remain not homogeneous with Lava
OK, that does surprise me. I could imagine that running a test with lots of output to the serial port could slow things down, but if you're redirecting everything then it should be making no difference at all. What kind of variance are you seeing?
Cheers,
On 26 January 2018 at 15:04, Florence ROUGER JUNG < florence.rouger-jung@st.com> wrote:
See following attachments which contains some results of 4 commands executed in same Lava job or without Lava
Each test takes around 3/4 minutes. Is it some periodically action from LAVA to check if DUT is always available (ping/ssh ... ) which can explain these differences of results ?
There is no periodic action from LAVA during execution of a test shell. LAVA sends nothing to the DUT once the device starts execution of the test shell.
Please explain *what* the differences are. The numbers in the files you attached are essentially meaningless as we're not familiar with what IOZone is meant to produce. It looks just like random divergence as would happen by chance, to me.
Florence
-----Original Message----- From: Steve McIntyre [mailto:steve.mcintyre@linaro.org] Sent: vendredi 26 janvier 2018 15:14 To: Florence ROUGER JUNG florence.rouger-jung@st.com Cc: Remi Duraffort remi.duraffort@linaro.org; Neil Williams < neil.williams@linaro.org>; lava-users@lists.linaro.org Subject: Re: [Lava-users] IOZone test to measure performance of DUT
On Fri, Jan 26, 2018 at 02:04:52PM +0000, Florence ROUGER JUNG wrote:
That's right, at first I used following command and there was some
output to the terminal
iozone $OPT -f /mnt_emmc/tmp/iozone.tmp | tee result.logBut now I redirect output iozone $OPT -f /mnt_emmc/tmp/iozone.tmp > iozone.log 2>&1
performance
are better but remain not homogeneous with Lava
OK, that does surprise me. I could imagine that running a test with lots of output to the serial port could slow things down, but if you're redirecting everything then it should be making no difference at all. What kind of variance are you seeing?
Cheers,
Steve McIntyre steve.mcintyre@linaro.org http://www.linaro.org/ Linaro.org | Open source software for ARM SoCs
On 26 January 2018 at 08:14, Steve McIntyre steve.mcintyre@linaro.org wrote:
On Fri, Jan 26, 2018 at 02:04:52PM +0000, Florence ROUGER JUNG wrote:
That's right, at first I used following command and there was some
output to the terminal
iozone $OPT -f /mnt_emmc/tmp/iozone.tmp | tee result.logBut now I redirect output iozone $OPT -f /mnt_emmc/tmp/iozone.tmp > iozone.log 2>&1 performance are better but remain not homogeneous with Lava
OK, that does surprise me. I could imagine that running a test with lots of output to the serial port could slow things down, but if you're redirecting everything then it should be making no difference at all. What kind of variance are you seeing?
I've been working on a long test run definition with a lot of serial console output (Piglit), here is what i found [1], looks like is related to this thread.
Anibal
[1] https://lists.linaro.org/pipermail/lava-users/2018-January/000735.html
Cheers,
Steve McIntyre steve.mcintyre@linaro.org http://www.linaro.org/ Linaro.org | Open source software for ARM SoCs
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
On 29 January 2018 at 16:48, Anibal Limon anibal.limon@linaro.org wrote:
On 26 January 2018 at 08:14, Steve McIntyre steve.mcintyre@linaro.org wrote:
On Fri, Jan 26, 2018 at 02:04:52PM +0000, Florence ROUGER JUNG wrote:
That's right, at first I used following command and there was some
output to the terminal
iozone $OPT -f /mnt_emmc/tmp/iozone.tmp | tee result.logBut now I redirect output iozone $OPT -f /mnt_emmc/tmp/iozone.tmp > iozone.log 2>&1 performance are better but remain not homogeneous with Lava
OK, that does surprise me. I could imagine that running a test with lots of output to the serial port could slow things down, but if you're redirecting everything then it should be making no difference at all. What kind of variance are you seeing?
I've been working on a long test run definition with a lot of serial console output (Piglit), here is what i found [1], looks like is related to this thread.
Anibal: It is not related - this thread is concerning output from scripts running on the DUT and writing to the DUT filesystem without sending output anywhere else.
Your Piglit report is different - you are reporting activity on the dispatcher (telnet & lava-run). Steve addresses the difference in the paragraph above your reply.
Florence: please provide more information on what variance is actually being seen? The numbers quoted so far are meaningless without the context of what amount of variance is expected and what the numbers actually represent.
Hi Neil,
IOzone is a filesystem benchmark tool. The benchmark generates and measures a variety of file operations As there are several parameters/possibilities, first I reduced command to perform few operations, only one file size set to 102400 kB and 5 record size (4k / 16k / 512k / 1024k / 16384k) Unit for results provided by iozone command are in kBytes/sec for read /reread /write and rewrite operations For each measure I expect a variability less than 10%
I extract some specific results (from file.txt provided previous week) Measure (kBytes/sec) done with File size = 102400 kB and reclen = 512k With Lava
Average
Ecart Type
Variability
Try1
Try2
Try3
Try4
write
11783
6564
11733
9203
9821
2483
25,3%
rewrite
12771
5818
12630
10038
10314
3250
31,5%
read
41083
24049
40799
27819
33438
8801
26,3%
reread
41161
23488
40862
27586
33274
9090
27,3%
Without Lava
Average
Ecart Type
Variability
Try1
Try2
Try3
Try4
write
13434
13623
13476
13487
13505
82
0,6%
rewrite
13479
13688
13498
13520
13546
96
0,7%
read
40785
40794
40870
40829
40820
39
0,1%
reread
40882
40722
40874
40829
40827
74
0,2%
See following complete log of one execution (try2 without Lava) iozone -az -i0 -i1 -I -e -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -f /mnt_emmc//tmp/iozone.tmp Iozone: Performance Test of File I/O Version $Revision: 3.465 $ Compiled for 32 bit mode. Build: linux-arm
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins Al Slater, Scott Rhine, Mike Wisner, Ken Goss Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, Vangel Bojaxhi, Ben England, Vikentsi Lapa, Alexey Skidanov.
Run began: Thu Dec 14 08:30:40 2017
Auto Mode Cross over of record size disabled. O_DIRECT feature enabled Include fsync in write timing File size set to 102400 kB Record Size 4 kB Record Size 16 kB Record Size 512 kB Record Size 1024 kB Record Size 16384 kB Command line used: iozone -az -i0 -i1 -I -e -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -f /mnt_emmc//tmp/iozone.tmp Output is in kBytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 kBytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. random random bkwd record stride kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 102400 4 8567 10445 14289 14290 102400 16 13654 14436 23454 23418 102400 512 13623 13688 40794 40722 102400 1024 14294 14393 41432 41456 102400 16384 13743 14035 42327 42425 iozone test complete.
Florence
From: Lava-users [mailto:lava-users-bounces@lists.linaro.org] On Behalf Of Neil Williams Sent: mardi 30 janvier 2018 08:23 To: Anibal Limon anibal.limon@linaro.org Cc: lava-users@lists.linaro.org Subject: Re: [Lava-users] IOZone test to measure performance of DUT
On 29 January 2018 at 16:48, Anibal Limon <anibal.limon@linaro.orgmailto:anibal.limon@linaro.org> wrote:
On 26 January 2018 at 08:14, Steve McIntyre <steve.mcintyre@linaro.orgmailto:steve.mcintyre@linaro.org> wrote: On Fri, Jan 26, 2018 at 02:04:52PM +0000, Florence ROUGER JUNG wrote:
That's right, at first I used following command and there was some output to the terminal iozone $OPT -f /mnt_emmc/tmp/iozone.tmp | tee result.log
But now I redirect output iozone $OPT -f /mnt_emmc/tmp/iozone.tmp > iozone.log 2>&1 performance are better but remain not homogeneous with Lava
OK, that does surprise me. I could imagine that running a test with lots of output to the serial port could slow things down, but if you're redirecting everything then it should be making no difference at all. What kind of variance are you seeing?
I've been working on a long test run definition with a lot of serial console output (Piglit), here is what i found [1], looks like is related to this thread.
Anibal: It is not related - this thread is concerning output from scripts running on the DUT and writing to the DUT filesystem without sending output anywhere else.
Your Piglit report is different - you are reporting activity on the dispatcher (telnet & lava-run). Steve addresses the difference in the paragraph above your reply.
Florence: please provide more information on what variance is actually being seen? The numbers quoted so far are meaningless without the context of what amount of variance is expected and what the numbers actually represent.
--
Neil Williams ============= neil.williams@linaro.orgmailto:neil.williams@linaro.org http://www.linux.codehelp.co.uk/
-
Anibal Limon -
Florence ROUGER JUNG -
Neil Williams -
Remi Duraffort -
Steve McIntyre