Hello,
On Sat, 6 Apr 2013 15:40:15 +0300 Paul Sokolovsky Paul.Sokolovsky@linaro.org wrote:
[]
Ok, so here're these 2 builds:
gcc-4.8~svn196132
panda-es02 https://validation.linaro.org/lava-server/scheduler/job/50993
panda-es05 https://validation.linaro.org/lava-server/scheduler/job/50994
#50993 went well.
#50994 midway in compilation started to get invalid data (grep for "is not valid in preprocessor expressions"), then got kernel fault, then got caught in reboot loop, apparently due to:
[ 6.631256] thermal_init_thermal_state: Getting initial temp for cpu domain [ 6.638702] thermal_request_temp [ 6.642150] omap_fatal_zone:FATAL ZONE (hot spot temp: 128490)
- all these behaviors were seen by me before (actually, previously, I
didn't see such explicit messages from kernel that it's a thermal faults).
So, CBuild/LAVA can do (successful) builds, but some builds fail due to thermal issues. Actually, let me load up all boards with the same build now, towards assessing thermal failure rate more scientifically.
Well, let's count:
https://validation.linaro.org/lava-server/dashboard/streams/anonymous/cbuild... OK
https://validation.linaro.org/lava-server/dashboard/streams/anonymous/cbuild... Failed, thermal (see above)
https://validation.linaro.org/lava-server/dashboard/streams/anonymous/cbuild... "../../../gcc-4.8~svn196132/gcc/gengtype.c:4106:39: error: cannot convert 'flisT*' to 'flist*' in assignment" flipped bit in file (so that before, including on local Panda), then other random failures.
https://validation.linaro.org/lava-server/dashboard/streams/anonymous/cbuild... "rsync: getaddrinfo: toolchain64 2000: Temporary failure in name resolution" - network flip
https://validation.linaro.org/lava-server/dashboard/streams/anonymous/cbuild... OK
https://validation.linaro.org/lava-server/dashboard/streams/anonymous/cbuild... Lot of "malloc: ../bash/jobs.c:743: assertion botched"
https://validation.linaro.org/lava-server/dashboard/streams/anonymous/cbuild... Reboot during configure, then reboot cycle.
So, out of 7 builds, only 2 were successful, the rest fail due to thermal issues (apparently, all but the one with network issues). We can try to consider why LAVA-based builds have such low yield rate comparing to native Cbuild builds (one explanation is that LAVA does pretty heavy lifting to install OS, etc., so when build starts, CPU is already pretty hot), but it's clear that using kernel with voltage/frequency scaling disabled simply doesn't work that well for *builds* (vs benchmarks).
So, we can consider preparing OS image which uses the same basic OS, but normal kernel, and use that for builds. Would TCWG be able to prepare such image? If not, we can add it to the list of tasks for (now combined) LAVA/Infra team, but with all the other tasks we have, it may take some time to get to.