OpenEmbbedded as a test suite next steps

[Ken Werner]

On 03/08/2012 09:03 AM, Marcin Juszkiewicz wrote:
> W dniu 07.03.2012 22:47, Ken Werner pisze:
>> On 03/06/2012 01:26 AM, Michael Hope wrote:
>
>>> Talking:
>>> Say Hi to Validation re: EC2 and plans
>>> Say Hi to the ARM landing team re: vexpress upstream support
>>> Say Hi to Beth Flanagan re: Yocto's existing auto builders and any hints
>>
>> Yep, it looks like there are several options:
>>   a) the Yocto autobuilder
>>   b) Jenkins
>>   c) LAVA
>>   d) something homegrown
>>
>> I talked to fabo and zyga and agreed that - as a starting point - I'll
>> provide a shell script that automates the process and we'll go from
>> there. I've made contact with Elizabeth and briefly looked into the
>> yocto autobuilder whis is based on buildbot. Obviously it's meant for
>> regression testing the various yocto images. I think we want to keep
>> oe-core+linaro-meta stable and exchange the toolchain which is kind of
>> the other way round what they are doing. But I'll keep an eye on it.
>
> I had setup at least two buildbots building OE images. For testing of
> Linaro toolchain it can be used too I think. If buildbot has ability to
> pull bzr for changes then you can get automatic tests after each
> commit/merge.

Sounds good. I'll come back to you in case I give it a go. :)

>> Just to give you an overview. The build of the sato and qt4e images
>> takes about two hours my machine (24x "E5649  @ 2.53GHz" with 32gb of
>> RAM) and creates about 37GiB of object files, binaries, packages and
>> images. This is excluding the size of the sources (and the time for
>> fetching them in case they are not there). I haven't spent time on
>> optimizing my build environment (tmpfs etc), so I guess there is room
>> for improvement. I cannot even say whether it's I/O or CPU bound.
>> Sometimes it appears to be the latter - when building Qt for example.
>> But sometimes only one CPU gets used and it's waiting for a source being
>> unpacked that other tasks depend on.
>
> BB_NUMBER_THREADS=12
> PARALLEL_MAKE=8

Yep, I've already played with these and I'm currently using:
  BB_NUMBER_THREADS=24
  PARALLEL_MAKE=24
It sounds way too much since in theory this would spin off 24x24 GCCs 
but in practice this rarely happens because some task is waiting on a 
dependency.

> Or other values for them and your machine will have most of CPU time in
> use. First one control how many BitBake threads are called at same time,
> second is given as "-jX" argument to make. With both used you do not
> have to wait as there is always something to do (up to moment when
> do_rootfs is called as this is usually last task).
>
> INHERIT += "rm_work"
>
> This will remove contents of WORKDIR after recipe's build. With this
> enabled (and downloads outsite of tmp/) you can probably fit in tmpfs on
> your machine.

Ah, thanks - I forgot about that. It saves about 16GB of space and 
running on tmpfs saves about 30 minutes on my setup. Thanks!

--
Ken