Platform tests for kernel CI-loop

[Deepak Saxena]

Hi all,

This session is scheduled for tomorrow at 9am in Grand Sierra H. I
hope to see all the LT leads there.

On 11 October 2011 16:44, Deepak Saxena <dsaxena at linaro.org> wrote:
> Hi all,
>
> I have created a blueprint [1] for connect to discuss the
> topic of platform level testing and want us to have some
> sense of what we want before going into a face to face
> discussion. I would like all the landing team leads
> to attend this session, so I've marked you as essential.
>
> The overall problem we are trying to solve is how
> do we ensure that hardware enablement (USB,
> MMC, etc) does not break across kernel versions.
> We've had several cases where a patch got merged
> into kernel.org that broke a device driver and
> we didn't catch it until just before our release.
> This causes us to scramble and reduces the
> quality of the work we're delivering to our members.
> With the CI loop in place, we now have the opportunity
> to catch these type of issues early on, before they
> even make it into the -rc kernels. My hope is that
> by the end of the summit session, we have a enough
> information to go back and develop a high level
> roadmap of  test cases to deliver. The questions I'd
> like us to think about before the connect session include:
>
> 1) What are the different devices on each board that
>    we want to test and how do we test them?
>
>   We need to go through board by board and determine
>   which I/O devices can be tested and come up with
>   common methods to test them across all our existing
>   platforms. One of the questions that comes up for me
>   is what level of testing do we want? We can do simple
>   discovery tests such as look for sysfs device nodes and
>   poke at values in there or we can right small applications
>   that test specific functionality (mounting block devices
>   and running benchmark tests and also testing ioctls for
>   example).
>
> 2) Who develops these tests?
>
>    I think the answer to this is to have Landing Team
>    engineers developing the board-specific tests with a
>    KWG engineer assigned to coordinate overall direction
>    of the work.
>
> 3) Do we integrate those tests into an existing
>    framework (LTP for example) or develop a
>    separate framework for these tests? Related
>    to this question is whether some of the vendors
>    will provide us test cases we can just pull into
>    Lava?
>
> ~Deepak
>
> [1] https://blueprints.launchpad.net/linux-linaro/+spec/linaro-kernel-platform-testing
>