Hi Graeme,

On 26 November 2014 at 09:44, G Gregory graeme.gregory@linaro.org wrote:

Hi,

So over the last couple of days I have been looking into the test suites we currently have available in the leg-kernel CI/test cycle and come to the following conclusions.

1. aapits - tests the core of the acpica code, does not test anything

in the kernel or the BIOS. Is unmaintained by acpica folks and Tomasz patches have sat idle on bug tracker for 3/4 of a year. This test suite does not test anything we are interested in I propose we drop it from CI.

2. acpica-tools - tests the iasl compiler, is run on every tools

packaging for rpm/deb. I suggest the tools CI/packaging cycles are a better location for these tests and we drop them from CI

3. acpi-smoke-test - This is the basic test that we booted from ACPI,

I think this is useful to keep as a quick getout of testing if it fails.

4. FWTS - current target of our testing for BIOS testing. This we keep.

So I am suggesting we only have tests that exercise the kernel or BIOS

I agree on your suggestion to focus on points #3 and #4.

in some manner. We are still lacking in tests that exercise the kernel but I have no idea what these tests should be at current time.

I would anyway turn your very last sentence into a constructive one and suggest to add those tests from the Ubuntu image that Mark Brown is running on LSK. Specifically, if it makes sense (excerpt from his doc):

++++++++++++

Basic smoke tests ==============

Purpose:

Validate that basic command line tools work properly

Steps:

Expected outcome is that the program exit code is 0

pwd

uname -a

vmstat

ifconfig -a

lscpu

lsusb

lsb_release -a

Basic networking tests =================

Purpose:

Validate that basic wired networking works well

Steps:

Expected outcome is that the program exit code is 0

netstat -an

ifconfig -a

route

ifconfig lo up

route

ifconfig eth0 up

ifconfig eth0 down

dhclient -v eth0

route

ping -c 5 192.168.1.10

apt-get update

apt-get install -q -y curl

curl -x http://192.168.1.10:3128 http://samplemedia.linaro.org/MPEG4/big_buck_bunny_720p_MPEG4_MP3_25fps_3300...

/dev/null

**** maybe we can skip the video playback, we are not concerned by tablet/mobile :-)

LTP ===

Linux test project description can be found at http://ltp.sourceforge.net/

Purpose:

Verify that the syscalls work properly

Steps:

Download latest LTP sources (currently ltp-full-20130503)

Build the sources on the target device

Run tests: ./runltp -f syscalls -p -q

+++++++

Would this help increase confidence on the ACPI kernel for ARM?

which subset would you recommend?

Thanks

Graeme

thanks!

Andrea