Hi All,
So I’ve spent a little time checking into how one might take advantage of ‘useful’ test cases which are part of various distro packages.
Specifically this often runs the make check or akin target.
These are often wired into the various package build systems that distros have. The nice thing about using this mechanism is:
1) the package already has been base lined.
2) packages run a range of completely useless tests for testing the kernel (like checking internal apis to a package) to things which actually trigger kernel activity like running valgrind or driving some network activity.
So in my case I’ve been using gentoo to dork around since gentoo has a pretty nice process and happily is able to email testing results. More about why this is a good attribute in a moment.
The use scenario ends up looking like:
RC released -> build/deploy new kernel -> emerge package_foo; email package_foo result; emerge package_bar; email package_bar result; post process results and integrate into qa-reports.
Where this might be viewed as kind of weird is this class of testing really doesn’t belong in LAVA. One doesn’t need (or want) to do whole fresh image deployment. I know what you’re thinking, the system will be dirty after a run is made so one MUST do a redeploy or the results will be invalid.
That might be the case for some environments, but it’s not the case for gentoo. Package build and test is already ‘clean’ for the purposes of a system under test because in the land of gentoo we sandbox. If we didn’t distro package testing would be incredibly painful. So really the only ‘danger’ is that the local environment does need to build and boot a to the kernel under test but that’s not a big deal, since if something fails to build/boot, well kernelci will complain. This isn’t about testing that type of situation.
Anyway I should have some test test 4.14 results in a few hours as some examples. I’m thinking an upstream report is probably super super simple, unless a regression pops up.
Comments?
Tom
> On Mon, Oct 16, 2017 at 09:48:30AM +0000, Linaro QA wrote:
>> Summary
>> ------------------------------------------------------------------------
>>
>> kernel: 4.4.92
>> git repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
>> git branch: linux-4.4.y
>> git commit: 69f53f5d37d53ba17ca744947226b4cdadb90c13
>> git describe: v4.4.92
>> Test details: https://qa-reports.linaro.org/lkft/linux-stable-rc-4.4-oe/build/v4.4.92
>>
>> Regressions (compared to build v4.4.91-48-gc0489dd5896d)
>> ------------------------------------------------------------------------
>>
>> x15 - arm:
>> ltp-syscalls-tests:
>> * fcntl36
>>
>> * test src: git://github.com/linux-test-project/ltp.git
>
> Hm, what does this mean? Did this regress due to a breakage of your
> system, as this link:
> https://qa-reports.linaro.org/lkft/linux-stable-rc-4.4-oe/build/v4.4.92/tes…
> shows all "No log available for this test" :(
No test logs are expected with current status of qa-reports.
We have complete log for ltp syscalls runs in a single file and work
in progress for split logs for each test case.
Here is the log for the failure test case.
fcntl36.c:303: INFO: OFD read lock vs OFD write lock
fcntl36.c:381: PASS: Access between threads synchronized
fcntl36.c:303: INFO: OFD write lock vs POSIX write lock
fcntl36.c:381: PASS: Access between threads synchronized
fcntl36.c:303: INFO: OFD read lock vs POSIX write lock
fcntl36.c:205: FAIL: Unexpected data offset 12304 value 9
fcntl36.c:205: FAIL: Unexpected data offset 13568 value 47
fcntl36.c:303: INFO: OFD write lock vs POSIX read lock
fcntl36.c:381: PASS: Access between threads synchronized
fcntl36.c:303: INFO: OFD write lock vs OFD write lock
fcntl36.c:381: PASS: Access between threads synchronized
fcntl36.c:303: INFO: OFD r/w lock vs POSIX write lock
fcntl36.c:381: PASS: Access between threads synchronized
fcntl36.c:303: INFO: OFD r/w lock vs POSIX read lock
fcntl36.c:381: PASS: Access between threads synchronized
Summary:
passed 6
failed 2
skipped 0
warnings 0
Linux version 4.4.92 (buildslave@x86-64-08) (gcc version 6.2.1
20161016 (Linaro GCC 6.2-2016.11) ) #1 SMP Thu Oct 12 10:59:44 UTC
2017
LAVA job id:
https://lkft.validation.linaro.org/scheduler/job/46680#L2847
Bug reported for this failure please refer this link.
LKFT: 4.4: X15: LTP syscall test case fcntl36.c:205: FAIL: Unexpected
data offset 12304 value 9
https://bugs.linaro.org/show_bug.cgi?id=3339
This bug is not consistence,
I have re tested fcntl36.
1. test PASS in independent run "./fcntl36"
2. test FAIL with "strace ./fcntl36"
Re-tested log,
https://lkft.validation.linaro.org/scheduler/job/47795#L904
Pastebin log for above case if you do not like full boot log.
https://pastebin.com/XWuSDaMe
- Naresh
>
> confused.
>
> greg k-h