On Thu Feb 29, 2024 at 12:55 AM EET, Helen Koike wrote:
Dear Kernel Community,
This patch introduces a `.gitlab-ci` file along with a `ci/` folder, defining a basic test pipeline triggered by code pushes to a GitLab-CI instance. This initial version includes static checks (checkpatch and smatch for now) and build tests across various architectures and configurations. It leverages an integrated cache for efficient build times and introduces a flexible 'scenarios' mechanism for subsystem-specific extensions.
tl;dr: check this video to see a quick demo: https://youtu.be/TWiTjhjOuzg, but don't forget to check the "Motivation for this work" below. Your feedback, whether a simple thumbs up or down, is crucial to determine if it is worthwhile to pursue this initiative.
GitLab is an Open Source platform that includes integrated CI/CD. The pipeline provided in this patch is designed to work out-of-the-box with any GitLab instance, including the gitlab.com Free Tier. If you reach the limits of the Free Tier, consider using community instances like https://gitlab.freedesktop.org/. Alternatively, you can set up a local runner for more flexibility. The bootstrap-gitlab-runner.sh script included with this patch simplifies this process, enabling you to run tests on your preferred infrastructure, including your own machine.
For detailed information, please refer to the documentation included in the patch, or check the rendered version here: https://koike.pages.collabora.com/-/linux/-/jobs/298498/artifacts/artifacts/... .
Motivation for this Work
We all know tests are a major topic in the community, so let's mention the specificities of this approach:
- **Built-in User Interface:** GitLab CI/CD is growing in popularity and has an
user-friendly interface. Our experience with the upstream DRM-CI in the kernel tree (see this blog post [https://www.collabora.com/news-and-blog/blog/2024/02/08/drm-ci-a-gitlab-ci-p...] ) has provided insights into how such a system can benefit the wider community.
- **Distributed Infrastructure:**
The proposed GitLab-CI pipeline is designed with a distributed infrastructure model, being possible to run in any gitlab instance.
- **Reduce regressions:** Fostering a culture where people habitually run
validated tests and post their results can prevent many issues in post-merge tests.
- **Collaborative Testing Environment:** The kernel community is already
engaged in numerous testing efforts, including various GitLab-CI pipelines such as DRM-CI, which I maintain, along with other solutions like KernelCI and BPF-CI. This proposal is designed to further stimulate contributions to the evolving testing landscape. Our goal is to establish a comprehensive suite of common tools and files.
- **Ownership of QA:**
Discrepancies between kernel code and outdated tests often lead to misattributed failures, complicating regression tracking. This issue, often arising from neglected or deprioritized test updates, creates uncertainty about the source of failures. Adopting an "always green pipeline" approach, as detailed in this patch's documentation, encourages timely maintenance and validation of tests. This ensures that testing accurately reflects the current state of the kernel, thereby improving the effectiveness of our QA processes.
Additionally, if we discover that this method isn't working for us, we can easily remove it from the codebase, as it is primarily contained within the ci/ folder.
Not to criticize but I can do tests I ever want with either Github or Gitlab simply by bootstrapping BuildRoot on top of whatever the CI runner has. So I essentially need just enough deps to make a BR build, and that's it. And e.g. could run x86 tests on ARM ISA runner with zero issues. And can even have emulated TPM chip in the QEMU VM by building swtpm.
I had this for some time running actually Gitlab runner. It does not currently build QEMU but then it also did that:
https://gitlab.com/jarkkojs/linux-tpmdd-test
Essentially just executing this sequence:
git clone https://gitlab.com/jarkkojs/linux-tpmdd-test.git cd linux-tpmdd-test cmake -Bbuild && make -Cbuild buildroot-prepare make -Cbuild/buildroot/build build/buildroot/build/images/run-tests.sh
I use TCL's "expect" to make conclusions from the output :-)
I'm assuming that this has a bigger point that I can understand right now but makes me a bit puzzled given that it is quite trivial problem to my understanding (if you want to pursue to it). Like one work week maybe but not more than that...
Especially it feels weird that it needs kernel to be patched at all and when I did read the motivation but it has sort of whitepaperish stuff that does not really explain me the edge of this compared to e.g. to my ad-hoc but still very usable solution (which is agnostic to runner's CPU architecture, can emulated hardware and works in any possible hosting with CI).
So maybe my review comment it this: do not assume that this would be entirely new thing. It is not, and I'm sure some other people have done this too in the past rather than just me. Instead this should explain why this is so great that even kernel tree needs to be patched?
BR, Jarkko