Right now I am focused on the mocking and getting that to work. I am still have some very strange issues with mocking some function calls including kernel panics. Your plan seems logical but just how useful is it without mocking? -----Original Message----- From: Brendan Higgins [mailto:brendanhiggins@google.com] Sent: Monday, September 23, 2019 5:42 PM To: David Gow davidgow@google.com Cc: David Chiang davidchiang@google.com; Siebert, David (US) David.Siebert@L3Harris.com; Kees Cook keescook@chromium.org; Mike Salvatore mike.salvatore@canonical.com; Pei Huang peihuang@google.com; Sagi Shahar sagis@google.com; Sangsu Ha sangsu.ha@samsung.com; Shuah Khan shuah@kernel.org; kunit-dev@googlegroups.com; open list:KERNEL SELFTEST FRAMEWORK linux-kselftest@vger.kernel.org Subject: Re: kunit: what do we do with the 'kunit/alpha/master' branch?
On Tue, Sep 17, 2019 at 11:41 AM David Gow davidgow@google.com wrote:
TL;DR: We expect KUnit to be accepted upstream into Linus' branch in the next week or two, and we now need to figure out what we are going to do with our non-upstream 'kunit/alpha/master' branch.
Given that it has been about a week and we haven't heard any comments, complaints, or concerns about this. I assume that there are no strong opinions against this, and people will be generally okay with this strategy.
As mentioned previously, we are expecting to see KUnit make it into torvalds/master this merge window (the next week or so), so we will probably update/rename kunit/alpha/master shortly thereafter.
Cheers
Hello everyone,
We've put together a rough proposal of what we should do with our not-upstream branch, known to most people using it as 'kunit/alpha/master'[1], now that KUnit's acceptance into mainline appears to be imminent (the KUnit MVP patchset is now in linux-next, and the merge window just opened).
========== Background ==========
KUnit development is currently split between two versions: the 'kunit/alpha/master'[1] git branch, and the version being submitted to the upstream Linux kernel. While there are some good reasons to continue to have two separate versions of KUnit, at present there is some uncertainty around the difference between these versions, and in which circumstances each version is useful.
At present, the 'kunit/alpha/master' branch serves a few different purposes. It is a place for code not-yet-ready for upstream -- such as the mocking framework -- while being developed, while also acting as a stable version for customers who do not wish to follow along with the changes made during the upstreaming process. Adding to the confusion, the name 'kunit/alpha/master' refers to an early (alpha) version of KUnit, and the version of KUnit being upstreamed has now diverged significantly from this version, requiring significant differences in documentation, and requiring a number of changes to tests when porting from one version to the other. Finally, it is not clear how the 'kunit/alpha/master' version should evolve as features it contains are upstreamed.
On the other hand, the version being upstreamed has its own complications. It contains significantly fewer features (as features such as the mocking frameworks will be upstreamed individually), and so is less useful for the average customer. Until each feature is upstreamed, it is iterated on rapidly to address comments from the kernel community, so in-progress features are not stable enough to reasonably build on. Finally, it exists only as a set of patches on mailing lists, rather than as a maintained git repository (due to the fact that the patches themselves are changing rapidly), making it difficult for early adopters to incorporate into their own trees.
Whilst we believe there to be enough (at times conflicting) goals above to justify having multiple versions of KUnit, we want to ensure that they are meeting their goals, and that we have a process to ensure that code finds its way into the correct version, that we can deprecate and remove failed experiments or superseded versions, and that we can keep pace with upstream kernel releases.
============ The Proposal ============
We propose having two tracks of development: the upstream kernel (comprising both code that has been upstreamed, and code which is in the process of being upstreamed -- i.e. is being reviewed on the mailing lists), and an 'experimental' branch, which contains features which are yet to be submitted upstream.
Periodically, the 'experimental' branch will be replaced with a new 'experimental' branch based on the latest upstream version. This will allow us to focus on code based on upstream versions by deprecating earlier, experimental versions of features. The older experimental branch will stay around, but receive no new features or support: customers will need to eventually migrate to either the upstream kernel (if all of the features they require have been upstreamed), or to the latest 'experimental' branch (if they still require non-upstreamed features).
Where practical, 'experimental' branches will try to provide a stable interface (and/or compatibility layers/shims) such that users can write tests against an experimental version and expect our changes to break them infrequently. We don't intend to make any solid guarantee, however: so users will need to be able to deal with some breakage.
We expect most changes to KUnit to be submitted directly upstream, and reserve the 'experimental' branch for features where: a) the design is yet to be proven by early adopters, and is likely to change; b) the feature is large enough that having a solid staging environment in which several developers can work is necessary; and/or c) it depends on existing features only available in the 'experimental' branch.
An example of features suited to the 'experimental' branch are the mocking systems[2].
===================== Lifetime of a feature =====================
A new feature (that for one of the above reasons may require some experimentation before being sent upstream) would typically be developed as follows:
- An initial version of the feature is developed against the latest
'experimental' branch. 2. The feature is tested by users of the 'experimental' branch. 3. It is revised as needed within the 'experimental' branch. If breaking changes are required, and supporting some form of compatibility layer is practical, such a layer is implemented to avoid breaking existing 'experimental' customers. 4. Once the feature is deemed ready, it is submitted to the mailing lists for the upstream kernel. Any compatibility layers for earlier experimental version are removed -- this is a 'clean' version of the change. 5. Development of the change continues on the kernel mailing lists: any particularly important changes or bugfixes are ported to the 'experimental' branch if feasible, but complete convergence is not expected at this stage. 6. When the feature is accepted upstream, a new 'experimental' branch is created based on the new upstream kernel. Development on the old 'experimental' branch ceases. 7. The upstream version of the feature has now superseded the experimental version. All future development is done based on the upstream version, either in the new 'experimental' branch, or directly against the upstream kernel. 8. Any customers still using the experimental version of the feature will need to update their code to work with the upstream version in order to update to the new (supported) 'experimental' branch, or to the upstream kernel.
=========================== What does this mean for me? ===========================
...as a user?
If you're using the upstream version of KUnit, nothing! Features will continue to be pushed upstream when they're ready.
If you're using the 'kunit/alpha/master' branch, then you'll need to switch to the new 'experimental-<version>' branches. Every so often, we'll create a new 'experimental-<version>' branch, and you'll need to rebase your code on top of that to keep receiving updated features and bugfixes. While we intend to make this as painless as possible, there may be breaking changes which require you to rework some of your tests.
Once a feature exists in the upstream kernel, the version of that feature in the 'experimental' branches should stabilise somewhat.
...as a KUnit developer?
For most changes/bugfixes which apply cleanly to the upstream version, you should directly submit these upstream as before. If it is feasible and useful to backport them to the latest 'experimental' branch (e.g., they are bugfixes to code that the experimental branch already has), please do so.
For more major features (or anything which depends on yet-to-be-upstreamed code), you should develop them in the latest 'experimental' branch. For the most part, you can treat this the same way as the old 'kunit/alpha/master' branch, but note that changes will need to be rebased on top of a newer experimental branch periodically. It may make sense to use this as an opportunity to tidy the code up somewhat: by removing compatibility code for deprecated features, or to merge 'fixup' commits. Once the code has been upstreamed, the next rebase of the 'experimental' branch should only use the upstream version: there should not be a separate 'experimental' version of upstream code save for the development of a new 'experimental' feature or bugfix.
For more details, see the 'Lifetime of a feature' section above.
============= In Conclusion =============
We'll continue to be working with upstream as before, but we intend to rename and more explicitly specify the purpose of the 'kunit/alpha/master' branch as above. This will require a bit more work from people using and developing against this 'experimental' branch, but allows us to continue to use it for features such as the mocking framework while limiting its divergence from upstream.
Do let us know if this sounds reasonable to you, if it will affect your use case, or if you have any questions.
Cheers, -- David & Brendan
[1] https://kunit.googlesource.com/linux/+/refs/heads/kunit/alpha/master [2] https://kunit.googlesource.com/linux/+/kunit/alpha/master/Documentatio n/test/usage.rst#187
CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of the intended recipient and may contain material that is proprietary, confidential, privileged or otherwise legally protected or restricted under applicable government laws. Any review, disclosure, distributing or other use without expressed permission of the sender is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies without reading, printing, or saving.