Hi folks,
During LPC Linus brought up a concern that linux-next is too lightly tested, and pull requests that end up reaching him still might be functionally broken (and kill his laptop).
linux-next works great as an integration branch as far as conflicts between trees and build issues go, but it's volatile state makes it very difficult to test it. For example, a single for-next branch that wasn't tested can ruin testing for everyone else.
I've proposed a new "linus-next" tree which is composed of pull requests that were sent to Linus. This (in theory) will give us a higher quality tree that will be composed only of pull requests that were sent to Linus but not picked up yes. Basically, we'll be frontrunning Linus as he goes on with his merges.
The tree is located at git://git.kernel.org/pub/scm/linux/kernel/git/sashal/linus-next.git and the branch is 'linus-next'.
As far as testing goes, I think that it should be equivalent to what Linus's tree gets, but I'm open to suggestions.
Thank you!
Hi Sasha,
We're happy to assist from the LKFT side.
Our main focus is validating LTS kernels, while testing mainline and next kernels remains more of a best-effort approach. We'll start by enabling the same set of builds and running them on virtual hardware, then move on to boot tests on the DUTs. If needed, we can discuss expanding DUT testing later.
We'll run this once per day to help manage costs.
Regarding the merge SHAs - will Linus be pulling yours into his tree? If not, we might end up testing the same kernel with different SHAs. Are you two coordinating to ensure consistency?
Long term, is the plan to shift testing from Linus' tree to yours, once we've ramped up on your side?
Also, how would you like to collaborate on regression reports?
Just trying to get a better sense of the workload.
Thanks for your efforts on this!
Cheers, Anders
On Mon, 23 Sept 2024 at 18:59, Sasha Levin sashal@kernel.org wrote:
Hi folks,
During LPC Linus brought up a concern that linux-next is too lightly tested, and pull requests that end up reaching him still might be functionally broken (and kill his laptop).
linux-next works great as an integration branch as far as conflicts between trees and build issues go, but it's volatile state makes it very difficult to test it. For example, a single for-next branch that wasn't tested can ruin testing for everyone else.
I've proposed a new "linus-next" tree which is composed of pull requests that were sent to Linus. This (in theory) will give us a higher quality tree that will be composed only of pull requests that were sent to Linus but not picked up yes. Basically, we'll be frontrunning Linus as he goes on with his merges.
The tree is located at git://git.kernel.org/pub/scm/linux/kernel/git/sashal/linus-next.git and the branch is 'linus-next'.
As far as testing goes, I think that it should be equivalent to what Linus's tree gets, but I'm open to suggestions.
Thank you!
-- Thanks, Sasha
On Tue, Sep 24, 2024 at 04:42:17PM +0200, Anders Roxell wrote:
Hi Sasha,
We're happy to assist from the LKFT side.
Our main focus is validating LTS kernels, while testing mainline and next kernels remains more of a best-effort approach. We'll start by enabling the same set of builds and running them on virtual hardware, then move on to boot tests on the DUTs. If needed, we can discuss expanding DUT testing later.
We'll run this once per day to help manage costs.
Thank you!
For more context, it would inderectly help the LTS trees as well: we often end up releasing commits that came vie pull requests to Linus during the merge window right after -rc1 is released, so this process will help improve the testing story behind those.
Regarding the merge SHAs - will Linus be pulling yours into his tree? If not, we might end up testing the same kernel with different SHAs. Are you two coordinating to ensure consistency?
Not yet, this is "experimental" to see if it catches real issues that Linus would have missed. It's very possible that a year for now I'll be asking to remove this tree :)
Long term, is the plan to shift testing from Linus' tree to yours, once we've ramped up on your side?
In a sense: I think that the longer term is to prevent Linus from pulling branches that would break tests, so it would make sense to focus more in linus-next
Also, how would you like to collaborate on regression reports?
What works best for you here? I don't really mind looking at mails/dashboards/etc.
Just trying to get a better sense of the workload.
Thanks for your efforts on this!
Thanks for looking into this!
Hi Sasha,
I appreciate there has been delay in my response. As we discussed on IRC, you can check the results here: https://qa-reports.linaro.org/lkft/sashal-linus-next. Let us know if you need help navigating it—it’s not the most intuitive dashboard.
We can explore a text-based report later if needed.
We want to ensure this service adds value. If it does, we'd be grateful if you could share your positive experience. If it doesn’t meet your needs, please let us know so we can consider whether to continue offering it.
Cheers, Anders
On Tue, 24 Sept 2024 at 22:42, Sasha Levin sashal@kernel.org wrote:
On Tue, Sep 24, 2024 at 04:42:17PM +0200, Anders Roxell wrote:
Hi Sasha,
We're happy to assist from the LKFT side.
Our main focus is validating LTS kernels, while testing mainline and next kernels remains more of a best-effort approach. We'll start by enabling the same set of builds and running them on virtual hardware, then move on to boot tests on the DUTs. If needed, we can discuss expanding DUT testing later.
We'll run this once per day to help manage costs.
Thank you!
For more context, it would inderectly help the LTS trees as well: we often end up releasing commits that came vie pull requests to Linus during the merge window right after -rc1 is released, so this process will help improve the testing story behind those.
Regarding the merge SHAs - will Linus be pulling yours into his tree? If not, we might end up testing the same kernel with different SHAs. Are you two coordinating to ensure consistency?
Not yet, this is "experimental" to see if it catches real issues that Linus would have missed. It's very possible that a year for now I'll be asking to remove this tree :)
Long term, is the plan to shift testing from Linus' tree to yours, once we've ramped up on your side?
In a sense: I think that the longer term is to prevent Linus from pulling branches that would break tests, so it would make sense to focus more in linus-next
Also, how would you like to collaborate on regression reports?
What works best for you here? I don't really mind looking at mails/dashboards/etc.
Just trying to get a better sense of the workload.
Thanks for your efforts on this!
Thanks for looking into this!
-- Thanks, Sasha
-
Anders Roxell -
Sasha Levin