Btw, not sure if any of you have seen the 0-day kbuild setup that intel has..
https://lists.01.org/mailman/listinfo/kbuild
runs various builds for different archs on every commit with different configs, randconfig, etc. And various checks with sparse, smatch, etc. Seems kinda useful, and would be a worthwhile goal to get arm arch to the point of "it just compiles and boots" like x86 is, vs arm which has a lot higher tendency to be broken if you don't have the right kernel config, etc. I guess on x86, they boot test all the kernels too on VMs. Perhaps we could go one better with something tied in to lava?
BR, -R
On Sun, Jan 20, 2013 at 08:57:51AM -0600, Rob Clark wrote:
Btw, not sure if any of you have seen the 0-day kbuild setup that intel has..
https://lists.01.org/mailman/listinfo/kbuild
runs various builds for different archs on every commit with different configs, randconfig, etc. And various checks with sparse, smatch, etc. Seems kinda useful, and would be a worthwhile goal to get arm arch to the point of "it just compiles and boots" like x86 is, vs arm which has a lot higher tendency to be broken if you don't have the right kernel config, etc. I guess on x86, they boot test all the kernels too on VMs. Perhaps we could go one better with something tied in to lava?
I know Vince Sanders (in CC) used to have something like this set up, regularly building lots of variations of ARM kernels back when he was at Simtec. Maybe he could help / give advice...
Cheers,
On 01/20/2013 06:57 AM, Rob Clark wrote:
Btw, not sure if any of you have seen the 0-day kbuild setup that intel has..
https://lists.01.org/mailman/listinfo/kbuild
runs various builds for different archs on every commit with different configs, randconfig, etc. And various checks with sparse, smatch, etc. Seems kinda useful, and would be a worthwhile goal to get arm arch to the point of "it just compiles and boots" like x86 is, vs arm which has a lot higher tendency to be broken if you don't have the right kernel config, etc. I guess on x86, they boot test all the kernels too on VMs. Perhaps we could go one better with something tied in to lava?
+1 on this, Rob.
Just about every time I go to bump kernel revisions for the linaro.android tree, and actually try to build and boot on a board, I regularly hit compile and boot issues.
Although for the boot issues, there's still the hard problem for hardware-simpletons like me in being able to figure out what configs I actually need to enable (especially as those options change over time). I suspect these sorts of problems are unlikely to be something we can test away, and instead need a more critical eye as to how the kconfig options are added (so there's less of a random set of options that you have to pick the right combination, and more of the existing physical dependencies being properly stating in the kconfig - but I realize this is more complicated to do then say).
thanks -john
On Mon, Jan 21, 2013 at 12:11 PM, John Stultz john.stultz@linaro.org wrote:
On 01/20/2013 06:57 AM, Rob Clark wrote:
Btw, not sure if any of you have seen the 0-day kbuild setup that intel has..
https://lists.01.org/mailman/listinfo/kbuild
runs various builds for different archs on every commit with different configs, randconfig, etc. And various checks with sparse, smatch, etc. Seems kinda useful, and would be a worthwhile goal to get arm arch to the point of "it just compiles and boots" like x86 is, vs arm which has a lot higher tendency to be broken if you don't have the right kernel config, etc. I guess on x86, they boot test all the kernels too on VMs. Perhaps we could go one better with something tied in to lava?
+1 on this, Rob.
Just about every time I go to bump kernel revisions for the linaro.android tree, and actually try to build and boot on a board, I regularly hit compile and boot issues.
Although for the boot issues, there's still the hard problem for hardware-simpletons like me in being able to figure out what configs I actually need to enable (especially as those options change over time). I suspect these sorts of problems are unlikely to be something we can test away, and instead need a more critical eye as to how the kconfig options are added (so there's less of a random set of options that you have to pick the right combination, and more of the existing physical dependencies being properly stating in the kconfig - but I realize this is more complicated to do then say).
well, at least for build test, randconfig is actually good for tracking down missing 'select XYZ's.. although I suppose I wouldn't expect every random config to boot, at least not in a meaningful way..
BR, -R
thanks -john
linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
On 01/20/2013 08:57 AM, Rob Clark wrote:
Btw, not sure if any of you have seen the 0-day kbuild setup that intel has..
https://lists.01.org/mailman/listinfo/kbuild
runs various builds for different archs on every commit with different configs, randconfig, etc. And various checks with sparse, smatch, etc. Seems kinda useful, and would be a worthwhile goal to get arm arch to the point of "it just compiles and boots" like x86 is, vs arm which has a lot higher tendency to be broken if you don't have the right kernel config, etc. I guess on x86, they boot test all the kernels too on VMs. Perhaps we could go one better with something tied in to lava?
Seems pretty interesting. This might be a nice joint-effort thing we(LAVA) could do with the infrastructure team.
On 22 January 2013 04:09, Andy Doan andy.doan@linaro.org wrote:
On 01/20/2013 08:57 AM, Rob Clark wrote:
Btw, not sure if any of you have seen the 0-day kbuild setup that intel has..
https://lists.01.org/mailman/listinfo/kbuild
runs various builds for different archs on every commit with different configs, randconfig, etc. And various checks with sparse, smatch, etc. Seems kinda useful, and would be a worthwhile goal to get arm arch to the point of "it just compiles and boots" like x86 is, vs arm which has a lot higher tendency to be broken if you don't have the right kernel config, etc. I guess on x86, they boot test all the kernels too on VMs. Perhaps we could go one better with something tied in to lava?
Seems pretty interesting. This might be a nice joint-effort thing we(LAVA) could do with the infrastructure team.
Interesting...
My first reaction to this was "sounds like a CI problem", so we should make sure that our CI/LAVA tools can cope. Of course, working on CI means that everything looks like a CI problem :-)
I don't know if LAVA has priority queues, but this sounds like a good candidate for a low priority task; use spare compute time to automatically experiment, while users specific builds and tests get priority.
I expect that we will need some decent logic to do constrained random generation of configurations since there is no point testing configurations that definitely won't work. The logic would be easy enough, but the constraints need to be written. Do we have this information? If not, would the people who have it be able to provide it in a machine readable form?
It also sounds like a workflow problem waiting to be solved: * Claim triage of issue. * Which project does the bug belong to? o CI config generation o Kernel o Build tools o ... * Create a bug report, seeding it with data from the build. * Mark issue as one of: o Triaged o Viewed (I looked at this and I don't know what to do with it kick it back out into the triage queue, but for me, show as viewed).
I imagine that without the right UI to assist in these steps (+ any automation we can do) this will turn into a nice idea that gets little use. I am all for it as long as we can do it properly. It looks a lot like an email interface, but I like the idea of claiming an issue to avoid duplication of effort and claim/un-claim to be an action that isn't a mailing list announcement.
James Tunnicliffe james.tunnicliffe@linaro.org writes:
On 22 January 2013 04:09, Andy Doan andy.doan@linaro.org wrote:
On 01/20/2013 08:57 AM, Rob Clark wrote:
Btw, not sure if any of you have seen the 0-day kbuild setup that intel has..
https://lists.01.org/mailman/listinfo/kbuild
runs various builds for different archs on every commit with different configs, randconfig, etc. And various checks with sparse, smatch, etc. Seems kinda useful, and would be a worthwhile goal to get arm arch to the point of "it just compiles and boots" like x86 is, vs arm which has a lot higher tendency to be broken if you don't have the right kernel config, etc. I guess on x86, they boot test all the kernels too on VMs. Perhaps we could go one better with something tied in to lava?
Seems pretty interesting. This might be a nice joint-effort thing we(LAVA) could do with the infrastructure team.
Interesting...
My first reaction to this was "sounds like a CI problem",
My first reaction is "we can do this, but the hard part will be getting people to care about the results" :-)
so we should make sure that our CI/LAVA tools can cope. Of course, working on CI means that everything looks like a CI problem :-)
I don't know if LAVA has priority queues, but this sounds like a good candidate for a low priority task; use spare compute time to automatically experiment, while users specific builds and tests get priority.
It has simple support for high/medium/low priorities (thanks collabora!) but we don't really use them much. It has the potential problem that if all boards get soaked up by long low priority jobs and a high priority job comes in, it has to wait for one of the low prio jobs to finish.
I expect that we will need some decent logic to do constrained random generation of configurations since there is no point testing configurations that definitely won't work.
Well, make randconfig should always produce something that compiles, right? (I realize that today it likely _doesn't_ but that's the place to put this "constrained random generation" aiui).
The logic would be easy enough, but the constraints need to be written. Do we have this information? If not, would the people who have it be able to provide it in a machine readable form?
See above.
(My understanding might be wrong though)
It also sounds like a workflow problem waiting to be solved:
- Claim triage of issue.
- Which project does the bug belong to? o CI config generation o Kernel o Build tools o ...
- Create a bug report, seeding it with data from the build.
- Mark issue as one of: o Triaged o Viewed (I looked at this and I don't know what to do with it kick it back out into the triage queue, but for me, show as viewed).
I imagine that without the right UI to assist in these steps (+ any automation we can do) this will turn into a nice idea that gets little use. I am all for it as long as we can do it properly. It looks a lot like an email interface, but I like the idea of claiming an issue to avoid duplication of effort and claim/un-claim to be an action that isn't a mailing list announcement.
Ah, I see you did have the same worry as me then :-)
Cheers, mwh
-
Andy Doan -
James Tunnicliffe -
John Stultz -
Michael Hudson-Doyle -
Rob Clark -
Steve McIntyre