Hi, I wanted to sync up on where things are with this, and as I understand, there's some confusion still about how we should get the available kernel and/or images for testing fast models.
First off, it seems there is no way to just take a kernel .axf as we thought because the boot args are wrapped into it as well. Is there really no way to inject that stuff after build time?
Is it worth revisiting whether we should have proper hwpacks for fast models? I know there's the 2G max sd size issue with that, but if that's something arm can fix, or if we have another way around it, would that help?
Finally, I feel like we've chased a pretty messy route to get fast models supported here, which will ultimately break completely when we try to get android running also. Please correct me if I'm wrong here, but it seems the "recommended" approach is currently: lava takes as input: git tree for the kernel we want to build, defconfig to use, and rootfs tarball lava builds the kernel (lava doesn't do this currently, complicating things for us quite a bit - would be better if this step could be done externally, like in jenkins where we do the kernel ci builds) lava pushes the build axf to another system where we have the fast models software installed, provisions the rootfs for booting over nfs lava boots the system under fast models on this other machine, runs tests, gathers results, etc
Is that pretty close Zygmunt? Is there something more straightforward and less fragile that we can do here?
Thanks, Paul Larson
On Mon, Feb 13, 2012 at 9:54 PM, Paul Larson paul.larson@linaro.org wrote:
Hi, I wanted to sync up on where things are with this, and as I understand, there's some confusion still about how we should get the available kernel and/or images for testing fast models.
First off, it seems there is no way to just take a kernel .axf as we thought because the boot args are wrapped into it as well. Is there really no way to inject that stuff after build time?
As explained in my other email this is now handled by linux-system-semi.axf which is a small boot loader for the real kernel.
Is it worth revisiting whether we should have proper hwpacks for fast models? I know there's the 2G max sd size issue with that, but if that's something arm can fix, or if we have another way around it, would that help?
My current desire is to implement NFS support. We can extend that to NBD support if there is interest in such features. As for fixing SD card limit, I don't know how that works. Perhaps that's a question for ARM folks.
Finally, I feel like we've chased a pretty messy route to get fast models supported here, which will ultimately break completely when we try to get android running also. Please correct me if I'm wrong here, but it seems the "recommended" approach is currently: lava takes as input: git tree for the kernel we want to build, defconfig to use, and rootfs tarball lava builds the kernel (lava doesn't do this currently, complicating things for us quite a bit - would be better if this step could be done externally, like in jenkins where we do the kernel ci builds) lava pushes the build axf to another system where we have the fast models software installed, provisions the rootfs for booting over nfs lava boots the system under fast models on this other machine, runs tests, gathers results, etc
Is that pretty close Zygmunt? Is there something more straightforward and less fragile that we can do here?
That's pretty much it. We don't need to build the kernel assuming we're getting a pre-build kernel from somewhere with all the modules so that we can prepare a working rootfs.
ZK
Thanks, Paul Larson
But what you said earlier was that there are still a lot of complications around the nfs booting, especially with specifying where to boot from. I suspect nfs is not going to be a viable option at all for things like android. So what about revisiting why we cant use a real image before we make this more difficult than it really needs to be? I think the main thing was the image size, but since we've started looking at our main use cases for fast models, what stands out is that we really could cover our testing in nano and android - which should both fit fine in < 2GB.
Thanks, Paul On Feb 13, 2012 4:03 PM, "Zygmunt Krynicki" zygmunt.krynicki@linaro.org wrote:
On Mon, Feb 13, 2012 at 9:54 PM, Paul Larson paul.larson@linaro.org wrote:
Hi, I wanted to sync up on where things are with this, and as I
understand,
there's some confusion still about how we should get the available kernel and/or images for testing fast models.
First off, it seems there is no way to just take a kernel .axf as we
thought
because the boot args are wrapped into it as well. Is there really no
way
to inject that stuff after build time?
As explained in my other email this is now handled by linux-system-semi.axf which is a small boot loader for the real kernel.
Is it worth revisiting whether we should have proper hwpacks for fast models? I know there's the 2G max sd size issue with that, but if that's something arm can fix, or if we have another way around it, would that
help?
My current desire is to implement NFS support. We can extend that to NBD support if there is interest in such features. As for fixing SD card limit, I don't know how that works. Perhaps that's a question for ARM folks.
Finally, I feel like we've chased a pretty messy route to get fast models supported here, which will ultimately break completely when we try to get android running also. Please correct me if I'm wrong here, but it seems
the
"recommended" approach is currently: lava takes as input: git tree for the kernel we want to build, defconfig
to
use, and rootfs tarball lava builds the kernel (lava doesn't do this currently, complicating
things
for us quite a bit - would be better if this step could be done
externally,
like in jenkins where we do the kernel ci builds) lava pushes the build axf to another system where we have the fast models software installed, provisions the rootfs for booting over nfs lava boots the system under fast models on this other machine, runs
tests,
gathers results, etc
Is that pretty close Zygmunt? Is there something more straightforward and less fragile that we can do here?
That's pretty much it. We don't need to build the kernel assuming we're getting a pre-build kernel from somewhere with all the modules so that we can prepare a working rootfs.
ZK
Thanks, Paul Larson
On Mon, Feb 13, 2012 at 11:13 PM, Paul Larson paul.larson@linaro.org wrote:
But what you said earlier was that there are still a lot of complications around the nfs booting, especially with specifying where to boot from.
This is all fixed since we can manipulate kernel bootcmd arguments. We just say "root=/dev/nfs nfsroot=ip:/path" while starting the model (with a pre-built kernel)
I
suspect nfs is not going to be a viable option at all for things like android. So what about revisiting why we cant use a real image before we make this more difficult than it really needs to be? I think the main thing was the image size, but since we've started looking at our main use cases for fast models, what stands out is that we really could cover our testing in nano and android - which should both fit fine in < 2GB.
We don't have any boot loaders that work from SD. You can still keep the rootfs if you really want to but I don't see how that is more complicated than NFS setup (which we already have). Thanks ZK
Thanks, Paul
On Feb 13, 2012 4:03 PM, "Zygmunt Krynicki" zygmunt.krynicki@linaro.org wrote:
On Mon, Feb 13, 2012 at 9:54 PM, Paul Larson paul.larson@linaro.org wrote:
Hi, I wanted to sync up on where things are with this, and as I understand, there's some confusion still about how we should get the available kernel and/or images for testing fast models.
First off, it seems there is no way to just take a kernel .axf as we thought because the boot args are wrapped into it as well. Is there really no way to inject that stuff after build time?
As explained in my other email this is now handled by linux-system-semi.axf which is a small boot loader for the real kernel.
Is it worth revisiting whether we should have proper hwpacks for fast models? I know there's the 2G max sd size issue with that, but if that's something arm can fix, or if we have another way around it, would that help?
My current desire is to implement NFS support. We can extend that to NBD support if there is interest in such features. As for fixing SD card limit, I don't know how that works. Perhaps that's a question for ARM folks.
Finally, I feel like we've chased a pretty messy route to get fast models supported here, which will ultimately break completely when we try to get android running also. Please correct me if I'm wrong here, but it seems the "recommended" approach is currently: lava takes as input: git tree for the kernel we want to build, defconfig to use, and rootfs tarball lava builds the kernel (lava doesn't do this currently, complicating things for us quite a bit - would be better if this step could be done externally, like in jenkins where we do the kernel ci builds) lava pushes the build axf to another system where we have the fast models software installed, provisions the rootfs for booting over nfs lava boots the system under fast models on this other machine, runs tests, gathers results, etc
Is that pretty close Zygmunt? Is there something more straightforward and less fragile that we can do here?
That's pretty much it. We don't need to build the kernel assuming we're getting a pre-build kernel from somewhere with all the modules so that we can prepare a working rootfs.
ZK
Thanks, Paul Larson
On 13 February 2012 22:32, Zygmunt Krynicki zygmunt.krynicki@linaro.org wrote:
On Mon, Feb 13, 2012 at 11:13 PM, Paul Larson paul.larson@linaro.org wrote: I
suspect nfs is not going to be a viable option at all for things like android. So what about revisiting why we cant use a real image before we make this more difficult than it really needs to be? I think the main thing was the image size, but since we've started looking at our main use cases for fast models, what stands out is that we really could cover our testing in nano and android - which should both fit fine in < 2GB.
We don't have any boot loaders that work from SD.
Yeah. The SD approach would simply be that instead of root=/dev/nfs and a rootfs expanded out into somewhere served over nfs, you have the rootfs in an image file and say root=/dev/mmblk0p2 [or whatever]. This should be fine as long as the image can be kept under 2GB (bearing in mind that if you have the KVM host (ie fast models guest) use an SD card image then that image must contain within itself the root fs image for the KVM guest as well, so really you're working with more like a 1GB limit for each of host and guest). You also need to be able to insert the qemu-linaro binary under test into the KVM host rootfs, but that shouldn't be much harder either way I guess; slightly more faff to loopback-mount an sd image and manipulate it but not much.
I have no idea how android on fast models will work, but presumably it's still just a kernel you start with some arguments? If it doesn't start using the same kernel boot process as Linux then the boot wrapper will need significant work before you can boot an Android system at all.
-- PMM
On Mon, Feb 13, 2012 at 2:32 PM, Zygmunt Krynicki < zygmunt.krynicki@linaro.org> wrote:
On Mon, Feb 13, 2012 at 11:13 PM, Paul Larson paul.larson@linaro.org wrote:
But what you said earlier was that there are still a lot of complications around the nfs booting, especially with specifying where to boot from.
This is all fixed since we can manipulate kernel bootcmd arguments. We just say "root=/dev/nfs nfsroot=ip:/path" while starting the model (with a pre-built kernel)
I
suspect nfs is not going to be a viable option at all for things like android. So what about revisiting why we cant use a real image before we make this more difficult than it really needs to be? I think the main
thing
was the image size, but since we've started looking at our main use cases for fast models, what stands out is that we really could cover our
testing
in nano and android - which should both fit fine in < 2GB.
We don't have any boot loaders that work from SD. You can still keep the rootfs if you really want to but I don't see how that is more complicated than NFS setup (which we already have).
I think you misunderstood what I'm getting at. What I'm suggesting is
that there should just be a proper hwpack for vexpress-a7/15+fastmodel. This wouldn't have to be supported forever, just until we have real hardware that can be used. My understanding from the discussion before is that this would be possible, but might require some modifications to the bootloader. I suspect this modification would be needed for the forthcoming android image as well though. Once we had this hwpack, it could be used to create a regular sd image that can be booted in fast models, with the restriction that it cannot be > 2GB. nano and android should both fit well within that constraint though, and would be adequate for the testing that they want to do in fast models by way of lava.
The benefit here, is that we don't have to go through all of this again to cope with android in a few {days, weeks, months?} when we have something for android.
If ARM can fix the 2GB limit, then what that means is *if* we someday need to test using a larger image, we can. But for now, it doesn't seem to be critical. I did mention this to Roger at the connect, but I'll also pursue this further with ARM just in case.
-Paul Larson
On Mon, Feb 13, 2012 at 11:43 PM, Paul Larson paul.larson@linaro.org wrote:
On Mon, Feb 13, 2012 at 2:32 PM, Zygmunt Krynicki zygmunt.krynicki@linaro.org wrote:
On Mon, Feb 13, 2012 at 11:13 PM, Paul Larson paul.larson@linaro.org wrote:
But what you said earlier was that there are still a lot of complications around the nfs booting, especially with specifying where to boot from.
This is all fixed since we can manipulate kernel bootcmd arguments. We just say "root=/dev/nfs nfsroot=ip:/path" while starting the model (with a pre-built kernel)
I
suspect nfs is not going to be a viable option at all for things like android. So what about revisiting why we cant use a real image before we make this more difficult than it really needs to be? I think the main thing was the image size, but since we've started looking at our main use cases for fast models, what stands out is that we really could cover our testing in nano and android - which should both fit fine in < 2GB.
We don't have any boot loaders that work from SD. You can still keep the rootfs if you really want to but I don't see how that is more complicated than NFS setup (which we already have).
I think you misunderstood what I'm getting at. What I'm suggesting is that there should just be a proper hwpack for vexpress-a7/15+fastmodel.
Sure, if we do then this is good. But this would be solving a problem that does not exist at all with NFS based approach. We just need to figure out which tree is the tree that has our A15 kernel. We build those, stick them out as "hwpack" or whatever (it will never see l-m-c fortunately). Then LAVA gets to use it as is.
The alternative is to venture on a bootloader development quest to support booting from SD. Use QEMU (which will not work on ARMv8 unless someone fixed that already) to create a pre-installed image. Lastly LAVA would consume this image, manipulate it further (which _is_ annoying as oneiric kernel has no support for partitions on loopback devices) only to spawn fast models with a SD card image instead of unpacked root filesystem.
This
wouldn't have to be supported forever, just until we have real hardware that can be used. My understanding from the discussion before is that this would be possible, but might require some modifications to the bootloader.
I cannot judge how easy/hard this task is.
I
suspect this modification would be needed for the forthcoming android image as well though. Once we had this hwpack, it could be used to create a regular sd image that can be booted in fast models, with the restriction that it cannot be > 2GB. nano and android should both fit well within that constraint though, and would be adequate for the testing that they want to do in fast models by way of lava.
The benefit here, is that we don't have to go through all of this again to cope with android in a few {days, weeks, months?} when we have something for android.
If ARM can fix the 2GB limit, then what that means is *if* we someday need to test using a larger image, we can. But for now, it doesn't seem to be critical. I did mention this to Roger at the connect, but I'll also pursue this further with ARM just in case.
I don't see the advantage of that over what we have already.
Thanks ZK
On 13 February 2012 22:43, Paul Larson paul.larson@linaro.org wrote:
I think you misunderstood what I'm getting at. What I'm suggesting is that there should just be a proper hwpack for vexpress-a7/15+fastmodel. This wouldn't have to be supported forever, just until we have real hardware that can be used.
A hwpack supporting the fast model would be good, but my understanding of the workflow to use it would be: (1) use l-m-c to turn hwpack+rootfs into sd card image (2) unpack kernel and initrd from sd card image (3) boot on model using split kernel+initrd and the sd card image rootfs
My understanding from the discussion before is that this would be possible, but might require some modifications to the bootloader.
I'm not sure what bootloader you're talking about here. When booting a kernel on fast models there is no boot loader involved (this is why we have the boot-wrapper code).
If ARM can fix the 2GB limit, then what that means is *if* we someday need to test using a larger image, we can. But for now, it doesn't seem to be critical. I did mention this to Roger at the connect, but I'll also pursue this further with ARM just in case.
The reason to file feature requests formally is that the fast models are produced by a completely different part of ARM from that which Roger is in charge of. Obviously he can raise things internally if necessary [and it is worth keeping him in the loop] but mostly you're just adding a longer and lossier channel for the information :-)
-- PMM
I'm not sure what I can add to this discussion but I wanted to mention a couple of things I noticed in the discussion:
1) l-m-c / a-l-m-c will need modifying if you want to create an image for a7/a15 models. Currently, the media create scripts have a hard coded list of boards they support, you have to modify the script if you want to add a new board. This isn't hard, but it isn't a single line change either. The tools people need to be in the loop to do this as they'd have to either make the changes or at the very least approve and integrate them. Of course, with the new hwpack format that we agreed last week, this will be easier, but that won't be available for many weeks yet.
2) Creating a "proper" hwpack isn't as trivial a task as it might seem. And maintaining our current hwpacks properly takes Tixy several days for every release.
3) Android doesn't use hwpacks, it builds its own kernel. So that's actually easier, but I wanted to mention it as it didn't seem clear in the mails before this.
On 13 February 2012 23:05, Peter Maydell peter.maydell@linaro.org wrote:
On 13 February 2012 22:43, Paul Larson paul.larson@linaro.org wrote:
I think you misunderstood what I'm getting at. What I'm suggesting is
that
there should just be a proper hwpack for vexpress-a7/15+fastmodel. This wouldn't have to be supported forever, just until we have real hardware
that
can be used.
A hwpack supporting the fast model would be good, but my understanding of the workflow to use it would be: (1) use l-m-c to turn hwpack+rootfs into sd card image (2) unpack kernel and initrd from sd card image (3) boot on model using split kernel+initrd and the sd card image rootfs
My understanding from the discussion before is that this would be possible, but might require some modifications to the bootloader.
I'm not sure what bootloader you're talking about here. When booting a kernel on fast models there is no boot loader involved (this is why we have the boot-wrapper code).
If ARM can fix the 2GB limit, then what that means is *if* we someday
need
to test using a larger image, we can. But for now, it doesn't seem to be critical. I did mention this to Roger at the connect, but I'll also
pursue
this further with ARM just in case.
The reason to file feature requests formally is that the fast models are produced by a completely different part of ARM from that which Roger is in charge of. Obviously he can raise things internally if necessary [and it is worth keeping him in the loop] but mostly you're just adding a longer and lossier channel for the information :-)
-- PMM
On 13 February 2012 22:03, Zygmunt Krynicki zygmunt.krynicki@linaro.org wrote:
On Mon, Feb 13, 2012 at 9:54 PM, Paul Larson paul.larson@linaro.org wrote:
Is it worth revisiting whether we should have proper hwpacks for fast models? I know there's the 2G max sd size issue with that, but if that's something arm can fix, or if we have another way around it, would that help?
As for fixing SD card limit, I don't know how that works. Perhaps that's a question for ARM folks.
As I said before when this came up, if we want features fixed in Fast Models we need to *raise them formally with our ARM support channel*. This is the only way to ensure that they are prioritised appropriately by the group who produces that product.
-- PMM
How do we engage with our official ARM support channels?
ZK
On Mon, Feb 13, 2012 at 11:35 PM, Peter Maydell peter.maydell@linaro.org wrote:
On 13 February 2012 22:03, Zygmunt Krynicki zygmunt.krynicki@linaro.org wrote:
On Mon, Feb 13, 2012 at 9:54 PM, Paul Larson paul.larson@linaro.org wrote:
Is it worth revisiting whether we should have proper hwpacks for fast models? I know there's the 2G max sd size issue with that, but if that's something arm can fix, or if we have another way around it, would that help?
As for fixing SD card limit, I don't know how that works. Perhaps that's a question for ARM folks.
As I said before when this came up, if we want features fixed in Fast Models we need to *raise them formally with our ARM support channel*. This is the only way to ensure that they are prioritised appropriately by the group who produces that product.
-- PMM
On 13 February 2012 22:37, Zygmunt Krynicki zygmunt.krynicki@linaro.org wrote:
How do we engage with our official ARM support channels?
(1) Identify who in Linaro is talking to the ARM support team for things like obtaining licenses. [this not me and I'm not clear who it is.] (2) Write up the description of the problem / missing feature as a decent quality bug report. (3) Ask the person from (1) to forward the text from (2) to the ARM support folks.
-- PMM
linaro-validation@lists.linaro.org
-
Paul Larson -
Peter Maydell -
Ryan Harkin -
Zygmunt Krynicki