Update (sorry to steal the thread, google mail is not nice in this regard).
Prompted by Paul I've decided to try a bigLITTLE config. My test did not end up very good: 1) I've tried the RealView Express A15x1-A7x1 model 2) It has different options that can be passed to model shell, needles to say they are not compatible. I tweaked my semihosting line to look like this:
-C coretile.cluster0.cpu0.semihosting-cmd_line="--kernel $KERNEL -- console=ttyAMA0 mem=512M ip=dhcp root=/dev/nfs nfsroot=$NFSROOT,tcp rw earlyprintk $@"
3) I did not get the kernel to boot at all, I suspect that our boot wrapper is not compatible or I mis-configured the kernel or something in between. I did not even get a X11 terminal window as I usually always get. 4) I feel lame for not trying earlier
Ideas welcome ZK
On Wed, Feb 15, 2012 at 12:31 AM, Michael Hudson-Doyle michael.hudson@canonical.com wrote:
On Tue, 14 Feb 2012 20:24:51 +0100, Zygmunt Krynicki zygmunt.krynicki@linaro.org wrote:
On Tue, Feb 14, 2012 at 3:26 AM, Michael Hudson-Doyle michael.hudson@canonical.com wrote:
On Mon, 13 Feb 2012 22:27:25 +0100, Zygmunt Krynicki zygmunt.krynicki@linaro.org wrote:
Hi.
Fast model support is getting better. It seems that with the excellent patches by Peter Maydell we can now boot some kernels (I've only tried one tree, additional trees welcome :-). I'm currently building a from-scratch environment to ensure everything is accounted for and I understand how pieces interact.
Having said that I'd like to summarize how LAVA handles fast models:
Technically the solution is not unlike QEMU which you are all familiar with. The key differences are:
- Only NFS boot makes sense. There are no other sensible method that
I know of. We may also use a SD card (virtual obviously) but it is constrained to two gigabytes of data.
As mentioned in the other thread, it would be good to at least let ARM know that removing this limit would help us (if we can figure out how to do this).
We may figure out how to do this by reading the LISA source code that came with the model. That's a big task though (maybe grepping for mmc0 is a low hanging fruit, I did not check)
That's not what I was suggesting! We should try to persuade ARM to do that. It may be that they can't do it in a reasonable timeframe, or maybe it's simply not a need that has been explained to them yet and is something they can do in a week.
- The way we actually boot is complicated. There is no uboot, fast
model interpreter actually starts an .axf file that can do anything (some examples include running tests and benchmarks without actually setting the kernel or anything like that). There is no way to easily load the kernel and pass a command line. To work around that we're using a special axf file that uses fast model semihosting features to load the kernel/initrd from a host filesystem as well as to setup the command line that will be passed to the booting kernel. This allows us to freely configure NFS services and point our virtual kernel at appropriate IP addresses and pathnames.
So I guess I'd like to understand how this works in a bit more detail. Can you brain dump on the topic for a few minutes? :) What is "fast model semihosting"?
It's a way to have "syscalls" that connect the "bare hardware" (be it physical or emulated) to an external debugger or other monitor. You can find a short introduction in this blog [1]. For us it means we get to write bare-metal assembly that does the equivalent of open(), read(), write() and close(). The files are being opened are on the machine that runs the fast model. You can also print debugging statements straight to the console this way (we probably could write semihosting console driver if there is no such code yet) to get all of the output to the same tty that runs the model (model_shell). A more detailed explanation of this topic can be found in [2]
Fast model semihosting simply refers to using semihosting facilities in a fast model interpreter.
Thanks for that. Sounds a tiny little bit like it's using a JTAG-for-fast-models type facility?
- After the machine starts up we immediately open a TPC/IP connection
to a local TCP socket. We know which port is being used so we can easily allocate them up front. This port is now the traditional LAVA serial console.
I guess there is a risk here that we will miss early boot messages? This might not matter much.
There are other options but currently this seems to work quite okay.
Fair enough.
Once we've found someone at ARM we can officially complain at about fast models, an option to have serial comms happen on the process stdin/stdout would be nice.
I think the reason they don't happen in the console is that by default we get four telnet ports to connect to (definitely more than one) so the logical question they'll ask is "which port should we redirect". Maybe there is an option buried somewhere to make that happen but so far I have not found it.
Again, I'm not saying that this is something we should do...
The rest of this looks like QEMU:
- you can access the filesystem easily (to gather results)
- we can use QEMU to chroot into the NFS root to install additional
software (emulation via a fast model is extremely slow)
In my testing, the pip install bzr+lp:lava-test step did not really work under QEMU. Maybe it does now, or maybe we can install a tarball or something.
I installed lava-test using release tarball. That has worked pretty well.
OK. That makes sense.
In general I think that:
- We need to reconsider how to do testing on very slow machines
You mean a "drive from the outside" approach like lava-android-test uses may make sense?
- What can be invoked on the host (part of installation, unless that
wants to build stuff, result parsing and tracking)
Yeah, this sort of thing is a grey area currently. More below.
- What has to be invoked on the target (test code, system probes)
It's important to make the intent very clear. If we define that cmd_install installs something while in "master image" on the "target" then we should not break that.
Well. We can't avoid breaking that if there *is no master image*.
Currently the dispatcher has the concept of a "reliable session", which is meant to be a target-like environment where things like compilation are possible. For master image based deployments, this is "booted into the master image, chrooted into a mounted testrootfs". For qemu, it is currently "boot the test image and hope that works", but it could be "chrooted into the testrootfs mounted on the host with qemu-arm-static in the right place", but that was less reliable than the other approach when I was testing this.
I think that it would be sensible to add "host_chroot" mode that applies nicely to qemu and fast models. Very slow things that don't care about the architecture could be invoked in that mode without sacrificing performance.
This code exists already. See _chroot_into_rootfs_session in lava_dispatcher.client.qemu.LavaQEMUClient and surrounds. The problem is that qemu is some distance from perfect...
Maybe we can limit the things that lava-test install does to things that work under qemu -- I guess installing via dpkg usually works (unless it's something like mono) and gcc probably works ok? Maybe we can do something like scratchbox where gcc is magically a cross compiler running directly on the host?
Cheers, mwh