Hello everyone,
during the last month, and also during this one, the Infrastructure team has been working on a new feature for linaro-android-media-create. We are introducing an "Android hardwarepack", similar to the normal hardwarepack of linaro-media-create, with the difference that the Android one is just a configuration file, that will store all the board parameters (instead of having them hard-coded inside the code).
Blueprint, more details and initial code can be found here: https://blueprints.launchpad.net/linaro-image-tools/+spec/android-media-crea...
We wanted to know what kind of impact these new features might have on your side and on LAVA. The old linaro-android-media-create behavior will be maintained (at least for a while), meaning that it will be possible to run l-a-m-c without the hwpack file passed on the command line, but in the future using hwpack will be the default approach.
On the LAVA side, do jobs need to be reconfigured or will this be transparent for you? Any other things that might broke or change due to the new features?
Thanks!
-- Milo Casagrande Infrastructure Engineer Linaro.org <www.linaro.org> │ Open source software for ARM SoCs
On Wed, Dec 5, 2012 at 11:43 AM, Milo Casagrande milo.casagrande@linaro.org wrote:
Hello everyone,
during the last month, and also during this one, the Infrastructure team has been working on a new feature for linaro-android-media-create. We are introducing an "Android hardwarepack", similar to the normal hardwarepack of linaro-media-create, with the difference that the Android one is just a configuration file, that will store all the board parameters (instead of having them hard-coded inside the code).
Blueprint, more details and initial code can be found here: https://blueprints.launchpad.net/linaro-image-tools/+spec/android-media-crea...
We wanted to know what kind of impact these new features might have on your side and on LAVA. The old linaro-android-media-create behavior will be maintained (at least for a while), meaning that it will be possible to run l-a-m-c without the hwpack file passed on the command line, but in the future using hwpack will be the default approach.
On the LAVA side, do jobs need to be reconfigured or will this be transparent for you? Any other things that might broke or change due to the new features?
We currently support pre-built images as well as submitting jobs with all the parts with LAVA running lamc. So yes, I guess LAVA needs to be updated to support the new, optional hwpack-part.
Thanks!
-- Milo Casagrande Infrastructure Engineer Linaro.org <www.linaro.org> │ Open source software for ARM SoCs
linaro-validation mailing list linaro-validation@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-validation
On Wed, Dec 05, 2012 at 12:43:19PM +0100, Alexander Sack wrote:
On Wed, Dec 5, 2012 at 11:43 AM, Milo Casagrande milo.casagrande@linaro.org wrote:
Hello everyone,
during the last month, and also during this one, the Infrastructure team has been working on a new feature for linaro-android-media-create. We are introducing an "Android hardwarepack", similar to the normal hardwarepack of linaro-media-create, with the difference that the Android one is just a configuration file, that will store all the board parameters (instead of having them hard-coded inside the code).
Blueprint, more details and initial code can be found here: https://blueprints.launchpad.net/linaro-image-tools/+spec/android-media-crea...
We wanted to know what kind of impact these new features might have on your side and on LAVA. The old linaro-android-media-create behavior will be maintained (at least for a while), meaning that it will be possible to run l-a-m-c without the hwpack file passed on the command line, but in the future using hwpack will be the default approach.
So, will linaro-android-media-create need boot, system, userdata + a hwpack file?
On the LAVA side, do jobs need to be reconfigured or will this be transparent for you? Any other things that might broke or change due to the new features?
We currently support pre-built images as well as submitting jobs with all the parts with LAVA running lamc. So yes, I guess LAVA needs to be updated to support the new, optional hwpack-part.
Currently there is a single point where the LAVA dispatcher calls linaro-android-media-create directly: when deploying builds that come in separate parts (boot, system and userdata) for fastmodels.
For physical boards, however, currently the partitions corresponding to those parts are populated directly from the build tarballs ({boot,system,userdata}.tar.bz2 etc). In this case, if the hwpack file is going to be a new input to this process, we would need to know what has to be done with that hwpack file. As I understand from https://wiki.linaro.org/AndroidHardwarePacksV3, the hwpack file seems to contain stuff that is going to be used to create u-boot scripts and the like, right?
On Wed, Dec 5, 2012 at 1:34 PM, Antonio Terceiro antonio.terceiro@linaro.org wrote:
On Wed, Dec 05, 2012 at 12:43:19PM +0100, Alexander Sack wrote:
On Wed, Dec 5, 2012 at 11:43 AM, Milo Casagrande milo.casagrande@linaro.org wrote:
Hello everyone,
during the last month, and also during this one, the Infrastructure team has been working on a new feature for linaro-android-media-create. We are introducing an "Android hardwarepack", similar to the normal hardwarepack of linaro-media-create, with the difference that the Android one is just a configuration file, that will store all the board parameters (instead of having them hard-coded inside the code).
Blueprint, more details and initial code can be found here: https://blueprints.launchpad.net/linaro-image-tools/+spec/android-media-crea...
We wanted to know what kind of impact these new features might have on your side and on LAVA. The old linaro-android-media-create behavior will be maintained (at least for a while), meaning that it will be possible to run l-a-m-c without the hwpack file passed on the command line, but in the future using hwpack will be the default approach.
So, will linaro-android-media-create need boot, system, userdata + a hwpack file?
On the LAVA side, do jobs need to be reconfigured or will this be transparent for you? Any other things that might broke or change due to the new features?
We currently support pre-built images as well as submitting jobs with all the parts with LAVA running lamc. So yes, I guess LAVA needs to be updated to support the new, optional hwpack-part.
Currently there is a single point where the LAVA dispatcher calls linaro-android-media-create directly: when deploying builds that come in separate parts (boot, system and userdata) for fastmodels.
For physical boards, however, currently the partitions corresponding to those parts are populated directly from the build tarballs ({boot,system,userdata}.tar.bz2 etc). In this case, if the hwpack file is going to be a new input to this process, we would need to know what has to be done with that hwpack file. As I understand from https://wiki.linaro.org/AndroidHardwarePacksV3, the hwpack file seems to contain stuff that is going to be used to create u-boot scripts and the like, right?
On high level, I would guess its just another argument to lamc ... and then I would expect lamc bails out if it detects an image is of the new format but we dont pass a hwpack and the other way around (e.g. you pass hwpack, but the image doesnt announce that it supports the new format.
-- Antonio Terceiro Software Engineer - Linaro http://www.linaro.org
linaro-validation mailing list linaro-validation@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-validation
On Wed, Dec 05, 2012 at 02:24:50PM +0100, Alexander Sack wrote:
On Wed, Dec 5, 2012 at 1:34 PM, Antonio Terceiro
For physical boards, however, currently the partitions corresponding to those parts are populated directly from the build tarballs ({boot,system,userdata}.tar.bz2 etc). In this case, if the hwpack file is going to be a new input to this process, we would need to know what has to be done with that hwpack file. As I understand from https://wiki.linaro.org/AndroidHardwarePacksV3, the hwpack file seems to contain stuff that is going to be used to create u-boot scripts and the like, right?
On high level, I would guess its just another argument to lamc ... and then I would expect lamc bails out if it detects an image is of the new format but we dont pass a hwpack and the other way around (e.g. you pass hwpack, but the image doesnt announce that it supports the new format.
My point was: lamc is *only* used for creating a single image file for fastmodels. For physical boards running master images, the dispatcher *does not* run lamc at all, what means it should probably mimic whatever lamc does with the hwpack file
When we have SD muxe, then we might use lamc for all devices, but until then the dispatcher would have to be taught what to do with the hwpack file.
On 12/05/2012 07:47 AM, Antonio Terceiro wrote:
On Wed, Dec 05, 2012 at 02:24:50PM +0100, Alexander Sack wrote:
On Wed, Dec 5, 2012 at 1:34 PM, Antonio Terceiro
For physical boards, however, currently the partitions corresponding to those parts are populated directly from the build tarballs ({boot,system,userdata}.tar.bz2 etc). In this case, if the hwpack file is going to be a new input to this process, we would need to know what has to be done with that hwpack file. As I understand from https://wiki.linaro.org/AndroidHardwarePacksV3, the hwpack file seems to contain stuff that is going to be used to create u-boot scripts and the like, right?
On high level, I would guess its just another argument to lamc ... and then I would expect lamc bails out if it detects an image is of the new format but we dont pass a hwpack and the other way around (e.g. you pass hwpack, but the image doesnt announce that it supports the new format.
My point was: lamc is *only* used for creating a single image file for fastmodels. For physical boards running master images, the dispatcher *does not* run lamc at all, what means it should probably mimic whatever lamc does with the hwpack file
When we have SD muxe, then we might use lamc for all devices, but until then the dispatcher would have to be taught what to do with the hwpack file.
Antonio is correct. For our "master devices" which account for probably 90% of our Android jobs, we do our own special stuff with tarballs. So we have options:
1) change nothing - assuming the {boot,system,userdata}.tar.bz2 still have everything. sounds unlikely
2) change our master device logic to do whatever new process is required
3) extend our prebuilt image support to android (we currently only do pre-built images for ubuntu)
Question: with this new hwpack - it sounds like we'll now need to be given 4 parameters (hwpack, boot, system, and userdata)?
Hello everyone,
I start replying from the last email.
On Wed, Dec 5, 2012 at 4:33 PM, Andy Doan andy.doan@linaro.org wrote:
Antonio is correct. For our "master devices" which account for probably 90% of our Android jobs, we do our own special stuff with tarballs. So we have options:
- change nothing - assuming the {boot,system,userdata}.tar.bz2 still have
everything. sounds unlikely
They should have everything as usual: with this kind of hwpack (it is just a configuration file) we are passing parameters that right now are stored in the code of l-a-m-c. Parameters that we are extracting and storing in configuration files.
change our master device logic to do whatever new process is required
extend our prebuilt image support to android (we currently only do
pre-built images for ubuntu)
Question: with this new hwpack - it sounds like we'll now need to be given 4 parameters (hwpack, boot, system, and userdata)?
In the beginning, and for a while, we will keep the old behavior (no need to pass the hwpack configuration file), but yes, in the future it will be necessary to pass also the hwpack on the command line to l-a-m-c (I think l-a-m-c needs the "dev" target as well, so it should be 5 parameters). So it might be good to start "porting" the jobs/tasks that call l-a-m-c directly.
-- Milo Casagrande Infrastructure Engineer Linaro.org <www.linaro.org> │ Open source software for ARM SoCs
On 5 December 2012 17:33, Andy Doan andy.doan@linaro.org wrote:
On 12/05/2012 07:47 AM, Antonio Terceiro wrote:
On Wed, Dec 05, 2012 at 02:24:50PM +0100, Alexander Sack wrote:
On Wed, Dec 5, 2012 at 1:34 PM, Antonio Terceiro
For physical boards, however, currently the partitions corresponding to those parts are populated directly from the build tarballs ({boot,system,userdata}.tar.bz2 etc). In this case, if the hwpack file is going to be a new input to this process, we would need to know what has to be done with that hwpack file. As I understand from https://wiki.linaro.org/AndroidHardwarePacksV3, the hwpack file seems to contain stuff that is going to be used to create u-boot scripts and the like, right?
On high level, I would guess its just another argument to lamc ... and then I would expect lamc bails out if it detects an image is of the new format but we dont pass a hwpack and the other way around (e.g. you pass hwpack, but the image doesnt announce that it supports the new format.
My point was: lamc is *only* used for creating a single image file for fastmodels. For physical boards running master images, the dispatcher *does not* run lamc at all, what means it should probably mimic whatever lamc does with the hwpack file
When we have SD muxe, then we might use lamc for all devices, but until then the dispatcher would have to be taught what to do with the hwpack file.
Antonio is correct. For our "master devices" which account for probably 90% of our Android jobs, we do our own special stuff with tarballs. So we have options:
- change nothing - assuming the {boot,system,userdata}.tar.bz2 still have
everything. sounds unlikely
change our master device logic to do whatever new process is required
extend our prebuilt image support to android (we currently only do
pre-built images for ubuntu)
Pre-built image support is available on Android as well: http://snapshots.linaro.org/android/~linaro-android/vexpress-jb-gcc47-armlt-...
It's controlled by BUILD_FS_IMAGE in build configuration.
imo, LAVA shouldn't do its own special stuff but use l-a-m-c or the pre-built image.
Question: with this new hwpack - it sounds like we'll now need to be given 4 parameters (hwpack, boot, system, and userdata)?
On Fri, Dec 07, 2012 at 09:24:37AM +0200, Fathi Boudra wrote:
On 5 December 2012 17:33, Andy Doan andy.doan@linaro.org wrote:
On 12/05/2012 07:47 AM, Antonio Terceiro wrote:
When we have SD muxe, then we might use lamc for all devices, but until then the dispatcher would have to be taught what to do with the hwpack file.
Antonio is correct. For our "master devices" which account for probably 90% of our Android jobs, we do our own special stuff with tarballs. So we have options:
- change nothing - assuming the {boot,system,userdata}.tar.bz2 still have
everything. sounds unlikely
change our master device logic to do whatever new process is required
extend our prebuilt image support to android (we currently only do
pre-built images for ubuntu)
Pre-built image support is available on Android as well: http://snapshots.linaro.org/android/~linaro-android/vexpress-jb-gcc47-armlt-...
It's controlled by BUILD_FS_IMAGE in build configuration.
imo, LAVA shouldn't do its own special stuff but use l-a-m-c or the pre-built image.
I guess everyone agrees with you. :-)
The problem is, without an SD mux, there is no way for the host to flash images to the SD card on the target. For this we must have a known-good system running on the target, what we call a master image. The LAVA makes the master image system flash the test image parts to additional SD card partitions (i.e. we must keep the partitions with kernel/rootfs for the master system intact).
That is the case even for pre-built images. LAVA has to extract the contents of its partitions and have the master system flash them to the SD card before rebooting into the test system.
On 12/07/2012 08:07 AM, Antonio Terceiro wrote:
On Fri, Dec 07, 2012 at 09:24:37AM +0200, Fathi Boudra wrote:
On 5 December 2012 17:33, Andy Doan andy.doan@linaro.org wrote:
On 12/05/2012 07:47 AM, Antonio Terceiro wrote:
When we have SD muxe, then we might use lamc for all devices, but until then the dispatcher would have to be taught what to do with the hwpack file.
Antonio is correct. For our "master devices" which account for probably 90% of our Android jobs, we do our own special stuff with tarballs. So we have options:
- change nothing - assuming the {boot,system,userdata}.tar.bz2 still have
everything. sounds unlikely
change our master device logic to do whatever new process is required
extend our prebuilt image support to android (we currently only do
pre-built images for ubuntu)
Pre-built image support is available on Android as well: http://snapshots.linaro.org/android/~linaro-android/vexpress-jb-gcc47-armlt-...
It's controlled by BUILD_FS_IMAGE in build configuration.
imo, LAVA shouldn't do its own special stuff but use l-a-m-c or the pre-built image.
I guess everyone agrees with you. :-)
+ infinity
The problem is, without an SD mux, there is no way for the host to flash images to the SD card on the target. For this we must have a known-good system running on the target, what we call a master image. The LAVA makes the master image system flash the test image parts to additional SD card partitions (i.e. we must keep the partitions with kernel/rootfs for the master system intact).
That is the case even for pre-built images. LAVA has to extract the contents of its partitions and have the master system flash them to the SD card before rebooting into the test system.
To add to the difficulty, Android's unitird's init.rc (I think that's the file) is set up to mount to specific partition *numbers*. This doesn't work at all for LAVA so we have to use horrible sed/awk style stuff to make sure Android's "system" partition is really partition 5 and not 2 :(
On Wed, Dec 5, 2012 at 2:47 PM, Antonio Terceiro antonio.terceiro@linaro.org wrote:
My point was: lamc is *only* used for creating a single image file for fastmodels. For physical boards running master images, the dispatcher *does not* run lamc at all, what means it should probably mimic whatever lamc does with the hwpack file
If the dispatcher does not need boards parameter, like boot arguments , partition layout or something else, then there is no need to change anything. Only where l-a-m-c is invoked should be necessary.
I have no idea of the dispatcher internals, so I do not know what it actually does when given the image file.
-- Milo Casagrande Infrastructure Engineer Linaro.org <www.linaro.org> │ Open source software for ARM SoCs
linaro-validation@lists.linaro.org
-
Alexander Sack -
Andy Doan -
Antonio Terceiro -
Fathi Boudra -
Milo Casagrande