Hi all,
Scott asked me to take a look at the hwpack spec:
https://wiki.linaro.org/Platform/UserPlatforms/Specs/10.11/HardwarePacks
I took the liberty of editing it somewhat to make the definition of a hardware pack clearer, and remove some contradiction in the implementation suggestions.
I have some questions about the goals of the spec though:
1. Do we support installing more than one hardware pack on an image? 2. What is the purpose of the hwpack.deb that is mentioned in places? 3. Do we want to be able to pull in new versions of a hwpack on request, or should it just be a case of updating the image, with a hwpack-install call if you want to install a newer version that pulls in new packages? 4. Do we want to support cross-installation in any way? 5. What are the use cases for tags? I can only see X/no X in the spec. 6. What are the use cases for support information?
There is also one larger question, which is that I disagree that we shouldn't provide anything that will go in a hardware pack in the linaro images. I think that having the images be useful by themselves has lots of benefits.
- Being able to quickly spin up an image in QEMU makes for easier testing. - Requiring a hardware pack for every operation will make some things more tedious. - If everything in a hardware pack becomes more consolidated then hardware packs probably become less useful.
Thanks,
James
On Fri, Aug 20, 2010 at 04:26:46PM -0400, James Westby wrote:
- What are the use cases for tags? I can only see X/no X in the spec.
One other I see is OpenMAX + Gstreamer versus pure Gstreamer versus OpenCORE. I guess the generic use case I see there is being able to identify what sort of software components the hardware pack provides support for -- a hardware pack for X would require some X-specific glue code that wouldn't be required in a different environment.
Note that even X/no-X may be simplistic, because the no-X case can be varied and potentially require different contents in the hardware packs.
On Fri, 20 Aug 2010 18:49:58 -0300, Christian Robottom Reis kiko@linaro.org wrote:
On Fri, Aug 20, 2010 at 04:26:46PM -0400, James Westby wrote:
- What are the use cases for tags? I can only see X/no X in the spec.
One other I see is OpenMAX + Gstreamer versus pure Gstreamer versus OpenCORE. I guess the generic use case I see there is being able to identify what sort of software components the hardware pack provides support for -- a hardware pack for X would require some X-specific glue code that wouldn't be required in a different environment.
Note that even X/no-X may be simplistic, because the no-X case can be varied and potentially require different contents in the hardware packs.
So would it be simpler to just allow exclusion of specific packages from the hardware pack? As written there are likely to be less than 10 packages, and all the combinations of tags is going to far exceed that.
Thanks,
James
On Fri, Aug 20, 2010 at 05:54:20PM -0400, James Westby wrote:
On Fri, 20 Aug 2010 18:49:58 -0300, Christian Robottom Reis kiko@linaro.org wrote:
On Fri, Aug 20, 2010 at 04:26:46PM -0400, James Westby wrote:
- What are the use cases for tags? I can only see X/no X in the spec.
One other I see is OpenMAX + Gstreamer versus pure Gstreamer versus OpenCORE. I guess the generic use case I see there is being able to identify what sort of software components the hardware pack provides support for -- a hardware pack for X would require some X-specific glue code that wouldn't be required in a different environment.
Note that even X/no-X may be simplistic, because the no-X case can be varied and potentially require different contents in the hardware packs.
So would it be simpler to just allow exclusion of specific packages from the hardware pack? As written there are likely to be less than 10 packages, and all the combinations of tags is going to far exceed that.
The simplest form would be to make it so the hardware pack would be installable on a number of different stacks and it would just work. But that would mean that the packs would contain a number of potentially conflicting packages.
I'm not sure I understand your question, though. Are you asking if packages could be excluded at hardware-pack install time or at creation time?
One thing which is nice about tags is that it's clear what platforms are supported in which configurations. So you can look at board X and see ah, it has a no-X gstreamer hardware pack, but no X gstreamer pack, which means at the moment there's no X acceleration for it.
On Fri, 20 Aug 2010 19:15:31 -0300, Christian Robottom Reis kiko@canonical.com wrote:
I'm not sure I understand your question, though. Are you asking if packages could be excluded at hardware-pack install time or at creation time?
I mean at install time.
The only use case I have seen so far is --no-X. That's fine to support, we can just mark whether a package is X related in some way, and so exclude it if necessary.
I would like to know of any other use cases though.
One thing which is nice about tags is that it's clear what platforms are supported in which configurations. So you can look at board X and see ah, it has a no-X gstreamer hardware pack, but no X gstreamer pack, which means at the moment there's no X acceleration for it.
Using X to mean x.org and an arbitrary board is mighty confusing :-)
Your use case above implies some more tools than are currently in the spec. How much nicer is it than just seeing which packages are installed?
Thanks,
James
On Fri, Aug 20, 2010 at 07:37:43PM -0400, James Westby wrote:
I would like to know of any other use cases though.
What about the OpenMAX plus gstreamer versus gstreamer versus OpenCORE (plus OpenMAX) use case I mentioned?
Using X to mean x.org and an arbitrary board is mighty confusing :-)
Oops!
Your use case above implies some more tools than are currently in the spec. How much nicer is it than just seeing which packages are installed?
Not much nicer, and there are other ways of solving the visibility problem, so maybe I am just trying to confuse you <wink>
On Fri, 20 Aug 2010 21:02:46 -0300, Christian Robottom Reis kiko@canonical.com wrote:
What about the OpenMAX plus gstreamer versus gstreamer versus OpenCORE (plus OpenMAX) use case I mentioned?
Yes, sorry, I forgot that one.
Could you expand a little in to what packages would be involved, which of those would in a hardware pack, and what combinations you would want to install them in?
Not much nicer, and there are other ways of solving the visibility problem, so maybe I am just trying to confuse you <wink>
I would never rule that out!
Thanks,
James
On Fri, Aug 20, 2010, James Westby wrote:
- Do we want to be able to pull in new versions of a hwpack on
request, or should it just be a case of updating the image, with a hwpack-install call if you want to install a newer version that pulls in new packages?
This would come for free if hwpacks are implemented as a light layer on top of tasks and meta-packages (the latter mostly because tasks aren't really supported in PPAs).
On Fri, Aug 20, 2010 at 10:26 PM, James Westby james.westby@linaro.orgwrote:
Hi all,
Scott asked me to take a look at the hwpack spec:
https://wiki.linaro.org/Platform/UserPlatforms/Specs/10.11/HardwarePacks
I took the liberty of editing it somewhat to make the definition of a hardware pack clearer, and remove some contradiction in the implementation suggestions.
I have some questions about the goals of the spec though:
- Do we support installing more than one hardware pack on an image?
in theory yes, but practically I don't expect this to become a major use case. If it's easier assuming that there is just one in the first phase, lets do that.
- What is the purpose of the hwpack.deb that is mentioned in places?
this is scotts baby i think. personally i am fine without a hwpack.deb. I think the idea was that configs etc. like apt source lines accompanying the hwpack could be shipped in there. Personally I think its good enough to start with this meta info being optionally in the tarballs somewhere.
- Do we want to be able to pull in new versions of a hwpack on
request, or should it just be a case of updating the image, with a hwpack-install call if you want to install a newer version that pulls in new packages?
we dont want to have magic that pulls a new hwpack at this stage. hwpack-install is used to install and upgrade from a local/remote-url that points to the hwpack tarball.
However, as (iirc) mentioned in the spec, a hardware pack optionally can also ship custom apt lines, so magic upgrades without a new hwpack tarball can happen for hwpacks that come with such a location (e.g. a ppa or a partner hosted apt repository etc.).
- Do we want to support cross-installation in any way?
yes, however, we will run the hwpack-install inside a armel chroot, so thats not a technical problem for the hwpack-install binary. The magic for that will probably happen in linaro-media-create.
- What are the use cases for tags? I can only see X/no X in the spec.
tags are low priority. It came up with an ability to not install parts of the pack (like no x drivers if you dont care about X in a headless install for instance.
Other tags I envision could tag packages according to what use case they support (e.g. multimedia) or whether its a free/proprietary etc.
- What are the use cases for support information?
We want to label hwpacks as "unsupported" or "community" so we can offer them to download on snapshots.linaro.org while sending a clear message that those are not official linaro hardware packs because they don't come from a linaro consolidated kernel tree for instance.
On Mon, 23 Aug 2010 17:06:18 +0200, Alexander Sack asac@linaro.org wrote:
in theory yes, but practically I don't expect this to become a major use case. If it's easier assuming that there is just one in the first phase, lets do that.
The big problem I see is knowing which hwpacks should supercede all others. Doing it by name makes sense for upgrades, but there could be many that could install a kernel.
Maybe this doesn't actually matter, but my suspicion is that it will.
this is scotts baby i think. personally i am fine without a hwpack.deb. I think the idea was that configs etc. like apt source lines accompanying the hwpack could be shipped in there. Personally I think its good enough to start with this meta info being optionally in the tarballs somewhere.
I think that's fine. The spec is just confusing as it suddenly starts discussing it in the middle.
we dont want to have magic that pulls a new hwpack at this stage. hwpack-install is used to install and upgrade from a local/remote-url that points to the hwpack tarball.
However, as (iirc) mentioned in the spec, a hardware pack optionally can also ship custom apt lines, so magic upgrades without a new hwpack tarball can happen for hwpacks that come with such a location (e.g. a ppa or a partner hosted apt repository etc.).
That's upgrades of the packages, not upgrades of the hwpack.
- What are the use cases for tags? I can only see X/no X in the spec.
tags are low priority. It came up with an ability to not install parts of the pack (like no x drivers if you dont care about X in a headless install for instance.
Ok, tags are deferred from the initial implementation. Once we have some experience using them we will be able to define the user interaction much better.
- What are the use cases for support information?
We want to label hwpacks as "unsupported" or "community" so we can offer them to download on snapshots.linaro.org while sending a clear message that those are not official linaro hardware packs because they don't come from a linaro consolidated kernel tree for instance.
Does that information have to go inside the hwpack. What would display that information? What would behave differently?
Thanks,
James
On Mon, Aug 23, 2010 at 5:14 PM, James Westby james.westby@linaro.orgwrote:
On Mon, 23 Aug 2010 17:06:18 +0200, Alexander Sack asac@linaro.org wrote:
in theory yes, but practically I don't expect this to become a major use case. If it's easier assuming that there is just one in the first phase, lets do that.
The big problem I see is knowing which hwpacks should supercede all others. Doing it by name makes sense for upgrades, but there could be many that could install a kernel.
Yeah. hwpacks should have a unique id in their meta info. In that way you can figure this.
Maybe this doesn't actually matter, but my suspicion is that it will.
this is scotts baby i think. personally i am fine without a hwpack.deb. I think the idea was that configs etc. like apt source lines accompanying
the
hwpack could be shipped in there. Personally I think its good enough to start with this meta info being optionally in the tarballs somewhere.
I think that's fine. The spec is just confusing as it suddenly starts discussing it in the middle.
good. whatever is easiest.
we dont want to have magic that pulls a new hwpack at this stage. hwpack-install is used to install and upgrade from a local/remote-url
that
points to the hwpack tarball.
However, as (iirc) mentioned in the spec, a hardware pack optionally can also ship custom apt lines, so magic upgrades without a new hwpack
tarball
can happen for hwpacks that come with such a location (e.g. a ppa or a partner hosted apt repository etc.).
That's upgrades of the packages, not upgrades of the hwpack.
exactly.
- What are the use cases for tags? I can only see X/no X in the spec.
tags are low priority. It came up with an ability to not install parts of the pack (like no x drivers if you dont care about X in a headless
install
for instance.
Ok, tags are deferred from the initial implementation. Once we have some experience using them we will be able to define the user interaction much better.
Thanks. Thats fine.
- What are the use cases for support information?
We want to label hwpacks as "unsupported" or "community" so we can offer them to download on snapshots.linaro.org while sending a clear message
that
those are not official linaro hardware packs because they don't come from
a
linaro consolidated kernel tree for instance.
Does that information have to go inside the hwpack. What would display that information? What would behave differently?
The canonical meta info like this should go into the hwpack itself. On top we would also want to have that in the hwpack filename, so its obvious without introspection in the download.
On Mon, 23 Aug 2010 18:14:18 +0200, Alexander Sack asac@linaro.org wrote:
Yeah. hwpacks should have a unique id in their meta info. In that way you can figure this.
No, what I mean is that we may want to have any hardware pack containing a kernel to supercede any other containing a kernel. Or instead we might just want the latest to "win".
hwpacks should have a name, so we can do upgrades, the question is what other things we might want. Do we want relationships like packages? (Depends, Conflicts, etc.)
I'm inclined to allow installation of multiple hwpacks at this stage, and just leave those other questions for later.
Thanks,
James
On Mon, Aug 23, 2010 at 6:21 PM, James Westby james.westby@canonical.comwrote:
On Mon, 23 Aug 2010 18:14:18 +0200, Alexander Sack asac@linaro.org wrote:
Yeah. hwpacks should have a unique id in their meta info. In that way you can figure this.
No, what I mean is that we may want to have any hardware pack containing a kernel to supercede any other containing a kernel. Or instead we might just want the latest to "win".
hwpacks should have a name, so we can do upgrades, the question is what other things we might want. Do we want relationships like packages? (Depends, Conflicts, etc.)
I'm inclined to allow installation of multiple hwpacks at this stage, and just leave those other questions for later.
I thought a bit more about this and i think the single hwpacks policy makes
the "clean up" part mentioned in last sentence of user story 2 easier to implement.
So, hwpack-install can just remember the packages shipped as part of the previous hwpack and then removes those not shipped in the new hwpack.
Remember that on arm we usually don't have a boot menu where you can select one of many kernels ... so having two kernels installed that are suitable for the board would make things kind of random.
On Mon, 23 Aug 2010 19:52:47 +0200, Alexander Sack asac@linaro.org wrote:
I thought a bit more about this and i think the single hwpacks policy makes the "clean up" part mentioned in last sentence of user story 2 easier to implement.
Right.
Maybe we want hardware pack types in the future, so that you can have 1 of the "kernel" type, one of the "graphics" type, etc.
This is where I see some blurring of the lines between a hwpack and the packages that it contains though, so something feels wrong about that strategy.
I will update the spec to indicate that only a single hwpack can be installed at one time.
Thanks,
James
On Mon, 23 Aug 2010 17:06:18 +0200, Alexander Sack asac@linaro.org wrote:
- What are the use cases for support information?
We want to label hwpacks as "unsupported" or "community" so we can offer them to download on snapshots.linaro.org while sending a clear message that those are not official linaro hardware packs because they don't come from a linaro consolidated kernel tree for instance.
How is this different from the support status of the packages that the hardware pack installs? Should it be derived from that data.
Thanks,
James
On Mon, Aug 23, 2010 at 8:29 PM, James Westby james.westby@linaro.orgwrote:
On Mon, 23 Aug 2010 17:06:18 +0200, Alexander Sack asac@linaro.org wrote:
- What are the use cases for support information?
We want to label hwpacks as "unsupported" or "community" so we can offer them to download on snapshots.linaro.org while sending a clear message
that
those are not official linaro hardware packs because they don't come from
a
linaro consolidated kernel tree for instance.
How is this different from the support status of the packages that the hardware pack installs? Should it be derived from that data.
Do we already have a linaro support status for packages implemented? or are you refering to the ubuntu style main/universe/package-set support status here?
On Mon, 23 Aug 2010 20:31:37 +0200, Alexander Sack asac@linaro.org wrote:
Do we already have a linaro support status for packages implemented? or are you refering to the ubuntu style main/universe/package-set support status here?
I'm asking both conceptually and concretely.
In theory, how does the support status differ from the support status of the package? I assume that we wouldn't have a supported hardware pack based on an unsupported kernel, but would we ever want to release an unsupported hardware pack containing a supported kernel?
Concretely if we have rules then this could be based on the "Supported" metadata in the package indices. This is probably not available in PPAs etc., so may not work, but let's decide if it's even something we want to do first.
Thanks,
James
On Mon, 2010-08-23 at 17:06 +0200, Alexander Sack wrote:
2. What is the purpose of the hwpack.deb that is mentioned in places?this is scotts baby i think. personally i am fine without a hwpack.deb. I think the idea was that configs etc. like apt source lines accompanying the hwpack could be shipped in there. Personally I think its good enough to start with this meta info being optionally in the tarballs somewhere.
This was my idea. It was to put everything about the hardware pack in a deb, and install it first. On second thought, it seems overkill, and it should probably be scrubbed from the spec.
Scott
A further thought occurred to me the other day.
I assume that we have to put the source packages in the hwpack too, for at least the packages with some GPL code. Given that hwpacks also serve as a snapshot of the archive, we can't rely on a pointer to the original archive to serve the purpose.
Knowing which packages to include source for will be tricky, so I would implement this as source for all of them, at least to start with.
Thanks,
James
On Mon, Sep 13, 2010 at 11:11 PM, James Westby james.westby@linaro.org wrote:
I assume that we have to put the source packages in the hwpack too, for
...
Knowing which packages to include source for will be tricky, so I would implement this as source for all of them, at least to start with.
Right, feels like a good thing to ship those sources in a separate -source tarball next to the binary hwpack.
I assume we wouldn't necessarily fail if sources cannot be downloaded/found? A warning feels appropriate to me as a first step. Later, we could think about a hwpack config option similar to "fail-no-sources=yes".
Hi James,
On Mon, Sep 13, 2010 at 05:11:19PM -0400, James Westby wrote:
A further thought occurred to me the other day.
I assume that we have to put the source packages in the hwpack too, for at least the packages with some GPL code. Given that hwpacks also serve as a snapshot of the archive, we can't rely on a pointer to the original archive to serve the purpose.
Knowing which packages to include source for will be tricky, so I would implement this as source for all of them, at least to start with.
Is it intended that pre-release hwpacks will be long-lived? I expected the same rules to apply as to pre-release images: ephemeral objects used during development that would be replaced at release time by images built from the final archive, leaving no lingering obligation to provide easy access to source for images we're no longer distributing. Are the hwpacks not going to follow this same release cycle?
On Mon, 13 Sep 2010 23:59:42 +0200, Alexander Sack asac@linaro.org wrote:
On Mon, Sep 13, 2010 at 11:11 PM, James Westby james.westby@linaro.org wrote:
I assume that we have to put the source packages in the hwpack too, for
...
Knowing which packages to include source for will be tricky, so I would implement this as source for all of them, at least to start with.
Right, feels like a good thing to ship those sources in a separate -source tarball next to the binary hwpack.
That makes sense.
I assume we wouldn't necessarily fail if sources cannot be downloaded/found? A warning feels appropriate to me as a first step. Later, we could think about a hwpack config option similar to "fail-no-sources=yes".
I would fail and then add the option to turn it off.
Thanks,
James
On Tue, Sep 14, 2010 at 12:14 AM, Steve Langasek steve.langasek@linaro.org wrote:
Hi James,
On Mon, Sep 13, 2010 at 05:11:19PM -0400, James Westby wrote:
A further thought occurred to me the other day.
I assume that we have to put the source packages in the hwpack too, for at least the packages with some GPL code. Given that hwpacks also serve as a snapshot of the archive, we can't rely on a pointer to the original archive to serve the purpose.
Knowing which packages to include source for will be tricky, so I would implement this as source for all of them, at least to start with.
Is it intended that pre-release hwpacks will be long-lived? I expected the same rules to apply as to pre-release images: ephemeral objects used during development that would be replaced at release time by images built from the final archive, leaving no lingering obligation to provide easy access to source for images we're no longer distributing. Are the hwpacks not going to follow this same release cycle?
Right, I think legally we might really not be required to do this; however, technically it still makes sense to me to have the sources used to produce a certain hwpack. Especially since we use ppa's or some other archives for hwpacks which have no guarantees to keep history etc.
On Mon, 13 Sep 2010 15:14:04 -0700, Steve Langasek steve.langasek@linaro.org wrote:
Is it intended that pre-release hwpacks will be long-lived? I expected the same rules to apply as to pre-release images: ephemeral objects used during development that would be replaced at release time by images built from the final archive, leaving no lingering obligation to provide easy access to source for images we're no longer distributing. Are the hwpacks not going to follow this same release cycle?
I believe that they will, but I can't say for sure, Alexander is the one with the ideas.
It appears that there will be things built from PPAs too, and I'm not sure the images to do this too? We might want to have a policy on the PPAs that we include in hwpacks in order to be able to do this.
Thanks,
James
On Tue, Sep 14, 2010 at 09:16:15AM +0200, Alexander Sack wrote:
Is it intended that pre-release hwpacks will be long-lived? I expected the same rules to apply as to pre-release images: ephemeral objects used during development that would be replaced at release time by images built from the final archive, leaving no lingering obligation to provide easy access to source for images we're no longer distributing. Are the hwpacks not going to follow this same release cycle?
Right, I think legally we might really not be required to do this; however, technically it still makes sense to me to have the sources used to produce a certain hwpack. Especially since we use ppa's or some other archives for hwpacks which have no guarantees to keep history etc.
Yes, if we're going to be building hwpacks out of ppas, getting the sources out of the PPA afterwards would be a problem; so best to keep them with the hwpack.
-
Alexander Sack -
Christian Robottom Reis -
Christian Robottom Reis -
James Westby -
James Westby -
Loïc Minier -
Scott Bambrough -
Steve Langasek