When using LAVA inside a docker container, the LXC support adds lots of unnecessary overhead since the docker images are already made to include the necessary tools. So having another container is pointless. Even worse, LXC doesn't work inside docker anyways.
The LXC support should be made optional for a given LAVA install.
Until LXC can be disabled, projects like lava-docker[1] simply cannot support fastboot devices which is a major problem.
Kevin
On Fri, Jan 26, 2018 at 10:37:16AM -0800, Kevin Hilman wrote:
When using LAVA inside a docker container, the LXC support adds lots of unnecessary overhead since the docker images are already made to include the necessary tools. So having another container is pointless. Even worse, LXC doesn't work inside docker anyways.
The LXC support should be made optional for a given LAVA install.
Until LXC can be disabled, projects like lava-docker[1] simply cannot support fastboot devices which is a major problem.
Hi Kevin,
We're already working on this, mocking up bits of the lxc interfaces so they'll work transparently in a docker. It's an obvious thing to go with the the docker-dispatcher. See
https://git.linaro.org/lava/lava-lxc-mocker.git/
for initial development work. Senthil can tell you more...
Cheers,
Steve McIntyre steve.mcintyre@linaro.org writes:
On Fri, Jan 26, 2018 at 10:37:16AM -0800, Kevin Hilman wrote:
When using LAVA inside a docker container, the LXC support adds lots of unnecessary overhead since the docker images are already made to include the necessary tools. So having another container is pointless. Even worse, LXC doesn't work inside docker anyways.
The LXC support should be made optional for a given LAVA install.
Until LXC can be disabled, projects like lava-docker[1] simply cannot support fastboot devices which is a major problem.
Hi Kevin,
We're already working on this, mocking up bits of the lxc interfaces so they'll work transparently in a docker. It's an obvious thing to go with the the docker-dispatcher. See
https://git.linaro.org/lava/lava-lxc-mocker.git/
for initial development work. Senthil can tell you more...
Hmm, that's an interesting idea, and I understand the need to pretend/fake that LXC exists if you already have jobs that use LXC.
However, for LAVA installs that have never used LXC (for multiple reasons), it adds yet another layer of confusion (and potential bugs) that is not needed.
I don't want my jobs to pretend they're in an LXC container when they are not, and never will be because I'm already running LAVA inside a container.
In addition, I'm not convinced that faking an LXC container is actually a good idea and don't understand how you expect to ensure compatibility for jobs that are expecting to be virtualized (think virtualized networking?) Anyways, sounds like a can of worms to me.
We have a growing base of users for lava-docker, and have never (and will never) support LXC jobs, so it makes a lot more sense to have an option to completely disable LXC support.
Thanks,
Kevin
On Tuesday 30 January 2018 04:47 AM, Kevin Hilman wrote:
Steve McIntyre steve.mcintyre@linaro.org writes:
We're already working on this, mocking up bits of the lxc interfaces so they'll work transparently in a docker. It's an obvious thing to go with the the docker-dispatcher. See
https://git.linaro.org/lava/lava-lxc-mocker.git/
for initial development work. Senthil can tell you more...
This is tracked via https://projects.linaro.org/browse/LAVA-1201
Hmm, that's an interesting idea, and I understand the need to pretend/fake that LXC exists if you already have jobs that use LXC.
As you can see from the above card that is the basic idea where we do not want the lava-master to know what kind of worker is running.
I don't want my jobs to pretend they're in an LXC container when they are not, and never will be because I'm already running LAVA inside a container.
Such an use-case is the idea behind the text explained in "TBD" section of the above card. In this case we plan to control it via two things:
1. Absence of LXC protocol related content in the job yaml. 2. A setting in lava-dispatcher that governs whether running fastboot and adb is allowed directly on this dispatcher. This is required for LAVA instances that does not run within virtualized containers and hosts more than one DUT.
The above implementation will require changing things like:
* Provide equivalent pipeline actions that does not use an LXC connection or lxc-attach * Constructing a different job action pipeline from what will be used for a normal LXC protocol based job.
Though we started with the above idea, later our design discussions led to lava-lxc-mocker.
We will re-iterate to see if we can support new jobs that will completely use direct access without LXC, which will be taken up once we have lava-lxc-mocker use case done, which is still valid for providing compatibility for reusing jobs that uses LXC protocol.
Thank You.
On 30 January 2018 at 01:41, Senthil Kumaran S senthil.kumaran@linaro.org wrote:
On Tuesday 30 January 2018 04:47 AM, Kevin Hilman wrote:
Steve McIntyre steve.mcintyre@linaro.org writes:
We're already working on this, mocking up bits of the lxc interfaces so they'll work transparently in a docker. It's an obvious thing to go with the the docker-dispatcher. See
https://git.linaro.org/lava/lava-lxc-mocker.git/
for initial development work. Senthil can tell you more...
This is tracked via https://projects.linaro.org/browse/LAVA-1201
Hmm, that's an interesting idea, and I understand the need to pretend/fake that LXC exists if you already have jobs that use LXC.
As you can see from the above card that is the basic idea where we do not want the lava-master to know what kind of worker is running.
Just to mention here, the mockup support is for compatibility when kernel engineers want to reproduce a LAVA test job on a local device *without* making lots of changes to the test job that would potentially change the behaviour being triaged. e.g. when a regression is spotted in LKFT on a device which needs LXC support on LKFT. In this situation, the engineer has the choice of using docker (with a slight but known & reproducible variation from LKFT) or setting up an instance to use LXC - without needing any changes in the test job submission and only minimal changes in the device configuration to account for local PDU ports etc.
It's a different use case from yours where the test jobs being submitted can be designed to not need LXC in the first place. The test job YAML will be very different from the first use case.
As Senthil outlines, we are looking at implementing support for both use cases.
I don't want my jobs to pretend they're in an LXC container when they are not, and never will be because I'm already running LAVA inside a container.
Such an use-case is the idea behind the text explained in "TBD" section of the above card. In this case we plan to control it via two things:
- Absence of LXC protocol related content in the job yaml.
- A setting in lava-dispatcher that governs whether running fastboot and adb is allowed directly on this dispatcher. This is required for LAVA instances that does not run within virtualized containers and hosts more than one DUT.
The above implementation will require changing things like:
- Provide equivalent pipeline actions that does not use an LXC
connection or lxc-attach
- Constructing a different job action pipeline from what will be used
for a normal LXC protocol based job.
Though we started with the above idea, later our design discussions led to lava-lxc-mocker.
We will re-iterate to see if we can support new jobs that will completely use direct access without LXC, which will be taken up once we have lava-lxc-mocker use case done, which is still valid for providing compatibility for reusing jobs that uses LXC protocol.
Thank You.
Senthil Kumaran S http://www.stylesen.org/ http://www.sasenthilkumaran.com/
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
On Tue, Jan 30, 2018 at 12:38 AM, Neil Williams neil.williams@linaro.org wrote:
On 30 January 2018 at 01:41, Senthil Kumaran S senthil.kumaran@linaro.org wrote:
On Tuesday 30 January 2018 04:47 AM, Kevin Hilman wrote:
Steve McIntyre steve.mcintyre@linaro.org writes:
We're already working on this, mocking up bits of the lxc interfaces so they'll work transparently in a docker. It's an obvious thing to go with the the docker-dispatcher. See
https://git.linaro.org/lava/lava-lxc-mocker.git/
for initial development work. Senthil can tell you more...
This is tracked via https://projects.linaro.org/browse/LAVA-1201
Hmm, that's an interesting idea, and I understand the need to pretend/fake that LXC exists if you already have jobs that use LXC.
As you can see from the above card that is the basic idea where we do not want the lava-master to know what kind of worker is running.
Just to mention here, the mockup support is for compatibility when kernel engineers want to reproduce a LAVA test job on a local device *without* making lots of changes to the test job that would potentially change the behaviour being triaged. e.g. when a regression is spotted in LKFT on a device which needs LXC support on LKFT. In this situation, the engineer has the choice of using docker (with a slight but known & reproducible variation from LKFT) or setting up an instance to use LXC - without needing any changes in the test job submission and only minimal changes in the device configuration to account for local PDU ports etc.
It's a different use case from yours where the test jobs being submitted can be designed to not need LXC in the first place. The test job YAML will be very different from the first use case.
Understood.
As Senthil outlines, we are looking at implementing support for both use cases.
Great news, thanks for the clarification.
I'll be a willing beta tester of the LXC-free case.
Kevin
I don't want my jobs to pretend they're in an LXC container when they are not, and never will be because I'm already running LAVA inside a container.
Such an use-case is the idea behind the text explained in "TBD" section of the above card. In this case we plan to control it via two things:
- Absence of LXC protocol related content in the job yaml.
- A setting in lava-dispatcher that governs whether running fastboot and adb is allowed directly on this dispatcher. This is required for LAVA instances that does not run within virtualized containers and hosts more than one DUT.
The above implementation will require changing things like:
- Provide equivalent pipeline actions that does not use an LXC
connection or lxc-attach
- Constructing a different job action pipeline from what will be used
for a normal LXC protocol based job.
Though we started with the above idea, later our design discussions led to lava-lxc-mocker.
We will re-iterate to see if we can support new jobs that will completely use direct access without LXC, which will be taken up once we have lava-lxc-mocker use case done, which is still valid for providing compatibility for reusing jobs that uses LXC protocol.
Thank You.
Senthil Kumaran S http://www.stylesen.org/ http://www.sasenthilkumaran.com/
Lava-users mailing list Lava-users@lists.linaro.org https://lists.linaro.org/mailman/listinfo/lava-users
--
Neil Williams
neil.williams@linaro.org http://www.linux.codehelp.co.uk/
-
Kevin Hilman -
Neil Williams -
Senthil Kumaran S -
Steve McIntyre