I have been thinking how to simplify cross-testing our toolchain for both automated and development/debugging builds, and among various options the most universal I came up with is ARM hardware + ssh + binfmt_misc + sshfs. I wonder if anyone has already tried this or can suggest alternatives which are as universal.
Given: - host x86_64 development machine - cross-compiler - target hardware with fast network to the host - host and target have ssh - testsuite (gcc/glibc/gdb/etc)
Here is how it is going to work
1. On host we create a simple wrapper script that will pass through its arguments as command to execute on target via ssh: === #!/bin/sh ssh -p 22NN $TARGET_BOARD "$@" ===
2. We register this script in binfmt_misc to be used as interpreter for target binaries. Value of $TARGET_BOARD will be picked up from the environment and can be set to different boards for different testsuite runs.
3. The target board needs to be prepared for a particular testsuite run: -- Runtime libraries need to be either copied or mounted via sshfs from the host. It is an open question how best to install several sets of libraries (for parallel runs) so that each set appears to be main system libraries. My current thinking is a separate ssh server inside chroot per each test run. -- Test directory needs to be sshfs mounted on target from host so that the target could see test executables. -- Preparation/finalization of the board can either be done explicitly before/after testing. Or it can be done on demand by the aforementioned script: the script checks whether a multiplexed ssh socket exists, and, if not, it prepares the board and starts a multiplexed ssh connection.
4. Testing is fired up as if it is normal "native" testing. Whenever kernel is given an ARM binary to execute -- it passes it off to wrapper, which passes it off to the target board via ssh. The board sees same filesystem as host and happily executes binaries against toolchain runtime libraries.
Comments or rotten tomatoes?
Thank you,
-- Maxim Kuvyrkov www.linaro.org
On 21/05/14 16:08, Maxim Kuvyrkov wrote:
I have been thinking how to simplify cross-testing our toolchain for both automated and development/debugging builds, and among various options the most universal I came up with is ARM hardware + ssh + binfmt_misc + sshfs. I wonder if anyone has already tried this or can suggest alternatives which are as universal.
Given:
- host x86_64 development machine
- cross-compiler
- target hardware with fast network to the host
- host and target have ssh
- testsuite (gcc/glibc/gdb/etc)
Here is how it is going to work
- On host we create a simple wrapper script that will pass through its arguments as command to execute on target via ssh:
=== #!/bin/sh ssh -p 22NN $TARGET_BOARD "$@" ===
We register this script in binfmt_misc to be used as interpreter for target binaries. Value of $TARGET_BOARD will be picked up from the environment and can be set to different boards for different testsuite runs.
The target board needs to be prepared for a particular testsuite run:
-- Runtime libraries need to be either copied or mounted via sshfs from the host. It is an open question how best to install several sets of libraries (for parallel runs) so that each set appears to be main system libraries. My current thinking is a separate ssh server inside chroot per each test run. -- Test directory needs to be sshfs mounted on target from host so that the target could see test executables. -- Preparation/finalization of the board can either be done explicitly before/after testing. Or it can be done on demand by the aforementioned script: the script checks whether a multiplexed ssh socket exists, and, if not, it prepares the board and starts a multiplexed ssh connection.
Is this set-up for NX? Issue is how do we share the target board between different users? We can of-course initiate a lava hacking session but the amount of time we will have to wait to get the session active might be too long depending on the target classes availability.
Thanks Kugan
- Testing is fired up as if it is normal "native" testing. Whenever kernel is given an ARM binary to execute -- it passes it off to wrapper, which passes it off to the target board via ssh. The board sees same filesystem as host and happily executes binaries against toolchain runtime libraries.
Comments or rotten tomatoes?
Thank you,
-- Maxim Kuvyrkov www.linaro.org
linaro-toolchain mailing list linaro-toolchain@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-toolchain
If we add models as targets then that alleviates the sharing problem a bit - for test runs that aren't going to be too slow on models. And you'll need to watch out for nested '"' in $@, but I guess you know that.
I guess this approach is similar to what 'make check test-wrapper="...../glibc/scripts/cross-test-ssh.sh hostname"' is supposed to do for glibc. Ryan mentioned that glibc cross-test doesn't work, and it certainly fell over when I tried it. That makes me wonder whether there are hidden difficulties here, but that vague bit of evidence aside, the approach sounds sensible to me.
On 21 May 2014 07:46, Kugan kugan.vivekanandarajah@linaro.org wrote:
On 21/05/14 16:08, Maxim Kuvyrkov wrote:
I have been thinking how to simplify cross-testing our toolchain for both automated and development/debugging builds, and among various options the most universal I came up with is ARM hardware + ssh + binfmt_misc + sshfs. I wonder if anyone has already tried this or can suggest alternatives which are as universal.
Given:
- host x86_64 development machine
- cross-compiler
- target hardware with fast network to the host
- host and target have ssh
- testsuite (gcc/glibc/gdb/etc)
Here is how it is going to work
- On host we create a simple wrapper script that will pass through its arguments as command to execute on target via ssh:
=== #!/bin/sh ssh -p 22NN $TARGET_BOARD "$@" ===
We register this script in binfmt_misc to be used as interpreter for target binaries. Value of $TARGET_BOARD will be picked up from the environment and can be set to different boards for different testsuite runs.
The target board needs to be prepared for a particular testsuite run:
-- Runtime libraries need to be either copied or mounted via sshfs from the host. It is an open question how best to install several sets of libraries (for parallel runs) so that each set appears to be main system libraries. My current thinking is a separate ssh server inside chroot per each test run. -- Test directory needs to be sshfs mounted on target from host so that the target could see test executables. -- Preparation/finalization of the board can either be done explicitly before/after testing. Or it can be done on demand by the aforementioned script: the script checks whether a multiplexed ssh socket exists, and, if not, it prepares the board and starts a multiplexed ssh connection.
Is this set-up for NX? Issue is how do we share the target board between different users? We can of-course initiate a lava hacking session but the amount of time we will have to wait to get the session active might be too long depending on the target classes availability.
Thanks Kugan
- Testing is fired up as if it is normal "native" testing. Whenever kernel is given an ARM binary to execute -- it passes it off to wrapper, which passes it off to the target board via ssh. The board sees same filesystem as host and happily executes binaries against toolchain runtime libraries.
Comments or rotten tomatoes?
Thank you,
-- Maxim Kuvyrkov www.linaro.org
linaro-toolchain mailing list linaro-toolchain@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-toolchain
linaro-toolchain mailing list linaro-toolchain@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-toolchain
On Wed, May 21, 2014 at 5:54 AM, Bernie Ogden bernie.ogden@linaro.orgwrote:
If we add models as targets then that alleviates the sharing problem a bit - for test runs that aren't going to be too slow on models. And you'll need to watch out for nested '"' in $@, but I guess you know that.
Ultimately we need to validate on real hardware. From a release perspective it's necessary. From a development perspective being able to validate with models is effective until you need to have the final validation.
I guess this approach is similar to what 'make check test-wrapper="...../glibc/scripts/cross-test-ssh.sh hostname"' is supposed to do for glibc. Ryan mentioned that glibc cross-test doesn't work, and it certainly fell over when I tried it. That makes me wonder whether there are hidden difficulties here, but that vague bit of evidence aside, the approach sounds sensible to me.
It's not that it doesn't work. It's just that it's complicated to setup because it involves setting up a network file system.
On 21 May 2014 07:46, Kugan kugan.vivekanandarajah@linaro.org wrote:
On 21/05/14 16:08, Maxim Kuvyrkov wrote:
I have been thinking how to simplify cross-testing our toolchain for
both automated and development/debugging builds, and among various options the most universal I came up with is ARM hardware + ssh + binfmt_misc + sshfs. I wonder if anyone has already tried this or can suggest alternatives which are as universal.
Given:
- host x86_64 development machine
- cross-compiler
- target hardware with fast network to the host
- host and target have ssh
- testsuite (gcc/glibc/gdb/etc)
Here is how it is going to work
- On host we create a simple wrapper script that will pass through its
arguments as command to execute on target via ssh:
=== #!/bin/sh ssh -p 22NN $TARGET_BOARD "$@" ===
- We register this script in binfmt_misc to be used as interpreter for
target binaries. Value of $TARGET_BOARD will be picked up from the environment and can be set to different boards for different testsuite runs.
- The target board needs to be prepared for a particular testsuite run:
-- Runtime libraries need to be either copied or mounted via sshfs
from the host. It is an open question how best to install several sets of libraries (for parallel runs) so that each set appears to be main system libraries. My current thinking is a separate ssh server inside chroot per each test run.
-- Test directory needs to be sshfs mounted on target from host so
that the target could see test executables.
-- Preparation/finalization of the board can either be done
explicitly before/after testing. Or it can be done on demand by the aforementioned script: the script checks whether a multiplexed ssh socket exists, and, if not, it prepares the board and starts a multiplexed ssh connection.
Is this set-up for NX? Issue is how do we share the target board between different users? We can of-course initiate a lava hacking session but the amount of time we will have to wait to get the session active might be too long depending on the target classes availability.
Thanks Kugan
- Testing is fired up as if it is normal "native" testing. Whenever
kernel is given an ARM binary to execute -- it passes it off to wrapper, which passes it off to the target board via ssh. The board sees same filesystem as host and happily executes binaries against toolchain runtime libraries.
Comments or rotten tomatoes?
Thank you,
-- Maxim Kuvyrkov www.linaro.org
linaro-toolchain mailing list linaro-toolchain@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-toolchain
linaro-toolchain mailing list linaro-toolchain@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-toolchain
linaro-toolchain mailing list linaro-toolchain@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-toolchain
On May 21, 2014, at 10:54 PM, Bernie Ogden bernie.ogden@linaro.org wrote:
If we add models as targets then that alleviates the sharing problem a bit - for test runs that aren't going to be too slow on models. And you'll need to watch out for nested '"' in $@, but I guess you know that.
The point is to run it primarily on real hardware. Binfmt_misc is already working nicely if you want to do cross-testing with QEMU.
I guess this approach is similar to what 'make check test-wrapper="...../glibc/scripts/cross-test-ssh.sh hostname"' is supposed to do for glibc. Ryan mentioned that glibc cross-test doesn't work, and it certainly fell over when I tried it. That makes me wonder whether there are hidden difficulties here, but that vague bit of evidence aside, the approach sounds sensible to me.
As Ryan mentioned, it all works, but is a pain to setup. And "yes" the idea is very similar with the main distinction of using kernel's binfmt_misc instead of explicit test-wrapper.
-- Maxim Kuvyrkov www.linaro.org
On 21 May 2014 07:46, Kugan kugan.vivekanandarajah@linaro.org wrote:
On 21/05/14 16:08, Maxim Kuvyrkov wrote:
I have been thinking how to simplify cross-testing our toolchain for both automated and development/debugging builds, and among various options the most universal I came up with is ARM hardware + ssh + binfmt_misc + sshfs. I wonder if anyone has already tried this or can suggest alternatives which are as universal.
Given:
- host x86_64 development machine
- cross-compiler
- target hardware with fast network to the host
- host and target have ssh
- testsuite (gcc/glibc/gdb/etc)
Here is how it is going to work
- On host we create a simple wrapper script that will pass through its arguments as command to execute on target via ssh:
=== #!/bin/sh ssh -p 22NN $TARGET_BOARD "$@" ===
We register this script in binfmt_misc to be used as interpreter for target binaries. Value of $TARGET_BOARD will be picked up from the environment and can be set to different boards for different testsuite runs.
The target board needs to be prepared for a particular testsuite run:
-- Runtime libraries need to be either copied or mounted via sshfs from the host. It is an open question how best to install several sets of libraries (for parallel runs) so that each set appears to be main system libraries. My current thinking is a separate ssh server inside chroot per each test run. -- Test directory needs to be sshfs mounted on target from host so that the target could see test executables. -- Preparation/finalization of the board can either be done explicitly before/after testing. Or it can be done on demand by the aforementioned script: the script checks whether a multiplexed ssh socket exists, and, if not, it prepares the board and starts a multiplexed ssh connection.
Is this set-up for NX? Issue is how do we share the target board between different users? We can of-course initiate a lava hacking session but the amount of time we will have to wait to get the session active might be too long depending on the target classes availability.
Thanks Kugan
- Testing is fired up as if it is normal "native" testing. Whenever kernel is given an ARM binary to execute -- it passes it off to wrapper, which passes it off to the target board via ssh. The board sees same filesystem as host and happily executes binaries against toolchain runtime libraries.
Comments or rotten tomatoes?
Thank you,
-- Maxim Kuvyrkov www.linaro.org
linaro-toolchain mailing list linaro-toolchain@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-toolchain
linaro-toolchain mailing list linaro-toolchain@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-toolchain
On May 21, 2014, at 6:46 PM, Kugan kugan.vivekanandarajah@linaro.org wrote:
Is this set-up for NX? Issue is how do we share the target board between different users? We can of-course initiate a lava hacking session but the amount of time we will have to wait to get the session active might be too long depending on the target classes availability.
This is just a discussion at this point, and I didn't attempt a prototype yet.
User sharing will be controlled by the fact that different users will be ssh'ing in under their respective usernames to the board and operating as normal users (modulo sudo to start a new ssh server).
-- Maxim Kuvyrkov www.linaro.org
Hi,
This is essentially what sbrsh was created for a decade ago:
http://www.scratchbox.org/documentation/user/scratchbox-1.0/html/sbrsh.html http://packages.qa.debian.org/s/sbrsh.html
You run register sbrsh with binfmt-misc (or just prefix sbrsh command) and run sbrshd on the target device. You have a configuration file for sbrsh that tells what to mount from where.
Haven't used it for a long long time...
Riku
On 21 May 2014 09:08, Maxim Kuvyrkov maxim.kuvyrkov@linaro.org wrote:
I have been thinking how to simplify cross-testing our toolchain for both automated and development/debugging builds, and among various options the most universal I came up with is ARM hardware + ssh + binfmt_misc + sshfs. I wonder if anyone has already tried this or can suggest alternatives which are as universal.
Given:
- host x86_64 development machine
- cross-compiler
- target hardware with fast network to the host
- host and target have ssh
- testsuite (gcc/glibc/gdb/etc)
Here is how it is going to work
- On host we create a simple wrapper script that will pass through its
arguments as command to execute on target via ssh:
#!/bin/sh ssh -p 22NN $TARGET_BOARD "$@" ===
- We register this script in binfmt_misc to be used as interpreter for
target binaries. Value of $TARGET_BOARD will be picked up from the environment and can be set to different boards for different testsuite runs.
- The target board needs to be prepared for a particular testsuite run:
-- Runtime libraries need to be either copied or mounted via sshfs from the host. It is an open question how best to install several sets of libraries (for parallel runs) so that each set appears to be main system libraries. My current thinking is a separate ssh server inside chroot per each test run. -- Test directory needs to be sshfs mounted on target from host so that the target could see test executables. -- Preparation/finalization of the board can either be done explicitly before/after testing. Or it can be done on demand by the aforementioned script: the script checks whether a multiplexed ssh socket exists, and, if not, it prepares the board and starts a multiplexed ssh connection.
- Testing is fired up as if it is normal "native" testing. Whenever
kernel is given an ARM binary to execute -- it passes it off to wrapper, which passes it off to the target board via ssh. The board sees same filesystem as host and happily executes binaries against toolchain runtime libraries.
Comments or rotten tomatoes?
Thank you,
-- Maxim Kuvyrkov www.linaro.org
linaro-toolchain mailing list linaro-toolchain@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-toolchain
Thanks Riku!
I was hoping that someone would point me to an existing implementation. I will check it out.
-- Maxim Kuvyrkov www.linaro.org
On May 21, 2014, at 11:55 PM, Riku Voipio riku.voipio@linaro.org wrote:
Hi,
This is essentially what sbrsh was created for a decade ago:
http://www.scratchbox.org/documentation/user/scratchbox-1.0/html/sbrsh.html http://packages.qa.debian.org/s/sbrsh.html
You run register sbrsh with binfmt-misc (or just prefix sbrsh command) and run sbrshd on the target device. You have a configuration file for sbrsh that tells what to mount from where.
Haven't used it for a long long time...
Riku
On 21 May 2014 09:08, Maxim Kuvyrkov maxim.kuvyrkov@linaro.org wrote: I have been thinking how to simplify cross-testing our toolchain for both automated and development/debugging builds, and among various options the most universal I came up with is ARM hardware + ssh + binfmt_misc + sshfs. I wonder if anyone has already tried this or can suggest alternatives which are as universal.
Given:
- host x86_64 development machine
- cross-compiler
- target hardware with fast network to the host
- host and target have ssh
- testsuite (gcc/glibc/gdb/etc)
Here is how it is going to work
- On host we create a simple wrapper script that will pass through its arguments as command to execute on target via ssh:
=== #!/bin/sh ssh -p 22NN $TARGET_BOARD "$@" ===
We register this script in binfmt_misc to be used as interpreter for target binaries. Value of $TARGET_BOARD will be picked up from the environment and can be set to different boards for different testsuite runs.
The target board needs to be prepared for a particular testsuite run:
-- Runtime libraries need to be either copied or mounted via sshfs from the host. It is an open question how best to install several sets of libraries (for parallel runs) so that each set appears to be main system libraries. My current thinking is a separate ssh server inside chroot per each test run. -- Test directory needs to be sshfs mounted on target from host so that the target could see test executables. -- Preparation/finalization of the board can either be done explicitly before/after testing. Or it can be done on demand by the aforementioned script: the script checks whether a multiplexed ssh socket exists, and, if not, it prepares the board and starts a multiplexed ssh connection.
- Testing is fired up as if it is normal "native" testing. Whenever kernel is given an ARM binary to execute -- it passes it off to wrapper, which passes it off to the target board via ssh. The board sees same filesystem as host and happily executes binaries against toolchain runtime libraries.
Comments or rotten tomatoes?
Thank you,
-- Maxim Kuvyrkov www.linaro.org
linaro-toolchain mailing list linaro-toolchain@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-toolchain
linaro-toolchain@lists.linaro.org
-
Bernie Ogden -
Kugan -
Maxim Kuvyrkov -
Riku Voipio -
Ryan Arnold