As one action from last kernel WG meeting, I have investigate the availability of boards Remote Access.
Right now, we don't have anything setup for remote access. Either we have to have the boards in hand or we have to check the https://wiki.linaro.org/Internal/Hardware and see who has the needed board(s) and see whether we can run the tests on that board9s).
The Validation's team is looking at supporting building and testing the landing team kernels, at that time we may have something we can work with. The kernel engineer will build and create the kernel image that needs to be tested, then he/she would run the setup that is availble to the Landing teams. (this is still work in progress, we don't know the exact details yet).
On the long run, maybe we should open a Stakeholders Proposal to address Remote Access and make it easier to the engineers to reserve a board and use it for a short while.
Regards, Mounir
What kind of remote access usage would you need?
Since we're on the kernel list, I guess you'd want to test kernels remotely; that could be a bit tricky though
We have a need for a Panda and a Beagle to compare & fix performance. As I recall from last meeting, the Panda's USB I/O has 2x performance degradation from Beagle. Per or Mian may have more details on that.
On Thu, Feb 24, 2011 at 6:31 PM, Loïc Minier loic.minier@linaro.org wrote:
What kind of remote access usage would you need?
Since we're on the kernel list, I guess you'd want to test kernels remotely; that could be a bit tricky though
-- Loďc Minier
On Thu, Feb 24, 2011, Mounir Bsaibes wrote:
We have a need for a Panda and a Beagle to compare & fix performance. As I recall from last meeting, the Panda's USB I/O has 2x performance degradation from Beagle. Per or Mian may have more details on that.
Ok; interesting!
My initial fear was that the kernel testing might trigger unbootable boards and that it would be tricky to work with unstable kernels remotely; for USB performance testing, I can see that we could set it up for remote testing, but we need to have something plugged to USB and be careful to use the same hardware on both boards so that it compares usefully.
On 02/25/2011 10:58 AM, Somebody in the thread at some point said:
On Thu, Feb 24, 2011, Mounir Bsaibes wrote:
Hi -
We have a need for a Panda and a Beagle to compare& fix performance. As I recall from last meeting, the Panda's USB I/O has 2x performance degradation from Beagle. Per or Mian may have more details on that.
Ok; interesting!
My initial fear was that the kernel testing might trigger unbootable boards and that it would be tricky to work with unstable kernels remotely; for USB performance testing, I can see that we could set it up for remote testing, but we need to have something plugged to USB and be careful to use the same hardware on both boards so that it compares usefully.
1&1 (webhosts) have an interesting approach to this for their unmanaged boxes, they have the same problem with people doing kernel or bootloader upgrades making their box unworkable.
You can ssh into a second "buddy" box who has a serial connection to the target box's serial port (in the 1&1 case they set grub even to issue serial menus). The buddy box also controls the target via a smart power switch.
When you cycle the target power using a web interface to the buddy box, you can also select to force a canned boot path sent over bootp, so even a trashed box with no bootloader can be recovered.
When you select a canned boot, you can use your ssh connection to the buddy box to see boot messages and login to the target, so it's an ssh <-> serial bridge basically.
They deploy this on all their unmanaged servers, they just have a few buddy boxes each handling many servers.
It's not far away from being workable on embedded boards, maybe by having an adapter card that sites two SD cards in one slot, you can enable the normal one or a backup read-only one based on a line controlled by its buddy box.
-Andy
On Fri, Feb 25, 2011 at 10:58 AM, Loïc Minier loic.minier@linaro.org wrote:
On Thu, Feb 24, 2011, Mounir Bsaibes wrote:
We have a need for a Panda and a Beagle to compare & fix performance. As I recall from last meeting, the Panda's USB I/O has 2x performance degradation from Beagle. Per or Mian may have more details on that.
Ok; interesting!
My initial fear was that the kernel testing might trigger unbootable boards and that it would be tricky to work with unstable kernels remotely; for USB performance testing, I can see that we could set it up for remote testing, but we need to have something plugged to USB and be careful to use the same hardware on both boards so that it compares usefully.
If we never upgrade the bootloader, and make sure that the known-good bootscripts/images are never modified/removed, then we should generally be able to avoid bricking.
On vexpress for example, we have a known-good U-Boot in flash which can be used to recover the board if needed.
Where random hardware has to be plugged in, that could be a bit harder to arrange...
Cheers ---Dave
Pro
-- Loïc Minier
linaro-kernel mailing list linaro-kernel@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-kernel
On Fri, Feb 25, 2011, Andy Green wrote:
You can ssh into a second "buddy" box who has a serial connection to the target box's serial port (in the 1&1 case they set grub even to issue serial menus). The buddy box also controls the target via a smart power switch.
Yup; I'm familiar with this idea (well, except my hosting provider doesn't have a serial line, they just start an Ubuntu live CD with VNC and SSH) and we do have remote controlled power, but...
When you cycle the target power using a web interface to the buddy box, you can also select to force a canned boot path sent over bootp, so even a trashed box with no bootloader can be recovered.
not all boards boot when power-cycled (!) and you can't test the default bootloader with this approach (e.g. if your card only boots from SD and you want to test u-boot-linaro on SD, you might not be able to recover from a broken bootloader), but for the kernel that would be fine. I think almost all our current boards boot when power-cycled though.
It's not far away from being workable on embedded boards, maybe by having an adapter card that sites two SD cards in one slot, you can enable the normal one or a backup read-only one based on a line controlled by its buddy box.
adapter card > interesting idea! Does this require us to build custom hardware, or does this exist on the shelves already?
On Fri, Feb 25, 2011, Dave Martin wrote:
On vexpress for example, we have a known-good U-Boot in flash which can be used to recover the board if needed.
That's good if you have flash, yup (unless you're testing the bootloader in flash, but we currently never want to do this).
On 02/25/2011 01:13 PM, Somebody in the thread at some point said:
Hi -
When you cycle the target power using a web interface to the buddy box, you can also select to force a canned boot path sent over bootp, so even a trashed box with no bootloader can be recovered.
not all boards boot when power-cycled (!) and you can't test the default bootloader with this approach (e.g. if your card only boots from SD and you want to test u-boot-linaro on SD, you might not be able to recover from a broken bootloader), but for the kernel that would be fine. I think almost all our current boards boot when power-cycled though.
For PC hardware you can usually either set this on the board or BIOS; all the embedded boards I know of will start up on power coming. I'm willing to believe ones that don't might exist but I doubt they can maintain that sang froid in the face of their power-on button also being overridden at the same time ^^
It's not far away from being workable on embedded boards, maybe by having an adapter card that sites two SD cards in one slot, you can enable the normal one or a backup read-only one based on a line controlled by its buddy box.
adapter card> interesting idea! Does this require us to build custom hardware, or does this exist on the shelves already?
I never saw one; however this exists for $0.71 in quantity
http://www.fairchildsemi.com/ds/FS/FSSD06.pdf
so designing, laying out, and prototyping one is no problem for me to do from here with my existing tools if you should be interested.
I designed a similar emulation board about 12 years ago for smartcard readers, it's the same basic deal.
I guess it should present itself to the buddy box as a USB <-> serial adapter and use one of the unused serial handshake lines from PC direction to control which card is enabled, and another to switch the target power. The actual serial traffic would be the board debug port. These adapter chips are likewise cheap and simple and off the shelf. You could control it with stty without any special driver then.
-Andy
Dave Martin wrote:
On Fri, Feb 25, 2011 at 10:58 AM, Loďc Minier loic.minier@linaro.org wrote:
On Thu, Feb 24, 2011, Mounir Bsaibes wrote:
We have a need for a Panda and a Beagle to compare & fix performance. As I recall from last meeting, the Panda's USB I/O has 2x performance degradation from Beagle. Per or Mian may have more details on that.
Ok; interesting!
My initial fear was that the kernel testing might trigger unbootable boards and that it would be tricky to work with unstable kernels remotely; for USB performance testing, I can see that we could set it up for remote testing, but we need to have something plugged to USB
and
be careful to use the same hardware on both boards so that it
compares
usefully.
If we never upgrade the bootloader, and make sure that the known-good bootscripts/images are never modified/removed, then we should generally be able to avoid bricking.
On vexpress for example, we have a known-good U-Boot in flash which can be used to recover the board if needed.
Where random hardware has to be plugged in, that could be a bit harder to arrange...
I work remotely with my Pandaboard - I have the UART plugged into a desktop and a USB relay board for the power supply. I have a screen session with picocom running on the desktop all the time.
I keep known-working loaders on the SD card with a "uImage-flasher" image alongside. (Essentially a kernel with drivers enabled for the EHCI controller and the USB-LAN chip + initramfs + tiny scripts that pick up a kernel image from a hardcoded location using tftp and replace the one currently on the SD card.
To get into the flasher image, I just halt manually in u-boot and load uimage-flasher. Else, u-boot is configured to pick up 'uImage' by default.
Issues: - Can't work remotely if you're working on x-loader code (flashing a wrong image MLO/u-boot could mean a physical visit to the board) --- Okay for u-boot if you want to load u-boot from within a known- working u-boot. - Someone 'borrows' the board without asking. --- I've had folks walk in, see the board apparently unused, and borrow it for a - leaving me wondering why the board stopped responding. :(
- Anand
On 25 February 2011 04:24, Mounir Bsaibes mounir.bsaibes@linaro.org wrote:
We have a need for a Panda and a Beagle to compare & fix performance. As I recall from last meeting, the Panda's USB I/O has 2x performance degradation from Beagle. Per or Mian may have more details on that.
It came up during the last weekly kernel meeting. Mian is working within the USB area but he doesn't have access to a Panda nor a Beagle. To investigate this bug it would probably involve updating the kernel and not likely any boot loaders. I am working on generic MMC updates and have the need to test and benchmark across several Linaro boards too.
/Per
On Thu, Feb 24, 2011 at 6:31 PM, Loïc Minier loic.minier@linaro.org wrote:
What kind of remote access usage would you need?
Since we're on the kernel list, I guess you'd want to test kernels remotely; that could be a bit tricky though
-- Loďc Minier
linaro-kernel mailing list linaro-kernel@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-kernel
linaro-kernel@lists.linaro.org
-
Anand Gadiyar -
Andy Green -
Dave Martin -
Loïc Minier -
Mounir Bsaibes -
Per Forlin