Another crazy option would be to expose LAVA Job Dispatcher directly and allow people to run jobs. In this case one job would use abrek and some other tools to invoke tests, process results and send them to the dashboard while other job (one for android) would not run abrek at all, instead it would call some other helper, while still reusing identical components for "process results and send to result storage" phases. This is still in flux but has some advantages:
0) Jobs are simple text files that can be stored and shared with others
1) Jobs can encapsulate device information like which android device to connect to and how.
2) Jobs can still "call" to other parts of the LAVA stack such as result submission
3) Jobs can be extended locally (by LAVA plugins) so that anyone can develop specialized use cases for their very specific needs without
altering the stack or having to write something completely custom.

I think exposing Job Dispatcher directly would not be a good idea for validation farm, where test jobs are queued for execution via Scheduler (see LAVA architecture). Bypassing the job queue on the Dispatcher level should only be allowed in exceptional cases, i.e. canceling jobs for server/board update or similar. There might be scenarios where the unrestricted direct control is desirable, but that should be only allowed for local development environments.
I hope you don't really think this is a crazy idea, because it's *exactly* what we discussed doing, and I think it is still the direction forward.  Even back at the sprint, we talked about the fact that though we don't know exactly what our android images will look like yet (so specifics are hard to plan around), we do have a pretty good idea that things like abrek are not going to be a good fit there.  Additionally, there are other tests that may eventually need to run from the outside.  For that matter, the boot test itself is exactly one of those things.  If a system doesn't boot, we still want to capture the serial log and save it for later debugging.  We certainly need, and have already discussed the fact that the dispatcher has a server component that drives execution of such things.  This isn't, in any way, bypassing the dispatcher, but rather using a different interface of it to run a non-abrek based test.

Despite that, this is still a useful exercise, and many thanks to Jim for putting this together to show us a proof of concept for how we can run tests remotely on android systems!

Thanks,
Paul Larson