New subject: SUMMARY: Re: New Development

29 Nov 2016


      OK now that we have lists and IRC in place, it's time to start
using them.  Bryan (at least) has posted some patches for review
but if Greybus is going to provide some greater long-term usefulness
it will need to be adapted to be more easily used in contexts outside
Project Ara.
There was some informal conversation about this at ELCE.  I have a
few of my own thoughts but mainly I just want to get some discussion
started to see if we have some pieces we can agree on, and take things
from there.  Here's a short summary of what's below:
- Retargeting Greybus for use in IoT seems reasonable
- We will need to adjust the abstract model currently implemented for
  Greybus to match the new target
- Certain core functionality should be useful as-is; but some other
  features will need to be re-thought and redefined.
- We will need to evolve the Greybus kernel code to define what is
  truly "core" functionality, and what that core needs from its
  environment.
- We will need to ensure the Greybus API is stable, and that our
  versioning design is sound.
One thing that came up multiple times in October was the notion that
Greybus could serve as the basis for communication among entities in
an Internet of Things system.  I agree with this, at least in part.
We have set up a system with some well-defined mechanisms for
encapsulating messages, along with a model of remote procedure calls
that provide a pretty robust way of getting things done across a
network.  The core code implements a lot of features (like timeouts
and message cancellation) that are very useful, and which should
really be done in a generic way.  We've got code and conventions in
the existing protocol drivers that provide good examples of how
to use the core functionality.  I think it's reasonable to pursue
adapting Greybus to be usable in IoT environments.
In any case, one thing that occurred to me during that week and
those discussions was that we really need to define a new *target*
for Greybus.  That is, we no longer have the Ara phone as the
hardware target, and we need to somehow define another ultimate
goal for the hardware or environment in which Greybus operates.
This should be more abstract than a hardware definition; instead
it ought to define properties of and services provided by the
entities communicating in a Greybus system.  In some ways we
already have that (Interfaces and Modules are abstract, but
the details of what they represent may no longer match what
is required).
I think the things that Greybus defines now that can be pretty
universally useful are the core concepts:
- generic Greybus host device driver interface
- connections with fixed protocols that define how messages
  carried over them are interpreted
- the format of messages, and the generic message header
- the overall messaging model, including sending, receiving
  cancelling messages
- the operations RPC model, including completion semantics
  (asynchronous completion, unidirectional operations, first
  error result prevails, and the ability to have multiple
  outstanding operations)
...
From there we get into some things that are almost certainly
useful, but there may need to be some changes:
- The grouping of connections into bundles, which attach to
  Linux device drivers.  (I think we need this, but there
  remains some work to do, especially in relation to the
  next item.)
- Manifests to describe module/interface functionality
- Control connection and how it's used
And we can keep going into other things that become more and
more likely to be unnecessary or just plain wrong when we
are targeting something different from an Ara phone:
- SVC and the SVC protocol
The above is not at all exhaustive; I just wanted to try to
identify a few specific things we can maybe start to agree
on.
Currently, the control protocol and SVC protocol are very
much biased toward the Ara hardware.  (Other protocols are
too, like time sync and maybe firmware update.)  Furthermore,
pieces of these have necessarily made their way into core
Greybus code.  As we define a new non-Ara target, we need
to figure out what parts really belong in the Greybus core,
and which things need to be supplied either by external
entities or by protocol drivers.  One way to do that might
be to implement the core code as a library, but to be honest
I think the real work lies in teasing things apart so the
core can be defined to supply certain well-known services,
with hooks to allow other capabilities.  For a (contrived)
example of what I mean, we may need to provide hooks for
setting up connection routing rather than implementing
it in the core code.  In any case, I think we need to look
at what constitutes the true "core" of Greybus services
and what that core needs from its environment in order to
be used.
If Greybus is going to provide a more general service we need
to ensure it provides a stable API.  We have structured things
this way since the beginning, but I fear we may have gotten
sloppy because we basically always packaged everything together.
As it is, all the versioning is in place but to my knowledge
we've never really tested it to make sure it works.
I had another point which I've now forgotten...  In any case,
I really just hope to get some conversation going.
What do you think?
-Alex