You have listed how your implementation is similar to Greybus. You say what is not so great is streaming, i.e. the bulk data transfer needed to implement xmit_sync() and xmit_async() above. Greybus is too much RPC based. RPCs are actually what you want for most of the operations listed above, but i agree for data, in order to keep the transport fully loaded, you want double buffering. However, that appears to be possible with the current Greybus code.
gb_operation_unidirectional_timeout() says:
Yes, these are request messages that don't require a response. The acknowledgement is about when the host *sent it*, not when it got received. They're rarely used but I could see them being used this way. Still, you might be limited to 255 or so in-flight messages.
I don't actually see how you can have multiple messages in-flight, but maybe i'm missing something. It appears that upper layers pass the message down and then block on a completion. The signalling of that completion only happens when the message is on the wire. So it is all synchronous. In order to have multiple messages in-flight, the lower layer would have to copy the message, signal the completion, and then send the copy whenever the transport was free.
The network stack is however async by nature. The ndo_start_xmit call passes an skbuf. The data in the skbuf is setup for DMA transfer, and then ndo_start_xmit returns. Later, when the DMA has completed, the driver calls dev_kfree_skb() to say it has finished with the skb.
Ideally we want a similar async mechanism, which is why i suggested gb_operation_unidirectional_async(). Pass a message to Greybus, none blocking, and have a callback for when the message has hit the wire and the skb can be friend. The low level can then keep a list of skb's so it can quickly do back to back transfers over the transport to keep it busy.
Andrew