[Linaro-validation] Syntax for custom LAVA tests
james.tunnicliffe at linaro.org
Tue Mar 27 14:04:43 UTC 2012
On 27 March 2012 14:37, Zygmunt Krynicki <zygmunt.krynicki at linaro.org> wrote:
> W dniu 27.03.2012 15:04, James Tunnicliffe pisze:
>> On 27 March 2012 12:48, Zygmunt Krynicki <zygmunt.krynicki at linaro.org> wrote:
>>> W dniu 27.03.2012 13:32, Paul Sokolovsky pisze:
>>>> Yong Qin is working on the blueprint
>>>> to add arbitrary custom client-side scripts to Android Build. He
>>>> submitted first implementation of it as
>>>> and documented as
>>>> https://wiki.linaro.org/Platform/Android/AndroidBuild-LavaIntegration .
>>>> Unfortunately, I'm not thrilled with that implementation, more
>>>> specifically, its "user interface" (i.e. any parts which user directly
>>>> faces) by the following reasons:
>>>> 1. The idea behind Android Build's build config was that they're short
>>>> and easy for human to parse, essentially one glance-over would enough
>>>> to get a good idea what is built here, even for outsider. Consequently,
>>>> the configs should not be overloaded with details not related to
>>>> building. If there's a need for integration with other systems, we have
>>>> good pattern of externalizing such details and then just referring to
>>>> them with a single variable in a build config.
>>>> 2. The whole approach in
>>>> seems like trying to encode hierarchical structure in the shell syntax,
>>>> which is not much supporting of that. The end result looks pretty much
>>>> like representation of graph structure in raw assembler - it's
>>>> spaghetti mix of data pieces and labels, requiring long time to wrap
>>>> hand around to understand it, and cumbersome and error-prone to write.
>>>> So, I would like to propose alternative syntax solving the issues
>>>> above. I probably should start with saying that if the talk is about
>>>> LAVA, then using native LAVA JSON request immediately comes to mind.
>>>> Well, I guess human-writability wasn't a design goal for that, so I
>>>> skip it. It still makes sense to stick to general-purpose hierarchical
>>>> structure syntax though. Except that JSON has 2 problems: a) it doesn't
>>>> support comments natively, so we'll need to pre-process it; b) error
>>>> reporting/localization may be still no ideal.
>>> Hi, just jumping into the conversation briefly to look at small
>>> technical aspect. I have not really been tracking this command effort
>>> and I don't understand what it's about.
>>> On JSON: I think that comment 2 is inaccurate. We have very precise
>>> syntax error reporting (down to line/column and text range on some
>>> pinpoints the part of json document that does not match the schema, same
>>> for the schema actually).
>>> Anyway, you have my full support for native json formats. I think that
>>> if comments are an issue I can provide a parser that simply ignores
>>> comments. We could then keep the human readable documents and strictly
>>> machine readable, schema-backed data.
>> There is always YAML - very much human readable, supports hierarchical
>> structures, references and comments.
> CC-ing the list again.
> My long established stance is that YAML is too baroque to be used as a
> data interchange format intended for humans to _write_.
> Compare YAML spec: http://www.yaml.org/spec/1.2/spec.html
> With JSON spec: http://www.json.org/
> Make sure to scroll down on the YAML spec, that's _is_ what people need
> to remember to understand what they write. The JSON spec is entirely
> defined in that railroad graph that fills my screen.
> IMHO YAML is filled so many feature that it is virtually impossible to
> remember the full syntax. Last time I've checked there was still no
> single python parser that supported everything the spec has to offer.
> While JSON is strict to the point that it's sometimes annoying it does
> not let you make a mistake by accident, there's like 5 syntax elements
> to remember. YAML is full of, comparatively, exotic features that while
> otherwise harmless can easily confuse users (seemingly random syntax has
> meaning) and tools (oh, my parser does not support that feature).
> The second argument against YAML is that JSON remains usable on the web
> and you don't need two interfaces. Keeping everything in one format is a
> good idea IMHO. This may be of no relevance here but if you look at the
> entire LAVA ecosystem so far there's no YAML there, just JSON.
> Lastly, we have also built nice validation for JSON so that not only
> syntax but also the full structure can be checked automatically and
> reliably and I'm not aware of any for YAML at this time. The fact that
> YAML has support for loops and pointers makes the tools we have useless.
> For the sake of human-readability _and_ writeability, a small subset of
> YAML might be better than JSON but the full spec IMHO suffers just a
> much as JSON, yet for opposite reasons.
> My vote is -1.
All good points. I have used it in the past in a very basic way
because it was easy to write and all the data arrived in Python in a
nice predictable format. I didn't use the references stuff, just:
I only suggested it because of the comments made in previous email
about comments :-)
More information about the linaro-validation