W dniu 27.03.2012 15:04, James Tunnicliffe pisze:
On 27 March 2012 12:48, Zygmunt Krynicki zygmunt.krynicki@linaro.org wrote:
W dniu 27.03.2012 13:32, Paul Sokolovsky pisze:
Hello,
Yong Qin is working on the blueprint https://blueprints.launchpad.net/lava-android-test/+spec/modify-android-buil... to add arbitrary custom client-side scripts to Android Build. He submitted first implementation of it as https://code.launchpad.net/~liuyq0307/linaro-android-build-tools/run-custom/... and documented as https://wiki.linaro.org/Platform/Android/AndroidBuild-LavaIntegration .
Unfortunately, I'm not thrilled with that implementation, more specifically, its "user interface" (i.e. any parts which user directly faces) by the following reasons:
- The idea behind Android Build's build config was that they're short
and easy for human to parse, essentially one glance-over would enough to get a good idea what is built here, even for outsider. Consequently, the configs should not be overloaded with details not related to building. If there's a need for integration with other systems, we have good pattern of externalizing such details and then just referring to them with a single variable in a build config.
- The whole approach in
https://wiki.linaro.org/Platform/Android/AndroidBuild-LavaIntegration seems like trying to encode hierarchical structure in the shell syntax, which is not much supporting of that. The end result looks pretty much like representation of graph structure in raw assembler - it's spaghetti mix of data pieces and labels, requiring long time to wrap hand around to understand it, and cumbersome and error-prone to write.
So, I would like to propose alternative syntax solving the issues above. I probably should start with saying that if the talk is about LAVA, then using native LAVA JSON request immediately comes to mind. Well, I guess human-writability wasn't a design goal for that, so I skip it. It still makes sense to stick to general-purpose hierarchical structure syntax though. Except that JSON has 2 problems: a) it doesn't support comments natively, so we'll need to pre-process it; b) error reporting/localization may be still no ideal.
Hi, just jumping into the conversation briefly to look at small technical aspect. I have not really been tracking this command effort and I don't understand what it's about.
On JSON: I think that comment 2 is inaccurate. We have very precise syntax error reporting (down to line/column and text range on some errors) and even better format reporting (the javascript expression that pinpoints the part of json document that does not match the schema, same for the schema actually).
Anyway, you have my full support for native json formats. I think that if comments are an issue I can provide a parser that simply ignores comments. We could then keep the human readable documents and strictly machine readable, schema-backed data.
There is always YAML - very much human readable, supports hierarchical structures, references and comments.
CC-ing the list again.
My long established stance is that YAML is too baroque to be used as a data interchange format intended for humans to _write_.
Compare YAML spec: http://www.yaml.org/spec/1.2/spec.html With JSON spec: http://www.json.org/
Make sure to scroll down on the YAML spec, that's _is_ what people need to remember to understand what they write. The JSON spec is entirely defined in that railroad graph that fills my screen.
IMHO YAML is filled so many feature that it is virtually impossible to remember the full syntax. Last time I've checked there was still no single python parser that supported everything the spec has to offer. While JSON is strict to the point that it's sometimes annoying it does not let you make a mistake by accident, there's like 5 syntax elements to remember. YAML is full of, comparatively, exotic features that while otherwise harmless can easily confuse users (seemingly random syntax has meaning) and tools (oh, my parser does not support that feature).
The second argument against YAML is that JSON remains usable on the web and you don't need two interfaces. Keeping everything in one format is a good idea IMHO. This may be of no relevance here but if you look at the entire LAVA ecosystem so far there's no YAML there, just JSON.
Lastly, we have also built nice validation for JSON so that not only syntax but also the full structure can be checked automatically and reliably and I'm not aware of any for YAML at this time. The fact that YAML has support for loops and pointers makes the tools we have useless.
For the sake of human-readability _and_ writeability, a small subset of YAML might be better than JSON but the full spec IMHO suffers just a much as JSON, yet for opposite reasons.
My vote is -1.
Thanks ZK
On 27 March 2012 14:37, Zygmunt Krynicki zygmunt.krynicki@linaro.org wrote:
W dniu 27.03.2012 15:04, James Tunnicliffe pisze:
On 27 March 2012 12:48, Zygmunt Krynicki zygmunt.krynicki@linaro.org wrote:
W dniu 27.03.2012 13:32, Paul Sokolovsky pisze:
Hello,
Yong Qin is working on the blueprint https://blueprints.launchpad.net/lava-android-test/+spec/modify-android-buil... to add arbitrary custom client-side scripts to Android Build. He submitted first implementation of it as https://code.launchpad.net/~liuyq0307/linaro-android-build-tools/run-custom/... and documented as https://wiki.linaro.org/Platform/Android/AndroidBuild-LavaIntegration .
Unfortunately, I'm not thrilled with that implementation, more specifically, its "user interface" (i.e. any parts which user directly faces) by the following reasons:
- The idea behind Android Build's build config was that they're short
and easy for human to parse, essentially one glance-over would enough to get a good idea what is built here, even for outsider. Consequently, the configs should not be overloaded with details not related to building. If there's a need for integration with other systems, we have good pattern of externalizing such details and then just referring to them with a single variable in a build config.
- The whole approach in
https://wiki.linaro.org/Platform/Android/AndroidBuild-LavaIntegration seems like trying to encode hierarchical structure in the shell syntax, which is not much supporting of that. The end result looks pretty much like representation of graph structure in raw assembler - it's spaghetti mix of data pieces and labels, requiring long time to wrap hand around to understand it, and cumbersome and error-prone to write.
So, I would like to propose alternative syntax solving the issues above. I probably should start with saying that if the talk is about LAVA, then using native LAVA JSON request immediately comes to mind. Well, I guess human-writability wasn't a design goal for that, so I skip it. It still makes sense to stick to general-purpose hierarchical structure syntax though. Except that JSON has 2 problems: a) it doesn't support comments natively, so we'll need to pre-process it; b) error reporting/localization may be still no ideal.
Hi, just jumping into the conversation briefly to look at small technical aspect. I have not really been tracking this command effort and I don't understand what it's about.
On JSON: I think that comment 2 is inaccurate. We have very precise syntax error reporting (down to line/column and text range on some errors) and even better format reporting (the javascript expression that pinpoints the part of json document that does not match the schema, same for the schema actually).
Anyway, you have my full support for native json formats. I think that if comments are an issue I can provide a parser that simply ignores comments. We could then keep the human readable documents and strictly machine readable, schema-backed data.
There is always YAML - very much human readable, supports hierarchical structures, references and comments.
CC-ing the list again.
My long established stance is that YAML is too baroque to be used as a data interchange format intended for humans to _write_.
Compare YAML spec: http://www.yaml.org/spec/1.2/spec.html With JSON spec: http://www.json.org/
Make sure to scroll down on the YAML spec, that's _is_ what people need to remember to understand what they write. The JSON spec is entirely defined in that railroad graph that fills my screen.
IMHO YAML is filled so many feature that it is virtually impossible to remember the full syntax. Last time I've checked there was still no single python parser that supported everything the spec has to offer. While JSON is strict to the point that it's sometimes annoying it does not let you make a mistake by accident, there's like 5 syntax elements to remember. YAML is full of, comparatively, exotic features that while otherwise harmless can easily confuse users (seemingly random syntax has meaning) and tools (oh, my parser does not support that feature).
The second argument against YAML is that JSON remains usable on the web and you don't need two interfaces. Keeping everything in one format is a good idea IMHO. This may be of no relevance here but if you look at the entire LAVA ecosystem so far there's no YAML there, just JSON.
Lastly, we have also built nice validation for JSON so that not only syntax but also the full structure can be checked automatically and reliably and I'm not aware of any for YAML at this time. The fact that YAML has support for loops and pointers makes the tools we have useless.
For the sake of human-readability _and_ writeability, a small subset of YAML might be better than JSON but the full spec IMHO suffers just a much as JSON, yet for opposite reasons.
My vote is -1.
All good points. I have used it in the past in a very basic way because it was easy to write and all the data arrived in Python in a nice predictable format. I didn't use the references stuff, just:
# Comment thing: property: value another property: - list - of - values
I only suggested it because of the comments made in previous email about comments :-)
linaro-validation@lists.linaro.org
-
James Tunnicliffe -
Zygmunt Krynicki