W dniu 18.10.2012 16:28, Abner Silva pisze:

Hello,

Sorry for my delay. I have been busy with other things.

Hey Abner, good to see you here again!

It's good to see people discussing it. I'm glad.

Here are some comments about this initial spec:

[Terminologies]

As a section says, it's not going to change the test/testcase/testrun/testresult terminology for now. It would be good to have each of these terminologies well explained somewhere. It might be crystal clear to you guys, but for other people it might be confusing when attending to the Linaro Connect session, since these same terminologies are commonly used to represent other QA artifacts.

They are somewhat explained in the glossary section of linaro-dashboard-bundle python library: http://linaro-dashboard-bundle.readthedocs.org/en/latest/glossary.html

I'd gladly accept patches to that library to reword / improve it [edit: I cannot accept patches there anymore but I'm sure we can correct that]

[Test case concept]

The first thing that passed through my head when I started thinking about TC (sorry, I don't know what to call it considering LAVA naming) inside LAVA, was that it would be a different Object/artifact/file, and not really represented by the test definition concept/file LAVA already have nowadays.

Is that what you guys are planning to do? To consider the test def that we have today as the TC entity? If yes, I probably have some questions about it.

I think that a test definition (currently) merely specifies test (suite) and it can have many test cases as an effect of running. Having said that I'd like to see a test definition that clearly states all of the test cases it can produce, with sensible description/meta data for each of them.

I guess it depends on the number of test cases, behemoths such as LTP with test cases in the thousands are probably going to not offer per-test case meta-data just yet.

Still, whatever we choose to do, initially I would recommend a discovery phase where we enumerate all the things that we have encountered and classified. As we gather visibility on that data we can improve the definitions of our terminology. In the longer term I would like to see common vocabulary that spans "development camps" so that when LAVA developers talk to Checkbox developers and Phoronix developers (for example) there are no misunderstandings.

Thanks ZK