From: Frank Rowand frank.rowand@sony.com
An August 2021 RFC patch [1] to create the KTAP Specification resulted in some discussion of possible items to add to the specification. The conversation ended without completing the document.
Progress resumed with a December 2021 RFC patch [2] to add a KTAP Specification file (Version 1) to the Linux kernel. Many of the suggestions from the August 2021 discussion were not included in Version 1. This patch series is intended to revisit some of the suggestions from the August 2021 discussion.
Patch 1 changes the Specification version to "2-rc" to indicate that following patches are not yet accepted into a final version 2.
Patch 2 is an example of a simple change to the Specification. The change does not change the content of the Specification, but updates a formatting directive as suggested by the Documentation maintainer.
I intend to take some specific suggestions from the August 2021 discussion to create stand-alone RFC patches to the Specification instead of adding them as additional patches in this series. The intent is to focus discussion on a single area of the Specification in each patch email thread.
[1] https://lore.kernel.org/r/CA+GJov6tdjvY9x12JsJT14qn6c7NViJxqaJk+r-K1YJzPggFD... [2] https://lore.kernel.org/r/20211207190251.18426-1-davidgow@google.com
Frank Rowand (2): Documentation: dev-tools: KTAP spec change version to 2-rc Documentation: dev-tools: use literal block instead of code-block
Documentation/dev-tools/ktap.rst | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-)
From: Frank Rowand frank.rowand@sony.com
Prepare KTAP Specification for the process of creating Version 2. The version will remain 2-rc until the final commit to complete Version 2.
Signed-off-by: Frank Rowand frank.rowand@sony.com --- Documentation/dev-tools/ktap.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Documentation/dev-tools/ktap.rst b/Documentation/dev-tools/ktap.rst index 5ee735c6687f..37b5dc61bfb8 100644 --- a/Documentation/dev-tools/ktap.rst +++ b/Documentation/dev-tools/ktap.rst @@ -1,7 +1,7 @@ .. SPDX-License-Identifier: GPL-2.0
=================================================== -The Kernel Test Anything Protocol (KTAP), version 1 +The Kernel Test Anything Protocol (KTAP), version 2-rc ===================================================
TAP, or the Test Anything Protocol is a format for specifying test results used
On Thu, Mar 17, 2022 at 4:26 AM frowand.list@gmail.com wrote:
From: Frank Rowand frank.rowand@sony.com
Prepare KTAP Specification for the process of creating Version 2. The version will remain 2-rc until the final commit to complete Version 2.
Signed-off-by: Frank Rowand frank.rowand@sony.com
This seems okay to me, though I'd generally rather this stay in a branch rather than hitting torvalds/master while there aren't any substantive changes to the actual spec. (Basically, let's not rush naming this "2-rc" for 5.18 if there aren't any other changes: a 2-rc which is practically the same as 1 is probably going to be more confusing than helpful if it sticks around for a whole kernel release.
(Also, when would we want to update the various mentions of "KTAP version 1" in the document to "KTAP version 2" or "KTAP version 2-rc"? Now, when 2 is released, at the first breaking change to that example, etc?)
Cheers, -- David
Documentation/dev-tools/ktap.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Documentation/dev-tools/ktap.rst b/Documentation/dev-tools/ktap.rst index 5ee735c6687f..37b5dc61bfb8 100644 --- a/Documentation/dev-tools/ktap.rst +++ b/Documentation/dev-tools/ktap.rst @@ -1,7 +1,7 @@ .. SPDX-License-Identifier: GPL-2.0
=================================================== -The Kernel Test Anything Protocol (KTAP), version 1
+The Kernel Test Anything Protocol (KTAP), version 2-rc
Nit: Sphinx complains that we now need more '=' signs:
Documentation/dev-tools/ktap.rst:3: WARNING: Title overline too short. ===================================================
The Kernel Test Anything Protocol (KTAP), version 2-rc ===================================================
TAP, or the Test Anything Protocol is a format for specifying test results used
Frank Rowand frank.rowand@sony.com
On 3/17/22 03:43, David Gow wrote:
On Thu, Mar 17, 2022 at 4:26 AM frowand.list@gmail.com wrote:
From: Frank Rowand frank.rowand@sony.com
Prepare KTAP Specification for the process of creating Version 2. The version will remain 2-rc until the final commit to complete Version 2.
Signed-off-by: Frank Rowand frank.rowand@sony.com
This seems okay to me, though I'd generally rather this stay in a branch rather than hitting torvalds/master while there aren't any substantive changes to the actual spec. (Basically, let's not rush naming this "2-rc" for 5.18 if there aren't any other changes: a 2-rc which is practically the same as 1 is probably going to be more confusing than helpful if it sticks around for a whole kernel release.
Sounds good. In my reply (a few minutes ago) to your other email, I offer to host a branch. The branch approach sounds good to me.
(Also, when would we want to update the various mentions of "KTAP version 1" in the document to "KTAP version 2" or "KTAP version 2-rc"? Now, when 2 is released, at the first breaking change to that example, etc?)
There are only a few references to version 1. I can create a patch to change them to version 2 (other than the version of the document itself).
If we are working on a branch, it should not make a big difference when the internal uses of "version 1" are changed to "version 2". If I do it right away then it won't get overlooked.
-Frank
Cheers, -- David
Documentation/dev-tools/ktap.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Documentation/dev-tools/ktap.rst b/Documentation/dev-tools/ktap.rst index 5ee735c6687f..37b5dc61bfb8 100644 --- a/Documentation/dev-tools/ktap.rst +++ b/Documentation/dev-tools/ktap.rst @@ -1,7 +1,7 @@ .. SPDX-License-Identifier: GPL-2.0
=================================================== -The Kernel Test Anything Protocol (KTAP), version 1
+The Kernel Test Anything Protocol (KTAP), version 2-rc
Nit: Sphinx complains that we now need more '=' signs:
Documentation/dev-tools/ktap.rst:3: WARNING: Title overline too short.
The Kernel Test Anything Protocol (KTAP), version 2-rc
TAP, or the Test Anything Protocol is a format for specifying test results used
Frank Rowand frank.rowand@sony.com
From: Frank Rowand frank.rowand@sony.com
KTAP Specification: Change code-block directives to straightforward literal blocks since the blocks do not contain code.
Suggested-by: Jonathan Corbet corbet@lwn.net Signed-off-by: Frank Rowand frank.rowand@sony.com --- Documentation/dev-tools/ktap.rst | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-)
diff --git a/Documentation/dev-tools/ktap.rst b/Documentation/dev-tools/ktap.rst index 37b5dc61bfb8..b9a57ceddd4f 100644 --- a/Documentation/dev-tools/ktap.rst +++ b/Documentation/dev-tools/ktap.rst @@ -115,34 +115,32 @@ The diagnostic data field is optional, and results which have neither a directive nor any diagnostic data do not need to include the "#" field separator.
-Example result lines include: - -.. code-block:: none +Example result lines include::
ok 1 test_case_name
The test "test_case_name" passed.
-.. code-block:: none +::
not ok 1 test_case_name
The test "test_case_name" failed.
-.. code-block:: none +::
ok 1 test # SKIP necessary dependency unavailable
The test "test" was SKIPPED with the diagnostic message "necessary dependency unavailable".
-.. code-block:: none +::
not ok 1 test # TIMEOUT 30 seconds
The test "test" timed out, with diagnostic data "30 seconds".
-.. code-block:: none +::
ok 5 check return code # rcode=0
@@ -202,7 +200,7 @@ allowed to be either indented or not indented.
An example of a test with two nested subtests:
-.. code-block:: none +::
KTAP version 1 1..1 @@ -215,7 +213,7 @@ An example of a test with two nested subtests:
An example format with multiple levels of nested testing:
-.. code-block:: none +::
KTAP version 1 1..2 @@ -250,7 +248,7 @@ nested version line, uses a line of the form
Example KTAP output -------------------- -.. code-block:: none +::
KTAP version 1 1..1
On Thu, Mar 17, 2022 at 4:26 AM frowand.list@gmail.com wrote:
From: Frank Rowand frank.rowand@sony.com
KTAP Specification: Change code-block directives to straightforward literal blocks since the blocks do not contain code.
Suggested-by: Jonathan Corbet corbet@lwn.net Signed-off-by: Frank Rowand frank.rowand@sony.com
This looks good to me.
I'd personally rather push this through independently of the KTAP 2.0 spec updates, as it's really just a minor formatting change to the spec, and it has no impact on the actual KTAP format.
So, if we can accept this independently, that'd be swell.
Reviewed-by: David Gow davidgow@google.com
Cheers, -- David
Documentation/dev-tools/ktap.rst | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-)
diff --git a/Documentation/dev-tools/ktap.rst b/Documentation/dev-tools/ktap.rst index 37b5dc61bfb8..b9a57ceddd4f 100644 --- a/Documentation/dev-tools/ktap.rst +++ b/Documentation/dev-tools/ktap.rst @@ -115,34 +115,32 @@ The diagnostic data field is optional, and results which have neither a directive nor any diagnostic data do not need to include the "#" field separator.
-Example result lines include:
-.. code-block:: none +Example result lines include::
ok 1 test_case_nameThe test "test_case_name" passed.
-.. code-block:: none +::
not ok 1 test_case_nameThe test "test_case_name" failed.
-.. code-block:: none +::
ok 1 test # SKIP necessary dependency unavailableThe test "test" was SKIPPED with the diagnostic message "necessary dependency unavailable".
-.. code-block:: none +::
not ok 1 test # TIMEOUT 30 secondsThe test "test" timed out, with diagnostic data "30 seconds".
-.. code-block:: none +::
ok 5 check return code # rcode=0@@ -202,7 +200,7 @@ allowed to be either indented or not indented.
An example of a test with two nested subtests:
-.. code-block:: none +::
KTAP version 1 1..1@@ -215,7 +213,7 @@ An example of a test with two nested subtests:
An example format with multiple levels of nested testing:
-.. code-block:: none +::
KTAP version 1 1..2@@ -250,7 +248,7 @@ nested version line, uses a line of the form
Example KTAP output
-.. code-block:: none +::
KTAP version 1 1..1-- Frank Rowand frank.rowand@sony.com
Hi Jon,
On 3/17/22 03:43, David Gow wrote:
On Thu, Mar 17, 2022 at 4:26 AM frowand.list@gmail.com wrote:
From: Frank Rowand frank.rowand@sony.com
KTAP Specification: Change code-block directives to straightforward literal blocks since the blocks do not contain code.
Suggested-by: Jonathan Corbet corbet@lwn.net Signed-off-by: Frank Rowand frank.rowand@sony.com
This looks good to me.
I'd personally rather push this through independently of the KTAP 2.0 spec updates, as it's really just a minor formatting change to the spec, and it has no impact on the actual KTAP format.
So, if we can accept this independently, that'd be swell.
Reviewed-by: David Gow davidgow@google.com
I got distracted with other tasks, just now returning to this.
There has been no other comment in the last month, so I think it is fair to go ahead and ask you to pull this patch.
Can you pull just this patch (patch 2/2) or would you prefer that I do a version 2 which does not contain patch 1/2?
-Frank
Cheers, -- David
Documentation/dev-tools/ktap.rst | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-)
diff --git a/Documentation/dev-tools/ktap.rst b/Documentation/dev-tools/ktap.rst index 37b5dc61bfb8..b9a57ceddd4f 100644 --- a/Documentation/dev-tools/ktap.rst +++ b/Documentation/dev-tools/ktap.rst @@ -115,34 +115,32 @@ The diagnostic data field is optional, and results which have neither a directive nor any diagnostic data do not need to include the "#" field separator.
-Example result lines include:
-.. code-block:: none +Example result lines include::
ok 1 test_case_nameThe test "test_case_name" passed.
-.. code-block:: none +::
not ok 1 test_case_nameThe test "test_case_name" failed.
-.. code-block:: none +::
ok 1 test # SKIP necessary dependency unavailableThe test "test" was SKIPPED with the diagnostic message "necessary dependency unavailable".
-.. code-block:: none +::
not ok 1 test # TIMEOUT 30 secondsThe test "test" timed out, with diagnostic data "30 seconds".
-.. code-block:: none +::
ok 5 check return code # rcode=0@@ -202,7 +200,7 @@ allowed to be either indented or not indented.
An example of a test with two nested subtests:
-.. code-block:: none +::
KTAP version 1 1..1@@ -215,7 +213,7 @@ An example of a test with two nested subtests:
An example format with multiple levels of nested testing:
-.. code-block:: none +::
KTAP version 1 1..2@@ -250,7 +248,7 @@ nested version line, uses a line of the form
Example KTAP output
-.. code-block:: none +::
KTAP version 1 1..1-- Frank Rowand frank.rowand@sony.com
Frank Rowand frowand.list@gmail.com writes:
Hi Jon,
I got distracted with other tasks, just now returning to this.
There has been no other comment in the last month, so I think it is fair to go ahead and ask you to pull this patch.
Can you pull just this patch (patch 2/2) or would you prefer that I do a version 2 which does not contain patch 1/2?
I've applied it, thanks.
jon
On Thu, Mar 17, 2022 at 4:26 AM frowand.list@gmail.com wrote:
From: Frank Rowand frank.rowand@sony.com
An August 2021 RFC patch [1] to create the KTAP Specification resulted in some discussion of possible items to add to the specification. The conversation ended without completing the document.
Progress resumed with a December 2021 RFC patch [2] to add a KTAP Specification file (Version 1) to the Linux kernel. Many of the suggestions from the August 2021 discussion were not included in Version 1. This patch series is intended to revisit some of the suggestions from the August 2021 discussion.
Thanks for kicking this off again. There were definitely a lot of good ideas in those threads which we haven't got to yet.
I think there is an interesting line to walk between keeping KTAP sufficiently "TAP-like" (particularly w/r/t being able to reuse existing TAP parsers), and actually adding features, but I don't recall seeing many such issues in the previous threads.
Patch 1 changes the Specification version to "2-rc" to indicate that following patches are not yet accepted into a final version 2.
I'm okay with this, though I'd want us to be a little careful with the timing so we don't end up with, for example, 5.18 having a KTAP spec called 2-rc which is functionally indistinguishable from v1.
Patch 2 is an example of a simple change to the Specification. The change does not change the content of the Specification, but updates a formatting directive as suggested by the Documentation maintainer.
Thanks -- personally, I'd rather this change _does_ go in straight away, even before the 2-rc renaming.
I intend to take some specific suggestions from the August 2021 discussion to create stand-alone RFC patches to the Specification instead of adding them as additional patches in this series. The intent is to focus discussion on a single area of the Specification in each patch email thread.
Seems like a sensible way to structure the discussion. It could get a little bit messy if there end up being merge conflicts, but the whole thing could be collapsed into a single patchset later if that ended up making more sense. (Though that might remove the need for the "rc" version, depending on exactly when and how it happened.)
I'd also be curious to see patches to tests and/or test parsers to show off any particularly compatibility-breaking and/or interesting changes, though I don't think that _has_ to be a prerequisite for discussion or the spec.
[1] https://lore.kernel.org/r/CA+GJov6tdjvY9x12JsJT14qn6c7NViJxqaJk+r-K1YJzPggFD... [2] https://lore.kernel.org/r/20211207190251.18426-1-davidgow@google.com
Frank Rowand (2): Documentation: dev-tools: KTAP spec change version to 2-rc Documentation: dev-tools: use literal block instead of code-block
Documentation/dev-tools/ktap.rst | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-)
-- Frank Rowand frank.rowand@sony.com
Cheers, -- David
On 3/17/22 03:42, David Gow wrote:
On Thu, Mar 17, 2022 at 4:26 AM frowand.list@gmail.com wrote:
From: Frank Rowand frank.rowand@sony.com
An August 2021 RFC patch [1] to create the KTAP Specification resulted in some discussion of possible items to add to the specification. The conversation ended without completing the document.
Progress resumed with a December 2021 RFC patch [2] to add a KTAP Specification file (Version 1) to the Linux kernel. Many of the suggestions from the August 2021 discussion were not included in Version 1. This patch series is intended to revisit some of the suggestions from the August 2021 discussion.
Thanks for kicking this off again. There were definitely a lot of good ideas in those threads which we haven't got to yet.
I think there is an interesting line to walk between keeping KTAP sufficiently "TAP-like" (particularly w/r/t being able to reuse existing TAP parsers), and actually adding features, but I don't recall seeing many such issues in the previous threads.
Patch 1 changes the Specification version to "2-rc" to indicate that following patches are not yet accepted into a final version 2.
I'm okay with this, though I'd want us to be a little careful with the timing so we don't end up with, for example, 5.18 having a KTAP spec called 2-rc which is functionally indistinguishable from v1.
I finally have some time to return to this.
I could host a branch on my kernel.org "frowand" linux kernel. When agreement is reached on a patch on this mail list, I would add it to the branch. When the discussion determines that it is time to release a version 2 of the specification I would add one more commit that only updates the version.
Does that sound like a good way to proceed?
Patch 2 is an example of a simple change to the Specification. The change does not change the content of the Specification, but updates a formatting directive as suggested by the Documentation maintainer.
Thanks -- personally, I'd rather this change _does_ go in straight away, even before the 2-rc renaming.
I intend to take some specific suggestions from the August 2021 discussion to create stand-alone RFC patches to the Specification instead of adding them as additional patches in this series. The intent is to focus discussion on a single area of the Specification in each patch email thread.
Seems like a sensible way to structure the discussion. It could get a little bit messy if there end up being merge conflicts, but the whole thing could be collapsed into a single patchset later if that ended up making more sense. (Though that might remove the need for the "rc" version, depending on exactly when and how it happened.)
Yes, if I host a branch then no need for the preliminary rc version.
I'd also be curious to see patches to tests and/or test parsers to show off any particularly compatibility-breaking and/or interesting changes, though I don't think that _has_ to be a prerequisite for discussion or the spec.
That is a good suggestion.
-Frank
[1] https://lore.kernel.org/r/CA+GJov6tdjvY9x12JsJT14qn6c7NViJxqaJk+r-K1YJzPggFD... [2] https://lore.kernel.org/r/20211207190251.18426-1-davidgow@google.com
Frank Rowand (2): Documentation: dev-tools: KTAP spec change version to 2-rc Documentation: dev-tools: use literal block instead of code-block
Documentation/dev-tools/ktap.rst | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-)
-- Frank Rowand frank.rowand@sony.com
Cheers, -- David
On Sat, Apr 23, 2022 at 7:16 AM Frank Rowand frowand.list@gmail.com wrote:
On 3/17/22 03:42, David Gow wrote:
On Thu, Mar 17, 2022 at 4:26 AM frowand.list@gmail.com wrote:
From: Frank Rowand frank.rowand@sony.com
An August 2021 RFC patch [1] to create the KTAP Specification resulted in some discussion of possible items to add to the specification. The conversation ended without completing the document.
Progress resumed with a December 2021 RFC patch [2] to add a KTAP Specification file (Version 1) to the Linux kernel. Many of the suggestions from the August 2021 discussion were not included in Version 1. This patch series is intended to revisit some of the suggestions from the August 2021 discussion.
Thanks for kicking this off again. There were definitely a lot of good ideas in those threads which we haven't got to yet.
I think there is an interesting line to walk between keeping KTAP sufficiently "TAP-like" (particularly w/r/t being able to reuse existing TAP parsers), and actually adding features, but I don't recall seeing many such issues in the previous threads.
Patch 1 changes the Specification version to "2-rc" to indicate that following patches are not yet accepted into a final version 2.
I'm okay with this, though I'd want us to be a little careful with the timing so we don't end up with, for example, 5.18 having a KTAP spec called 2-rc which is functionally indistinguishable from v1.
I finally have some time to return to this.
I could host a branch on my kernel.org "frowand" linux kernel. When agreement is reached on a patch on this mail list, I would add it to the branch. When the discussion determines that it is time to release a version 2 of the specification I would add one more commit that only updates the version.
Does that sound like a good way to proceed?
Yeah: that sounds good to me.
-- David
linux-kselftest-mirror@lists.linaro.org
-
David Gow -
Frank Rowand -
frowand.list＠gmail.com
-
Jonathan Corbet