Re: [Lts-dev] lkft/hikey-mainline-rebase-4.4.x, build f5ec4ffd9c9ddf29aaf339a8619a5cd145f62342: 945 tests, 6 failed, 939 passed
On Tue, Jun 20, 2017 at 12:03:02AM +0000, Linaro QA wrote:
More feedback on these...
One thing I just noticed is that the reports are being sent to every recipient individually. This is really bad because it breaks up the discussion, if someone replies it won't go to everyone who got the original report which increases confusion and leads to duplication of work.
Details: https://qa-reports.linaro.org/lkft/hikey-mainline-rebase-4.4.x/build/f5ec4ff...
The web part of things doesn't seem to have any way of getting to logs. It'd be good to at least have links to the LAVA jobs as a placeholder for now.
kernel-describe: v3.19-rc7-71416-gf5ec4ff
This is obvious nonsense, this is v4.4 based not v3.19 based. I suspect that this can be fixed by adding Linus' tree as a remote to the git tree used to generate these and keeping it up to date.
format: Lava-Test Test Definition 1.0
Do we need this in the e-mail? It doesn't seem relevant.
description: Dummy tests
...
name: dummy-tests
Having the description before the name seems kind of odd, as does them not being adjacent.
However looking at the test counts and comparing with the web interface it seems like this isn't actually the results for a single testsuite but rather the collected results for all the testsuites that were run for this tree. If that's the case then probably this can just be removed but then...
Failures
hi6220-hikey:
- test_lru_map
...when we list test results we should probably say which testsuite they are from...
| kselftest/test_maps | | fail |
...as is done in some parts of the reports.
On 20 June 2017 at 12:42, Mark Brown broonie@kernel.org wrote:
On Tue, Jun 20, 2017 at 12:03:02AM +0000, Linaro QA wrote:
More feedback on these...
One thing I just noticed is that the reports are being sent to every recipient individually. This is really bad because it breaks up the discussion, if someone replies it won't go to everyone who got the original report which increases confusion and leads to duplication of work.
OK, I'll think of some better solution here.
Details: https://qa-reports.linaro.org/lkft/hikey-mainline-rebase-4.4.x/build/f5ec4ff...
The web part of things doesn't seem to have any way of getting to logs. It'd be good to at least have links to the LAVA jobs as a placeholder for now.
yes, this is known problem. The fixes are in review and should be rolled out this week.
kernel-describe: v3.19-rc7-71416-gf5ec4ff
This is obvious nonsense, this is v4.4 based not v3.19 based. I suspect that this can be fixed by adding Linus' tree as a remote to the git tree used to generate these and keeping it up to date.
Fathi, any clues? I take this value from the build script.
format: Lava-Test Test Definition 1.0
Do we need this in the e-mail? It doesn't seem relevant.
This is already fixed and will be rolled out this week
description: Dummy tests
...
As above.
name: dummy-tests
Having the description before the name seems kind of odd, as does them not being adjacent.
However looking at the test counts and comparing with the web interface it seems like this isn't actually the results for a single testsuite but rather the collected results for all the testsuites that were run for this tree. If that's the case then probably this can just be removed but then...
You're correct, the email collects results from all test suites available. So this metadata is unnecessary and will be removed (it's already done, just waiting for roll out).
Failures
hi6220-hikey:
- test_lru_map
...when we list test results we should probably say which testsuite they are from...
+1, will be fixed.
Thanks for the feedback.
milosz
| kselftest/test_maps | | fail |
...as is done in some parts of the reports.
Kernel-build-reports mailing list Kernel-build-reports@lists.linaro.org https://lists.linaro.org/mailman/listinfo/kernel-build-reports
On Tue, Jun 20, 2017 at 01:33:38PM +0100, Milosz Wasilewski wrote:
On 20 June 2017 at 12:42, Mark Brown broonie@kernel.org wrote:
format: Lava-Test Test Definition 1.0
Do we need this in the e-mail? It doesn't seem relevant.
This is already fixed and will be rolled out this week
Thanks for looking at these! It seems like there's a lot of things here that are already fixed and waiting to be rolled out. Especially while we are in the fairly early stages of the project is it worth looking at our process here so we can get a tighter feedback loop? That'd help with iterating on feedback hopefully.
On Tue, Jun 20, 2017 at 1:39 PM, Mark Brown broonie@kernel.org wrote:
On Tue, Jun 20, 2017 at 01:33:38PM +0100, Milosz Wasilewski wrote:
On 20 June 2017 at 12:42, Mark Brown broonie@kernel.org wrote:
format: Lava-Test Test Definition 1.0
Do we need this in the e-mail? It doesn't seem relevant.
This is already fixed and will be rolled out this week
Thanks for looking at these! It seems like there's a lot of things here that are already fixed and waiting to be rolled out. Especially while we are in the fairly early stages of the project is it worth looking at our process here so we can get a tighter feedback loop? That'd help with iterating on feedback hopefully.
I think the best approach is to keep the feedback coming on the ML to give everybody the chance to pitch in. We tweaked our release process so you will see feedback get incorporated much faster than before and that will tighten the feedback loop significantly.
anmar
On 20 June 2017 at 12:42, Mark Brown broonie@kernel.org wrote:
On Tue, Jun 20, 2017 at 12:03:02AM +0000, Linaro QA wrote:
More feedback on these...
One thing I just noticed is that the reports are being sent to every recipient individually. This is really bad because it breaks up the discussion, if someone replies it won't go to everyone who got the original report which increases confusion and leads to duplication of work.
I took a closer look at the email sending part. There is a setting whether to use multipart (html+txt) or just text version. If all recipients should be included in the address field of the message, this setting has no meaning. So there are three options: a) use text only and send to all recipients b) use multipart and sent to all recipients c) send separate emails (two of them) based on whether the recipient belongs to 'multipart' list or 'plain' list.
Which option should I use?
milosz
On Thu, Jun 22, 2017 at 09:45:32AM +0100, Milosz Wasilewski wrote:
I took a closer look at the email sending part. There is a setting whether to use multipart (html+txt) or just text version. If all recipients should be included in the address field of the message,
I think I'd expect this to be a global thing rather than a per tree/report thing - it's likely that there's some users will be getting multiple reports and probably you don't want to have to update their preferences in multiple places if they change their mind on anything. Each report has a list of recipients and separately each recipient has a preference on e-mail format.
this setting has no meaning. So there are three options: a) use text only and send to all recipients b) use multipart and sent to all recipients c) send separate emails (two of them) based on whether the recipient belongs to 'multipart' list or 'plain' list.
Which option should I use?
If it's a global thing then there's also the option of falling back to text only if any one of the recipients is text only which is probably what I'd end up going for.
On Fri, Jun 23, 2017 at 01:10:52PM +0100, Mark Brown wrote:
On Thu, Jun 22, 2017 at 09:45:32AM +0100, Milosz Wasilewski wrote:
I took a closer look at the email sending part. There is a setting whether to use multipart (html+txt) or just text version. If all recipients should be included in the address field of the message,
I think I'd expect this to be a global thing rather than a per tree/report thing - it's likely that there's some users will be getting multiple reports and probably you don't want to have to update their preferences in multiple places if they change their mind on anything. Each report has a list of recipients and separately each recipient has a preference on e-mail format.
this setting has no meaning. So there are three options: a) use text only and send to all recipients b) use multipart and sent to all recipients c) send separate emails (two of them) based on whether the recipient belongs to 'multipart' list or 'plain' list.
Which option should I use?
If it's a global thing then there's also the option of falling back to text only if any one of the recipients is text only which is probably what I'd end up going for.
note that this is more or less equivalent to having a single per-project setting, since a any subscriber that does not want HTML will stop everyone else from getting it.
On Wed, Jun 28, 2017 at 03:06:33PM -0300, Antonio Terceiro wrote:
On Fri, Jun 23, 2017 at 01:10:52PM +0100, Mark Brown wrote:
If it's a global thing then there's also the option of falling back to text only if any one of the recipients is text only which is probably what I'd end up going for.
note that this is more or less equivalent to having a single per-project setting, since a any subscriber that does not want HTML will stop everyone else from getting it.
Yes, that's the idea. My expectation is that it'll mainly be mailing lists that have this restriction anyway.
On Wed, Jun 28, 2017 at 07:18:01PM +0100, Mark Brown wrote:
On Wed, Jun 28, 2017 at 03:06:33PM -0300, Antonio Terceiro wrote:
On Fri, Jun 23, 2017 at 01:10:52PM +0100, Mark Brown wrote:
If it's a global thing then there's also the option of falling back to text only if any one of the recipients is text only which is probably what I'd end up going for.
note that this is more or less equivalent to having a single per-project setting, since a any subscriber that does not want HTML will stop everyone else from getting it.
Yes, that's the idea. My expectation is that it'll mainly be mailing lists that have this restriction anyway.
This makes the having a per-recipient setting pointless. This is what we will do:
- qa-reports will send a single notification email, with all the subscribers in To: so discussion can follow from there - we will remove the "allow HTML" setting, and just send plain text and HTML. Mailing lists can be configured to discard HTML, so they should.
On Wed, Jun 28, 2017 at 05:48:04PM -0300, Antonio Terceiro wrote:
On Wed, Jun 28, 2017 at 07:18:01PM +0100, Mark Brown wrote:
Yes, that's the idea. My expectation is that it'll mainly be mailing lists that have this restriction anyway.
This makes the having a per-recipient setting pointless. This is what we will do:
I'm not sure it makes things pointless - it's likely there will be people on multiple reports.
- qa-reports will send a single notification email, with all the subscribers in To: so discussion can follow from there
- we will remove the "allow HTML" setting, and just send plain text and HTML. Mailing lists can be configured to discard HTML, so they should.
That's not going to work for kernel stuff, vger just completely discards all mails with HTML. A kernel testing tool that can't send mail to the main kernel mail server isn't really viable.
On Fri, Jun 30, 2017 at 12:42:39PM +0100, Mark Brown wrote:
On Wed, Jun 28, 2017 at 05:48:04PM -0300, Antonio Terceiro wrote:
On Wed, Jun 28, 2017 at 07:18:01PM +0100, Mark Brown wrote:
Yes, that's the idea. My expectation is that it'll mainly be mailing lists that have this restriction anyway.
This makes the having a per-recipient setting pointless. This is what we will do:
I'm not sure it makes things pointless - it's likely there will be people on multiple reports.
Yes, but since a single recipient will be able to mandate what everyone else gets for each given project, it makes more sense to keep that setting in the project instead of in each of the recipients.
- qa-reports will send a single notification email, with all the subscribers in To: so discussion can follow from there
- we will remove the "allow HTML" setting, and just send plain text and HTML. Mailing lists can be configured to discard HTML, so they should.
That's not going to work for kernel stuff, vger just completely discards all mails with HTML. A kernel testing tool that can't send mail to the main kernel mail server isn't really viable.
ACK. I will make sure we can have plain text only emails, then.
-
Anmar Oueja -
Antonio Terceiro -
Mark Brown -
Milosz Wasilewski