Hi Charles - thanks for the write-up. I use alternative baselines quite regularly. My thoughts below.
On Tue, Feb 12, 2019 at 10:59:52AM -0200, Charles Oliveira wrote:
Hi,
Yesterday during SQUAD sync we discussed strategies for marking baselines instead of having them marked automatically as a previous builds. This has been suggested before here, and here.
By default, every finished build becomes the baseline of the next finished build. Two alternatives to tell SQUAD not to mark a finished build as baseline are as follows (to be implemented): a) go to build's page > build settings > check 'ignore as baseline' option; anyone with write permissions to that build's project is allowed to change its settings; b) calling qa-reports.l.o/api/createbuild passing 'is_baseline=false' as POST before calling qa-reports.l.o/api/submitjob In absence of 'is_baseline', the build will be marked as baseline by default, for backwards compatibility.
I think this would work for LKFT. If building a tagged build, let is_baseline be true, otherwise set it to false.
Before discussing edge cases, take example below (considering option b above is being used to submit builds to Squad):
+------------+-----------------+-------------+ | build | is_baseline | baseline | +------------+---------------- +-------------+ | build101 | false | build100 | | build102 | false | build100 | | build103 | ----- | build100 | | build104 | ----- | build103 | | build105 | false | build104 | | build106 | false | build104 | +------------+-----------------+-------------+
- If build104 is later manually marked as not being a baseline,
through UI, then build103 automatically becomes the current baseline.
- If in the event above, Squad will *NOT* re-run test comparisons for
build106 and build105 against build103.
I think we discussed other edge cases, but I can't remember them now.
I'm curious what use-cases you have for this feature? At first I thought it wouldn't be useful for LKFT, but in fact I think it would really simplify what we're currently doing to do reporting.
I'm also curious how the UI will deal with it so it's clear which builds are being compared to determine fixes/regressions.
Seems like a reasonable approach to me :)
Thanks! Dan
Charles _______________________________________________ Squad-dev mailing list Squad-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/squad-dev