In an attempt to better organize bugzilla (see my p.s. if you're asking "what about phabricator?"), I'd like to propose the following changes to how we track and monitor bugs.
First, how we use status. Using 'VERIFIED' is new, and allows us to resolve all the bugs that we're just waiting on:
Bugzilla supports the following statuses. This is how they're used in LKFT.
UNCONFIRMED - New bugs, not yet assigned.
CONFIRMED - Assigned for triaging to determine root cause, and report/escalate/fix accordingly.
IN_PROGRESS - Root cause has been identified and being worked on.
RESOLVED - Root cause fixed. i.e. patch sent
VERIFIED - Fix has been accepted, backported as necessary, and test can be removed from skiplist/known issues list.
To implement this, we'd need to set all of the LKFT bugs that are already RESOLVED to VERIFIED. There are 200, and it can be done in one fell swoop in bugzilla (multi bug change). Then, we'll have to regularly review the list of RESOLVED (but not VERIFIED) bugs for their status, and re-open or verify as necessary.
Second, how we use Component. Our components can be one of LTP, Kernel, kselftest, or General. Right now, we search for e.g. kernel bugs using the 'kernel' keyword, but we apply the kernel keyword for anything that needs to be updated in the kernel tree (including kselftest changes). If we filter using component instead, we can more easily find our actual kernel bugs, vs things in kselftest or ltp. Things in none of those are "General".
Keywords can then be used to be more granular than component, but are often not necessary and I've always found their usage a tad confusing. They can be treated as optional.
Lastly, priority. I don't think we're really good about setting priority, but I'd like to go through and set priority based on the following criteria.
- First, actual kernel bugs. - Second, by impact. Things that impact all boards and all branches are a higher priority than a single board/branch combination. - Third, by triviality. Fix the easy ones first. - Last, things that are difficult and small in terms of impact.
The criteria is a guideline and is flexible/subject to reason. An actual kernel bug in x86 that is very minor may be lower priority than a build problem that is causing instability in our environment.
Once we agree, I'll update https://collaborate.linaro.org/display/EP/Bugs with this information and go through all the bugs and apply these rules accordingly.
The 'bug lists' listed on that page will change to: - All bugs, sorted by priority - Bugs by component: - Kernel bugs, sorted by priority - LTP bugs, sorted by priority - Kselftest bugs, sorted by priority - General bugs, sorted by priority - Bugs by status: - New bugs (unconfirmed) - Confirmed bugs - In progress bugs - Resolved (but not verified) bugs
Thoughts/feedback please!
Thanks, Dan
p.s. apologies for the mixed signals with regard to using bugzilla vs phabricator vs refactoring how we use bugzilla. I've grown frustrated and need to do _something_ asap, and it's not clear to me what the future holds. This is something we can do immediately.
This has been implemented. You may have received a lot of email because I changed all 'resolved' bugs to 'verified' in the Kernel Functional Testing Product. Going forward, 'resolved' bugs will be monitored and moved to 'verified' once the change makes it into LKFT's sources.
Please see https://collaborate.linaro.org/display/EP/Bugs for details of how bugs will be tracked in LKFT going forward.
On Tue, Jun 26, 2018 at 4:15 PM, Dan Rue dan.rue@linaro.org wrote:
In an attempt to better organize bugzilla (see my p.s. if you're asking "what about phabricator?"), I'd like to propose the following changes to how we track and monitor bugs.
First, how we use status. Using 'VERIFIED' is new, and allows us to resolve all the bugs that we're just waiting on:
Bugzilla supports the following statuses. This is how they're used in LKFT. UNCONFIRMED - New bugs, not yet assigned. CONFIRMED - Assigned for triaging to determine root cause, and report/escalate/fix accordingly. IN_PROGRESS - Root cause has been identified and being worked on. RESOLVED - Root cause fixed. i.e. patch sent VERIFIED - Fix has been accepted, backported as necessary, and test can be removed from skiplist/known issues list.To implement this, we'd need to set all of the LKFT bugs that are already RESOLVED to VERIFIED. There are 200, and it can be done in one fell swoop in bugzilla (multi bug change). Then, we'll have to regularly review the list of RESOLVED (but not VERIFIED) bugs for their status, and re-open or verify as necessary.
Second, how we use Component. Our components can be one of LTP, Kernel, kselftest, or General. Right now, we search for e.g. kernel bugs using the 'kernel' keyword, but we apply the kernel keyword for anything that needs to be updated in the kernel tree (including kselftest changes). If we filter using component instead, we can more easily find our actual kernel bugs, vs things in kselftest or ltp. Things in none of those are "General".
Keywords can then be used to be more granular than component, but are often not necessary and I've always found their usage a tad confusing. They can be treated as optional.
Lastly, priority. I don't think we're really good about setting priority, but I'd like to go through and set priority based on the following criteria.
- First, actual kernel bugs.
- Second, by impact. Things that impact all boards and all branches are a higher priority than a single board/branch combination.
- Third, by triviality. Fix the easy ones first.
- Last, things that are difficult and small in terms of impact.
The criteria is a guideline and is flexible/subject to reason. An actual kernel bug in x86 that is very minor may be lower priority than a build problem that is causing instability in our environment.
Once we agree, I'll update https://collaborate.linaro.org/display/EP/Bugs with this information and go through all the bugs and apply these rules accordingly.
The 'bug lists' listed on that page will change to:
- All bugs, sorted by priority
- Bugs by component:
- Kernel bugs, sorted by priority
- LTP bugs, sorted by priority
- Kselftest bugs, sorted by priority
- General bugs, sorted by priority
- Bugs by status:
- New bugs (unconfirmed)
- Confirmed bugs
- In progress bugs
- Resolved (but not verified) bugs
Thoughts/feedback please!
Thanks, Dan
p.s. apologies for the mixed signals with regard to using bugzilla vs phabricator vs refactoring how we use bugzilla. I've grown frustrated and need to do _something_ asap, and it's not clear to me what the future holds. This is something we can do immediately.
-
Dan Rue