I've fleshed out a potential way of tracking patches at: https://wiki.linaro.org/WorkingGroups/ToolChain/PatchTracking#Method%201
It's not too bad if you're a developer. The extra steps are: * Create a ticket * Mark that ticket as affecting upstream * Change the status as the patch evolves * Mark where the patch lands when finished
This is all done through Launchpad's existing interface.
Thoughts?
-- Michael
On 17/08/10 00:23, Michael Hope wrote:
Thoughts?
This isn't what we discussed at all ????
Yes, keeping a bug open to track work begun upstream is probably a good policy, but it's not at all what we were discussing.
The patch tracker should ensure that all revisions in the bzr branch are submitted to upstream, or if not, ensure they are not lost in a rebase. Nothing more, nothing less.
This really wasn't supposed to be something complicated. I knocked together the first version in a matter of a few hours. Yes, looking back, I should have had a column that indicated which upstream version the patch was in, to help with the rebase, and yes, of course using a wiki page wouldn't scale as well as using a bug tracker, or provide as much room for notes and such, but it did what was required.
Here is the original spec we agreed to in Prague:
* Each bzr revision has a tracking ticket. * The tickets should be *created* automatically. - people cannot be relied upon to do it. - people cannot be relied upon to get it right. * The developer then edits the ticket manually.
The user then has to do something to make the ticket go away, but nothing at all to ensure that the patch is tracked.
We discussed reusing bug tickets for tracking patches, but since then I thought we'd agreed that overloading the tickets this way was fraught with trouble, and not necessary.
Can we please stick to this simple plan?
I really wanted this done weeks ago (when I originally implemented it), so I'm now getting frustrated that every time it looks like we're getting somewhere, it seems to have gone off on a tangent and lost sight of all the original requirements. :(
I'm going to write a "method 2" to explain what I need/want.
Andrew
On 17/08/10 10:39, Andrew Stubbs wrote:
I'm going to write a "method 2" to explain what I need/want.
Now done: https://wiki.linaro.org/WorkingGroups/ToolChain/PatchTracking#Method%202
Andrew
Hi Andrew. I'm confused - apart from a few differences, our methods seem to be the same.
The differences against Method 1 are: 1. Every revision has an associated ticket 2. There's a bot that automatically creates a ticket per revision 3. Final upstream status is tracked through the status field instead of the milestone
I agree that we should automate creating tickets for untracked revisions. I wanted to get the rest in place before adding this.
I'm not concerned on (1). There are a couple of problems vs re-using an existing ticket: * Two related revisions (such as the update changelog/bump revision pair done on a release) lead to two tickets where they could be one * If the change is a bug fix, we have to update two tickets - the tracking and the original to show where it is fixed. * A fix which is upstreamed in 4.6 and backported to our 4.4 and 4.5 leads to duplicate tracking tickets * Work which is done upstream starts with a ticket to track the work and email messages, then is commited to ours and perhaps backported, leading to three tickets with identical state
There's a lot of ticket duplication there.
(1) does have some advantages: * The original ticket isn't polluted with things the reporter may not care about, such as the FSF toolchain it is fixed in
Regarding your comments,
The patch tracking comments might get lost among the other comments
Neither here nor there on this one
Bugs that produce multiple commits will have to be tracked multiple times, which is just confusing.
Why is this confusing? Each patch is committed with a --fixes=... and is work against fixing that bug. Note that most work will be done in topic branches, so in many cases there will be one final tracked commit to the trunk.
Commits that fix multiple bugs are not straight-forward
In what way? The commit has multiple --fixes=... lines for each bug that it fixes. Each original bug then has a record of when it will be available upstream. This does mean that one should be the 'master' which has the upstream patch history, and the others should be marked as such.
(3) is neither here or there. I'm abusing the milestone system to give more information. Yours is fine - we can always add tags if it turns out more information is useful.
-- Michael
On Tue, Aug 17, 2010 at 10:28 PM, Andrew Stubbs ams@codesourcery.com wrote:
On 17/08/10 10:39, Andrew Stubbs wrote:
I'm going to write a "method 2" to explain what I need/want.
Now done: https://wiki.linaro.org/WorkingGroups/ToolChain/PatchTracking#Method%202
Andrew
On 17/08/10 23:38, Michael Hope wrote:
Hi Andrew. I'm confused - apart from a few differences, our methods seem to be the same.
The differences against Method 1 are:
- Every revision has an associated ticket
- There's a bot that automatically creates a ticket per revision
Yes, because these two differences are the basic requirements that we started with: that every change needs to be tracked, and that expecting real people to start the tracking process will lead to trouble.
Your tool to detect what tickets are associated with a revision comes up blank much of the time, and I don't see a way to fix it - not every revision uses --fixes, and even if it should it's easily forgotten.
- Final upstream status is tracked through the status field instead
of the milestone
No, my method uses milestones to show where a patch *should* go, and where it *has* gone. It differentiates between the two using the status.
I agree that we should automate creating tickets for untracked revisions. I wanted to get the rest in place before adding this.
The kernel guys made it very clear that they weren't interested unless it was automated. I take a similar view. If I have to add lots of extra bugs manually, then (a) I'll miss some, and (b) I'll get them wrong.
I'm not concerned on (1). There are a couple of problems vs re-using an existing ticket:
- Two related revisions (such as the update changelog/bump revision
pair done on a release) lead to two tickets where they could be one
Yes, two revisions, two tickets. Keep it simple.
- If the change is a bug fix, we have to update two tickets - the
tracking and the original to show where it is fixed.
No, the bug fix will have a link to the tracking ticket. That's the only place that information needs to be.
- A fix which is upstreamed in 4.6 and backported to our 4.4 and 4.5
leads to duplicate tracking tickets
Again, two revisions, two tickets. Very simple, easy to understand. It's quite probably the two revisions are not the exact same patch anyway.
- Work which is done upstream starts with a ticket to track the work
and email messages, then is commited to ours and perhaps backported, leading to three tickets with identical state
Yes, that's true.
There's a lot of ticket duplication there.
OK, here's a use case: I look at r123456, and I want to know what the upstream state is? How do I do that?
Method 1: if r123456 was a bug fix, and the committer remembered to use --fixes, then we're good. If r123456 does not have a ticket associated with it, what now?
Method 2: search launchpad gnu-linaro-tools bug tracker for "r123456"; there's guaranteed to be a match.
Regarding your comments,
The patch tracking comments might get lost among the other comments
Neither here nor there on this one
Imagine it's one of those bugs that has comments that go on forever. Ok, so you're looking for the upstream status, so you can do a search for it and find it. Now imagine that multiple commits were tracked against this bug; it's obvious that there are multiple when you search, but it's not clear which revisions have been posted upstream, and which have not - the follow up comments are hard to untangle, and rely on knowledge of what each patch does. Add to this that the bug got reopened due to a re-occurrence of the original problem, and it's going to be very hard to figure out.
Bugs that produce multiple commits will have to be tracked multiple times, which is just confusing.
Why is this confusing? Each patch is committed with a --fixes=... and is work against fixing that bug. Note that most work will be done in topic branches, so in many cases there will be one final tracked commit to the trunk.
It's confusing because there will be multiple patches being tracked in parallel in the same comment thread. The initial tracking entroes might be clear enough, but the follow-up comments are likely to be hard to differentiate, at least for those not intimately involved with the bug fix. Presumably there will also be multiple "affects gcc-linaro-tracking" lines, and which is which? It'll be hard to find a definitive final state for each patch.
Commits that fix multiple bugs are not straight-forward
In what way? The commit has multiple --fixes=... lines for each bug that it fixes. Each original bug then has a record of when it will be available upstream. This does mean that one should be the 'master' which has the upstream patch history, and the others should be marked as such.
It's certainly not the most difficult case, but there still potential for confusion here. Presumably even if the history is recorded in just one place, each bug still has an 'affects' that need updating.
(3) is neither here or there. I'm abusing the milestone system to give more information. Yours is fine - we can always add tags if it turns out more information is useful.
I think our system is the exact same here, except that I suggested having multiple milestones in the "never" milestone.
Andrew
On Wed, Aug 18, 2010 at 8:11 PM, Andrew Stubbs ams@codesourcery.com wrote:
On 17/08/10 23:38, Michael Hope wrote:
Hi Andrew. I'm confused - apart from a few differences, our methods seem to be the same.
The differences against Method 1 are: 1. Every revision has an associated ticket 2. There's a bot that automatically creates a ticket per revision
Yes, because these two differences are the basic requirements that we started with: that every change needs to be tracked, and that expecting real people to start the tracking process will lead to trouble.
Agreed. Both methods achieve that. Method 1 currently uses a report to show what is missing but will have a tracker bot added.
Your tool to detect what tickets are associated with a revision comes up blank much of the time, and I don't see a way to fix it - not every revision uses --fixes, and even if it should it's easily forgotten.
You can tell which revisions are untracked by the patchtracker report at http://ex.seabright.co.nz/helpers/patchtrack. Revisions that are currently untracked can be added to an existing ticket by adding a one line 'related:' comment to that ticket, or via a tracker bot. Future work will typically use the --fixes line.
3. Final upstream status is tracked through the status field instead of the milestone
No, my method uses milestones to show where a patch *should* go, and where it *has* gone. It differentiates between the two using the status.
Sorry, by 'should go' do you mean the ideal upstream location such as 4.6, and by 'has gone' mean where it ended up, such as 4.6 or in the rejected list?
The kernel guys made it very clear that they weren't interested unless it was automated. I take a similar view. If I have to add lots of extra bugs manually, then (a) I'll miss some, and (b) I'll get them wrong.
We shouldn't manually add tickets for new features. The patchtracker report ensures none are missed.
I'm not concerned on (1). There are a couple of problems vs re-using an existing ticket: * Two related revisions (such as the update changelog/bump revision pair done on a release) lead to two tickets where they could be one
Yes, two revisions, two tickets. Keep it simple.
...but one original piece of work. We won't spread things like this, but say we wanted to add feature X which was spread across five commits. If we have one ticket then we can easily answer the question 'what is the state of X' by checking one ticket instead of 5.
* If the change is a bug fix, we have to update two tickets - the tracking and the original to show where it is fixed.
No, the bug fix will have a link to the tracking ticket. That's the only place that information needs to be.
Say the bug was found in Firefox but caused by GCC. How can the original reporter easily tell the status of the ticket? If there are two tickets, one for the bug and one for the revision, then something has to slave the bug status off the revision status.
* A fix which is upstreamed in 4.6 and backported to our 4.4 and 4.5 leads to duplicate tracking tickets
Again, two revisions, two tickets. Very simple, easy to understand. It's quite probably the two revisions are not the exact same patch anyway.
Agreed.
* Work which is done upstream starts with a ticket to track the work and email messages, then is commited to ours and perhaps backported, leading to three tickets with identical state
Yes, that's true.
There's a lot of ticket duplication there.
OK, here's a use case: I look at r123456, and I want to know what the upstream state is? How do I do that?
Method 1: if r123456 was a bug fix, and the committer remembered to use --fixes, then we're good. If r123456 does not have a ticket associated with it, what now?
Either the tracker bot creates a new ticket, or you add the one line comment 'related: ...' to the bug that it fixes. The revision and bug are now linked.
Regarding your comments,
The patch tracking comments might get lost among the other comments
Neither here nor there on this one
Imagine it's one of those bugs that has comments that go on forever. Ok, so you're looking for the upstream status, so you can do a search for it and find it. Now imagine that multiple commits were tracked against this bug; it's obvious that there are multiple when you search, but it's not clear which revisions have been posted upstream, and which have not - the follow up comments are hard to untangle, and rely on knowledge of what each patch does. Add to this that the bug got reopened due to a re-occurrence of the original problem, and it's going to be very hard to figure out.
I agree that on messy fixes we should have multiple tickets to track separate upstream patches. Both methods allow this. Method 2 requires this.
Bugs that produce multiple commits will have to be tracked multiple times, which is just confusing.
Why is this confusing? Each patch is committed with a --fixes=... and is work against fixing that bug. Note that most work will be done in topic branches, so in many cases there will be one final tracked commit to the trunk.
It's confusing because there will be multiple patches being tracked in parallel in the same comment thread. The initial tracking entroes might be clear enough, but the follow-up comments are likely to be hard to differentiate, at least for those not intimately involved with the bug fix. Presumably there will also be multiple "affects gcc-linaro-tracking" lines, and which is which? It'll be hard to find a definitive final state for each patch.
Agreed on messy fixes. So far all of our (few) fixes have been small and gone upstream easily. We should have a simple process for typical uses and a scalable process for harder uses. Both achieve that.
Commits that fix multiple bugs are not straight-forward
In what way? The commit has multiple --fixes=... lines for each bug that it fixes. Each original bug then has a record of when it will be available upstream. This does mean that one should be the 'master' which has the upstream patch history, and the others should be marked as such.
It's certainly not the most difficult case, but there still potential for confusion here. Presumably even if the history is recorded in just one place, each bug still has an 'affects' that need updating.
True, but the current patchtrack report shows that and gives an easy cross check. Note that Method 2 has a similar problem - two bugs, one fix gives two bug tickets, one revision ticket. When the release is finally done you need to update the status of three different items.
-- Michael
On 18/08/10 22:36, Michael Hope wrote:
You can tell which revisions are untracked by the patchtracker report at http://ex.seabright.co.nz/helpers/patchtrack. Revisions that are currently untracked can be added to an existing ticket by adding a one line 'related:' comment to that ticket, or via a tracker bot. Future work will typically use the --fixes line.
OK, this probably addresses my biggest concern about this system.
I'm going to go away and work through the various use cases, and post something that is hopefully easier to read.
Andrew
I've had a go at adding the basic trackerbot functionality to: http://ex.seabright.co.nz/helpers/patchtrack
This page lists all revisions on a branch. Any revisions that don't have a corresponding ticket are marked in orange and have a 'Create one' link. Clicking this link takes you to a pre-filled out form that will then create the ticket.
It's a manual process at the moment as I wanted to sort out the formatting and content of the tickets before having the bot do it automatically.
Note that the database behind this only updates once an hour. If you make a change you unfortunately won't see the effect for some time.
Thoughts?
-- Michael
On 17/08/10 10:39, Andrew Stubbs wrote:
I really wanted this done weeks ago (when I originally implemented it), so I'm now getting frustrated that every time it looks like we're getting somewhere, it seems to have gone off on a tangent and lost sight of all the original requirements.:(
In the meantime, I've gone back to using my old system:
https://wiki.linaro.org/WorkingGroups/ToolChain/GCC4.5UpstreamPatches
The info can be transferred when the new system comes online.
Andrew
linaro-toolchain@lists.linaro.org
-
Andrew Stubbs -
Loïc Minier -
Michael Hope