The next big thing for the Toolchain WG is sending patches upstream and beginning to track patches in general. Some of the use cases are:
* Do a change upstream in 4.7, backport it to our 4.5, and backport it to our 4.6 when we make one * Do a change locally, track where it lands upstream, and figure out what we have to bring forward when we next rebase * Track what is Linaro specific and can't go upstream so we know what to bring forward
I'm currently planning to use the Launchpad bug tracker and some custom reports to manage this. It's important that we don't lose any patches.
Does anyone else have similar requirements, use for such a tool, or know of tools that already cover this?
-- Michael
On 10 Aug 13, Michael Hope wrote:
The next big thing for the Toolchain WG is sending patches upstream and beginning to track patches in general. Some of the use cases are:
- Do a change upstream in 4.7, backport it to our 4.5, and backport
it to our 4.6 when we make one
- Do a change locally, track where it lands upstream, and figure out
what we have to bring forward when we next rebase
- Track what is Linaro specific and can't go upstream so we know what
to bring forward
I'm currently planning to use the Launchpad bug tracker and some custom reports to manage this. It's important that we don't lose any patches.
So git-based workflows won't benefit from this then?
Does anyone else have similar requirements, use for such a tool, or know of tools that already cover this?
It would be very useful for tracking kernel/tools deltas in the Kernel and PM WG. I know the Ubuntu Kernel team has expressed an interest in such a tool too.
/Amit
On Fri, Aug 13, 2010 at 7:19 PM, Amit Kucheria amit.kucheria@linaro.org wrote:
So git-based workflows won't benefit from this then?
There's no particular dependence on any VC system. I will be making a report on each bzr commit to make sure there is a corresponding ticket that tracks the upstream state.
At the moment you do need a backing Launchpad ticket to record the state changes. Out of interest, can git record meta data with a commit, such as the upstream status, and modify that meta data after the commit
Does anyone else have similar requirements, use for such a tool, or know of tools that already cover this?
It would be very useful for tracking kernel/tools deltas in the Kernel and PM WG. I know the Ubuntu Kernel team has expressed an interest in such a tool too.
OK. I'll post by brief when it's done and we'll see if it meets others needs.
-- Michael
On 10 Aug 13, Michael Hope wrote:
On Fri, Aug 13, 2010 at 7:19 PM, Amit Kucheria amit.kucheria@linaro.org wrote:
So git-based workflows won't benefit from this then?
There's no particular dependence on any VC system. I will be making a report on each bzr commit to make sure there is a corresponding ticket that tracks the upstream state.
Sounds tedious. It probably isn't, so I'll wait for you to post your results.
At the moment you do need a backing Launchpad ticket to record the state changes. Out of interest, can git record meta data with a commit, such as the upstream status, and modify that meta data after the commit
Not in git-core AFAIK. I'm assuming you want to store free-form text with each commit. You could potentially use git hooks to store additional data is a separate file.
Does anyone else have similar requirements, use for such a tool, or know of tools that already cover this?
It would be very useful for tracking kernel/tools deltas in the Kernel and PM WG. I know the Ubuntu Kernel team has expressed an interest in such a tool too.
OK. I'll post by brief when it's done and we'll see if it meets others needs.
-- Michael
/Amit
On 13/08/10 10:08, Amit Kucheria wrote:
On 10 Aug 13, Michael Hope wrote:
On Fri, Aug 13, 2010 at 7:19 PM, Amit Kucheriaamit.kucheria@linaro.org wrote:
So git-based workflows won't benefit from this then?
There's no particular dependence on any VC system. I will be making a report on each bzr commit to make sure there is a corresponding ticket that tracks the upstream state.
Sounds tedious. It probably isn't, so I'll wait for you to post your results.
Tedious, yes, but there's no real way around that. We're planning to do everything possible to reduce the work, but in the end, somebody has to make a note "merge done here ..", or "merge not done because ..".
The idea is that an automated tool reads bzr log output (or git log output) and uses the launchpad API to create a ticket for each, seeded with as much info as it knows, and assigns it to the committer.
Having no manual step in starting the tracking process means no initial tedium, and no patches fall through the cracks.
If the patch has not been merged upstream, there's no further manual step to do, and the system tracks until forever that the patch exists and needs merging on rebase (this is a bigger deal in some projects than others, of course), and also indicates that you really should be doing something about it.
If the patch has been merged, or will never be merged, or whatever, then somebody can manually change the state, and again, we know what we need to do on rebase.
In the meantime, we can use the ticket to record whatever interesting information there might be regarding this patch: "awaiting ...", "sent for review here ...", "maybe I'll do this a different way ...".
Andrew
On Fri, Aug 13, 2010 at 9:08 PM, Amit Kucheria amit.kucheria@linaro.org wrote:
On 10 Aug 13, Michael Hope wrote:
On Fri, Aug 13, 2010 at 7:19 PM, Amit Kucheria amit.kucheria@linaro.org wrote:
So git-based workflows won't benefit from this then?
There's no particular dependence on any VC system. I will be making a report on each bzr commit to make sure there is a corresponding ticket that tracks the upstream state.
Sounds tedious. It probably isn't, so I'll wait for you to post your results.
There will be manual steps involved but I'll try to minimise them or automate them.
-- Michael
On Fri, 13 Aug 2010, Michael Hope wrote:
On Fri, Aug 13, 2010 at 7:19 PM, Amit Kucheria amit.kucheria@linaro.org wrote:
So git-based workflows won't benefit from this then?
There's no particular dependence on any VC system. I will be making a report on each bzr commit to make sure there is a corresponding ticket that tracks the upstream state.
At the moment you do need a backing Launchpad ticket to record the state changes. Out of interest, can git record meta data with a commit, such as the upstream status, and modify that meta data after the commit
Yes. With a recent enough Git version, you can attach so called notes to any commit and modify them at will:
http://www.kernel.org/pub/software/scm/git/docs/git-notes.html
Nicolas
On 10 Aug 13, Nicolas Pitre wrote:
On Fri, 13 Aug 2010, Michael Hope wrote:
On Fri, Aug 13, 2010 at 7:19 PM, Amit Kucheria amit.kucheria@linaro.org wrote:
So git-based workflows won't benefit from this then?
There's no particular dependence on any VC system. I will be making a report on each bzr commit to make sure there is a corresponding ticket that tracks the upstream state.
At the moment you do need a backing Launchpad ticket to record the state changes. Out of interest, can git record meta data with a commit, such as the upstream status, and modify that meta data after the commit
Yes. With a recent enough Git version, you can attach so called notes to any commit and modify them at will:
http://www.kernel.org/pub/software/scm/git/docs/git-notes.html
Nice!
/me learns something new today
/amit
I have started to put a Flow Chart together for Patch Tracking Process (very preliminary draft) , see if that is what we want and let me know what to add/modify to visualize the process at at a high level, first. Cheers, Mounir
On Fri, Aug 13, 2010 at 2:57 PM, Amit Kucheria amit.kucheria@linaro.orgwrote:
On 10 Aug 13, Nicolas Pitre wrote:
On Fri, 13 Aug 2010, Michael Hope wrote:
On Fri, Aug 13, 2010 at 7:19 PM, Amit Kucheria <
amit.kucheria@linaro.org> wrote:
So git-based workflows won't benefit from this then?
There's no particular dependence on any VC system. I will be making a report on each bzr commit to make sure there is a corresponding ticket that tracks the upstream state.
At the moment you do need a backing Launchpad ticket to record the state changes. Out of interest, can git record meta data with a commit, such as the upstream status, and modify that meta data after the commit
Yes. With a recent enough Git version, you can attach so called notes to any commit and modify them at will:
http://www.kernel.org/pub/software/scm/git/docs/git-notes.html
Nice!
/me learns something new today
/amit
linaro-dev mailing list linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
On Fri, Aug 13, 2010, Michael Hope wrote:
- Do a change upstream in 4.7, backport it to our 4.5, and backport
it to our 4.6 when we make one
- Do a change locally, track where it lands upstream, and figure out
what we have to bring forward when we next rebase
- Track what is Linaro specific and can't go upstream so we know what
to bring forward
I think it's relevant to consider two things around this tracking: - we're tracking a feature or a bug fix - we want to track updates to this new feature or this bug fix
I know I keep repeating this, but I *really* want that to be taken into account: patches go through multiple revisions, and new features are often tweaked after being checked in.
Of course we can consider commits after a feature has landed as unrelated bug fixes, but it kind of makes sense to group them together if we can, when preparing a backport for instance: this allows the backport to benefit from all known fixes which landed after the feature. For multiple revisions of the same patch, what I mean is that we might be pulling a patch from the upstream ML before it's committed, but then it get tweaked slightly and we want to pull that change, even if the patch was only ever committed once to the upstream repo. Another instance of this is when we are writing a patch locally, then upstreaming it: it might be committed in a slighlty different form, and we might want to update our local copy (or not), but in any case we want to check.
Perhaps what could work are: - distributing reviews of new checkins to upstream branches, making sure new tickets get opened when a new thing shows up that we want to grab, or updating an existing ticket when there's already a related one - distributing reviews of upstream MLs to identify new stuff to grab, or to propagate updates to patches we might have cherry-picked from the ML - having a process for checkins of our local new features / bug fixes, making sure they are pushed to all relevant upstream branches
After writing this up, I feel that writing the tool might be one hard thing, but getting the processes right seems quite hard too.
On 13/08/10 10:44, Loïc Minier wrote:
I think it's relevant to consider two things around this tracking:
- we're tracking a feature or a bug fix
There's no automatic way to determine this, but it could be tracked using launchpad tags. Updating the tags might be tedious though.
- we want to track updates to this new feature or this bug fix
Again, there's no automatic way to do this, but it's easy enough to do it manually. We would modify one of the tracking tickets to reference both parts of the patch (and close the other ticket, or mark it a duplicate).
Andrew
On Fri, Aug 13, 2010, Andrew Stubbs wrote:
I think it's relevant to consider two things around this tracking:
- we're tracking a feature or a bug fix
There's no automatic way to determine this, but it could be tracked using launchpad tags. Updating the tags might be tedious though.
- we want to track updates to this new feature or this bug fix
Again, there's no automatic way to do this, but it's easy enough to do it manually. We would modify one of the tracking tickets to reference both parts of the patch (and close the other ticket, or mark it a duplicate).
Sorry, wasn't clear, the part of my email which you quote above was meant to underline the properties of the patch tracking system that I care about rather than suggesting a design. The bottom of my original email has more ideas on how we could make sure we don't miss anything.
Before we dive into implementation questions such as Launchpad tags, or tickets, or new software to be developed, I think it's important to understand the fundamental goals of the tracker and the data it would be working on.
I personally think the processes should act on upstream mailing-list threads, on upstream checkins in various branches, and on our checkins in our branches. Launchpad bugs/tickets could indeed play an important part of that, but I suspect we'll need more on top of Launchpad, and the relation might be anywhere from zero Launchpad bugs for a patch which is in Linaro and upstream, up to many bugs for a single feature getting developed upstream, then backported to Linaro, or vice-versa.
For instance, we want to make sure we review all upstream commits and all upstream emails with patches, but we might not want to open one bug/ticket per upstream email or email thread.
On Fri, Aug 13, 2010 at 2:06 PM, Loïc Minier loic.minier@linaro.org wrote:
On Fri, Aug 13, 2010, Andrew Stubbs wrote:
I think it's relevant to consider two things around this tracking: - we're tracking a feature or a bug fix
There's no automatic way to determine this, but it could be tracked using launchpad tags. Updating the tags might be tedious though.
- we want to track updates to this new feature or this bug fix
Again, there's no automatic way to do this, but it's easy enough to do it manually. We would modify one of the tracking tickets to reference both parts of the patch (and close the other ticket, or mark it a duplicate).
Sorry, wasn't clear, the part of my email which you quote above was meant to underline the properties of the patch tracking system that I care about rather than suggesting a design. The bottom of my original email has more ideas on how we could make sure we don't miss anything.
Before we dive into implementation questions such as Launchpad tags, or tickets, or new software to be developed, I think it's important to understand the fundamental goals of the tracker and the data it would be working on.
I personally think the processes should act on upstream mailing-list threads, on upstream checkins in various branches, and on our checkins in our branches. Launchpad bugs/tickets could indeed play an important part of that, but I suspect we'll need more on top of Launchpad, and
patchwork[1][2][3] can already monitor mailing lists for patches and is used by several projects (Linux kernel, various sub-system maintainers, etc.) to make sure that patches posted to the mailing list are not dropped. (cc'ed Jeremy, author of patchwork)
Patchwork can also track updated patchsets and keep related patchsets (V1, V2, V3, ...) together, I believe.
What would be very useful is patchwork on steroids that can track pre-configured upstream git trees and figure out which patches already landed in those trees (with tag info). This could be done by searching based on author, commit one-line description and detailed description and perhaps even the contents of the commit itself. This info that then be automatically updated in the patch tracker.
The hard problem is when a commit goes upstream is a different form, by a different author, etc.
the relation might be anywhere from zero Launchpad bugs for a patch which is in Linaro and upstream, up to many bugs for a single feature getting developed upstream, then backported to Linaro, or vice-versa.
For instance, we want to make sure we review all upstream commits and all upstream emails with patches, but we might not want to open one bug/ticket per upstream email or email thread.
[1] http://ozlabs.org/~jk/projects/patchwork/ [2] patchwork.ozlabs.org [3] patchwork.kernel.org
On Fri, Aug 13, 2010, Amit Kucheria wrote:
patchwork[1][2][3] can already monitor mailing lists for patches and is used by several projects (Linux kernel, various sub-system maintainers, etc.) to make sure that patches posted to the mailing list are not dropped. (cc'ed Jeremy, author of patchwork)
Yup -- in fact I had patchwork in mind when suggesting one side of the tracker should track the upstream MLs
On Fri, 2010-08-13 at 14:35 +0300, Amit Kucheria wrote:
patchwork[1][2][3] can already monitor mailing lists for patches and is used by several projects (Linux kernel, various sub-system maintainers, etc.) to make sure that patches posted to the mailing list are not dropped. (cc'ed Jeremy, author of patchwork)
Jeff Kirsher from Intel gave a talk at OLS [1]. He was facing similar problems with the Intel LAN drivers. He described how he used patchwork to handle patches from internal people and the community. You guys might benefit from contacting him.
[1] http://www.linuxsymposium.org/2010/view_abstract.php?content_key=33
Cheers,
Scott
On 13/08/10 12:35, Amit Kucheria wrote:
patchwork[1][2][3] can already monitor mailing lists for patches and is used by several projects (Linux kernel, various sub-system maintainers, etc.) to make sure that patches posted to the mailing list are not dropped. (cc'ed Jeremy, author of patchwork)
This is not necessary. Launchpad merge proposals already handle this.
We could use it to monitor the output from the launchpad branch subscriptions, but can it cope with the idea that the patches are targeted at both upstream, *and* the next rebase, *but* only if the patch went into a later baseline upstream?
We're not tracking whether a patch has been reviewed or rejected here. We tracking where it exists, and does not exist.
What would be very useful is patchwork on steroids that can track pre-configured upstream git trees and figure out which patches already landed in those trees (with tag info). This could be done by searching based on author, commit one-line description and detailed description and perhaps even the contents of the commit itself. This info that then be automatically updated in the patch tracker.
That would be nice, assuming it worked, but I don't think we want to put much effort into implementing it, and I certainly don't want to wait until it is implemented to have any solution at all.
Andrew
On 13/08/10 12:06, Loïc Minier wrote:
Before we dive into implementation questions such as Launchpad tags, or tickets, or new software to be developed, I think it's important to understand the fundamental goals of the tracker and the data it would be working on.
We're way past that point now. We've been there, discussed that, and I would like to see an implementation right now.
I personally think the processes should act on upstream mailing-list threads, on upstream checkins in various branches, and on our checkins
This is different problem, and we should *not* confuse the two. Trying to mix the two can only succeed in delaying the implementation, and adding unnecessary complexity.
For instance, we want to make sure we review all upstream commits and all upstream emails with patches, but we might not want to open one bug/ticket per upstream email or email thread.
No, of course not, that would be silly.
Tracking upstream patches is interesting, but there are so many of them, across all the project we will eventually track, that you want the process to be as lightweight as possible.
I would also warn you against doing this on a large scale. There's no point in backporting every 4.6 patch to 4.5. Apart from using your entire engineering resource for patch porting work, you just end up with a 4.6 compiler, together with all the instability and other problems associated with using bleeding edge sources. We should only be interested in cherry picking specific performance improvements - even bug fixes can be left alone if the problem has never been observed.
In summary,
* tracking upstream patches is about eyeballing the list, and saying, "yes, no, no, no, yes, no .....". The "yes"s become tickets, or work items, or entries in a wiki.
* tracking local patches is about being able to ask, at any point in development, "what patches need upstreaming right now?", and "if I rebase to X, what patches will I need to forward port?"
Please don't try to unify the implementations.
Andrew
On Fri, Aug 13, 2010 at 11:06 PM, Loïc Minier loic.minier@linaro.org wrote:
On Fri, Aug 13, 2010, Andrew Stubbs wrote:
I think it's relevant to consider two things around this tracking: - we're tracking a feature or a bug fix
There's no automatic way to determine this, but it could be tracked using launchpad tags. Updating the tags might be tedious though.
- we want to track updates to this new feature or this bug fix
Again, there's no automatic way to do this, but it's easy enough to do it manually. We would modify one of the tracking tickets to reference both parts of the patch (and close the other ticket, or mark it a duplicate).
Sorry, wasn't clear, the part of my email which you quote above was meant to underline the properties of the patch tracking system that I care about rather than suggesting a design. The bottom of my original email has more ideas on how we could make sure we don't miss anything.
Before we dive into implementation questions such as Launchpad tags, or tickets, or new software to be developed, I think it's important to understand the fundamental goals of the tracker and the data it would be working on.
I personally think the processes should act on upstream mailing-list threads, on upstream checkins in various branches, and on our checkins in our branches. Launchpad bugs/tickets could indeed play an important part of that, but I suspect we'll need more on top of Launchpad, and the relation might be anywhere from zero Launchpad bugs for a patch which is in Linaro and upstream, up to many bugs for a single feature getting developed upstream, then backported to Linaro, or vice-versa.
For instance, we want to make sure we review all upstream commits and all upstream emails with patches, but we might not want to open one bug/ticket per upstream email or email thread.
I'm worried about this getting too big. Our immediate needs are: * Being able to send patches upstream and track what happened to them * Being able to do work upstream and track the upstream version where they landed
These two together lets us track what patches need to be brought forward on a rebase.
I wrote up the basic classes some time ago: https://wiki.linaro.org/WorkingGroups/ToolChain/PatchTracking
It's done as a set of reduced classes as they're easier to understand than use cases.
This captures the case where patches are accepted as is, modified before landing upstream, or rejected. It doesn't handle patches later being refined or obsoleted but that is outside scope and can be treated as a new feature.
There's a separate question about tracking other upstream changes and deciding what to backport. We shouldn't backport everything, only those that have either a performance gain or clear correctness gain.
Implementation-wise, I can see two good uses for Patchwork: * It provides a rolled-up summary of any of our patches. Easier than trawling through notes on a Launchpad bug * It (may) provide a shared way of making sure we've reviewed every patch for a potential backport.
Note that reviewing upstream patches for backport is a new feature and outside the scope of the Linaro patch tracker.
-- Michael
-
Amit Kucheria -
Andrew Stubbs -
Loïc Minier -
Michael Hope -
Mounir Bsaibes -
Nicolas Pitre -
Scott Bambrough