On 09/04/12 12:13, the mail apparently from Ricardo Salveti included:

Hi -

...

1. Can we have linux stable point release content in tilt-3.4? Rather than

my doing it, isn't it better to add it to llc-3.4 and merge it on the lt history tree periodically? That way every lt can get them from one place.

I don't see why merging the stable release contents would be an issue. We could keep updating the tree based on stable-only releases, as long as we still have at least one Landing Team interested on consuming it.

This would be another job that would probably be automated by Andrey's scripts.

Right it should usually be simple, although don't forget there is quite a lot of avant garde content in llct, such as Androidization. Just today I saw Xavier at TI find that merging of stable had a patch conflicting with llct Androidization content.

...

2. What's the deal with things that were the latest and greatest at that

time, ie, the best "CMA" or whatever series was in tracking, but after it got copied out to be linux-linaro-core-3.4, horrible bugs were fixed in linux-linaro-tracking? What's happening is that TI are sticking with these releases for a fair time as the basis for their release to customers.

The problem is that the main goal of pushing new content and branches at the linux-linaro-tracking is mainly to help people getting their own stuff at upstream (by providing an unified tree which can then be used for QA and such). So, from the maintainer perspective, he'll always be moving his own stuff based on the latest upstream available, and that's why they are always integrated at the latest linux-linaro-tracking as well (mostly following upstream).

Understood, and in itself that's obviously a good way.

If we decide to keep updating the older tree, that would probably need a substantial and not necessarily trivial work on backporting the new features and updates. Guess at least bugfixes would be ok, but I don't think would be trivial to identify just the fixes at the latest series.

I don't think we need to "keep updating" the older tree, in any continuous sense like the tracking one.

What's causing concern is specific cases where big problems that went out with the old tree were found and fixed in subsequent tracking work.

What happens now is the big problem stays landmine-style in the older tree until real customers trip over it and waste a lot of time characterizing it or trying to fix / resolve it. In fact it's already understood and fixed elsewhere.

What would preferably happen is that when the "domain experts" for a particular series in llct realize that the thing being fixed in tracking means there's something horrible in the stable release, they at least consider a minimal fix patch on the old tree, or ring a big bell on the lists saying, "don't use XYZ on 3.4, we just found it's unreliable and can crash with ABC. You need the implementation from 3.6 at least since there's no workaround". Then we can discuss which lucky person should look at backport, or if all the consumers who care about XYZ can migrate to 3.6.

At the moment none of that is happening which increases uncertainty about llct as a basis for medium-term release trees.

We don't need an SLA about it just the understanding that if a feature goes in llct, the guys writing it need to keep a little bit of love in their heart for also fixing the bigger problems later found in the code even in its old age.

...

I can see there's tension between tracking-style fix it for the future, and backport to old and crusty things, there's also issue of testing, but there must be some cases where this makes some sense. Again people looking after the feature tree for llct are best placed to make those calls about, "hm, that looks like it should maybe also go on the last couple of llc release trees".

What do you think about this?

I believe we can go on case by case, if we have people requesting us to backport new features or branches over previous releases. If you have any this point (or at least from TI), we can look and see what would be needed to update at the 3.4 based trees, but would also be nice if we can also get them involved on testing, so we know things are working properly.

The problem is we are not in a position to know what to ask for until we painfully stubbed our toes on it.

The "domain experts" who work on the topic content of llct are in a position to know since they are knee-deep in it daily.

And since the content went in at llct level, it needs managing and updating at llct level so all LTs get the benefit (and that point, LTs will be on history trees, and can simply merge).

Do you have any idea about how long TI will keep this 3.4 based tree maintained and supported? I saw you were about to move on working at the 3.6 based tree, but don't know if that's just to keep things sane (and near upstream), or because TI wants to have another well supported tree.

Right we have started to kick tyres on 3.6 tracking but that has some challenges that will take a while to fully overcome. tilt-3.4 has a lot of power-related tuning that will need porting, there's a move to proper DT, there will be an OMAP5 (not so much 4) focus, although we currently hope to inherit a lot of working OMAP4 stuff from mainline automagically.

So the thinking is tilt-3.4 will be around for a while as the mature OMAP4 solution with a lot of proven PM tuning and the 3.6 work is trying to sort out OMAP5 support as its priority.

-Andy

On Wed, Sep 5, 2012 at 7:55 AM, Andy Green andy.green@linaro.org wrote:

On 09/05/12 17:19, the mail apparently from Andy Green included:

...

On 09/04/12 12:13, the mail apparently from Ricardo Salveti included:

Hi -

...

...

1. Can we have linux stable point release content in tilt-3.4?

Rather than my doing it, isn't it better to add it to llc-3.4 and merge it on the lt history tree periodically? That way every lt can get them from one place.

I don't see why merging the stable release contents would be an issue. We could keep updating the tree based on stable-only releases, as long as we still have at least one Landing Team interested on consuming it.

This would be another job that would probably be automated by Andrey's scripts.

Right it should usually be simple, although don't forget there is quite a lot of avant garde content in llct, such as Androidization. Just today I saw Xavier at TI find that merging of stable had a patch conflicting with llct Androidization content.

So, it turns out that is a good example.

I researched the conflict and found a thread from RMK rejecting the patch 96714b5dfe283cd8ab13aac1f9ccb565064af152 that seems to have come in by Androidization series via llct.

http://lists.infradead.org/pipermail/linux-arm-kernel/2010-May/014116.html

We decided to take the kernel.org stable version of the patch 6019ae78aa65afe273da8c0dfeed8e89fb5edf8f which removes some locking evil in the Androidization version, which RMK noted opened up a horrible race.

Xavier then found a ghastly bug that had previously been impossible to track down disappeared.

So we now know that 96714b5dfe283cd8ab13aac1f9ccb565064af152 we had been happily pushing out on everyone in llct-3.4 is a terrible idea, not just for TILT but any kernel that has it in will suffer from very hard to reproduce mm instability under stress, and needs reverting in favour of the version that went in kernel.org stable.

But now we know about that flaw in llct-3.4 should we not do something about it?

Yeah, at least for stable related changes I believe it'd make a lot of sense to push those to llct-3.4.

Andrey, let's also coordinate the stable updates for llct-3.4 during this cycle, and then review the issues, if we get any, after the first merge/update.

Cheers,