The patch below does not apply to the 5.15-stable tree. If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to stable@vger.kernel.org.
To reproduce the conflict and resubmit, you may use the following commands:
git fetch https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ linux-5.15.y git checkout FETCH_HEAD git cherry-pick -x 310d6c15e9104c99d5d9d0ff8e5383a79da7d5e6 # <resolve conflicts, build, test, etc.> git commit -s git send-email --to 'stable@vger.kernel.org' --in-reply-to '2024071546-swerve-grew-de52@gregkh' --subject-prefix 'PATCH 5.15.y' HEAD^..
Possible dependencies:
310d6c15e910 ("mm/damon/core: merge regions aggressively when max_nr_regions is unmet")
thanks,
greg k-h
------------------ original commit in Linus's tree ------------------
From 310d6c15e9104c99d5d9d0ff8e5383a79da7d5e6 Mon Sep 17 00:00:00 2001 From: SeongJae Park sj@kernel.org Date: Mon, 24 Jun 2024 10:58:14 -0700 Subject: [PATCH] mm/damon/core: merge regions aggressively when max_nr_regions is unmet
DAMON keeps the number of regions under max_nr_regions by skipping regions split operations when doing so can make the number higher than the limit. It works well for preventing violation of the limit. But, if somehow the violation happens, it cannot recovery well depending on the situation. In detail, if the real number of regions having different access pattern is higher than the limit, the mechanism cannot reduce the number below the limit. In such a case, the system could suffer from high monitoring overhead of DAMON.
The violation can actually happen. For an example, the user could reduce max_nr_regions while DAMON is running, to be lower than the current number of regions. Fix the problem by repeating the merge operations with increasing aggressiveness in kdamond_merge_regions() for the case, until the limit is met.
[sj@kernel.org: increase regions merge aggressiveness while respecting min_nr_regions] Link: https://lkml.kernel.org/r/20240626164753.46270-1-sj@kernel.org [sj@kernel.org: ensure max threshold attempt for max_nr_regions violation] Link: https://lkml.kernel.org/r/20240627163153.75969-1-sj@kernel.org Link: https://lkml.kernel.org/r/20240624175814.89611-1-sj@kernel.org Fixes: b9a6ac4e4ede ("mm/damon: adaptively adjust regions") Signed-off-by: SeongJae Park sj@kernel.org Cc: stable@vger.kernel.org [5.15+] Signed-off-by: Andrew Morton akpm@linux-foundation.org
diff --git a/mm/damon/core.c b/mm/damon/core.c index 6392f1cc97a3..e66823d6b10b 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -1358,14 +1358,31 @@ static void damon_merge_regions_of(struct damon_target *t, unsigned int thres, * access frequencies are similar. This is for minimizing the monitoring * overhead under the dynamically changeable access pattern. If a merge was * unnecessarily made, later 'kdamond_split_regions()' will revert it. + * + * The total number of regions could be higher than the user-defined limit, + * max_nr_regions for some cases. For example, the user can update + * max_nr_regions to a number that lower than the current number of regions + * while DAMON is running. For such a case, repeat merging until the limit is + * met while increasing @threshold up to possible maximum level. */ static void kdamond_merge_regions(struct damon_ctx *c, unsigned int threshold, unsigned long sz_limit) { struct damon_target *t; + unsigned int nr_regions; + unsigned int max_thres;
- damon_for_each_target(t, c) - damon_merge_regions_of(t, threshold, sz_limit); + max_thres = c->attrs.aggr_interval / + (c->attrs.sample_interval ? c->attrs.sample_interval : 1); + do { + nr_regions = 0; + damon_for_each_target(t, c) { + damon_merge_regions_of(t, threshold, sz_limit); + nr_regions += damon_nr_regions(t); + } + threshold = max(1, threshold * 2); + } while (nr_regions > c->attrs.max_nr_regions && + threshold / 2 < max_thres); }
/*
Hi Greg,
On Mon, 15 Jul 2024 13:34:47 +0200 gregkh@linuxfoundation.org wrote:
The patch below does not apply to the 5.15-stable tree. If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to stable@vger.kernel.org.
To reproduce the conflict and resubmit, you may use the following commands:
git fetch https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ linux-5.15.y git checkout FETCH_HEAD git cherry-pick -x 310d6c15e9104c99d5d9d0ff8e5383a79da7d5e6
Similar to the failure of this patch for 6.1.y, I cannot reproduce the conflict on my setup. Attaching the patch for successfully cherry-picked one on my machine for any possible case.
[1] https://lore.kernel.org/2024071547-slum-anemic-a0cc@gregkh
Thanks, SJ
[...]
==== Attachment 0 (./0001-mm-damon-core-merge-regions-aggressively-when-max_nr.patch) ==== From cd2cd0ec28c79017a9326f235077eb904e4a2cdd Mon Sep 17 00:00:00 2001 From: SeongJae Park sj@kernel.org Date: Mon, 24 Jun 2024 10:58:14 -0700 Subject: [PATCH 5.15.y] mm/damon/core: merge regions aggressively when max_nr_regions is unmet
DAMON keeps the number of regions under max_nr_regions by skipping regions split operations when doing so can make the number higher than the limit. It works well for preventing violation of the limit. But, if somehow the violation happens, it cannot recovery well depending on the situation. In detail, if the real number of regions having different access pattern is higher than the limit, the mechanism cannot reduce the number below the limit. In such a case, the system could suffer from high monitoring overhead of DAMON.
The violation can actually happen. For an example, the user could reduce max_nr_regions while DAMON is running, to be lower than the current number of regions. Fix the problem by repeating the merge operations with increasing aggressiveness in kdamond_merge_regions() for the case, until the limit is met.
[sj@kernel.org: increase regions merge aggressiveness while respecting min_nr_regions] Link: https://lkml.kernel.org/r/20240626164753.46270-1-sj@kernel.org [sj@kernel.org: ensure max threshold attempt for max_nr_regions violation] Link: https://lkml.kernel.org/r/20240627163153.75969-1-sj@kernel.org Link: https://lkml.kernel.org/r/20240624175814.89611-1-sj@kernel.org Fixes: b9a6ac4e4ede ("mm/damon: adaptively adjust regions") Signed-off-by: SeongJae Park sj@kernel.org Cc: stable@vger.kernel.org [5.15+] Signed-off-by: Andrew Morton akpm@linux-foundation.org (cherry picked from commit 310d6c15e9104c99d5d9d0ff8e5383a79da7d5e6) --- mm/damon/core.c | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-)
diff --git a/mm/damon/core.c b/mm/damon/core.c index 7a4912d6e65f..3a18e1ad8ca9 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -507,14 +507,31 @@ static void damon_merge_regions_of(struct damon_target *t, unsigned int thres, * access frequencies are similar. This is for minimizing the monitoring * overhead under the dynamically changeable access pattern. If a merge was * unnecessarily made, later 'kdamond_split_regions()' will revert it. + * + * The total number of regions could be higher than the user-defined limit, + * max_nr_regions for some cases. For example, the user can update + * max_nr_regions to a number that lower than the current number of regions + * while DAMON is running. For such a case, repeat merging until the limit is + * met while increasing @threshold up to possible maximum level. */ static void kdamond_merge_regions(struct damon_ctx *c, unsigned int threshold, unsigned long sz_limit) { struct damon_target *t; - - damon_for_each_target(t, c) - damon_merge_regions_of(t, threshold, sz_limit); + unsigned int nr_regions; + unsigned int max_thres; + + max_thres = c->attrs.aggr_interval / + (c->attrs.sample_interval ? c->attrs.sample_interval : 1); + do { + nr_regions = 0; + damon_for_each_target(t, c) { + damon_merge_regions_of(t, threshold, sz_limit); + nr_regions += damon_nr_regions(t); + } + threshold = max(1, threshold * 2); + } while (nr_regions > c->attrs.max_nr_regions && + threshold / 2 < max_thres); }
/*
On Mon, Jul 15, 2024 at 10:02:09AM -0700, SeongJae Park wrote:
Hi Greg,
On Mon, 15 Jul 2024 13:34:47 +0200 gregkh@linuxfoundation.org wrote:
The patch below does not apply to the 5.15-stable tree. If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to stable@vger.kernel.org.
To reproduce the conflict and resubmit, you may use the following commands:
git fetch https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ linux-5.15.y git checkout FETCH_HEAD git cherry-pick -x 310d6c15e9104c99d5d9d0ff8e5383a79da7d5e6
Similar to the failure of this patch for 6.1.y, I cannot reproduce the conflict on my setup. Attaching the patch for successfully cherry-picked one on my machine for any possible case.
Same issue as 6.1, it breaks the build :(
On Mon, 15 Jul 2024 19:12:42 +0200 Greg KH gregkh@linuxfoundation.org wrote:
On Mon, Jul 15, 2024 at 10:02:09AM -0700, SeongJae Park wrote:
Hi Greg,
On Mon, 15 Jul 2024 13:34:47 +0200 gregkh@linuxfoundation.org wrote:
The patch below does not apply to the 5.15-stable tree. If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to stable@vger.kernel.org.
To reproduce the conflict and resubmit, you may use the following commands:
git fetch https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ linux-5.15.y git checkout FETCH_HEAD git cherry-pick -x 310d6c15e9104c99d5d9d0ff8e5383a79da7d5e6
Similar to the failure of this patch for 6.1.y, I cannot reproduce the conflict on my setup. Attaching the patch for successfully cherry-picked one on my machine for any possible case.
Same issue as 6.1, it breaks the build :(
Thank you for letting me know this. I confirmed the issue can be reproduced on my setup, and made a local fix. In addition to the build failure, however, this causes a build warning. The warning occurs on 6.1, too, so we're discussing it on another thread[1]. I will send a fixed version once the discussion for the warning on the thread is completed.
[1] https://lore.kernel.org/20240715195946.1043767-1-sj@kernel.org
Thanks, SJ
linux-stable-mirror@lists.linaro.org