June 2024 - Linux-kselftest-mirror

[PATCH v2 0/3] Userspace controls soft-offline pages

by Jiaqi Yan

Correctable memory errors are very common on servers with large amount of memory, and are corrected by ECC, but with two pain points to users: 1. Correction usually happens on the fly and adds latency overhead 2. Not-fully-proved theory states excessive correctable memory errors can develop into uncorrectable memory error. Soft offline is kernel's additional solution for memory pages having (excessive) corrected memory errors. Impacted page is migrated to healthy page if it is in use, then the original page is discarded for any future use. The actual policy on whether (and when) to soft offline should be maintained by userspace, especially in case of an 1G HugeTLB page. Soft-offline dissolves the HugeTLB page, either in-use or free, into chunks of 4K pages, reducing HugeTLB pool capacity by 1 hugepage. If userspace has not acknowledged such behavior, it may be surprised when later mmap hugepages MAP_FAILED due to lack of hugepages. In case of a transparent hugepage, it will be split into 4K pages as well; userspace will stop enjoying the transparent performance. In addition, discarding the entire 1G HugeTLB page only because of corrected memory errors sounds very costly and kernel better not doing under the hood. But today there are at least 2 such cases: 1. GHES driver sees both GHES_SEV_CORRECTED and CPER_SEC_ERROR_THRESHOLD_EXCEEDED after parsing CPER. 2. RAS Correctable Errors Collector counts correctable errors per PFN and when the counter for a PFN reaches threshold In both cases, userspace has no control of the soft offline performed by kernel's memory failure recovery. This patch series give userspace the control of softofflining any page: kernel only soft offlines raw page / transparent hugepage / HugeTLB hugepage if userspace has agreed to. The interface to userspace is a new sysctl called enable_soft_offline under /proc/sys/vm. By default enable_soft_line is 1 to preserve existing behavior in kernel. Changelog v1 => v2: * incorporate feedbacks from both Miaohe Lin <linmiaohe(a)huawei.com> and Jane Chu <jane.chu(a)oracle.com>. * make the switch to control all pages, instead of HugeTLB specific. * change the API from /sys/kernel/mm/hugepages/hugepages-${size}kB/softoffline_corrected_errors to /proc/sys/vm/enable_soft_offline. * minor update to test code. * update documentation of the user control API. * v2 is based on commit 83a7eefedc9b ("Linux 6.10-rc3"). Jiaqi Yan (3): mm/memory-failure: userspace controls soft-offlining pages selftest/mm: test enable_soft_offline behaviors docs: mm: add enable_soft_offline sysctl Documentation/admin-guide/sysctl/vm.rst | 15 + mm/memory-failure.c | 16 ++ tools/testing/selftests/mm/.gitignore | 1 + tools/testing/selftests/mm/Makefile | 1 + .../selftests/mm/hugetlb-soft-offline.c | 258 ++++++++++++++++++ tools/testing/selftests/mm/run_vmtests.sh | 4 + 6 files changed, 295 insertions(+) create mode 100644 tools/testing/selftests/mm/hugetlb-soft-offline.c -- 2.45.2.505.gda0bf45e8d-goog

3 months, 2 weeks

5
16
0 0

[PATCH AUTOSEL 5.10 06/13] selftests/openat2: Fix build warnings on ppc64

by Sasha Levin

From: Michael Ellerman <mpe(a)ellerman.id.au> [ Upstream commit 84b6df4c49a1cc2854a16937acd5fd3e6315d083 ] Fix warnings like: openat2_test.c: In function ‘test_openat2_flags’: openat2_test.c:303:73: warning: format ‘%llX’ expects argument of type ‘long long unsigned int’, but argument 5 has type ‘__u64’ {aka ‘long unsigned int’} [-Wformat=] By switching to unsigned long long for u64 for ppc64 builds. Signed-off-by: Michael Ellerman <mpe(a)ellerman.id.au> Reviewed-by: Muhammad Usama Anjum <usama.anjum(a)collabora.com> Signed-off-by: Shuah Khan <skhan(a)linuxfoundation.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> --- tools/testing/selftests/openat2/openat2_test.c | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/openat2/openat2_test.c b/tools/testing/selftests/openat2/openat2_test.c index 453152b58e7f0..1045df1a98c07 100644 --- a/tools/testing/selftests/openat2/openat2_test.c +++ b/tools/testing/selftests/openat2/openat2_test.c @@ -5,6 +5,7 @@ */ #define _GNU_SOURCE +#define __SANE_USERSPACE_TYPES__ // Use ll64 #include <fcntl.h> #include <sched.h> #include <sys/stat.h> -- 2.43.0

3 months, 2 weeks

1
0
0 0

[PATCH AUTOSEL 5.15 14/21] selftests/openat2: Fix build warnings on ppc64

by Sasha Levin

From: Michael Ellerman <mpe(a)ellerman.id.au> [ Upstream commit 84b6df4c49a1cc2854a16937acd5fd3e6315d083 ] Fix warnings like: openat2_test.c: In function ‘test_openat2_flags’: openat2_test.c:303:73: warning: format ‘%llX’ expects argument of type ‘long long unsigned int’, but argument 5 has type ‘__u64’ {aka ‘long unsigned int’} [-Wformat=] By switching to unsigned long long for u64 for ppc64 builds. Signed-off-by: Michael Ellerman <mpe(a)ellerman.id.au> Reviewed-by: Muhammad Usama Anjum <usama.anjum(a)collabora.com> Signed-off-by: Shuah Khan <skhan(a)linuxfoundation.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> --- tools/testing/selftests/openat2/openat2_test.c | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/openat2/openat2_test.c b/tools/testing/selftests/openat2/openat2_test.c index 7fb902099de45..f9d2b0ec77564 100644 --- a/tools/testing/selftests/openat2/openat2_test.c +++ b/tools/testing/selftests/openat2/openat2_test.c @@ -5,6 +5,7 @@ */ #define _GNU_SOURCE +#define __SANE_USERSPACE_TYPES__ // Use ll64 #include <fcntl.h> #include <sched.h> #include <sys/stat.h> -- 2.43.0

3 months, 2 weeks

1
0
0 0

[PATCH AUTOSEL 6.1 21/29] selftests/futex: pass _GNU_SOURCE without a value to the compiler

by Sasha Levin

From: John Hubbard <jhubbard(a)nvidia.com> [ Upstream commit cb708ab9f584f159798b60853edcf0c8b67ce295 ] It's slightly better to set _GNU_SOURCE in the source code, but if one must do it via the compiler invocation, then the best way to do so is this: $(CC) -D_GNU_SOURCE= ...because otherwise, if this form is used: $(CC) -D_GNU_SOURCE ...then that leads the compiler to set a value, as if you had passed in: $(CC) -D_GNU_SOURCE=1 That, in turn, leads to warnings under both gcc and clang, like this: futex_requeue_pi.c:20: warning: "_GNU_SOURCE" redefined Fix this by using the "-D_GNU_SOURCE=" form. Reviewed-by: Edward Liaw <edliaw(a)google.com> Reviewed-by: Davidlohr Bueso <dave(a)stgolabs.net> Signed-off-by: John Hubbard <jhubbard(a)nvidia.com> Signed-off-by: Shuah Khan <skhan(a)linuxfoundation.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> --- tools/testing/selftests/futex/functional/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/testing/selftests/futex/functional/Makefile b/tools/testing/selftests/futex/functional/Makefile index a392d0917b4e5..994fa3468f170 100644 --- a/tools/testing/selftests/futex/functional/Makefile +++ b/tools/testing/selftests/futex/functional/Makefile @@ -1,6 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 INCLUDES := -I../include -I../../ $(KHDR_INCLUDES) -CFLAGS := $(CFLAGS) -g -O2 -Wall -D_GNU_SOURCE -pthread $(INCLUDES) $(KHDR_INCLUDES) +CFLAGS := $(CFLAGS) -g -O2 -Wall -D_GNU_SOURCE= -pthread $(INCLUDES) $(KHDR_INCLUDES) LDLIBS := -lpthread -lrt LOCAL_HDRS := \ -- 2.43.0

3 months, 2 weeks

1
0
0 0

[PATCH AUTOSEL 6.1 20/29] selftests/openat2: Fix build warnings on ppc64

by Sasha Levin

From: Michael Ellerman <mpe(a)ellerman.id.au> [ Upstream commit 84b6df4c49a1cc2854a16937acd5fd3e6315d083 ] Fix warnings like: openat2_test.c: In function ‘test_openat2_flags’: openat2_test.c:303:73: warning: format ‘%llX’ expects argument of type ‘long long unsigned int’, but argument 5 has type ‘__u64’ {aka ‘long unsigned int’} [-Wformat=] By switching to unsigned long long for u64 for ppc64 builds. Signed-off-by: Michael Ellerman <mpe(a)ellerman.id.au> Reviewed-by: Muhammad Usama Anjum <usama.anjum(a)collabora.com> Signed-off-by: Shuah Khan <skhan(a)linuxfoundation.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> --- tools/testing/selftests/openat2/openat2_test.c | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/openat2/openat2_test.c b/tools/testing/selftests/openat2/openat2_test.c index 7fb902099de45..f9d2b0ec77564 100644 --- a/tools/testing/selftests/openat2/openat2_test.c +++ b/tools/testing/selftests/openat2/openat2_test.c @@ -5,6 +5,7 @@ */ #define _GNU_SOURCE +#define __SANE_USERSPACE_TYPES__ // Use ll64 #include <fcntl.h> #include <sched.h> #include <sys/stat.h> -- 2.43.0

3 months, 2 weeks

1
0
0 0

[PATCH AUTOSEL 6.6 24/35] selftests/futex: pass _GNU_SOURCE without a value to the compiler

by Sasha Levin

From: John Hubbard <jhubbard(a)nvidia.com> [ Upstream commit cb708ab9f584f159798b60853edcf0c8b67ce295 ] It's slightly better to set _GNU_SOURCE in the source code, but if one must do it via the compiler invocation, then the best way to do so is this: $(CC) -D_GNU_SOURCE= ...because otherwise, if this form is used: $(CC) -D_GNU_SOURCE ...then that leads the compiler to set a value, as if you had passed in: $(CC) -D_GNU_SOURCE=1 That, in turn, leads to warnings under both gcc and clang, like this: futex_requeue_pi.c:20: warning: "_GNU_SOURCE" redefined Fix this by using the "-D_GNU_SOURCE=" form. Reviewed-by: Edward Liaw <edliaw(a)google.com> Reviewed-by: Davidlohr Bueso <dave(a)stgolabs.net> Signed-off-by: John Hubbard <jhubbard(a)nvidia.com> Signed-off-by: Shuah Khan <skhan(a)linuxfoundation.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> --- tools/testing/selftests/futex/functional/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/testing/selftests/futex/functional/Makefile b/tools/testing/selftests/futex/functional/Makefile index a392d0917b4e5..994fa3468f170 100644 --- a/tools/testing/selftests/futex/functional/Makefile +++ b/tools/testing/selftests/futex/functional/Makefile @@ -1,6 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 INCLUDES := -I../include -I../../ $(KHDR_INCLUDES) -CFLAGS := $(CFLAGS) -g -O2 -Wall -D_GNU_SOURCE -pthread $(INCLUDES) $(KHDR_INCLUDES) +CFLAGS := $(CFLAGS) -g -O2 -Wall -D_GNU_SOURCE= -pthread $(INCLUDES) $(KHDR_INCLUDES) LDLIBS := -lpthread -lrt LOCAL_HDRS := \ -- 2.43.0

3 months, 2 weeks

1
0
0 0

[PATCH AUTOSEL 6.6 23/35] selftests/openat2: Fix build warnings on ppc64

by Sasha Levin

From: Michael Ellerman <mpe(a)ellerman.id.au> [ Upstream commit 84b6df4c49a1cc2854a16937acd5fd3e6315d083 ] Fix warnings like: openat2_test.c: In function ‘test_openat2_flags’: openat2_test.c:303:73: warning: format ‘%llX’ expects argument of type ‘long long unsigned int’, but argument 5 has type ‘__u64’ {aka ‘long unsigned int’} [-Wformat=] By switching to unsigned long long for u64 for ppc64 builds. Signed-off-by: Michael Ellerman <mpe(a)ellerman.id.au> Reviewed-by: Muhammad Usama Anjum <usama.anjum(a)collabora.com> Signed-off-by: Shuah Khan <skhan(a)linuxfoundation.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> --- tools/testing/selftests/openat2/openat2_test.c | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/openat2/openat2_test.c b/tools/testing/selftests/openat2/openat2_test.c index 7fb902099de45..f9d2b0ec77564 100644 --- a/tools/testing/selftests/openat2/openat2_test.c +++ b/tools/testing/selftests/openat2/openat2_test.c @@ -5,6 +5,7 @@ */ #define _GNU_SOURCE +#define __SANE_USERSPACE_TYPES__ // Use ll64 #include <fcntl.h> #include <sched.h> #include <sys/stat.h> -- 2.43.0

3 months, 2 weeks

1
0
0 0

[PATCH AUTOSEL 6.6 22/35] selftests: cachestat: Fix build warnings on ppc64

by Sasha Levin

From: Michael Ellerman <mpe(a)ellerman.id.au> [ Upstream commit bc4d5f5d2debf8bb65fba188313481549ead8576 ] Fix warnings like: test_cachestat.c: In function ‘print_cachestat’: test_cachestat.c:30:38: warning: format ‘%llu’ expects argument of type ‘long long unsigned int’, but argument 2 has type ‘__u64’ {aka ‘long unsigned int’} [-Wformat=] By switching to unsigned long long for u64 for ppc64 builds. Signed-off-by: Michael Ellerman <mpe(a)ellerman.id.au> Signed-off-by: Shuah Khan <skhan(a)linuxfoundation.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> --- tools/testing/selftests/cachestat/test_cachestat.c | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/cachestat/test_cachestat.c b/tools/testing/selftests/cachestat/test_cachestat.c index 4804c7dc7b312..ddb70d418c6a8 100644 --- a/tools/testing/selftests/cachestat/test_cachestat.c +++ b/tools/testing/selftests/cachestat/test_cachestat.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 #define _GNU_SOURCE +#define __SANE_USERSPACE_TYPES__ // Use ll64 #include <stdio.h> #include <stdbool.h> -- 2.43.0

3 months, 2 weeks

1
0
0 0

[PATCH AUTOSEL 6.9 28/44] selftests/futex: pass _GNU_SOURCE without a value to the compiler

by Sasha Levin

From: John Hubbard <jhubbard(a)nvidia.com> [ Upstream commit cb708ab9f584f159798b60853edcf0c8b67ce295 ] It's slightly better to set _GNU_SOURCE in the source code, but if one must do it via the compiler invocation, then the best way to do so is this: $(CC) -D_GNU_SOURCE= ...because otherwise, if this form is used: $(CC) -D_GNU_SOURCE ...then that leads the compiler to set a value, as if you had passed in: $(CC) -D_GNU_SOURCE=1 That, in turn, leads to warnings under both gcc and clang, like this: futex_requeue_pi.c:20: warning: "_GNU_SOURCE" redefined Fix this by using the "-D_GNU_SOURCE=" form. Reviewed-by: Edward Liaw <edliaw(a)google.com> Reviewed-by: Davidlohr Bueso <dave(a)stgolabs.net> Signed-off-by: John Hubbard <jhubbard(a)nvidia.com> Signed-off-by: Shuah Khan <skhan(a)linuxfoundation.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> --- tools/testing/selftests/futex/functional/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/testing/selftests/futex/functional/Makefile b/tools/testing/selftests/futex/functional/Makefile index a392d0917b4e5..994fa3468f170 100644 --- a/tools/testing/selftests/futex/functional/Makefile +++ b/tools/testing/selftests/futex/functional/Makefile @@ -1,6 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 INCLUDES := -I../include -I../../ $(KHDR_INCLUDES) -CFLAGS := $(CFLAGS) -g -O2 -Wall -D_GNU_SOURCE -pthread $(INCLUDES) $(KHDR_INCLUDES) +CFLAGS := $(CFLAGS) -g -O2 -Wall -D_GNU_SOURCE= -pthread $(INCLUDES) $(KHDR_INCLUDES) LDLIBS := -lpthread -lrt LOCAL_HDRS := \ -- 2.43.0

3 months, 2 weeks

1
0
0 0

[PATCH AUTOSEL 6.9 27/44] selftests/overlayfs: Fix build error on ppc64

by Sasha Levin

From: Michael Ellerman <mpe(a)ellerman.id.au> [ Upstream commit e8b8c5264d4ebd248f60a5cef077fe615806e7a0 ] Fix build error on ppc64: dev_in_maps.c: In function ‘get_file_dev_and_inode’: dev_in_maps.c:60:59: error: format ‘%llu’ expects argument of type ‘long long unsigned int *’, but argument 7 has type ‘__u64 *’ {aka ‘long unsigned int *’} [-Werror=format=] By switching to unsigned long long for u64 for ppc64 builds. Signed-off-by: Michael Ellerman <mpe(a)ellerman.id.au> Signed-off-by: Shuah Khan <skhan(a)linuxfoundation.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> --- tools/testing/selftests/filesystems/overlayfs/dev_in_maps.c | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/filesystems/overlayfs/dev_in_maps.c b/tools/testing/selftests/filesystems/overlayfs/dev_in_maps.c index 759f86e7d263e..2862aae58b79a 100644 --- a/tools/testing/selftests/filesystems/overlayfs/dev_in_maps.c +++ b/tools/testing/selftests/filesystems/overlayfs/dev_in_maps.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 #define _GNU_SOURCE +#define __SANE_USERSPACE_TYPES__ // Use ll64 #include <inttypes.h> #include <unistd.h> -- 2.43.0

3 months, 2 weeks

1
0
0 0

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror June 2024