There are a number of atomic_t usages in the kernel where atomic_t api is used strictly for counting sequence numbers and other statistical counters and not for managing object lifetime.
The purpose of these Sequence Number Ops is to clearly differentiate atomic_t counter usages from atomic_t usages that guard object lifetimes, hence prone to overflow and underflow errors.
The atomic_t api provides a wide range of atomic operations as a base api to implement atomic counters, bitops, spinlock interfaces. The usages also evolved into being used for resource lifetimes and state management. The refcount_t api was introduced to address resource lifetime problems related to atomic_t wrapping. There is a large overlap between the atomic_t api used for resource lifetimes and just counters, stats, and sequence numbers. It has become difficult to differentiate between the atomic_t usages that should be converted to refcount_t and the ones that can be left alone. Introducing seqnum_ops to wrap the usages that are stats, counters, sequence numbers makes it easier for tools that scan for underflow and overflow on atomic_t usages to detect overflow and underflows to scan just the cases that are prone to errors.
Sequence Number api provides interfaces for simple atomic_t counter usages that just count, and don't guard resource lifetimes. The seqnum_ops are built on top of atomic_t api, providing a smaller subset of atomic_t interfaces necessary to support atomic_t usages as simple counters. This api has init/set/inc/dec/read and doesn't support any other atomic_t ops with the intent to restrict the use of these interfaces as simple counting usages.
Sequence Numbers wrap around to INT_MIN when it overflows and should not be used to guard resource lifetimes, device usage and open counts that control state changes, and pm states. Overflowing to INT_MIN is consistent with the atomic_t api, which it is built on top of.
Using seqnum to guard lifetimes could lead to use-after free when it overflows and undefined behavior when used to manage state changes and device usage/open states.
In addition this patch series converts a few drivers to use the new api. The following criteria is used for select variables for conversion:
1. Variable doesn't guard object lifetimes, manage state changes e.g: device usage counts, device open counts, and pm states. 2. Variable is used for stats and counters. 3. The conversion doesn't change the overflow behavior. 4. Note: inc_return() usages are changed to _inc() followed by _read() Patches: 03/13, 04/13, 09/13, 10/13, 11/13 5. drivers/acpi and drivers/acpi/apei patches have been reviewed before the rename, however in addition to rename, inc_return() usages are changed to _inc() followed by _read() 6. test_async_driver_probe, char/ipmi, and edac patches have been reviewed and no changes other than the rename to seqnum_ops. 7. security/integrity/ima: Okay to depend on CONFIG_64BIT?
The work for this is a follow-on to the discussion and review of Introduce Simple atomic counters patch series:
//lore.kernel.org/lkml/cover.1602209970.git.skhan@linuxfoundation.org/
Based on the feedback to restrict and limit the scope: - dropped inc_return() - renamed interfaces to match the intent and also shorten the interface names.
Shuah Khan (13): seqnum_ops: Introduce Sequence Number Ops selftests: lib:test_seqnum_ops: add new test for seqnum_ops drivers/acpi: convert seqno seqnum_ops drivers/acpi/apei: convert seqno to seqnum_ops drivers/base/test/test_async_driver_probe: convert to use seqnum_ops drivers/char/ipmi: convert stats to use seqnum_ops drivers/edac: convert pci counters to seqnum_ops drivers/oprofile: convert stats to use seqnum_ops drivers/staging/rtl8723bs: convert stats to use seqnum_ops usb: usbip/vhci: convert seqno to seqnum_ops drivers/staging/rtl8188eu: convert stats to use seqnum_ops drivers/staging/unisys/visorhba: convert stats to use seqnum_ops security/integrity/ima: converts stats to seqnum_ops
Documentation/core-api/atomic_ops.rst | 4 + Documentation/core-api/index.rst | 1 + Documentation/core-api/seqnum_ops.rst | 126 ++++++++++++++ MAINTAINERS | 8 + drivers/acpi/acpi_extlog.c | 6 +- drivers/acpi/apei/ghes.c | 6 +- drivers/base/test/test_async_driver_probe.c | 26 +-- drivers/char/ipmi/ipmi_msghandler.c | 9 +- drivers/char/ipmi/ipmi_si_intf.c | 9 +- drivers/char/ipmi/ipmi_ssif.c | 9 +- drivers/edac/edac_pci.h | 5 +- drivers/edac/edac_pci_sysfs.c | 28 ++-- drivers/oprofile/buffer_sync.c | 9 +- drivers/oprofile/event_buffer.c | 3 +- drivers/oprofile/oprof.c | 3 +- drivers/oprofile/oprofile_stats.c | 11 +- drivers/oprofile/oprofile_stats.h | 11 +- drivers/oprofile/oprofilefs.c | 3 +- drivers/staging/rtl8188eu/core/rtw_mlme_ext.c | 23 ++- .../staging/rtl8188eu/include/rtw_mlme_ext.h | 3 +- drivers/staging/rtl8723bs/core/rtw_cmd.c | 3 +- drivers/staging/rtl8723bs/core/rtw_mlme_ext.c | 33 ++-- drivers/staging/rtl8723bs/include/rtw_cmd.h | 3 +- .../staging/rtl8723bs/include/rtw_mlme_ext.h | 3 +- .../staging/unisys/visorhba/visorhba_main.c | 37 +++-- drivers/usb/usbip/vhci.h | 3 +- drivers/usb/usbip/vhci_hcd.c | 9 +- drivers/usb/usbip/vhci_rx.c | 3 +- include/linux/oprofile.h | 3 +- include/linux/seqnum_ops.h | 154 ++++++++++++++++++ lib/Kconfig | 9 + lib/Makefile | 1 + lib/test_seqnum_ops.c | 154 ++++++++++++++++++ security/integrity/ima/ima.h | 5 +- security/integrity/ima/ima_api.c | 2 +- security/integrity/ima/ima_fs.c | 4 +- security/integrity/ima/ima_queue.c | 7 +- tools/testing/selftests/lib/Makefile | 1 + tools/testing/selftests/lib/config | 1 + .../testing/selftests/lib/test_seqnum_ops.sh | 10 ++ 40 files changed, 637 insertions(+), 111 deletions(-) create mode 100644 Documentation/core-api/seqnum_ops.rst create mode 100644 include/linux/seqnum_ops.h create mode 100644 lib/test_seqnum_ops.c create mode 100755 tools/testing/selftests/lib/test_seqnum_ops.sh
Add a new selftest for testing seqnum_ops. This test loads test_seqnum_ops test module and unloads it. The test module runs tests and prints results to dmesg.
There are a number of atomic_t usages in the kernel where atomic_t api is used strictly for counting sequence numbers and other statistical counters and not for managing object lifetime.
The purpose of these Sequence Number Ops is to clearly differentiate atomic_t counter usages from atomic_t usages that guard object lifetimes, hence prone to overflow and underflow errors.
The atomic_t api provides a wide range of atomic operations as a base api to implement atomic counters, bitops, spinlock interfaces. The usages also evolved into being used for resource lifetimes and state management. The refcount_t api was introduced to address resource lifetime problems related to atomic_t wrapping. There is a large overlap between the atomic_t api used for resource lifetimes and just counters, stats, and sequence numbers. It has become difficult to differentiate between the atomic_t usages that should be converted to refcount_t and the ones that can be left alone. Introducing seqnum_ops to wrap the usages that are stats, counters, sequence numbers makes it easier for tools that scan for underflow and overflow on atomic_t usages to detect overflow and underflows to scan just the cases that are prone to errors.
Sequence Number api provides interfaces for simple atomic_t counter usages that just count, and don't guard resource lifetimes. The seqnum_ops are built on top of atomic_t api, providing a smaller subset of atomic_t interfaces necessary to support atomic_t usages as simple counters. This api has init/set/inc/dec/read and doesn't support other atomic_t ops with the intent to restrict the use of these interfaces as simple counting usages.
Sequence Numbers wrap around to INT_MIN when it overflows and should not be used to guard resource lifetimes, device usage and open counts that control state changes, and pm states. Overflowing to INT_MIN is consistent with the atomic_t api, which it is built on top of.
Using seqnum to guard lifetimes could lead to use-after free when it overflows and undefined behavior when used to manage state changes and device usage/open states.
Signed-off-by: Shuah Khan skhan@linuxfoundation.org --- Documentation/core-api/seqnum_ops.rst | 9 +++++++++ MAINTAINERS | 1 + include/linux/seqnum_ops.h | 2 ++ tools/testing/selftests/lib/Makefile | 1 + tools/testing/selftests/lib/config | 1 + tools/testing/selftests/lib/test_seqnum_ops.sh | 10 ++++++++++ 6 files changed, 24 insertions(+) create mode 100755 tools/testing/selftests/lib/test_seqnum_ops.sh
diff --git a/Documentation/core-api/seqnum_ops.rst b/Documentation/core-api/seqnum_ops.rst index 7a396c2cda19..3a9ddba985f2 100644 --- a/Documentation/core-api/seqnum_ops.rst +++ b/Documentation/core-api/seqnum_ops.rst @@ -115,3 +115,12 @@ Decrements sequence number and doesn't return the new value. ::
seqnum32_dec() --> atomic_dec() seqnum64_dec() --> atomic64_dec() + +Where are the seqnum_ops and how to use and test them? +------------------------------------------------------ + +.. kernel-doc:: include/linux/seqnum_ops.h + +Please see lib/test_seqnum_ops.c for examples usages. +Please find selftest: testing/selftests/lib/test_seqnum_ops.sh +Please check dmesg for results after running test_seqnum_ops.sh. diff --git a/MAINTAINERS b/MAINTAINERS index c83a6f05610b..e6ae131836a5 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -15983,6 +15983,7 @@ L: linux-kernel@vger.kernel.org S: Maintained F: include/linux/seqnum_ops.h F: lib/test_seqnum_ops.c +F: tools/testing/selftests/lib/test_seqnum_ops.sh
SIMPLE FIRMWARE INTERFACE (SFI) S: Obsolete diff --git a/include/linux/seqnum_ops.h b/include/linux/seqnum_ops.h index b97c7f310beb..a1def2ad5bc2 100644 --- a/include/linux/seqnum_ops.h +++ b/include/linux/seqnum_ops.h @@ -28,6 +28,8 @@ * * Reference and API guide: * Documentation/core-api/seqnum_ops.rst for more information. + * lib/test_seqnum_ops.c - example usages + * tools/testing/selftests/lib/test_seqnum_ops.sh * */
diff --git a/tools/testing/selftests/lib/Makefile b/tools/testing/selftests/lib/Makefile index a105f094676e..1818444f0e97 100644 --- a/tools/testing/selftests/lib/Makefile +++ b/tools/testing/selftests/lib/Makefile @@ -5,5 +5,6 @@ all:
TEST_PROGS := printf.sh bitmap.sh prime_numbers.sh strscpy.sh +TEST_PROGS += test_seqnum_ops.sh
include ../lib.mk diff --git a/tools/testing/selftests/lib/config b/tools/testing/selftests/lib/config index b80ee3f6e265..674ed2a2ac82 100644 --- a/tools/testing/selftests/lib/config +++ b/tools/testing/selftests/lib/config @@ -3,3 +3,4 @@ CONFIG_TEST_BITMAP=m CONFIG_PRIME_NUMBERS=m CONFIG_TEST_STRSCPY=m CONFIG_TEST_BITOPS=m +CONFIG_TEST_SEQNUM_OPS=m diff --git a/tools/testing/selftests/lib/test_seqnum_ops.sh b/tools/testing/selftests/lib/test_seqnum_ops.sh new file mode 100755 index 000000000000..fdce16b220ba --- /dev/null +++ b/tools/testing/selftests/lib/test_seqnum_ops.sh @@ -0,0 +1,10 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# +# Copyright (c) 2020 Shuah Khan skhan@linuxfoundation.org +# Copyright (c) 2020 The Linux Foundation +# +# Tests the Sequence Number Ops interfaces using test_seqnum_ops +# kernel module +# +$(dirname $0)/../kselftest/module.sh "test_seqnum_ops" test_seqnum_ops
On Tue, Nov 10, 2020 at 12:53:26PM -0700, Shuah Khan wrote:
There are a number of atomic_t usages in the kernel where atomic_t api is used strictly for counting sequence numbers and other statistical counters and not for managing object lifetime.
The purpose of these Sequence Number Ops is to clearly differentiate atomic_t counter usages from atomic_t usages that guard object lifetimes, hence prone to overflow and underflow errors.
The atomic_t api provides a wide range of atomic operations as a base api to implement atomic counters, bitops, spinlock interfaces. The usages also evolved into being used for resource lifetimes and state management. The refcount_t api was introduced to address resource lifetime problems related to atomic_t wrapping. There is a large overlap between the atomic_t api used for resource lifetimes and just counters, stats, and sequence numbers. It has become difficult to differentiate between the atomic_t usages that should be converted to refcount_t and the ones that can be left alone. Introducing seqnum_ops to wrap the usages that are stats, counters, sequence numbers makes it easier for tools that scan for underflow and overflow on atomic_t usages to detect overflow and underflows to scan just the cases that are prone to errors.
Sequence Number api provides interfaces for simple atomic_t counter usages that just count, and don't guard resource lifetimes. The seqnum_ops are built on top of atomic_t api, providing a smaller subset of atomic_t interfaces necessary to support atomic_t usages as simple counters. This api has init/set/inc/dec/read and doesn't support any other atomic_t ops with the intent to restrict the use of these interfaces as simple counting usages.
Sequence Numbers wrap around to INT_MIN when it overflows and should not be used to guard resource lifetimes, device usage and open counts that control state changes, and pm states. Overflowing to INT_MIN is consistent with the atomic_t api, which it is built on top of.
If Sequence Numbers are subject to wraparound then they aren't reliable. Given that they aren't reliable, why use atomic instructions at all? Why not just use plain regular integers with READ_ONCE and WRITE_ONCE?
Alan Stern
On 11/10/20 1:44 PM, Alan Stern wrote:
On Tue, Nov 10, 2020 at 12:53:26PM -0700, Shuah Khan wrote:
There are a number of atomic_t usages in the kernel where atomic_t api is used strictly for counting sequence numbers and other statistical counters and not for managing object lifetime.
The purpose of these Sequence Number Ops is to clearly differentiate atomic_t counter usages from atomic_t usages that guard object lifetimes, hence prone to overflow and underflow errors.
The atomic_t api provides a wide range of atomic operations as a base api to implement atomic counters, bitops, spinlock interfaces. The usages also evolved into being used for resource lifetimes and state management. The refcount_t api was introduced to address resource lifetime problems related to atomic_t wrapping. There is a large overlap between the atomic_t api used for resource lifetimes and just counters, stats, and sequence numbers. It has become difficult to differentiate between the atomic_t usages that should be converted to refcount_t and the ones that can be left alone. Introducing seqnum_ops to wrap the usages that are stats, counters, sequence numbers makes it easier for tools that scan for underflow and overflow on atomic_t usages to detect overflow and underflows to scan just the cases that are prone to errors.
Sequence Number api provides interfaces for simple atomic_t counter usages that just count, and don't guard resource lifetimes. The seqnum_ops are built on top of atomic_t api, providing a smaller subset of atomic_t interfaces necessary to support atomic_t usages as simple counters. This api has init/set/inc/dec/read and doesn't support any other atomic_t ops with the intent to restrict the use of these interfaces as simple counting usages.
Sequence Numbers wrap around to INT_MIN when it overflows and should not be used to guard resource lifetimes, device usage and open counts that control state changes, and pm states. Overflowing to INT_MIN is consistent with the atomic_t api, which it is built on top of.
If Sequence Numbers are subject to wraparound then they aren't reliable. Given that they aren't reliable, why use atomic instructions at all? Why not just use plain regular integers with READ_ONCE and WRITE_ONCE?
You still need atomic update for these numbers. The intent is to provide atomic api for cases where the variable doesn't guard lifetimes and yet needs atomic instructions.
Several such usages where atomic_t is used for up counting, also use upper bounds. It is also an option to switch to seqnum64 to avoid wrap around in case there is a concern.
thanks, -- Shuah
On Tue, Nov 10, 2020 at 12:53:26PM -0700, Shuah Khan wrote:
There are a number of atomic_t usages in the kernel where atomic_t api is used strictly for counting sequence numbers and other statistical counters and not for managing object lifetime.
We already have something in Linux called a sequence counter, and it's different from this. ID counter? instance number? monotonic_t? stat_t?
On 11/10/20 9:33 PM, Matthew Wilcox wrote:
On Tue, Nov 10, 2020 at 12:53:26PM -0700, Shuah Khan wrote:
There are a number of atomic_t usages in the kernel where atomic_t api is used strictly for counting sequence numbers and other statistical counters and not for managing object lifetime.
We already have something in Linux called a sequence counter, and it's different from this. ID counter? instance number? monotonic_t? stat_t?
No results for monotonic_t or stat_t. Can you give me a pointer to what your referring to.
thanks, -- Shuah
On Wed, Nov 11, 2020 at 09:03:20AM -0700, Shuah Khan wrote:
On 11/10/20 9:33 PM, Matthew Wilcox wrote:
On Tue, Nov 10, 2020 at 12:53:26PM -0700, Shuah Khan wrote:
There are a number of atomic_t usages in the kernel where atomic_t api is used strictly for counting sequence numbers and other statistical counters and not for managing object lifetime.
We already have something in Linux called a sequence counter, and it's different from this. ID counter? instance number? monotonic_t? stat_t?
No results for monotonic_t or stat_t. Can you give me a pointer to what your referring to.
We have a seqcount_t. We need to call this something different. maybe we should call it stat_t (and for that usage, stat_add() as well as stat_inc() is a legitimate API to have).
linux-kselftest-mirror@lists.linaro.org
-
Alan Stern -
Matthew Wilcox -
Shuah Khan