On 2022-08-17 08:53, Thomas Deutschmann wrote:
I'll post my results when I finished this bisect session.
I bisected kernel with KV set to "5.16-rc1":
git bisect start # good: [2c85ebc57b3e1817b6ce1a6b703928e113a90442] Linux 5.10 git bisect good 2c85ebc57b3e1817b6ce1a6b703928e113a90442 # bad: [8bb7eca972ad531c9b149c0a51ab43a417385813] Linux 5.15 git bisect bad 8bb7eca972ad531c9b149c0a51ab43a417385813 # bad: [6bdf2fbc48f104a84606f6165aa8a20d9a7d9074] Merge tag 'nvme-5.13-2021-05-13' of git://git.infradead.org/nvme into block-5.13 git bisect bad 6bdf2fbc48f104a84606f6165aa8a20d9a7d9074 # good: [02f9fc286e039d0bef7284fb1200ee755b525bde] Merge tag 'pm-5.12-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm git bisect good 02f9fc286e039d0bef7284fb1200ee755b525bde # bad: [f351f4b63dac127079bbd77da64b2a61c09d522d] usb: xhci-mtk: fix oops when unbind driver git bisect bad f351f4b63dac127079bbd77da64b2a61c09d522d # good: [28b9aaac4cc5a11485b6f70656e4e9ead590cf5b] Merge tag 'clk-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/clk/linux git bisect good 28b9aaac4cc5a11485b6f70656e4e9ead590cf5b # good: [cf64c2a905e0dabcc473ca70baf275fb3a61fac4] Merge branch 'work.sparc32' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs git bisect good cf64c2a905e0dabcc473ca70baf275fb3a61fac4 # bad: [ea6be461cbedefaa881711a43f2842aabbd12fd4] Merge tag 'acpi-5.12-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm git bisect bad ea6be461cbedefaa881711a43f2842aabbd12fd4 # good: [1c9077cdecd027714736e70704da432ee2b946bb] Merge tag 'nfs-for-5.12-1' of git://git.linux-nfs.org/projects/anna/linux-nfs git bisect good 1c9077cdecd027714736e70704da432ee2b946bb # good: [efba6d3a7c4bb59f0750609fae0f9644d82304b6] Merge tag 'for-5.12/io_uring-2021-02-25' of git://git.kernel.dk/linux-block git bisect good efba6d3a7c4bb59f0750609fae0f9644d82304b6 # bad: [0b311e34d5033fdcca4c9b5f2d9165b3604704d3] Merge tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi git bisect bad 0b311e34d5033fdcca4c9b5f2d9165b3604704d3 # good: [5ceabb6078b80a8544ba86d6ee523ad755ae6d5e] Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs git bisect good 5ceabb6078b80a8544ba86d6ee523ad755ae6d5e # bad: [3ab6608e66b16159c3a3c2d7015b9c11cd3396c1] Merge tag 'block-5.12-2021-02-27' of git://git.kernel.dk/linux-block git bisect bad 3ab6608e66b16159c3a3c2d7015b9c11cd3396c1 # bad: [e941894eae31b52f0fd9bdb3ce20620afa152f45] io-wq: make buffered file write hashed work map per-ctx git bisect bad e941894eae31b52f0fd9bdb3ce20620afa152f45 # good: [4379bf8bd70b5de6bba7d53015b0c36c57a634ee] io_uring: remove io_identity git bisect good 4379bf8bd70b5de6bba7d53015b0c36c57a634ee # good: [1c0aa1fae1acb77c5f9917adb0e4cb4500b9f3a6] io_uring: flag new native workers with IORING_FEAT_NATIVE_WORKERS git bisect good 1c0aa1fae1acb77c5f9917adb0e4cb4500b9f3a6 # good: [0100e6bbdbb79404e56939313662b42737026574] arch: ensure parisc/powerpc handle PF_IO_WORKER in copy_thread() git bisect good 0100e6bbdbb79404e56939313662b42737026574 # good: [8b3e78b5955abb98863832453f5c74eca8f53c3a] io-wq: fix races around manager/worker creation and task exit git bisect good 8b3e78b5955abb98863832453f5c74eca8f53c3a # good: [eb2de9418d56b5e6ebf27bad51dbce3e22ee109b] io-wq: fix race around io_worker grabbing git bisect good eb2de9418d56b5e6ebf27bad51dbce3e22ee109b # first bad commit: [e941894eae31b52f0fd9bdb3ce20620afa152f45] io-wq: make buffered file write hashed work map per-ctx
From e941894eae31b52f0fd9bdb3ce20620afa152f45 From: Jens Axboe Date: Fri, 19 Feb 2021 12:33:30 -0700 Subject: io-wq: make buffered file write hashed work map per-ctx
Before the io-wq thread change, we maintained a hash work map and lock per-node per-ring. That wasn't ideal, as we really wanted it to be per ring. But now that we have per-task workers, the hash map ends up being just per-task. That'll work just fine for the normal case of having one task use a ring, but if you share the ring between tasks, then it's considerably worse than it was before.
Make the hash map per ctx instead, which provides full per-ctx buffered write serialization on hashed writes.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i...
But I think this result is misleading.
Like mentioned, the problem I experienced during this bisect session was different (not the FIFREEZE ioctl hang). This sounds more like the already fixed regressions caused by the commit above, i.e.
- https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=...
- https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=...
I will do another round with 2b7196a219bf (good) <-> 5.18 (bad).