5.10 fixup for 89c2b3b7491820 ("io_uring: reexpand under-reexpanded iters"). We can't just directly cherry-pick them as the code base is quite different, so we also need patch 1/2. Previous attempts to backport 2/2 directly were pulling in too many dependencies only adding more problems.
Pavel Begunkov (2): io_uring: don't re-import iovecs from callbacks io_uring: fix using under-expanded iters
fs/io_uring.c | 47 ++++++----------------------------------------- 1 file changed, 6 insertions(+), 41 deletions(-)
We can't re-import or modify iterators from iocb callbacks, it's not safe as it might be reverted and/or reexpanded while unwinding stack. It's also not safe to resubmit as io-wq thread will race with stack undwinding for the iterator and other data.
Disallow resubmission from callbacks, it can fail some cases that were handled before, but the possibility of such a failure was a part of the API from the beginning and so it should be fine.
Signed-off-by: Pavel Begunkov asml.silence@gmail.com --- fs/io_uring.c | 39 --------------------------------------- 1 file changed, 39 deletions(-)
diff --git a/fs/io_uring.c b/fs/io_uring.c index 4330603eae35..aded83f20a15 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -2579,45 +2579,6 @@ static void io_complete_rw_common(struct kiocb *kiocb, long res, #ifdef CONFIG_BLOCK static bool io_resubmit_prep(struct io_kiocb *req, int error) { - struct iovec inline_vecs[UIO_FASTIOV], *iovec = inline_vecs; - ssize_t ret = -ECANCELED; - struct iov_iter iter; - int rw; - - if (error) { - ret = error; - goto end_req; - } - - switch (req->opcode) { - case IORING_OP_READV: - case IORING_OP_READ_FIXED: - case IORING_OP_READ: - rw = READ; - break; - case IORING_OP_WRITEV: - case IORING_OP_WRITE_FIXED: - case IORING_OP_WRITE: - rw = WRITE; - break; - default: - printk_once(KERN_WARNING "io_uring: bad opcode in resubmit %d\n", - req->opcode); - goto end_req; - } - - if (!req->async_data) { - ret = io_import_iovec(rw, req, &iovec, &iter, false); - if (ret < 0) - goto end_req; - ret = io_setup_async_rw(req, iovec, inline_vecs, &iter, false); - if (!ret) - return true; - kfree(iovec); - } else { - return true; - } -end_req: req_set_fail_links(req); return false; }
[ upstream commit cd65869512ab5668a5d16f789bc4da1319c435c4 ]
The issue was first described and addressed in 89c2b3b7491820 ("io_uring: reexpand under-reexpanded iters"), but shortly after reimplemented as. cd65869512ab56 ("io_uring: use iov_iter state save/restore helpers").
Here we follow the approach from the second patch but without in-callback resubmissions, fixups for not yet supported in 5.10 short read retries and replacing iov_iter_state with iter copies to not pull even more dependencies, and because it's just much simpler.
Signed-off-by: Pavel Begunkov asml.silence@gmail.com --- fs/io_uring.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/fs/io_uring.c b/fs/io_uring.c index aded83f20a15..b2b5edee1512 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -3389,6 +3389,7 @@ static int io_read(struct io_kiocb *req, bool force_nonblock, struct iovec inline_vecs[UIO_FASTIOV], *iovec = inline_vecs; struct kiocb *kiocb = &req->rw.kiocb; struct iov_iter __iter, *iter = &__iter; + struct iov_iter iter_cp; struct io_async_rw *rw = req->async_data; ssize_t io_size, ret, ret2; bool no_async; @@ -3399,6 +3400,7 @@ static int io_read(struct io_kiocb *req, bool force_nonblock, ret = io_import_iovec(READ, req, &iovec, iter, !force_nonblock); if (ret < 0) return ret; + iter_cp = *iter; io_size = iov_iter_count(iter); req->result = io_size; ret = 0; @@ -3434,7 +3436,7 @@ static int io_read(struct io_kiocb *req, bool force_nonblock, if (req->file->f_flags & O_NONBLOCK) goto done; /* some cases will consume bytes even on error returns */ - iov_iter_revert(iter, io_size - iov_iter_count(iter)); + *iter = iter_cp; ret = 0; goto copy_iov; } else if (ret < 0) { @@ -3517,6 +3519,7 @@ static int io_write(struct io_kiocb *req, bool force_nonblock, struct iovec inline_vecs[UIO_FASTIOV], *iovec = inline_vecs; struct kiocb *kiocb = &req->rw.kiocb; struct iov_iter __iter, *iter = &__iter; + struct iov_iter iter_cp; struct io_async_rw *rw = req->async_data; ssize_t ret, ret2, io_size;
@@ -3526,6 +3529,7 @@ static int io_write(struct io_kiocb *req, bool force_nonblock, ret = io_import_iovec(WRITE, req, &iovec, iter, !force_nonblock); if (ret < 0) return ret; + iter_cp = *iter; io_size = iov_iter_count(iter); req->result = io_size;
@@ -3587,7 +3591,7 @@ static int io_write(struct io_kiocb *req, bool force_nonblock, } else { copy_iov: /* some cases will consume bytes even on error returns */ - iov_iter_revert(iter, io_size - iov_iter_count(iter)); + *iter = iter_cp; ret = io_setup_async_rw(req, iovec, inline_vecs, iter, false); if (!ret) return -EAGAIN;
On Fri, Jun 03, 2022 at 01:17:03PM +0100, Pavel Begunkov wrote:
5.10 fixup for 89c2b3b7491820 ("io_uring: reexpand under-reexpanded iters"). We can't just directly cherry-pick them as the code base is quite different, so we also need patch 1/2. Previous attempts to backport 2/2 directly were pulling in too many dependencies only adding more problems.
Pavel Begunkov (2): io_uring: don't re-import iovecs from callbacks io_uring: fix using under-expanded iters
fs/io_uring.c | 47 ++++++----------------------------------------- 1 file changed, 6 insertions(+), 41 deletions(-)
-- 2.36.1
All now queued up, thanks.
greg k-h
linux-stable-mirror@lists.linaro.org
-
Greg Kroah-Hartman -
Pavel Begunkov