The function ovl_dir_real_file() currently uses the semaphore of the inode to synchronize write to the upperfile cache field.
However, this function will get called by ovl_ioctl_set_flags(), which utilizes the inode semaphore too. In this case ovl_dir_real_file() will try to claim a lock that is owned by a function in its call stack, which won't get released before ovl_dir_real_file() returns.
Define a dedicated semaphore for the upperfile cache, so that the deadlock won't happen.
Fixes: 61536bed2149 ("ovl: support [S|G]ETFLAGS and FS[S|G]ETXATTR ioctls for directories") Cc: stable@vger.kernel.org # v5.10 Signed-off-by: Icenowy Zheng icenowy@aosc.io --- Changes in v2: - Fixed missing replacement in error handling path. Changes in v3: - Use mutex instead of semaphore.
fs/overlayfs/readdir.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/fs/overlayfs/readdir.c b/fs/overlayfs/readdir.c index 01620ebae1bd..3980f9982f34 100644 --- a/fs/overlayfs/readdir.c +++ b/fs/overlayfs/readdir.c @@ -56,6 +56,7 @@ struct ovl_dir_file { struct list_head *cursor; struct file *realfile; struct file *upperfile; + struct mutex upperfile_mutex; };
static struct ovl_cache_entry *ovl_cache_entry_from_node(struct rb_node *n) @@ -874,8 +875,6 @@ struct file *ovl_dir_real_file(const struct file *file, bool want_upper) * Need to check if we started out being a lower dir, but got copied up */ if (!od->is_upper) { - struct inode *inode = file_inode(file); - realfile = READ_ONCE(od->upperfile); if (!realfile) { struct path upperpath; @@ -883,10 +882,10 @@ struct file *ovl_dir_real_file(const struct file *file, bool want_upper) ovl_path_upper(dentry, &upperpath); realfile = ovl_dir_open_realfile(file, &upperpath);
- inode_lock(inode); + mutex_lock(&od->upperfile_mutex); if (!od->upperfile) { if (IS_ERR(realfile)) { - inode_unlock(inode); + mutex_unlock(&od->upperfile_mutex); return realfile; } smp_store_release(&od->upperfile, realfile); @@ -896,7 +895,7 @@ struct file *ovl_dir_real_file(const struct file *file, bool want_upper) fput(realfile); realfile = od->upperfile; } - inode_unlock(inode); + mutex_unlock(&od->upperfile_mutex); } }
@@ -959,6 +958,7 @@ static int ovl_dir_open(struct inode *inode, struct file *file) od->realfile = realfile; od->is_real = ovl_dir_is_real(file->f_path.dentry); od->is_upper = OVL_TYPE_UPPER(type); + mutex_init(&od->upperfile_mutex); file->private_data = od;
return 0;
On Tue, Jan 5, 2021 at 2:36 AM Icenowy Zheng icenowy@aosc.io wrote:
The function ovl_dir_real_file() currently uses the semaphore of the inode to synchronize write to the upperfile cache field.
Although the inode lock is a rw_sem it is referred to as the "inode lock" and you also left semaphore in the commit subject. No need to re-post. This can be fixed on commit.
However, this function will get called by ovl_ioctl_set_flags(), which utilizes the inode semaphore too. In this case ovl_dir_real_file() will try to claim a lock that is owned by a function in its call stack, which won't get released before ovl_dir_real_file() returns.
Define a dedicated semaphore for the upperfile cache, so that the deadlock won't happen.
Fixes: 61536bed2149 ("ovl: support [S|G]ETFLAGS and FS[S|G]ETXATTR ioctls for directories") Cc: stable@vger.kernel.org # v5.10 Signed-off-by: Icenowy Zheng icenowy@aosc.io
Changes in v2:
- Fixed missing replacement in error handling path.
Changes in v3:
- Use mutex instead of semaphore.
fs/overlayfs/readdir.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/fs/overlayfs/readdir.c b/fs/overlayfs/readdir.c index 01620ebae1bd..3980f9982f34 100644 --- a/fs/overlayfs/readdir.c +++ b/fs/overlayfs/readdir.c @@ -56,6 +56,7 @@ struct ovl_dir_file { struct list_head *cursor; struct file *realfile; struct file *upperfile;
struct mutex upperfile_mutex;
That's a very specific name. This mutex protects members of struct ovl_dir_file, which could evolve into struct ovl_file one day (because no reason to cache only dir upper file), so I would go with a more generic name, but let's leave it to Miklos to decide.
He could have a different idea altogether for fixing this bug.
Thanks, Amir.
On Tue, Jan 5, 2021 at 8:47 AM Amir Goldstein amir73il@gmail.com wrote:
On Tue, Jan 5, 2021 at 2:36 AM Icenowy Zheng icenowy@aosc.io wrote:
The function ovl_dir_real_file() currently uses the semaphore of the inode to synchronize write to the upperfile cache field.
Although the inode lock is a rw_sem it is referred to as the "inode lock" and you also left semaphore in the commit subject. No need to re-post. This can be fixed on commit.
However, this function will get called by ovl_ioctl_set_flags(), which utilizes the inode semaphore too. In this case ovl_dir_real_file() will try to claim a lock that is owned by a function in its call stack, which won't get released before ovl_dir_real_file() returns.
Define a dedicated semaphore for the upperfile cache, so that the deadlock won't happen.
Fixes: 61536bed2149 ("ovl: support [S|G]ETFLAGS and FS[S|G]ETXATTR ioctls for directories") Cc: stable@vger.kernel.org # v5.10 Signed-off-by: Icenowy Zheng icenowy@aosc.io
Changes in v2:
- Fixed missing replacement in error handling path.
Changes in v3:
- Use mutex instead of semaphore.
fs/overlayfs/readdir.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/fs/overlayfs/readdir.c b/fs/overlayfs/readdir.c index 01620ebae1bd..3980f9982f34 100644 --- a/fs/overlayfs/readdir.c +++ b/fs/overlayfs/readdir.c @@ -56,6 +56,7 @@ struct ovl_dir_file { struct list_head *cursor; struct file *realfile; struct file *upperfile;
struct mutex upperfile_mutex;That's a very specific name. This mutex protects members of struct ovl_dir_file, which could evolve into struct ovl_file one day (because no reason to cache only dir upper file), so I would go with a more generic name, but let's leave it to Miklos to decide.
He could have a different idea altogether for fixing this bug.
Miklos,
Please fast track this or an alternative fix. It fixes an easy to reproduce deadlock introduced in 5.10. Icenowy Zheng has written a simple xfstest reproducer, but it wasn't posted - best to avoid hanging tester's machines until a fix is merged...
Thanks, Amir.
On Tue, Jan 05, 2021 at 08:47:41AM +0200, Amir Goldstein wrote:
On Tue, Jan 5, 2021 at 2:36 AM Icenowy Zheng icenowy@aosc.io wrote:
The function ovl_dir_real_file() currently uses the semaphore of the inode to synchronize write to the upperfile cache field.
Although the inode lock is a rw_sem it is referred to as the "inode lock" and you also left semaphore in the commit subject. No need to re-post. This can be fixed on commit.
However, this function will get called by ovl_ioctl_set_flags(), which utilizes the inode semaphore too. In this case ovl_dir_real_file() will try to claim a lock that is owned by a function in its call stack, which won't get released before ovl_dir_real_file() returns.
Define a dedicated semaphore for the upperfile cache, so that the deadlock won't happen.
Fixes: 61536bed2149 ("ovl: support [S|G]ETFLAGS and FS[S|G]ETXATTR ioctls for directories") Cc: stable@vger.kernel.org # v5.10 Signed-off-by: Icenowy Zheng icenowy@aosc.io
Changes in v2:
- Fixed missing replacement in error handling path.
Changes in v3:
- Use mutex instead of semaphore.
fs/overlayfs/readdir.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/fs/overlayfs/readdir.c b/fs/overlayfs/readdir.c index 01620ebae1bd..3980f9982f34 100644 --- a/fs/overlayfs/readdir.c +++ b/fs/overlayfs/readdir.c @@ -56,6 +56,7 @@ struct ovl_dir_file { struct list_head *cursor; struct file *realfile; struct file *upperfile;
struct mutex upperfile_mutex;That's a very specific name. This mutex protects members of struct ovl_dir_file, which could evolve into struct ovl_file one day (because no reason to cache only dir upper file), so I would go with a more generic name, but let's leave it to Miklos to decide.
He could have a different idea altogether for fixing this bug.
How about this (untested) patch?
It's a cleanup as well as a fix, but maybe we should separate the cleanup from the fix...
Thanks, Miklos ---
fs/overlayfs/readdir.c | 23 +++++++---------------- 1 file changed, 7 insertions(+), 16 deletions(-)
--- a/fs/overlayfs/readdir.c +++ b/fs/overlayfs/readdir.c @@ -865,7 +865,7 @@ struct file *ovl_dir_real_file(const str
struct ovl_dir_file *od = file->private_data; struct dentry *dentry = file->f_path.dentry; - struct file *realfile = od->realfile; + struct file *old, *realfile = od->realfile;
if (!OVL_TYPE_UPPER(ovl_path_type(dentry))) return want_upper ? NULL : realfile; @@ -874,29 +874,20 @@ struct file *ovl_dir_real_file(const str * Need to check if we started out being a lower dir, but got copied up */ if (!od->is_upper) { - struct inode *inode = file_inode(file); - realfile = READ_ONCE(od->upperfile); if (!realfile) { struct path upperpath;
ovl_path_upper(dentry, &upperpath); realfile = ovl_dir_open_realfile(file, &upperpath); + if (IS_ERR(realfile)) + return realfile;
- inode_lock(inode); - if (!od->upperfile) { - if (IS_ERR(realfile)) { - inode_unlock(inode); - return realfile; - } - smp_store_release(&od->upperfile, realfile); - } else { - /* somebody has beaten us to it */ - if (!IS_ERR(realfile)) - fput(realfile); - realfile = od->upperfile; + old = cmpxchg_release(&od->upperfile, NULL, realfile); + if (old) { + fput(realfile); + realfile = old; } - inode_unlock(inode); } }
On Wed, Jan 20, 2021 at 12:20 PM Miklos Szeredi miklos@szeredi.hu wrote:
On Tue, Jan 05, 2021 at 08:47:41AM +0200, Amir Goldstein wrote:
On Tue, Jan 5, 2021 at 2:36 AM Icenowy Zheng icenowy@aosc.io wrote:
The function ovl_dir_real_file() currently uses the semaphore of the inode to synchronize write to the upperfile cache field.
Although the inode lock is a rw_sem it is referred to as the "inode lock" and you also left semaphore in the commit subject. No need to re-post. This can be fixed on commit.
However, this function will get called by ovl_ioctl_set_flags(), which utilizes the inode semaphore too. In this case ovl_dir_real_file() will try to claim a lock that is owned by a function in its call stack, which won't get released before ovl_dir_real_file() returns.
Define a dedicated semaphore for the upperfile cache, so that the deadlock won't happen.
Fixes: 61536bed2149 ("ovl: support [S|G]ETFLAGS and FS[S|G]ETXATTR ioctls for directories") Cc: stable@vger.kernel.org # v5.10 Signed-off-by: Icenowy Zheng icenowy@aosc.io
Changes in v2:
- Fixed missing replacement in error handling path.
Changes in v3:
- Use mutex instead of semaphore.
fs/overlayfs/readdir.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/fs/overlayfs/readdir.c b/fs/overlayfs/readdir.c index 01620ebae1bd..3980f9982f34 100644 --- a/fs/overlayfs/readdir.c +++ b/fs/overlayfs/readdir.c @@ -56,6 +56,7 @@ struct ovl_dir_file { struct list_head *cursor; struct file *realfile; struct file *upperfile;
struct mutex upperfile_mutex;That's a very specific name. This mutex protects members of struct ovl_dir_file, which could evolve into struct ovl_file one day (because no reason to cache only dir upper file), so I would go with a more generic name, but let's leave it to Miklos to decide.
He could have a different idea altogether for fixing this bug.
How about this (untested) patch?
Much better :)
It's a cleanup as well as a fix, but maybe we should separate the cleanup from the fix...
Thanks, Miklos
fs/overlayfs/readdir.c | 23 +++++++---------------- 1 file changed, 7 insertions(+), 16 deletions(-)
--- a/fs/overlayfs/readdir.c +++ b/fs/overlayfs/readdir.c @@ -865,7 +865,7 @@ struct file *ovl_dir_real_file(const str
struct ovl_dir_file *od = file->private_data; struct dentry *dentry = file->f_path.dentry;
struct file *realfile = od->realfile;
struct file *old, *realfile = od->realfile; if (!OVL_TYPE_UPPER(ovl_path_type(dentry))) return want_upper ? NULL : realfile;@@ -874,29 +874,20 @@ struct file *ovl_dir_real_file(const str * Need to check if we started out being a lower dir, but got copied up */ if (!od->is_upper) {
struct inode *inode = file_inode(file);realfile = READ_ONCE(od->upperfile); if (!realfile) { struct path upperpath; ovl_path_upper(dentry, &upperpath); realfile = ovl_dir_open_realfile(file, &upperpath);
if (IS_ERR(realfile))return realfile;
inode_lock(inode);if (!od->upperfile) {if (IS_ERR(realfile)) {inode_unlock(inode);return realfile;}smp_store_release(&od->upperfile, realfile);} else {/* somebody has beaten us to it */if (!IS_ERR(realfile))fput(realfile);realfile = od->upperfile;
old = cmpxchg_release(&od->upperfile, NULL, realfile);if (old) {fput(realfile);realfile = old; }
inode_unlock(inode); } }
在 2021-01-20星期三的 11:20 +0100，Miklos Szeredi写道：
On Tue, Jan 05, 2021 at 08:47:41AM +0200, Amir Goldstein wrote:
On Tue, Jan 5, 2021 at 2:36 AM Icenowy Zheng icenowy@aosc.io wrote:
The function ovl_dir_real_file() currently uses the semaphore of the inode to synchronize write to the upperfile cache field.
Although the inode lock is a rw_sem it is referred to as the "inode lock" and you also left semaphore in the commit subject. No need to re-post. This can be fixed on commit.
However, this function will get called by ovl_ioctl_set_flags(), which utilizes the inode semaphore too. In this case ovl_dir_real_file() will try to claim a lock that is owned by a function in its call stack, which won't get released before ovl_dir_real_file() returns.
Define a dedicated semaphore for the upperfile cache, so that the deadlock won't happen.
Fixes: 61536bed2149 ("ovl: support [S|G]ETFLAGS and FS[S|G]ETXATTR ioctls for directories") Cc: stable@vger.kernel.org # v5.10 Signed-off-by: Icenowy Zheng icenowy@aosc.io
Changes in v2:
- Fixed missing replacement in error handling path.
Changes in v3:
- Use mutex instead of semaphore.
fs/overlayfs/readdir.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/fs/overlayfs/readdir.c b/fs/overlayfs/readdir.c index 01620ebae1bd..3980f9982f34 100644 --- a/fs/overlayfs/readdir.c +++ b/fs/overlayfs/readdir.c @@ -56,6 +56,7 @@ struct ovl_dir_file { struct list_head *cursor; struct file *realfile; struct file *upperfile; + struct mutex upperfile_mutex;
That's a very specific name. This mutex protects members of struct ovl_dir_file, which could evolve into struct ovl_file one day (because no reason to cache only dir upper file), so I would go with a more generic name, but let's leave it to Miklos to decide.
He could have a different idea altogether for fixing this bug.
How about this (untested) patch?
It's a cleanup as well as a fix, but maybe we should separate the cleanup from the fix...
If you are going to post this, feel free to add
Tested-by: Icenowy Zheng icenowy@aosc.io
(And if you remove the IS_ERR(realfile) part, the tested-by tag still applies.)
Thanks, Miklos
fs/overlayfs/readdir.c | 23 +++++++---------------- 1 file changed, 7 insertions(+), 16 deletions(-)
--- a/fs/overlayfs/readdir.c +++ b/fs/overlayfs/readdir.c @@ -865,7 +865,7 @@ struct file *ovl_dir_real_file(const str struct ovl_dir_file *od = file->private_data; struct dentry *dentry = file->f_path.dentry; - struct file *realfile = od->realfile; + struct file *old, *realfile = od->realfile; if (!OVL_TYPE_UPPER(ovl_path_type(dentry))) return want_upper ? NULL : realfile; @@ -874,29 +874,20 @@ struct file *ovl_dir_real_file(const str * Need to check if we started out being a lower dir, but got copied up */ if (!od->is_upper) { - struct inode *inode = file_inode(file);
realfile = READ_ONCE(od->upperfile); if (!realfile) { struct path upperpath; ovl_path_upper(dentry, &upperpath); realfile = ovl_dir_open_realfile(file, &upperpath); + if (IS_ERR(realfile)) + return realfile; - inode_lock(inode); - if (!od->upperfile) { - if (IS_ERR(realfile)) { - inode_unlock(inode); - return realfile; - } - smp_store_release(&od->upperfile, realfile); - } else { - /* somebody has beaten us to it */ - if (!IS_ERR(realfile)) - fput(realfile); - realfile = od->upperfile; + old = cmpxchg_release(&od->upperfile, NULL, realfile); + if (old) { + fput(realfile); + realfile = old; } - inode_unlock(inode); } }
On Thu, Jan 21, 2021 at 4:43 AM Icenowy Zheng icenowy@aosc.io wrote:
在 2021-01-20星期三的 11:20 +0100，Miklos Szeredi写道：
On Tue, Jan 05, 2021 at 08:47:41AM +0200, Amir Goldstein wrote:
On Tue, Jan 5, 2021 at 2:36 AM Icenowy Zheng icenowy@aosc.io wrote:
The function ovl_dir_real_file() currently uses the semaphore of the inode to synchronize write to the upperfile cache field.
Although the inode lock is a rw_sem it is referred to as the "inode lock" and you also left semaphore in the commit subject. No need to re-post. This can be fixed on commit.
However, this function will get called by ovl_ioctl_set_flags(), which utilizes the inode semaphore too. In this case ovl_dir_real_file() will try to claim a lock that is owned by a function in its call stack, which won't get released before ovl_dir_real_file() returns.
Define a dedicated semaphore for the upperfile cache, so that the deadlock won't happen.
Fixes: 61536bed2149 ("ovl: support [S|G]ETFLAGS and FS[S|G]ETXATTR ioctls for directories") Cc: stable@vger.kernel.org # v5.10 Signed-off-by: Icenowy Zheng icenowy@aosc.io
Changes in v2:
- Fixed missing replacement in error handling path.
Changes in v3:
- Use mutex instead of semaphore.
fs/overlayfs/readdir.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/fs/overlayfs/readdir.c b/fs/overlayfs/readdir.c index 01620ebae1bd..3980f9982f34 100644 --- a/fs/overlayfs/readdir.c +++ b/fs/overlayfs/readdir.c @@ -56,6 +56,7 @@ struct ovl_dir_file { struct list_head *cursor; struct file *realfile; struct file *upperfile;
struct mutex upperfile_mutex;That's a very specific name. This mutex protects members of struct ovl_dir_file, which could evolve into struct ovl_file one day (because no reason to cache only dir upper file), so I would go with a more generic name, but let's leave it to Miklos to decide.
He could have a different idea altogether for fixing this bug.
How about this (untested) patch?
It's a cleanup as well as a fix, but maybe we should separate the cleanup from the fix...
If you are going to post this, feel free to add
Tested-by: Icenowy Zheng icenowy@aosc.io
Okay, thanks.
(And if you remove the IS_ERR(realfile) part, the tested-by tag still applies.)
Dropping the IS_ERR(realfile) here would mean having to add the same check before relevant fput() calls, which would make it more complex not less.
Or did you mean something else?
Thanks, Miklos
在 2021-01-21星期四的 09:07 +0100，Miklos Szeredi写道：
On Thu, Jan 21, 2021 at 4:43 AM Icenowy Zheng icenowy@aosc.io wrote:
在 2021-01-20星期三的 11:20 +0100，Miklos Szeredi写道：
On Tue, Jan 05, 2021 at 08:47:41AM +0200, Amir Goldstein wrote:
On Tue, Jan 5, 2021 at 2:36 AM Icenowy Zheng icenowy@aosc.io wrote:
The function ovl_dir_real_file() currently uses the semaphore of the inode to synchronize write to the upperfile cache field.
Although the inode lock is a rw_sem it is referred to as the "inode lock" and you also left semaphore in the commit subject. No need to re-post. This can be fixed on commit.
However, this function will get called by ovl_ioctl_set_flags(), which utilizes the inode semaphore too. In this case ovl_dir_real_file() will try to claim a lock that is owned by a function in its call stack, which won't get released before ovl_dir_real_file() returns.
Define a dedicated semaphore for the upperfile cache, so that the deadlock won't happen.
Fixes: 61536bed2149 ("ovl: support [S|G]ETFLAGS and FS[S|G]ETXATTR ioctls for directories") Cc: stable@vger.kernel.org # v5.10 Signed-off-by: Icenowy Zheng icenowy@aosc.io
Changes in v2:
- Fixed missing replacement in error handling path.
Changes in v3:
- Use mutex instead of semaphore.
fs/overlayfs/readdir.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/fs/overlayfs/readdir.c b/fs/overlayfs/readdir.c index 01620ebae1bd..3980f9982f34 100644 --- a/fs/overlayfs/readdir.c +++ b/fs/overlayfs/readdir.c @@ -56,6 +56,7 @@ struct ovl_dir_file { struct list_head *cursor; struct file *realfile; struct file *upperfile; + struct mutex upperfile_mutex;
That's a very specific name. This mutex protects members of struct ovl_dir_file, which could evolve into struct ovl_file one day (because no reason to cache only dir upper file), so I would go with a more generic name, but let's leave it to Miklos to decide.
He could have a different idea altogether for fixing this bug.
How about this (untested) patch?
It's a cleanup as well as a fix, but maybe we should separate the cleanup from the fix...
If you are going to post this, feel free to add
Tested-by: Icenowy Zheng icenowy@aosc.io
Okay, thanks.
(And if you remove the IS_ERR(realfile) part, the tested-by tag still applies.)
Dropping the IS_ERR(realfile) here would mean having to add the same check before relevant fput() calls, which would make it more complex not less.
Or did you mean something else?
I mean "seperate the cleanup from the fix".
This is only for when you do the seperation.
Thanks, Miklos
linux-stable-mirror@lists.linaro.org
-
Amir Goldstein -
Icenowy Zheng -
Miklos Szeredi