Linaro-mm-sig June 2026

linaro-mm-sig@lists.linaro.org

135 participants
482 discussions

[PATCH v3 00/10] Add dmabuf read/write via io_uring

by Pavel Begunkov

The patch set allows to register a dmabuf to an io_uring instance for a specified file and use it with io_uring read / write requests. The infrastructure is not tied to io_uring and there could be more users in the future. A similar idea was attempted some years ago by Keith [1], from where I borrowed a good number of changes, and later was brough up by Tushar and Vishal from Intel. It's an opt-in feature for files, and they need to implement a new file operation to use it. Only NVMe block devices are supported in this series. The user API is built on top of io_uring's "registered buffers", where a dmabuf is registered in a special way, but after it can be used as any other "registered buffer" with IORING_OP_{READ,WRITE}_FIXED requests. It's created via a new file operation and the resulted map is then passed through the I/O stack in a new iterator type. There is some additional infrastructure to bind it all, which also counts requests using a dmabuf map and managing lifetimes, which is used to implement map invalidation. It was tested for GPU <-> NVMe transfers. Also, as it maintains a long-term dma mapping, it helps with the IOMMU cost. The numbers below are for udmabuf reads previously run by Anuj for different IOMMU modes: - STRICT: before = 570 KIOPS, after = 5.01 MIOPS - LAZY: before = 1.93 MIOPS, after = 5.01 MIOPS - PASSTHROUGH: before = 5.01 MIOPS, after = 5.01 MIOPS There are some liburing tests that can serve as an example: git: https://github.com/isilence/liburing.git rw-dmabuf-tests-v3 url: https://github.com/isilence/liburing/tree/rw-dmabuf-tests-v3 [1] https://lore.kernel.org/io-uring/20220805162444.3985535-1-kbusch@fb.com/ v3: - Rework io_uring registration - Move token/map infrastructure code out of blk-mq - Simplify callbacks: remove a separate blk-mq table, which was mostly just forwarding calls (to nvme). - Don't skip dma sync depending on request direction - Fix a couple of hangs - Rename s/dma/dmabuf/ - Other small changes v2: - Don't pass raw dma addresses, wrap it into a driver specific object - Split into two objects: token and map - Implement move_notify Pavel Begunkov (10): file: add callback for creating long-term dmabuf maps iov_iter: add iterator type for dmabuf maps block: move bvec init into __bio_clone block: introduce dma map backed bio type lib: add dmabuf token infrastructure block: forward create_dmabuf_token to drivers nvme-pci: implement dma_token backed requests io_uring/rsrc: introduce buf registration structure io_uring/rsrc: extend buffer update io_uring/rsrc: add dmabuf backed registered buffers block/bio.c | 28 +++- block/blk-merge.c | 14 ++ block/blk.h | 3 +- block/fops.c | 16 ++ drivers/nvme/host/pci.c | 282 ++++++++++++++++++++++++++++++++ include/linux/bio.h | 19 ++- include/linux/blk-mq.h | 9 + include/linux/blk_types.h | 8 +- include/linux/fs.h | 2 + include/linux/io_dmabuf_token.h | 92 +++++++++++ include/linux/io_uring_types.h | 5 + include/linux/uio.h | 11 ++ include/uapi/linux/io_uring.h | 31 +++- io_uring/io_uring.c | 3 +- io_uring/rsrc.c | 266 +++++++++++++++++++++++++----- io_uring/rsrc.h | 30 +++- io_uring/rw.c | 4 +- lib/Kconfig | 4 + lib/Makefile | 2 + lib/io_dmabuf_token.c | 272 ++++++++++++++++++++++++++++++ lib/iov_iter.c | 29 +++- 21 files changed, 1071 insertions(+), 59 deletions(-) create mode 100644 include/linux/io_dmabuf_token.h create mode 100644 lib/io_dmabuf_token.c -- 2.53.0

5 days, 15 hours

Independence for dma_fences! v6

by Christian König

Hi everyone, dma_fences have ever lived under the tyranny dictated by the module lifetime of their issuer, leading to crashes should anybody still holding a reference to a dma_fence when the module of the issuer was unloaded. The basic problem is that when buffer are shared between drivers dma_fence objects can leak into external drivers and stay there even after they are signaled. The dma_resv object for example only lazy releases dma_fences. So what happens is that when the module who originally created the dma_fence unloads the dma_fence_ops function table becomes unavailable as well and so any attempt to release the fence crashes the system. Previously various approaches have been discussed, including changing the locking semantics of the dma_fence callbacks (by me) as well as using the drm scheduler as intermediate layer (by Sima) to disconnect dma_fences from their actual users, but none of them are actually solving all problems. Tvrtko did some really nice prerequisite work by protecting the returned strings of the dma_fence_ops by RCU. This way dma_fence creators where able to just wait for an RCU grace period after fence signaling before they could be save to free those data structures. Now this patch set here goes a step further and protects the whole dma_fence_ops structure by RCU, so that after the fence signals the pointer to the dma_fence_ops is set to NULL when there is no wait nor release callback given. All functionality which use the dma_fence_ops reference are put inside an RCU critical section, except for the deprecated issuer specific wait and of course the optional release callback. Additional to the RCU changes the lock protecting the dma_fence state previously had to be allocated external. This set here now changes the functionality to make that external lock optional and allows dma_fences to use an inline lock and be self contained. v4: Rebases the whole set on upstream changes, especially the cleanup from Philip in patch "drm/amdgpu: independence for the amdkfd_fence!". Adding two patches which brings the DMA-fence self tests up to date. The first selftest changes removes the mock_wait and so actually starts testing the default behavior instead of some hacky implementation in the test. This one got upstreamed independent of this set. The second drops the mock_fence as well and tests the new RCU and inline spinlock functionality. v5: Rebase on top of drm-misc-next instead of drm-tip, leave out all driver changes for now since those should go through the driver specific paths anyway. Address a few more review comments, especially some rebase mess and typos. And finally fix one more bug found by AMDs CI system. v6: Minor style changes, re-ordered patch #1, dropped the scheduler fence change for now Please review and comment, Christian.

5 days, 16 hours

Re: [RFC PATCH 10/30] vfio/pci: Export vfio dma-buf specific info for importers

by Jason Gunthorpe

On Thu, May 29, 2025 at 01:34:53PM +0800, Xu Yilun wrote: > Export vfio dma-buf specific info by attaching vfio_dma_buf_data in > struct dma_buf::priv. Provide a helper vfio_dma_buf_get_data() for > importers to fetch these data. Exporters identify VFIO dma-buf by > successfully getting these data. > > VFIO dma-buf supports disabling host access to these exported MMIO > regions when the device is converted to private. Exporters like KVM > need to identify this type of dma-buf to decide if it is good to use. > KVM only allows host unaccessible MMIO regions been mapped in private > roots. > > Export struct kvm * handler attached to the vfio device. This > allows KVM to do another sanity check. MMIO should only be assigned to > a CoCo VM if its owner device is already assigned to the same VM. This doesn't seem right, it should be encapsulated into the standard DMABUF API in some way. Jason

6 days, 1 hour

Why Saving Your Game Can Be One of the Most Stressful Moments in Horror

by explicit.gerbil.htxg＠hidingmail.net

Saving progress should feel comforting. You've survived a difficult section. You've made it through another encounter. Your progress is secure. In theory, a save point represents safety. Yet some of the most memorable horror games somehow make saving feel tense. I've always found that fascinating. In most genres, saving is a completely mechanical action. You barely think about it. Open a menu, press a button, move on. In horror games, however, save systems often become part of the experience itself. Sometimes they even become part of the fear. Safety Feels Temporary One thing horror games understand exceptionally well is that safety and danger become more meaningful when they exist side by side. A safe room feels comforting because the rest of the world doesn't. A save point feels valuable because progress can be lost. That contrast creates emotional weight. I still remember older survival horror games where reaching a save location felt like completing a journey. Not because the act of saving was exciting. Because getting there was. Every hallway between me and that room carried risk. Every enemy encounter threatened valuable resources. Every mistake felt expensive. The save point became more than a feature. It became a destination. Relief Can Be an Emotion Too People often discuss fear when talking about horror games. Relief deserves more attention. In many ways, relief is one of the most important emotions the genre creates. Without moments of relief, tension eventually becomes exhausting. Players need opportunities to recover. Save rooms often provide exactly that. The music changes. The atmosphere softens. The pressure decreases. For a few moments, players can breathe. Those quiet pauses create an interesting effect. Instead of weakening the horror, they strengthen it. The next dangerous section feels more intense because you've experienced a brief sense of comfort. The contrast matters. Fear isn't effective when it never changes. Limited Saving Changes Behavior Some horror games allow unlimited saves. Others place restrictions on them. Whether through limited resources, specific locations, or special conditions, these systems dramatically influence how players think. I've noticed that limited saving changes decision-making almost immediately. Players become cautious. Resources feel more valuable. Exploration becomes riskier. Every choice carries additional weight because mistakes have consequences. Suddenly, saving isn't just a technical feature. It's a strategic decision. Should I save now? Should I keep going? Can I survive the next section without doing it? These questions create tension even when no immediate threat exists. That's impressive design. The game generates anxiety through decision-making rather than direct danger. The Walk Back Is Sometimes Scarier One of my favorite horror gaming experiences involves leaving a save room. Entering feels great. Leaving feels terrible. The moment you step back into the unknown, the comfort disappears. The music fades. The safety vanishes. The uncertainty returns. What's interesting is that nothing may have changed. The environment is the same. The enemies are the same. The layout remains familiar. Yet emotionally, everything feels different. You know you're moving away from security. That awareness alone increases tension. I've experienced situations where opening the door to leave a save room felt more intimidating than entering a boss area. Not because I expected a specific threat. Because I was abandoning certainty. Modern Horror Uses Saving Differently Contemporary horror games often approach saving in a less restrictive way. Autosaves are common. Checkpoints are frequent. Players rarely lose significant amounts of progress. That shift has obvious advantages. It reduces frustration and makes games more accessible. At the same time, it changes the emotional role of saving. Older horror games often transformed save systems into part of the atmosphere. Modern games typically treat them as invisible support systems. Neither approach is inherently better. They're simply pursuing different goals. One emphasizes vulnerability. The other prioritizes convenience. Interestingly, both can still create effective horror when used thoughtfully. For another perspective on player vulnerability, see our [discussion about why limited resources increase tension]. Safe Rooms Become Emotional Anchors Some locations in horror games remain memorable long after the details of the story fade. Safe rooms are often among them. Players remember how those spaces felt. The familiar music. The calm atmosphere. The temporary sense of control. I've finished games years ago and forgotten specific enemy encounters, yet I can still picture certain save rooms clearly. That's remarkable when you think about it. These locations rarely contain action. Nothing dramatic happens there. Their importance comes entirely from emotion. They provide stability within unstable worlds. The player develops a relationship with them. Returning feels reassuring. Leaving feels uncomfortable. Few game mechanics achieve that kind of emotional significance. Fear Feels Bigger When Progress Matters A major reason save systems affect horror so strongly is simple. Consequences matter. If players feel that failure costs nothing, tension often decreases. When progress becomes valuable, fear gains additional weight. Suddenly every encounter feels important. Every decision matters. Every mistake carries consequences beyond the immediate moment. This doesn't mean horror games need harsh punishment systems. Excessive penalties can easily become frustrating. But a small amount of risk often makes emotional investment stronger. Players care more because they have something to lose. And caring is essential for fear. Without investment, horror struggles to have lasting impact. The Psychology of "Just One More Room" I've fallen into this trap countless times. You're standing near a save point. Logic says you should save and stop playing. Instead, you think: "I'll check one more room." Then another room. Then another hallway. Then another objective. Before long, you've wandered into a situation far more dangerous than expected. Horror games thrive on this kind of curiosity. Players constantly balance caution against exploration. The save point represents security. The unexplored area represents possibility. Most of the time, curiosity wins. That's why horror games remain so engaging despite the fear they create. Players aren't simply avoiding danger. They're actively seeking answers. Why Save Points Remain Memorable Not every horror game uses traditional save rooms anymore. Not every game limits progress in meaningful ways. Yet the underlying idea remains powerful. Players need moments of safety. Moments of relief. Moments where tension briefly relaxes before building again. Save points became iconic because they delivered those emotions consistently. They represented hope within hostile environments. A reminder that survival was possible. A chance to regroup before facing whatever came next. And perhaps that's why so many horror fans remember them so fondly. They weren't frightening on their own. They mattered because of everything waiting outside. After all, what makes a safe place feel truly safe if there was never any danger to escape from in the first place? https://horrorgamesfree.com

1 week

A Guide to Mastering Slope

by sarahcolter＠wshu.net

Have you ever felt the exhilarating rush of speed, the focus demanded by precision, and the utter satisfaction of overcoming a seemingly impossible challenge? If so, then you might just be ready to tackle Slope. This deceptively simple online game has captivated players with its addictive gameplay and endless potential for improvement. It's a perfect little escape for a quick break or a dedicated practice session. Let's dive into how to play and, more importantly, how to experience the thrill of Slope. Introduction: What is Slope? At its core, Slope is a rolling ball game. You control a ball that’s constantly moving downhill, navigating a procedurally generated landscape of interconnected platforms. Sounds straightforward, right? Don’t be fooled! The platforms narrow, the speed increases relentlessly, and the gaps between platforms widen as you progress. One wrong move and you’re plummeting into the abyss. https://slopegame-online.com The brilliance of Slope lies in its accessibility and its learning curve. It's incredibly easy to pick up and play, yet fiendishly difficult to master. You don't need complicated controllers or extensive tutorials. Just a keyboard, a steady hand, and a determined spirit. It's a perfect example of a simple concept executed brilliantly. Gameplay: Navigating the Perilous Path The controls are as minimal as the aesthetic: the left and right arrow keys are all you need to guide your ball. Use the left arrow key to steer the ball left, and the right arrow key to steer it right. That's it! But mastering those two keys is the key to survival. Your objective is to stay on the platforms for as long as possible. As you roll further down the slope, your speed increases, and the environment becomes increasingly challenging. Red blocks appear, acting as obstacles that instantly end your run if touched. The platforms themselves will start to shift and move, adding another layer of complexity to the game. The game ends when your ball falls off the platform. Your score is determined by the distance you travelled before your demise. The higher the score, the better! The goal is to constantly improve your score, pushing yourself to see just how far you can go. Tips and Tricks for Conquering the Slope While the controls are simple, mastering Slope requires practice and a good understanding of the game's mechanics. Here are a few tips to help you improve your game: Smooth Movements Are Key: Avoid jerky, panicked movements. Small, controlled adjustments are far more effective than large, sweeping ones. Think less about reacting and more about anticipating. Look Ahead: Try to focus a little further down the slope. Anticipating the next turn or gap will give you more time to react and adjust your trajectory. This is easier said than done when the speed is high, but it's a crucial skill to develop. Embrace the Momentum: Understand how your ball's momentum affects its movement. Learn how to use the slight curve of the platforms to your advantage, especially when approaching tight turns. Don't Fear the Edges: Sometimes, the safest route is to ride along the edge of the platform. This allows you to make small corrections and avoid potential obstacles in the center. Practice, Practice, Practice: There's no substitute for practice. The more you play, the better you'll become at anticipating the challenges and reacting accordingly. Don't get discouraged by early failures. Everyone starts somewhere. You can visit Slope to begin practicing! Learn from Your Mistakes: Analyze your failures. What caused you to fall off the platform? Were you too slow to react? Did you misjudge the turn? Understanding your mistakes will help you avoid them in the future. Conclusion: The Enduring Appeal of an Endless Game Slope isn't about complicated storylines, stunning graphics, or intricate controls. It's about the pure, unadulterated challenge of pushing your skills to the limit. It's about the satisfaction of mastering a seemingly impossible task. It's about the addictive loop of playing, failing, learning, and improving. And its accessible nature means anyone can jump in and experience that thrill. So, go ahead, give it a try. You might just find yourself hooked on the endless decline. The simple controls and challenging gameplay make it a strangely addictive experience. It's a testament to the idea that sometimes, the simplest games are the most engaging.

1 week, 3 days

[PATCH] dma-buf: fix stale @lock references in struct dma_buf documentation

by Xiang Gao

From: Xiang Gao <gaoxiang17(a)xiaomi.com> The kernel-doc comments for vmapping_counter and vmap_ptr in struct dma_buf reference "@lock" as the protecting lock, but struct dma_buf no longer has a "lock" member. The mutex was removed in favor of using the dma_resv lock exclusively. The implementation correctly uses dma_resv_assert_held(dmabuf->resv) in dma_buf_vmap() and dma_buf_vunmap(), so update the documentation to reference @resv instead. Signed-off-by: gaoxiang17 <gaoxiang17(a)xiaomi.com> --- include/linux/dma-buf.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index 133b9e637b55..ef6d93fd7a2c 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -322,13 +322,13 @@ struct dma_buf { * @vmapping_counter: * * Used internally to refcnt the vmaps returned by dma_buf_vmap(). - * Protected by @lock. + * Protected by @resv. */ unsigned vmapping_counter; /** * @vmap_ptr: - * The current vmap ptr if @vmapping_counter > 0. Protected by @lock. + * The current vmap ptr if @vmapping_counter > 0. Protected by @resv. */ struct iosys_map vmap_ptr; -- 2.34.1

1 week, 4 days

[PATCH] dma-buf: explicitely forbid exposing pages from get_user_pages()

by Christian König

We recently had another incident where two drivers put pages they got from get_user_pages() into a DMA-buf and cause quite a number of problems. Explicitely document that this is not something exporters can do. Signed-off-by: Christian König <christian.koenig(a)amd.com> --- drivers/dma-buf/dma-buf.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index 71f37544a5c6..aa5af4f439c2 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -685,6 +685,14 @@ static struct file *dma_buf_getfile(size_t size, int flags) * * For the detailed semantics exporters are expected to implement see * &dma_buf_ops. + * + * It is explicitely forbidden for exporters to expose buffers they don't "own" + * as DMA-buf. This includes pages acquired by get_user_pages() or other import + * mechanism. Not following this rule can create numerous security problems. + * + * It is also strongly discouraged to expose the same backing store through + * multiple DMA-bufs at the same time. This eventually creates aliasing and + * cache coherency problems which are extremely hard to debug and fix. */ /** -- 2.43.0

1 week, 5 days

Wordle Unlimited Guide for Endless Word Puzzle Fun

by disastrous.heron.yxfs＠hidingmail.net

Introduction to Wordle Unlimited Experience Wordle Unlimited is a modern word puzzle game designed for players who enjoy continuous vocabulary challenges without daily limits. Unlike traditional word guessing games that restrict attempts to once per day, this version allows unlimited gameplay sessions. This creates an environment where learning, practice, and entertainment can happen at any time. https://wordleunlimitedgame.org/ The game has become popular among language learners, puzzle fans, and casual gamers because it combines simple rules with strategic thinking. Players must guess a hidden word within a set number of attempts, using feedback from each guess to improve accuracy. The unlimited format removes waiting time and allows repeated practice, making it suitable for skill development and relaxation. What Wordle Unlimited Means in Modern Gaming Wordle Unlimited refers to an expanded version of the original word guessing concept where users can play repeatedly without restriction. Each round generates a new hidden word, giving continuous opportunities to test vocabulary knowledge. This format is especially useful for users who want to improve language skills. Instead of waiting for a daily puzzle, players can engage in multiple rounds in one sitting. The structure remains consistent, which helps build familiarity and confidence over time. The game typically uses a grid system where each attempt reveals color coded feedback. Correct letters in correct positions, correct letters in wrong positions, and incorrect letters are all highlighted differently to guide future guesses. How to Play Wordle Unlimited Step by Step Playing Wordle Unlimited is straightforward and accessible for all skill levels. The objective is to identify the hidden word within a limited number of guesses. First, the player enters a starting word. This word is usually chosen to include common vowels and consonants to maximize information. After submission, the game provides feedback on each letter. Next, the player analyzes the feedback and adjusts the next guess accordingly. Letters that are correct and correctly placed should be kept in the same position. Letters that are correct but misplaced should be repositioned in the next attempt. Letters that are not part of the word should be avoided. This process continues until the correct word is discovered or attempts are exhausted. After each round, a new word is available immediately, allowing continuous play. Key Features That Define Wordle Unlimited Wordle Unlimited includes several features that make it appealing to a wide audience. One major feature is unlimited gameplay, which removes daily restrictions and supports continuous learning. Another important feature is instant feedback after each guess. This feedback system helps players improve logic and vocabulary skills quickly. It also encourages strategic thinking rather than random guessing. The game also maintains a simple interface, making it easy to understand for beginners. There are no complex controls or instructions, which allows users to focus entirely on word solving. Additionally, the game supports replayability. Each new round offers a different word challenge, which keeps the experience fresh and engaging over long periods. Effective Strategies for Better Results Success in Wordle Unlimited often depends on strategy rather than luck. One effective approach is starting with words that include multiple vowels and common consonants. This helps identify useful letters early in the game. Another strategy is pattern recognition. After receiving feedback, players should focus on common English word structures. Recognizing prefixes, suffixes, and common letter combinations can significantly improve guessing accuracy. It is also helpful to avoid repeating incorrect letters. Keeping track of eliminated letters reduces unnecessary guesses and increases efficiency. Advanced players often use logical elimination methods. By narrowing down possibilities step by step, they can solve puzzles in fewer attempts and improve overall performance. Benefits of Playing Wordle Unlimited Regularly Wordle Unlimited offers several cognitive and educational benefits. One major benefit is vocabulary improvement. Regular exposure to new words helps expand language knowledge over time. The game also enhances problem solving skills. Each round requires analysis, deduction, and logical thinking. These mental exercises contribute to sharper cognitive abilities. Another benefit is stress relief. The simple and repetitive structure of the game provides a relaxing experience that can help reduce mental fatigue. In addition, unlimited access allows flexible practice schedules. Players can engage for a few minutes or extended sessions depending on personal preference, making it suitable for different lifestyles. Why Wordle Unlimited Has Become So Popular The popularity of Wordle Unlimited comes from its balance of simplicity and challenge. It is easy to learn but difficult to master, which keeps users engaged. Social sharing also plays a role in its growth. Many players enjoy comparing results and discussing strategies with others, creating a sense of community around the game. The unlimited format is another key factor. Unlike limited daily puzzles, this version allows continuous engagement, which appeals to users who enjoy long gaming sessions. Its accessibility across devices also contributes to popularity. Players can enjoy the game on computers, tablets, or mobile devices without complicated setup processes. Conclusion on Wordle Unlimited Value Wordle Unlimited stands out as an engaging and educational word puzzle experience that combines entertainment with cognitive development. Its unlimited format allows continuous learning and practice, making it suitable for both beginners and advanced players. With simple rules, strategic depth, and instant feedback, the game provides a balanced experience that supports vocabulary growth and logical thinking. Whether used for relaxation or skill improvement, Wordle Unlimited remains a strong choice for anyone interested in word based challenges.

1 week, 5 days

What's Wave Rider game?

by Leo Enda

Wave Rider surprised me more than I expected. The game looks simple at first, but once the speed increases and the waves become unstable, every run starts feeling intense. Unlike many endless runner games that rely only on fast movement, Wave Rider creates pressure through unpredictable water physics and constantly changing obstacle patterns. Play Wave Rider online at https://wave-rider.io The gameplay is built around survival. Your board moves forward automatically while you focus on steering through dangerous sections of water. Floating barriers, underwater mines, sharp turns, and narrow lanes appear quickly, forcing players to react within seconds. What I liked most is how the game balances smooth surfing movement with sudden moments of chaos. Sometimes the water feels calm, then everything changes when multiple obstacles appear together. Wave Rider also rewards risky decisions. Pearls are often hidden near dangerous routes, making players choose between safe movement and higher scores. I found myself taking bigger risks after every run just to see how far I could push the distance counter. The durability system adds even more tension because repeated small crashes slowly reduce your chance of surviving longer sessions. Game Controls Left Arrow / A: Move left Right Arrow / D: Move right Spacebar or Up Arrow: Jump or glide over obstacles The controls are easy to learn, but mastering the timing takes practice. Quick reactions help, but smooth movement and patience usually lead to better runs. Players who panic and change direction too aggressively often crash faster than expected. Overall, Wave Rider feels like a solid arcade experience for anyone who enjoys reaction-based gameplay. The combination of flowing water, rising speed, and nonstop danger makes every session feel fresh and competitive.

1 week, 6 days

[PATCH v18 00/42] DEPT(DEPendency Tracker)

by Byungchul Park

I'm happy to see that DEPT reported real problems in practice: https://lore.kernel.org/lkml/6383cde5-cf4b-facf-6e07-1378a485657d@I-love.SA… https://lore.kernel.org/lkml/1674268856-31807-1-git-send-email-byungchul.pa… https://lore.kernel.org/all/b6e00e77-4a8c-4e05-ab79-266bf05fcc2d@igalia.com/ I’ve added documentation describing DEPT — this should help you understand what DEPT is and how it works. You can use DEPT simply by enabling CONFIG_DEPT and checking dmesg at runtime. --- Hi Linus and folks, I’ve been developing a tool to detect deadlock possibilities by tracking waits/events — rather than lock acquisition order — to cover all the synchronization mechanisms. To summarize the design rationale, starting from the problem statement, through analysis, to the solution: CURRENT STATUS -------------- Lockdep tracks lock acquisition order to identify deadlock conditions. Additionally, it tracks IRQ state changes — via {en,dis}able — to detect cases where locks are acquired unintentionally during interrupt handling. PROBLEM ------- Waits and their associated events that are never reachable can eventually lead to deadlocks. However, since Lockdep focuses solely on lock acquisition order, it has inherent limitations when handling waits and events. Moreover, by tracking only lock acquisition order, Lockdep cannot properly handle read locks or cross-event scenarios — such as wait_for_completion() and complete() — making it increasingly inadequate as a general-purpose deadlock detection tool. SOLUTION -------- Once again, waits and their associated events that are never reachable can eventually lead to deadlocks. The new solution, DEPT, focuses directly on waits and events. DEPT monitors waits and events, and reports them when any become unreachable. DEPT provides: * Correct handling of read locks. * Support for general waits and events. * Continuous operation, even after multiple reports. * Simple, intuitive annotation APIs. There are still false positives, and some are already being worked on for suppression. Especially splitting the folio class into several appropriate classes e.g. block device mapping class and regular file mapping class, is currently under active development by me and Yeoreum Yun. Anyway, these efforts will need to continue for a while, as we’ve seen with lockdep over two decades. DEPT is tagged as EXPERIMENTAL in Kconfig — meaning it’s not yet suitable for use as an automation tool. However, for those who are interested in using DEPT to analyze complex synchronization patterns and extract dependency insights, DEPT would be a great tool for the purpose. Thanks for your support and contributions to: Harry Yoo <harry.yoo(a)oracle.com> Gwan-gyeong Mun <gwan-gyeong.mun(a)intel.com> Yunseong Kim <ysk(a)kzalloc.com> Yeoreum Yun <yeoreum.yun(a)arm.com> FAQ --- Q. Is this the first attempt to solve this problem? A. No. The cross-release feature (commit b09be676e0ff2) attempted to address it — as a Lockdep extension. It was merged, but quickly reverted, because: While it uncovered valuable hidden issues, it also introduced false positives. Since these false positives mask further real problems with Lockdep — and developers strongly dislike them — the feature was rolled back. Q. Why wasn’t DEPT built as a Lockdep extension? A. Lockdep is the result of years of work by kernel developers — and is now very stable. But I chose to build DEPT separately, because: While reusing BFS(Breadth First Search) and Lockdep’s hashing is beneficial, the rest of the system must be rebuilt from scratch to align with DEPT’s wait-event model — since Lockdep was originally designed for tracking lock acquisition orders, not wait-event dependencies. Q. Do you plan to replace Lockdep entirely? A. Not at all — Lockdep still plays a vital role in validating correct lock usage. While its dependency-checking logic should eventually be superseded by DEPT, the rest of its functionality should stay. Q. Should we replace the dependency check immediately? A. Absolutely not. Lockdep’s stability is the result of years of hard work by kernel developers. Lockdep and DEPT should run side by side until DEPT matures. Q. Stronger detection often leads to more false positives — which was a major pain point when cross-release was added. Is DEPT designed to handle this? A. Yes. DEPT’s simple, generalized design enables flexible reporting — so while false positives still need fixing, they’re far less disruptive than they were under the Lockdep extension, cross-release. Q. Why not fix all false positives out-of-tree before merging? A. Since the affected subsystems span the entire kernel, like Lockdep, which has relied on annotations to avoid false positives over the last two decades, DEPT too will require the annotation efforts. Performing annotation work within the mainline will help us add annotations more appropriately and will also make DEPT a useful tool for a wider range of users more quickly. CONFIG_DEPT is marked EXPERIMENTAL, so it’s opt-in. Some users are already interested in using DEPT to analyze complex synchronization patterns and extract dependency insights. Byungchul --- Changes from v17: 1. Rebase on the mainline as of 2025 Dec 5. 2. Convert the documents' format from txt to rst. (feedbacked by Jonathan Corbet and Bagas Sanjaya) 3. Move the documents from 'Documentation/dependency' to 'Documentation/dev-tools'. (feedbakced by Jonathan Corbet) 4. Improve the documentation. (feedbacked by NeilBrown) 5. Use a common function, enter_from_user_mode(), instead of arch specific code, to notice context switch from user mode. (feedbacked by Dave Hansen, Mark Rutland, and Mark Brown) 6. Resolve the header dependency issue by using dept's internal header, instead of relocating 'struct llist_{head,node}' to another header. (feedbacked by Greg KH) 7. Improve page(or folio) usage type APIs. 8. Add rust helper for wait_for_completion(). (feedbacked by Guangbo Cui, Boqun Feng, and Danilo Krummrich) 9. Refine some commit messages. Changes from v16: 1. Rebase on v6.17. 2. Fix a false positive from rcu (by Yunseong Kim) 3. Introduce APIs to set page's usage, dept_set_page_usage() and dept_reset_page_usage() to avoid false positives. 4. Consider lock_page() as a potential wait unconditionally. 5. Consider folio_lock_killable() as a potential wait unconditionally. 6. Add support for tracking PG_writeback waits and events. 7. Fix two build errors due to the additional debug information added by dept. (by Yunseong Kim) Changes from v15: 1. Fix typo and improve comments and commit messages (feedbacked by ALOK TIWARI, Waiman Long, and kernel test robot). 2. Do not stop dept on detection of cicular dependency of recover event, allowing to keep reporting. 3. Add SK hynix to copyright. 4. Consider folio_lock() as a potential wait unconditionally. 5. Fix Kconfig dependency bug (feedbacked by kernel test rebot). 6. Do not suppress reports that involve classes even that have already involved in other reports, allowing to keep reporting. Changes from v14: 1. Rebase on the current latest, v6.15-rc6. 2. Refactor dept code. 3. With multi event sites for a single wait, even if an event forms a circular dependency, the event can be recovered by other event(or wake up) paths. Even though informing the circular dependency is worthy but it should be suppressed once informing it, if it doesn't lead an actual deadlock. So introduce APIs to annotate the relationship between event site and recover site, that are, event_site() and dept_recover_event(). 4. wait_for_completion() worked with dept map embedded in struct completion. However, it generates a few false positves since all the waits using the instance of struct completion, share the map and key. To avoid the false positves, make it not to share the map and key but each wait_for_completion() caller have its own key by default. Of course, external maps also can be used if needed. 5. Fix a bug about hardirq on/off tracing. 6. Implement basic unit test for dept. 7. Add more supports for dma fence synchronization. 8. Add emergency stop of dept e.g. on panic(). 9. Fix false positives by mmu_notifier_invalidate_*(). 10. Fix recursive call bug by DEPT_WARN_*() and DEPT_STOP(). 11. Fix trivial bugs in DEPT_WARN_*() and DEPT_STOP(). 12. Fix a bug that a spin lock, dept_pool_spin, is used in both contexts of irq disabled and enabled without irq disabled. 13. Suppress reports with classes, any of that already have been reported, even though they have different chains but being barely meaningful. 14. Print stacktrace of the wait that an event is now waking up, not only stacktrace of the event. 15. Make dept aware of lockdep_cmp_fn() that is used to avoid false positives in lockdep so that dept can also avoid them. 16. Do do_event() only if there are no ecxts have been delimited. 17. Fix a bug that was not synchronized for stage_m in struct dept_task, using a spin lock, dept_task()->stage_lock. 18. Fix a bug that dept didn't handle the case that multiple ttwus for a single waiter can be called at the same time e.i. a race issue. 19. Distinguish each kernel context from others, not only by system call but also by user oriented fault so that dept can work with more accuracy information about kernel context. That helps to avoid a few false positives. 20. Limit dept's working to x86_64 and arm64. Changes from v13: 1. Rebase on the current latest version, v6.9-rc7. 2. Add 'dept' documentation describing dept APIs. Changes from v12: 1. Refine the whole document for dept. 2. Add 'Interpret dept report' section in the document, using a deadlock report obtained in practice. Hope this version of document helps guys understand dept better. https://lore.kernel.org/lkml/6383cde5-cf4b-facf-6e07-1378a485657d@I-love.SA… https://lore.kernel.org/lkml/1674268856-31807-1-git-send-email-byungchul.pa… Changes from v11: 1. Add 'dept' documentation describing the concept of dept. 2. Rewrite the commit messages of the following commits for using weaker lockdep annotation, for better description. fs/jbd2: Use a weaker annotation in journal handling cpu/hotplug: Use a weaker annotation in AP thread (feedbacked by Thomas Gleixner) Changes from v10: 1. Fix noinstr warning when building kernel source. 2. dept has been reporting some false positives due to the folio lock's unfairness. Reflect it and make dept work based on dept annotaions instead of just wait and wake up primitives. 3. Remove the support for PG_writeback while working on 2. I will add the support later if needed. 4. dept didn't print stacktrace for [S] if the participant of a deadlock is not lock mechanism but general wait and event. However, it made hard to interpret the report in that case. So add support to print stacktrace of the requestor who asked the event context to run - usually a waiter of the event does it just before going to wait state. 5. Give up tracking raw_local_irq_{disable,enable}() since it totally messed up dept's irq tracking. So make it work in the same way as lockdep does. I will consider it once any false positives by those are observed again. 6. Change the manual rwsem_acquire_read(->j_trans_commit_map) annotation in fs/jbd2/transaction.c to the try version so that it works as much as it exactly needs. 7. Remove unnecessary 'inline' keyword in dept.c and add '__maybe_unused' to a needed place. Changes from v9: 1. Fix a bug. SDT tracking didn't work well because of my big mistake that I should've used waiter's map to indentify its class but it had been working with waker's one. FYI, PG_locked and PG_writeback weren't affected. They still worked well. (reported by YoungJun) Changes from v8: 1. Fix build error by adding EXPORT_SYMBOL(PG_locked_map) and EXPORT_SYMBOL(PG_writeback_map) for kernel module build - appologize for that. (reported by kernel test robot) 2. Fix build error by removing header file's circular dependency that was caused by "atomic.h", "kernel.h" and "irqflags.h", which I introduced - appolgize for that. (reported by kernel test robot) Changes from v7: 1. Fix a bug that cannot track rwlock dependency properly, introduced in v7. (reported by Boqun and lockdep selftest) 2. Track wait/event of PG_{locked,writeback} more aggressively assuming that when a bit of PG_{locked,writeback} is cleared there might be waits on the bit. (reported by Linus, Hillf and syzbot) 3. Fix and clean bad style code e.i. unnecessarily introduced a randome pattern and so on. (pointed out by Linux) 4. Clean code for applying dept to wait_for_completion(). Changes from v6: 1. Tie to task scheduler code to track sleep and try_to_wake_up() assuming sleeps cause waits, try_to_wake_up()s would be the events that those are waiting for, of course with proper dept annotations, sdt_might_sleep_weak(), sdt_might_sleep_strong() and so on. For these cases, class is classified at sleep entrance rather than the synchronization initialization code. Which would extremely reduce false alarms. 2. Remove the dept associated instance in each page struct for tracking dependencies by PG_locked and PG_writeback thanks to the 1. work above. 3. Introduce CONFIG_dept_AGGRESIVE_TIMEOUT_WAIT to suppress reports that waits with timeout set are involved, for those who don't like verbose reporting. 4. Add a mechanism to refill the internal memory pools on running out so that dept could keep working as long as free memory is available in the system. 5. Re-enable tracking hashed-waitqueue wait. That's going to no longer generate false positives because class is classified at sleep entrance rather than the waitqueue initailization. 6. Refactor to make it easier to port onto each new version of the kernel. 7. Apply dept to dma fence. 8. Do trivial optimizaitions. Changes from v5: 1. Use just pr_warn_once() rather than WARN_ONCE() on the lack of internal resources because WARN_*() printing stacktrace is too much for informing the lack. (feedback from Ted, Hyeonggon) 2. Fix trivial bugs like missing initializing a struct before using it. 3. Assign a different class per task when handling onstack variables for waitqueue or the like. Which makes dept distinguish between onstack variables of different tasks so as to prevent false positives. (reported by Hyeonggon) 4. Make dept aware of even raw_local_irq_*() to prevent false positives. (reported by Hyeonggon) 5. Don't consider dependencies between the events that might be triggered within __schedule() and the waits that requires __schedule(), real ones. (reported by Hyeonggon) 6. Unstage the staged wait that has prepare_to_wait_event()'ed *and* yet to get to __schedule(), if we encounter __schedule() in-between for another sleep, which is possible if e.g. a mutex_lock() exists in 'condition' of ___wait_event(). 7. Turn on CONFIG_PROVE_LOCKING when CONFIG_DEPT is on, to rely on the hardirq and softirq entrance tracing to make dept more portable for now. Changes from v4: 1. Fix some bugs that produce false alarms. 2. Distinguish each syscall context from another *for arm64*. 3. Make it not warn it but just print it in case dept ring buffer gets exhausted. (feedback from Hyeonggon) 4. Explicitely describe "EXPERIMENTAL" and "dept might produce false positive reports" in Kconfig. (feedback from Ted) Changes from v3: 1. dept shouldn't create dependencies between different depths of a class that were indicated by *_lock_nested(). dept normally doesn't but it does once another lock class comes in. So fixed it. (feedback from Hyeonggon) 2. dept considered a wait as a real wait once getting to __schedule() even if it has been set to TASK_RUNNING by wake up sources in advance. Fixed it so that dept doesn't consider the case as a real wait. (feedback from Jan Kara) 3. Stop tracking dependencies with a map once the event associated with the map has been handled. dept will start to work with the map again, on the next sleep. Changes from v2: 1. Disable dept on bit_wait_table[] in sched/wait_bit.c reporting a lot of false positives, which is my fault. Wait/event for bit_wait_table[] should've been tagged in a higher layer for better work, which is a future work. (feedback from Jan Kara) 2. Disable dept on crypto_larval's completion to prevent a false positive. Changes from v1: 1. Fix coding style and typo. (feedback from Steven) 2. Distinguish each work context from another in workqueue. 3. Skip checking lock acquisition with nest_lock, which is about correct lock usage that should be checked by lockdep. Changes from RFC(v0): 1. Prevent adding a wait tag at prepare_to_wait() but __schedule(). (feedback from Linus and Matthew) 2. Use try version at lockdep_acquire_cpus_lock() annotation. 3. Distinguish each syscall context from another. Byungchul Park (41): dept: implement DEPT(DEPendency Tracker) dept: add single event dependency tracker APIs dept: add lock dependency tracker APIs dept: tie to lockdep and IRQ tracing dept: add proc knobs to show stats and dependency graph dept: distinguish each kernel context from another dept: distinguish each work from another dept: add a mechanism to refill the internal memory pools on running out dept: record the latest one out of consecutive waits of the same class dept: apply sdt_might_sleep_{start,end}() to wait_for_completion()/complete() dept: apply sdt_might_sleep_{start,end}() to swait dept: apply sdt_might_sleep_{start,end}() to waitqueue wait dept: apply sdt_might_sleep_{start,end}() to hashed-waitqueue wait dept: apply sdt_might_sleep_{start,end}() to dma fence dept: track timeout waits separately with a new Kconfig dept: apply timeout consideration to wait_for_completion()/complete() dept: apply timeout consideration to swait dept: apply timeout consideration to waitqueue wait dept: apply timeout consideration to hashed-waitqueue wait dept: apply timeout consideration to dma fence wait dept: make dept able to work with an external wgen dept: track PG_locked with dept dept: print staged wait's stacktrace on report locking/lockdep: prevent various lockdep assertions when lockdep_off()'ed dept: add documents for dept cpu/hotplug: use a weaker annotation in AP thread dept: assign dept map to mmu notifier invalidation synchronization dept: assign unique dept_key to each distinct dma fence caller dept: make dept aware of lockdep_set_lock_cmp_fn() annotation dept: make dept stop from working on debug_locks_off() dept: assign unique dept_key to each distinct wait_for_completion() caller completion, dept: introduce init_completion_dmap() API dept: introduce a new type of dependency tracking between multi event sites dept: add module support for struct dept_event_site and dept_event_site_dep dept: introduce event_site() to disable event tracking if it's recoverable dept: implement a basic unit test for dept dept: call dept_hardirqs_off() in local_irq_*() regardless of irq state dept: introduce APIs to set page usage and use subclasses_evt for the usage dept: track PG_writeback with dept SUNRPC: relocate struct rcu_head to the first field of struct rpc_xprt mm: percpu: increase PERCPU_DYNAMIC_SIZE_SHIFT on DEPT and large PAGE_SIZE Yunseong Kim (1): rcu/update: fix same dept key collision between various types of RCU Documentation/dev-tools/dept.rst | 778 ++++++ Documentation/dev-tools/dept_api.rst | 125 + drivers/dma-buf/dma-fence.c | 23 +- include/asm-generic/vmlinux.lds.h | 13 +- include/linux/completion.h | 124 +- include/linux/dept.h | 402 +++ include/linux/dept_ldt.h | 78 + include/linux/dept_sdt.h | 68 + include/linux/dept_unit_test.h | 67 + include/linux/dma-fence.h | 74 +- include/linux/hardirq.h | 3 + include/linux/irq-entry-common.h | 4 + include/linux/irqflags.h | 21 +- include/linux/local_lock_internal.h | 1 + include/linux/lockdep.h | 105 +- include/linux/lockdep_types.h | 3 + include/linux/mm_types.h | 4 + include/linux/mmu_notifier.h | 26 + include/linux/module.h | 5 + include/linux/mutex.h | 1 + include/linux/page-flags.h | 217 +- include/linux/pagemap.h | 37 +- include/linux/percpu-rwsem.h | 2 +- include/linux/percpu.h | 4 + include/linux/rcupdate_wait.h | 13 +- include/linux/rtmutex.h | 1 + include/linux/rwlock_types.h | 1 + include/linux/rwsem.h | 1 + include/linux/sched.h | 118 + include/linux/seqlock.h | 2 +- include/linux/spinlock_types_raw.h | 3 + include/linux/srcu.h | 2 +- include/linux/sunrpc/xprt.h | 9 +- include/linux/swait.h | 3 + include/linux/wait.h | 3 + include/linux/wait_bit.h | 3 + init/init_task.c | 2 + init/main.c | 2 + kernel/Makefile | 1 + kernel/cpu.c | 2 +- kernel/dependency/Makefile | 5 + kernel/dependency/dept.c | 3499 ++++++++++++++++++++++++++ kernel/dependency/dept_hash.h | 10 + kernel/dependency/dept_internal.h | 314 +++ kernel/dependency/dept_object.h | 13 + kernel/dependency/dept_proc.c | 94 + kernel/dependency/dept_unit_test.c | 173 ++ kernel/exit.c | 1 + kernel/fork.c | 2 + kernel/locking/lockdep.c | 33 + kernel/module/main.c | 19 + kernel/rcu/rcu.h | 1 + kernel/rcu/update.c | 5 +- kernel/sched/completion.c | 62 +- kernel/sched/core.c | 9 + kernel/workqueue.c | 3 + lib/Kconfig.debug | 48 + lib/debug_locks.c | 2 + lib/locking-selftest.c | 2 + mm/filemap.c | 38 + mm/mm_init.c | 3 + mm/mmu_notifier.c | 31 +- rust/helpers/completion.c | 5 + 63 files changed, 6602 insertions(+), 121 deletions(-) create mode 100644 Documentation/dev-tools/dept.rst create mode 100644 Documentation/dev-tools/dept_api.rst create mode 100644 include/linux/dept.h create mode 100644 include/linux/dept_ldt.h create mode 100644 include/linux/dept_sdt.h create mode 100644 include/linux/dept_unit_test.h create mode 100644 kernel/dependency/Makefile create mode 100644 kernel/dependency/dept.c create mode 100644 kernel/dependency/dept_hash.h create mode 100644 kernel/dependency/dept_internal.h create mode 100644 kernel/dependency/dept_object.h create mode 100644 kernel/dependency/dept_proc.c create mode 100644 kernel/dependency/dept_unit_test.c base-commit: 43dfc13ca972988e620a6edb72956981b75ab6b0 -- 2.17.1

1 week, 6 days

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

Linaro-mm-sig June 2026