On Wed, May 07, 2025 at 03:49:41PM -0600, Uday Shankar wrote:
Add support in kublk for decoupled ublk_queues and ublk server threads. kublk now has two modes of operation:
- (preexisting mode) threads and queues are paired 1:1, and each thread services all the I/Os of one queue
- (new mode) thread and queue counts are independently configurable. threads service I/Os in a way that balances load across threads even if load is not balanced over queues. requires passing --round_robin
The new mode of operation is exercised by the new test_generic_08, which issues I/O against a single queue and verifies that each of the 8 ublk server threads handles exactly 1/8 of the total I/O count. Under the old mode of operation (i.e. without --round_robin), all I/O goes to one ublk server thread, and the test fails.
Hi Uday,
I just setup two ublk devices in the following way:
[root@ktest-40 ublk]# ./kublk add -t null -q 2 --nthreads=4 --round_robin dev id 0: nr_hw_queues 2 queue_depth 128 block size 512 dev_capacity 524288000 max rq size 1048576 daemon pid 8516 flags 0x842 state LIVE queue 0: affinity(0 ) queue 1: affinity(8 )
[root@ktest-40 ublk]# ./kublk add -t null -q 2 dev id 1: nr_hw_queues 2 queue_depth 128 block size 512 dev_capacity 524288000 max rq size 1048576 daemon pid 8539 flags 0x42 state LIVE queue 0: affinity(0 ) queue 1: affinity(8 )
Then run 'fio/t/io_uring -p0 /dev/ublkb[0|1]', and IOPS of /dev/ublkb0 is just 1/3 of /dev/ublkb1.
For ublk/loop, '-q 2 --nthreads=4 --round_robin' is still a bit slower than '-q 2', but the gap isn't too big(<10%).
Thanks, Ming