From: John Garry john.garry@huawei.com Sent: Wednesday, October 6, 2021 1:17 AM
On 06/10/2021 08:03, Dexuan Cui wrote:
After commit ea2f0f77538c, a 416-CPU VM running on Hyper-V hangs during boot because scsi_add_host_with_dma() sets shost->cmd_per_lun to a negative number: 'max_outstanding_req_per_channel' is 352, 'max_sub_channels' is (416 - 1) / 4 = 103, so in storvsc_probe(), scsi_driver.can_queue = 352 * (103 + 1) * (100 - 10) / 100 = 32947, which is bigger than SHRT_MAX (i.e. 32767).
Out of curiosity, are these values realistic? You're capping can_queue just because of a data size issue, so, if these values are realistic, seems a weak reason.
The calculated value of can_queue is not realistic. The blk-mq layer caps the number of tags at 10240, so the excessively large value calculated here didn't definitively break anything, though it can be poor from a performance tuning standpoint. The algorithm used here is fairly broken, particularly in VMs with large CPU counts. I have an effort underway to fix it, but its part of a bigger set of changes to also do a better job on the perf tuning aspects.
Fix the hang issue by capping scsi_driver.can_queue.
Add the below Fixed tag though ea2f0f77538c itself is good.
Fixes: ea2f0f77538c ("scsi: core: Cap scsi_host cmd_per_lun at can_queue") Cc: stable@vger.kernel.org Signed-off-by: Dexuan Cui decui@microsoft.com
drivers/scsi/storvsc_drv.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
diff --git a/drivers/scsi/storvsc_drv.c b/drivers/scsi/storvsc_drv.c index ebbbc1299c62..ba374908aec2 100644 --- a/drivers/scsi/storvsc_drv.c +++ b/drivers/scsi/storvsc_drv.c @@ -1976,6 +1976,16 @@ static int storvsc_probe(struct hv_device *device, (max_sub_channels + 1) * (100 - ring_avail_percent_lowater) / 100;
- /*
* v5.14 (see commit ea2f0f77538c) implicitly requires that* scsi_driver.can_queue should not exceed SHRT_MAX, otherwise* scsi_add_host_with_dma() sets shost->cmd_per_lun to a negative* number (note: the type of the "cmd_per_lun" field is "short"), and* the system may hang during early boot.*/The different data sizes for cmd_per_lun and can_queue are problematic here.
I'd be more inclined to set cmd_per_lun to the same data size as can_queue. We did discuss this when ea2f0f77538c was upstreamed (actually it was the other way around - setting can_queue to 16b).
I can see that making can_queue be 16 bits would make sense. And it also seems that both cmd_per_lun and can_queue should be unsigned, though I don't the implications of making such a change.
But in today's world where cmd_per_lun is "short" and can_queue is "int", ea2f0f77538c seems incorrect to me. The comparison should be done as "int", not "short", in order to prevent the truncation problem with can_queue that Dexuan's patch is trying to address. The result will always fit in back into the "short" cmd_per_lun since it is calculating a "min" function.
Thanks, John
- if (scsi_driver.can_queue > SHRT_MAX)
scsi_driver.can_queue = SHRT_MAX;
This fix works, but is a more of a temporary hack until I can finish a larger overhaul of the algorithm. But for now, I think the better fix is for ea2f0f77538c to do the comparison as "int" instead of "short".
Michael
host = scsi_host_alloc(&scsi_driver, sizeof(struct hv_host_device)); if (!host)