Pete!
On Fri, Jun 14 2024 at 21:42, Pete Swain wrote:
The flapping-irq detector still has a timebomb.
A pathological workload, or test script, can arm the spurious-irq timebomb described in 4f27c00bf80f ("Improve behaviour of spurious IRQ detect")
This leads to irqs being moved the much slower polled mode, despite the actual unhandled-irq rate being well under the 99.9k/100k threshold that the code appears to check.
How?
Queued completion handler, like nvme, servicing events as they appear in the queue, even if the irq corresponding to the event has not yet been seen.
queues frequently empty, so seeing "spurious" irqs whenever the last events of a threaded handler's while (events_queued()) process_them(); ends with those events' irqs posted while thread was scanning. In this case the while() has consumed last event(s), so next handler says IRQ_NONE.
In each run of "unhandled" irqs, exactly one IRQ_NONE response is promoted from IRQ_NONE to IRQ_HANDLED, by note_interrupt()'s SPURIOUS_DEFERRED logic.
Any 2+ unhandled-irq runs will increment irqs_unhandled. The time_after() check in note_interrupt() resets irqs_unhandled to 1 after an idle period, but if irqs are never spaced more than HZ/10 apart, irqs_unhandled keeps growing.
During processing of long completion queues, the non-threaded handlers will return IRQ_WAKE_THREAD, for potentially thousands of per-event irqs. These bypass note_interrupt()'s irq_count++ logic, so do not count as handled, and do not invoke the flapping-irq logic.
When the _counted_ irq_count reaches the 100k threshold, it's possible for irqs_unhandled > 99.9k to force a move to polling mode, even though many millions of _WAKE_THREAD irqs have been handled without being counted.
Solution: include IRQ_WAKE_THREAD events in irq_count. Only when IRQ_NONE responses outweigh (IRQ_HANDLED + IRQ_WAKE_THREAD) by the old 99:1 ratio will an irq be moved to polling mode.
Nice detective work. Though I'm not entirely sure whether that's the correct approach as it might misjudge the situation where IRQ_WAKE_THREAD is issued but the thread does not make progress at all.
Let me think about it some more.
Thanks,
tglx