During activation of secondary CPUs, lapic_online is invoked to initialize vectors. While lapic_online installs legacy vectors on all CPUs, it does not set the corresponding bits in per CPU bitmap maintained under irq_matrix. This may result in these legacy vectors getting allocated by irq_matrix_alloc and if that happens subsequent invocation of apic_update_vector will cause BUG like the one shown below:
[ 154.738226] kernel BUG at arch/x86/kernel/apic/vector.c:172! [ 154.805956] invalid opcode: 0000 [#1] SMP PTI [ 154.858092] CPU: 22 PID: 3569 Comm: ifup-eth Not tainted 5.8.0-20200716.x86_64 #1 [ 154.954939] Hardware name: Oracle Corporation ORACLE SERVER X6-2/ASM,MOTHERBOARD,1U [ 155.073636] RIP: 0010:apic_update_vector+0xa7/0x190 [ 155.131996] Code: 01 00 4a 8b 14 ed 80 69 01 a6 48 89 c8 4a 8d 04 e0 48 8b 04 10 48 85 c0 0f 84 d2 00 00 00 48 3d 00 f0 ff ff 0f 87 c6 00 00 00 <0f> 0b 41 8b 46 10 48 0f a3 05 6b 3e 7c 01 0f 92 c0 84 c0 0f 84 83 [ 155.356788] RSP: 0018:ffffb3848b417970 EFLAGS: 00010087 [ 155.419311] RAX: ffff9e9047c79000 RBX: 0000000000000000 RCX: 0000000000017040 [ 155.504719] RDX: ffff9e9fbf800000 RSI: 0000000000000182 RDI: ffff9e9fbe7936c0 [ 155.590127] RBP: ffffb3848b4179b0 R08: 0000000000000000 R09: 0004000000000000 [ 155.675533] R10: ffff000000000000 R11: 0000000000000246 R12: 0000000000000030 [ 155.760939] R13: 000000000000000a R14: ffff9e9fbe7939c0 R15: 0000000000000030 [ 155.846341] FS: 00007f6513279740(0000) GS:ffff9e979fb00000(0000) knlGS:0000000000000000 [ 155.943189] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 156.011947] CR2: 00007f6513280000 CR3: 00000007f2cbc003 CR4: 00000000003606e0 [ 156.097355] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 156.182761] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 156.268168] Call Trace: [ 156.297409] ? irq_matrix_alloc+0x8a/0x150 [ 156.346408] assign_vector_locked+0xd2/0x170 [ 156.397489] x86_vector_activate+0x1b5/0x320 [ 156.448570] __irq_domain_activate_irq+0x64/0xa0 [ 156.503808] __irq_domain_activate_irq+0x38/0xa0 [ 156.559050] irq_domain_activate_irq+0x2b/0x40 [ 156.612213] irq_activate+0x25/0x30 [ 156.653930] __setup_irq+0x58f/0x7b0 [ 156.696690] request_threaded_irq+0xf8/0x1b0 [ 156.747784] ixgbe_open+0x3af/0x600 [ixgbe] [ 156.797827] __dev_open+0xd8/0x160 [ 156.838503] dev_open+0x48/0x90 [ 156.876065] bond_enslave+0x2b6/0x12c0 [bonding] [ 156.931310] ? vsscanf+0x5af/0x8e0 [ 156.971986] ? sscanf+0x4e/0x70 [ 157.009546] bond_option_slaves_set+0x112/0x1c0 [bonding] [ 157.074148] __bond_opt_set+0xdc/0x320 [bonding] [ 157.129389] __bond_opt_set_notify+0x2c/0x90 [bonding] [ 157.190871] bond_opt_tryset_rtnl+0x56/0xa0 [bonding] [ 157.251315] bonding_sysfs_store_option+0x52/0x90 [bonding]
This patch marks these legacy vectors as assigned in irq_matrix so that corresponding bits in percpu bitmaps get set and these legacy vectors don't get reallocted.
Signed-off-by: Imran Khan imran.f.khan@oracle.com --- arch/x86/kernel/apic/vector.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c index 6dbdc7c22bb7..ea92b12614b9 100644 --- a/arch/x86/kernel/apic/vector.c +++ b/arch/x86/kernel/apic/vector.c @@ -806,6 +806,7 @@ static struct irq_desc *__setup_vector_irq(int vector) void lapic_online(void) { unsigned int vector; + struct irq_desc *desc = VECTOR_UNUSED;
lockdep_assert_held(&vector_lock);
@@ -821,8 +822,17 @@ void lapic_online(void) * must be installed on all CPUs. All non legacy interrupts can be * cleared. */ - for (vector = 0; vector < NR_VECTORS; vector++) - this_cpu_write(vector_irq[vector], __setup_vector_irq(vector)); + for (vector = 0; vector < NR_VECTORS; vector++) { + desc = __setup_vector_irq(vector); + this_cpu_write(vector_irq[vector], desc); + /* + * Mark legacy vectors assigned, so that + * irq_matrix_alloc does not see them as + * free in bitmap + */ + if (desc != VECTOR_UNUSED) + irq_matrix_assign(vector_matrix, vector); + } }
void lapic_offline(void)