From: Greg Kroah-Hartman gregkh@linuxfoundation.org
This is the start of the stable review cycle for the 5.4.105 release. There are 24 patches in this series, all will be posted as a response to this one. If anyone has any issues with these being applied, please let me know.
Responses should be made by Fri, 12 Mar 2021 13:23:09 +0000. Anything received after that time might be too late.
The whole patch series can be found in one patch at: https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/patch-5.4.105-rc1... or in the git tree and branch at: git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-5.4.y and the diffstat can be found below.
thanks,
greg k-h
------------- Pseudo-Shortlog of commits:
Greg Kroah-Hartman gregkh@linuxfoundation.org Linux 5.4.105-rc1
Pascal Terjan pterjan@google.com nvme-pci: add quirks for Lexar 256GB SSD
Julian Einwag jeinwag-nvme@marcapo.com nvme-pci: mark Seagate Nytro XM1440 as QUIRK_NO_NS_DESC_LIST.
Hans de Goede hdegoede@redhat.com HID: i2c-hid: Add I2C_HID_QUIRK_NO_IRQ_AFTER_RESET for ITE8568 EC on Voyo Winpad A15
Jisheng Zhang Jisheng.Zhang@synaptics.com mmc: sdhci-of-dwcmshc: set SDHCI_QUIRK2_PRESET_VALUE_BROKEN
AngeloGioacchino Del Regno angelogioacchino.delregno@somainline.org drm/msm/a5xx: Remove overwriting A5XX_PC_DBG_ECO_CNTL register
Aswath Govindraju a-govindraju@ti.com misc: eeprom_93xx46: Add quirk to support Microchip 93LC46B eeprom
Bjorn Helgaas bhelgaas@google.com PCI: Add function 1 DMA alias quirk for Marvell 9215 SATA controller
Chris Chiu chiu@endlessos.org ASoC: Intel: bytcr_rt5640: Add quirk for ARCHOS Cesium 140
Jasper St. Pierre jstpierre@mecheye.net ACPI: video: Add DMI quirk for GIGABYTE GB-BXBT-2807
Daniel Lee Kruse daniel.lee.kruse@protonmail.com media: cx23885: add more quirks for reset DMA on some AMD IOMMU
Ethan Warth redyoshi49q@gmail.com HID: mf: add support for 0079:1846 Mayflash/Dragonrise USB Gamecube Adapter
Hans de Goede hdegoede@redhat.com platform/x86: acer-wmi: Add ACER_CAP_KBD_DOCK quirk for the Aspire Switch 10E SW3-016
Hans de Goede hdegoede@redhat.com platform/x86: acer-wmi: Add support for SW_TABLET_MODE on Switch devices
Hans de Goede hdegoede@redhat.com platform/x86: acer-wmi: Add ACER_CAP_SET_FUNCTION_MODE capability flag
Hans de Goede hdegoede@redhat.com platform/x86: acer-wmi: Add new force_caps module parameter
Hans de Goede hdegoede@redhat.com platform/x86: acer-wmi: Cleanup accelerometer device handling
Hans de Goede hdegoede@redhat.com platform/x86: acer-wmi: Cleanup ACER_CAP_FOO defines
Tsuchiya Yuto kitakar@gmail.com mwifiex: pcie: skip cancel_work_sync() on reset failure path
Andrey Ryabinin arbn@yandex-team.com iommu/amd: Fix sleeping in atomic in increase_address_space()
Hans de Goede hdegoede@redhat.com ACPICA: Fix race in generic_serial_bus (I2C) and GPIO op_region parameter handling
Jeffle Xu jefflexu@linux.alibaba.com dm table: fix zoned iterate_devices based device capability checks
Jeffle Xu jefflexu@linux.alibaba.com dm table: fix DAX iterate_devices based device capability checks
Jeffle Xu jefflexu@linux.alibaba.com dm table: fix iterate_devices based device capability checks
Alexander Lobakin bloodyreaper@yandex.ru net: dsa: add GRO support via gro_cells
-------------
Diffstat:
Makefile | 4 +- drivers/acpi/acpica/acobject.h | 1 + drivers/acpi/acpica/evhandler.c | 7 ++ drivers/acpi/acpica/evregion.c | 64 +++++++--- drivers/acpi/acpica/evxfregn.c | 2 + drivers/acpi/video_detect.c | 7 ++ drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 2 - drivers/hid/hid-ids.h | 3 + drivers/hid/hid-mf.c | 2 + drivers/hid/hid-quirks.c | 2 + drivers/hid/i2c-hid/i2c-hid-core.c | 2 + drivers/iommu/amd_iommu.c | 10 +- drivers/md/dm-table.c | 174 ++++++++++++---------------- drivers/md/dm.c | 2 +- drivers/md/dm.h | 2 +- drivers/media/pci/cx23885/cx23885-core.c | 4 + drivers/misc/eeprom/eeprom_93xx46.c | 15 +++ drivers/mmc/host/sdhci-of-dwcmshc.c | 1 + drivers/net/wireless/marvell/mwifiex/pcie.c | 18 ++- drivers/net/wireless/marvell/mwifiex/pcie.h | 2 + drivers/nvme/host/pci.c | 6 +- drivers/pci/quirks.c | 3 + drivers/platform/x86/acer-wmi.c | 169 ++++++++++++++++++++++----- include/linux/eeprom_93xx46.h | 2 + net/dsa/Kconfig | 1 + net/dsa/dsa.c | 2 +- net/dsa/dsa_priv.h | 3 + net/dsa/slave.c | 10 +- sound/soc/intel/boards/bytcr_rt5640.c | 12 ++ 29 files changed, 371 insertions(+), 161 deletions(-)
On 3/10/21 6:24 AM, gregkh@linuxfoundation.org wrote:
From: Greg Kroah-Hartman gregkh@linuxfoundation.org
This is the start of the stable review cycle for the 5.4.105 release. There are 24 patches in this series, all will be posted as a response to this one. If anyone has any issues with these being applied, please let me know.
Responses should be made by Fri, 12 Mar 2021 13:23:09 +0000. Anything received after that time might be too late.
The whole patch series can be found in one patch at: https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/patch-5.4.105-rc1... or in the git tree and branch at: git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-5.4.y and the diffstat can be found below.
thanks,
greg k-h
Compiled and booted on my test system. No dmesg regressions.
Tested-by: Shuah Khan skhan@linuxfoundation.org
thanks, -- Shuah
On Wed, Mar 10, 2021 at 02:24:12PM +0100, gregkh@linuxfoundation.org wrote:
From: Greg Kroah-Hartman gregkh@linuxfoundation.org
This is the start of the stable review cycle for the 5.4.105 release. There are 24 patches in this series, all will be posted as a response to this one. If anyone has any issues with these being applied, please let me know.
Responses should be made by Fri, 12 Mar 2021 13:23:09 +0000. Anything received after that time might be too late.
Build results: total: 157 pass: 157 fail: 0 Qemu test results: total: 431 pass: 431 fail: 0
Tested-by: Guenter Roeck linux@roeck-us.net
Guenter
On Wed, Mar 10, 2021 at 02:24:12PM +0100, gregkh@linuxfoundation.org wrote:
From: Greg Kroah-Hartman gregkh@linuxfoundation.org
This is the start of the stable review cycle for the 5.4.105 release. There are 24 patches in this series, all will be posted as a response to this one. If anyone has any issues with these being applied, please let me know.
Compiled and booted with no regressions on x86_64.
Tested-by: Ross Schmidt ross.schm.dev@gmail.com
thanks,
Ross
+Alex,
On 3/10/2021 5:24 AM, gregkh@linuxfoundation.org wrote:
From: Greg Kroah-Hartman gregkh@linuxfoundation.org
This is the start of the stable review cycle for the 5.4.105 release. There are 24 patches in this series, all will be posted as a response to this one. If anyone has any issues with these being applied, please let me know.
Responses should be made by Fri, 12 Mar 2021 13:23:09 +0000. Anything received after that time might be too late.
The whole patch series can be found in one patch at: https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/patch-5.4.105-rc1... or in the git tree and branch at: git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-5.4.y and the diffstat can be found below.
thanks,
greg k-h
I believe you need to drop "net: dsa: add GRO support via gro_cells" as it causes the following kernel panic on a DSA-enabled platform:
Configuring rgmii_2 interface [ 10.170527] brcm-sf2 f0b00000.ethernet_switch rgmii_2: configuring for fixed/rgmii-txid link mode [ 10.179597] 8021q: adding VLAN 0 to HW filter on device rgmii_2 [ 10.185608] brcm-sf2 f0b00000.ethernet_switch rgmii_2: Link is Up - 1Gbps/Full - flow control off [ 10.198631] IPv6: ADDRCONF(NETDEV_CHANGE): rgmii_2: link becomes ready Configuring sit0 interface [ 10.254346] 8<--- cut here --- [ 10.257438] Unable to handle kernel paging request at virtual address d6df6190 [ 10.264685] pgd = (ptrval) [ 10.267411] [d6df6190] *pgd=80000000007003, *pmd=00000000 [ 10.272846] Internal error: Oops: 206 [#1] SMP ARM [ 10.277661] Modules linked in: [ 10.280739] CPU: 0 PID: 1886 Comm: sed Not tainted 5.4.105-1.0pre-geff642e2af2b #4 [ 10.288337] Hardware name: Broadcom STB (Flattened Device Tree) [ 10.294292] PC is at gro_cells_receive+0x90/0x11c [ 10.299020] LR is at dsa_switch_rcv+0x120/0x1d4 [ 10.303562] pc : [<c0a57a28>] lr : [<c0b4a65c>] psr: 600f0113 [ 10.309841] sp : c1d33cd0 ip : 000003e8 fp : c1d33ce4 [ 10.315078] r10: c8901000 r9 : c8901000 r8 : c0b4a53c [ 10.320314] r7 : c2208920 r6 : 00000000 r5 : 00000000 r4 : 00004000 [ 10.326855] r3 : d6df6188 r2 : c4927000 r1 : c8adc300 r0 : c22069dc [ 10.333398] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user [ 10.340547] Control: 30c5387d Table: 04ac4c80 DAC: fffffffd [ 10.346307] Process sed (pid: 1886, stack limit = 0x(ptrval)) [ 10.352066] Stack: (0xc1d33cd0 to 0xc1d34000) [ 10.356434] 3cc0: c8adc300 c4927000 c1d33d04 c1d33ce8 [ 10.364631] 3ce0: c0b4a65c c0a579a4 c1d33d24 c2208920 c1d33d24 00000000 c1d33d5c c1d33d08 [ 10.372827] 3d00: c0a0b38c c0b4a548 c021e070 c2204cc8 00000000 c89015c0 04b87700 c89015c0 [ 10.381023] 3d20: c2208920 c1d33d24 c1d33d24 00976ec2 04b87700 c8adc300 c89015c0 00000000 [ 10.389218] 3d40: c1d33d74 c1d32000 00000000 c230742c c1d33dac c1d33d60 c0a0b5c0 c0a0b180 [ 10.397414] 3d60: 00000000 c2204cc8 00000000 c1d33d6c c1d33d6c c1d33d80 c029daf8 00976ec2 [ 10.405610] 3d80: 00000800 c8901540 c89015c0 c8901540 00000000 00000001 0000016c 00000162 [ 10.413805] 3da0: c1d33dc4 c1d33db0 c0a0b7fc c0a0b3b8 00000000 c8adc300 c1d33dfc c1d33dc8 [ 10.422001] 3dc0: c0a0c660 c0a0b7e4 c8901540 c8adc300 c1d33dfc c1d33de0 c8901540 c8adc300 [ 10.430196] 3de0: 0000015e c8901000 00000001 0000016c c1d33e74 c1d33e00 c083df00 c0a0c4fc [ 10.438391] 3e00: 0000012c c22b0f14 c1d33e4c c1d33e18 c0fbd9b8 c0fbd9cc c0fbd9e0 c0fbd98c [ 10.446586] 3e20: 00000001 00000040 c8901500 00000001 00000000 00000000 00000000 00000000 [ 10.454780] 3e40: 00000000 00000000 c02f65a0 c8901540 00000001 00000040 c22b07e4 0000012c [ 10.462975] 3e60: d1003000 fffb942f c1d33edc c1d33e78 c0a0c94c c083dafc d051ad80 c2204cc8 [ 10.471170] 3e80: c2204cf0 c1d32000 c22b40b0 0e4a4000 c2076d80 c2203d00 c022bc70 c1d33e9c [ 10.479365] 3ea0: c1d33e9c c1d33ea4 c1d33ea4 00976ec2 c02f65a0 c220308c 00000003 c1d32000 [ 10.487561] 3ec0: c22b07e4 00000100 d1003000 00000008 c1d33f44 c1d33ee0 c020238c c0a0c6cc [ 10.495755] 3ee0: c1d33f14 c1d33ef0 00000001 00400000 c2203d00 fffb942f c206b2e4 c2076040 [ 10.503950] 3f00: c2076040 0000000a c2203080 c206b358 c1d33ee0 00000004 c90c9500 ffffe000 [ 10.512145] 3f20: 00000000 00000000 00000001 c9019000 d1003000 00000000 c1d33f5c c1d33f48 [ 10.520339] 3f40: c022bc70 c0202264 c2075fbc 00000000 c1d33f84 c1d33f60 c027feb4 c022bb88 [ 10.528535] 3f60: c226d668 c220565c d100200c c1d33fb0 d1002000 d1003000 c1d33fac c1d33f88 [ 10.536730] 3f80: c0202214 c027fe50 b6f1e0b6 000f0030 ffffffff 30c5387d 30c5387d 00000000 [ 10.544924] 3fa0: 00000000 c1d33fb0 c0201d8c c02021c4 b6ec1778 b6f32094 62632e73 62740000 [ 10.553120] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 00000000 00000000 00000006 [ 10.561316] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 ffffffff 00000000 00000000 [ 10.569508] Backtrace: [ 10.571970] [<c0a57998>] (gro_cells_receive) from [<c0b4a65c>] (dsa_switch_rcv+0x120/0x1d4) [ 10.580338] r5:c4927000 r4:c8adc300 [ 10.583929] [<c0b4a53c>] (dsa_switch_rcv) from [<c0a0b38c>] (__netif_receive_skb_list_core+0x218/0x238) [ 10.593343] r7:00000000 r6:c1d33d24 r5:c2208920 r4:c1d33d24 [ 10.599017] [<c0a0b174>] (__netif_receive_skb_list_core) from [<c0a0b5c0>] (netif_receive_skb_list_internal+0x214/0x2dc) [ 10.609909] r10:c230742c r9:00000000 r8:c1d32000 r7:c1d33d74 r6:00000000 r5:c89015c0 [ 10.617755] r4:c8adc300 [ 10.620300] [<c0a0b3ac>] (netif_receive_skb_list_internal) from [<c0a0b7fc>] (gro_normal_list.part.42+0x24/0x38) [ 10.630496] r10:00000162 r9:0000016c r8:00000001 r7:00000000 r6:c8901540 r5:c89015c0 [ 10.638342] r4:c8901540 [ 10.640885] [<c0a0b7d8>] (gro_normal_list.part.42) from [<c0a0c660>] (napi_complete_done+0x170/0x1d0) [ 10.650123] r5:c8adc300 r4:00000000 [ 10.653712] [<c0a0c4f0>] (napi_complete_done) from [<c083df00>] (bcm_sysport_poll+0x410/0x4b4) [ 10.662343] r9:0000016c r8:00000001 r7:c8901000 r6:0000015e r5:c8adc300 r4:c8901540 [ 10.670104] [<c083daf0>] (bcm_sysport_poll) from [<c0a0c94c>] (net_rx_action+0x28c/0x44c) [ 10.678301] r10:fffb942f r9:d1003000 r8:0000012c r7:c22b07e4 r6:00000040 r5:00000001 [ 10.686146] r4:c8901540 [ 10.688693] [<c0a0c6c0>] (net_rx_action) from [<c020238c>] (__do_softirq+0x134/0x414) [ 10.696542] r10:00000008 r9:d1003000 r8:00000100 r7:c22b07e4 r6:c1d32000 r5:00000003 [ 10.704387] r4:c220308c [ 10.706931] [<c0202258>] (__do_softirq) from [<c022bc70>] (irq_exit+0xf4/0x100) [ 10.714257] r10:00000000 r9:d1003000 r8:c9019000 r7:00000001 r6:00000000 r5:00000000 [ 10.722104] r4:ffffe000 [ 10.724650] [<c022bb7c>] (irq_exit) from [<c027feb4>] (__handle_domain_irq+0x70/0xc4) [ 10.732497] r5:00000000 r4:c2075fbc [ 10.736083] [<c027fe44>] (__handle_domain_irq) from [<c0202214>] (gic_handle_irq+0x5c/0xa0) [ 10.744453] r9:d1003000 r8:d1002000 r7:c1d33fb0 r6:d100200c r5:c220565c r4:c226d668 [ 10.752215] [<c02021b8>] (gic_handle_irq) from [<c0201d8c>] (__irq_usr+0x4c/0x60) [ 10.759713] Exception stack(0xc1d33fb0 to 0xc1d33ff8) [ 10.764776] 3fa0: b6ec1778 b6f32094 62632e73 62740000 [ 10.772973] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 00000000 00000000 00000006 [ 10.781167] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 ffffffff [ 10.787797] r9:00000000 r8:30c5387d r7:30c5387d r6:ffffffff r5:000f0030 r4:b6f1e0b6 [ 10.795559] Code: e30609dc e34c0220 e083300c e590c000 (e5930008) [ 10.801670] ---[ end trace 97c3942fa73eff4c ]--- [ 10.806300] Kernel panic - not syncing: Fatal exception in interrupt [ 10.812678] CPU2: stopping [ 10.815403] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 10.824641] Hardware name: Broadcom STB (Flattened Device Tree) [ 10.830573] Backtrace: [ 10.833036] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 10.840624] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 [ 10.846302] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 10.853546] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 10.860960] r10:00000000 r9:c2205400 r8:00000000 r7:00000002 r6:c22b0744 r5:00000004 [ 10.868807] r4:c22ce308 r3:00976ec2 [ 10.872395] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 10.879983] r10:00000000 r9:d1003000 r8:d1002000 r7:c9131f18 r6:d100200c r5:c220565c [ 10.887829] r4:c226d668 [ 10.890373] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 10.897871] Exception stack(0xc9131f18 to 0xc9131f60) [ 10.902934] 1f00: c020a47c 00000000 [ 10.911131] 1f20: 0e4ca000 600c0093 c9130000 c2204cf0 c2204d34 00000004 00000000 c20757b0 [ 10.919327] 1f40: 00000000 c9131f74 c9130000 c9131f68 00000000 c020a480 600c0013 ffffffff [ 10.927523] r9:c9130000 r8:00000000 r7:c9131f4c r6:ffffffff r5:600c0013 r4:c020a480 [ 10.935291] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 10.943403] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 10.951164] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 10.958752] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000002 [ 10.966598] r4:0000008a [ 10.969142] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 10.978210] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 10.985447] r5:00000000 r4:090eaa40 [ 10.989032] CPU3: stopping [ 10.991754] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 11.000992] Hardware name: Broadcom STB (Flattened Device Tree) [ 11.006924] Backtrace: [ 11.009384] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 11.016972] r7:c22a86d0 r6:00000000 r5:600f0193 r4:c22a86d0 [ 11.022649] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 11.029892] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 11.037307] r10:00000000 r9:c2205400 r8:00000000 r7:00000003 r6:c22b0744 r5:00000004 [ 11.045154] r4:c22ce308 r3:00976ec2 [ 11.048741] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 11.056329] r10:00000000 r9:d1003000 r8:d1002000 r7:c9133f18 r6:d100200c r5:c220565c [ 11.064176] r4:c226d668 [ 11.066719] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 11.074216] Exception stack(0xc9133f18 to 0xc9133f60) [ 11.079280] 3f00: c020a47c 00000000 [ 11.087476] 3f20: 0e4dd000 600f0093 c9132000 c2204cf0 c2204d34 00000008 00000000 c20757b0 [ 11.095671] 3f40: 00000000 c9133f74 c9132000 c9133f68 00000000 c020a480 600f0013 ffffffff [ 11.103867] r9:c9132000 r8:00000000 r7:c9133f4c r6:ffffffff r5:600f0013 r4:c020a480 [ 11.111633] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 11.119744] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 11.127506] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 11.135093] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000003 [ 11.142939] r4:0000008a [ 11.145484] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 11.154550] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 11.161788] r5:00000000 r4:090eaa40 [ 11.165372] CPU1: stopping [ 11.168094] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 11.177332] Hardware name: Broadcom STB (Flattened Device Tree) [ 11.183264] Backtrace: [ 11.185723] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 11.193311] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 [ 11.198988] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 11.206230] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 11.213644] r10:00000000 r9:c2205400 r8:00000000 r7:00000001 r6:c22b0744 r5:00000004 [ 11.221491] r4:c22ce308 r3:00976ec2 [ 11.225078] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 11.232666] r10:00000000 r9:d1003000 r8:d1002000 r7:c912ff18 r6:d100200c r5:c220565c [ 11.240512] r4:c226d668 [ 11.243054] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 11.250553] Exception stack(0xc912ff18 to 0xc912ff60) [ 11.255617] ff00: c020a47c 00000000 [ 11.263814] ff20: 0e4b7000 600c0093 c912e000 c2204cf0 c2204d34 00000002 00000000 c20757b0 [ 11.272010] ff40: 00000000 c912ff74 c912e000 c912ff68 00000000 c020a480 600c0013 ffffffff [ 11.280206] r9:c912e000 r8:00000000 r7:c912ff4c r6:ffffffff r5:600c0013 r4:c020a480 [ 11.287970] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 11.296081] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 11.303842] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 11.311430] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000001 [ 11.319275] r4:0000008a [ 11.321819] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 11.330886] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 11.338124] r5:00000000 r4:090eaa40 [ 11.341729] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---
it is not marked as fixing anything so I wonder how it landed in stable?
On Wed, Mar 10, 2021 at 08:19:45PM -0800, Florian Fainelli wrote:
+Alex,
On 3/10/2021 5:24 AM, gregkh@linuxfoundation.org wrote:
From: Greg Kroah-Hartman gregkh@linuxfoundation.org
This is the start of the stable review cycle for the 5.4.105 release. There are 24 patches in this series, all will be posted as a response to this one. If anyone has any issues with these being applied, please let me know.
Responses should be made by Fri, 12 Mar 2021 13:23:09 +0000. Anything received after that time might be too late.
The whole patch series can be found in one patch at: https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/patch-5.4.105-rc1... or in the git tree and branch at: git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-5.4.y and the diffstat can be found below.
thanks,
greg k-h
I believe you need to drop "net: dsa: add GRO support via gro_cells" as it causes the following kernel panic on a DSA-enabled platform:
Configuring rgmii_2 interface [ 10.170527] brcm-sf2 f0b00000.ethernet_switch rgmii_2: configuring for fixed/rgmii-txid link mode [ 10.179597] 8021q: adding VLAN 0 to HW filter on device rgmii_2 [ 10.185608] brcm-sf2 f0b00000.ethernet_switch rgmii_2: Link is Up - 1Gbps/Full - flow control off [ 10.198631] IPv6: ADDRCONF(NETDEV_CHANGE): rgmii_2: link becomes ready Configuring sit0 interface [ 10.254346] 8<--- cut here --- [ 10.257438] Unable to handle kernel paging request at virtual address d6df6190 [ 10.264685] pgd = (ptrval) [ 10.267411] [d6df6190] *pgd=80000000007003, *pmd=00000000 [ 10.272846] Internal error: Oops: 206 [#1] SMP ARM [ 10.277661] Modules linked in: [ 10.280739] CPU: 0 PID: 1886 Comm: sed Not tainted 5.4.105-1.0pre-geff642e2af2b #4 [ 10.288337] Hardware name: Broadcom STB (Flattened Device Tree) [ 10.294292] PC is at gro_cells_receive+0x90/0x11c [ 10.299020] LR is at dsa_switch_rcv+0x120/0x1d4 [ 10.303562] pc : [<c0a57a28>] lr : [<c0b4a65c>] psr: 600f0113 [ 10.309841] sp : c1d33cd0 ip : 000003e8 fp : c1d33ce4 [ 10.315078] r10: c8901000 r9 : c8901000 r8 : c0b4a53c [ 10.320314] r7 : c2208920 r6 : 00000000 r5 : 00000000 r4 : 00004000 [ 10.326855] r3 : d6df6188 r2 : c4927000 r1 : c8adc300 r0 : c22069dc [ 10.333398] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user [ 10.340547] Control: 30c5387d Table: 04ac4c80 DAC: fffffffd [ 10.346307] Process sed (pid: 1886, stack limit = 0x(ptrval)) [ 10.352066] Stack: (0xc1d33cd0 to 0xc1d34000) [ 10.356434] 3cc0: c8adc300 c4927000 c1d33d04 c1d33ce8 [ 10.364631] 3ce0: c0b4a65c c0a579a4 c1d33d24 c2208920 c1d33d24 00000000 c1d33d5c c1d33d08 [ 10.372827] 3d00: c0a0b38c c0b4a548 c021e070 c2204cc8 00000000 c89015c0 04b87700 c89015c0 [ 10.381023] 3d20: c2208920 c1d33d24 c1d33d24 00976ec2 04b87700 c8adc300 c89015c0 00000000 [ 10.389218] 3d40: c1d33d74 c1d32000 00000000 c230742c c1d33dac c1d33d60 c0a0b5c0 c0a0b180 [ 10.397414] 3d60: 00000000 c2204cc8 00000000 c1d33d6c c1d33d6c c1d33d80 c029daf8 00976ec2 [ 10.405610] 3d80: 00000800 c8901540 c89015c0 c8901540 00000000 00000001 0000016c 00000162 [ 10.413805] 3da0: c1d33dc4 c1d33db0 c0a0b7fc c0a0b3b8 00000000 c8adc300 c1d33dfc c1d33dc8 [ 10.422001] 3dc0: c0a0c660 c0a0b7e4 c8901540 c8adc300 c1d33dfc c1d33de0 c8901540 c8adc300 [ 10.430196] 3de0: 0000015e c8901000 00000001 0000016c c1d33e74 c1d33e00 c083df00 c0a0c4fc [ 10.438391] 3e00: 0000012c c22b0f14 c1d33e4c c1d33e18 c0fbd9b8 c0fbd9cc c0fbd9e0 c0fbd98c [ 10.446586] 3e20: 00000001 00000040 c8901500 00000001 00000000 00000000 00000000 00000000 [ 10.454780] 3e40: 00000000 00000000 c02f65a0 c8901540 00000001 00000040 c22b07e4 0000012c [ 10.462975] 3e60: d1003000 fffb942f c1d33edc c1d33e78 c0a0c94c c083dafc d051ad80 c2204cc8 [ 10.471170] 3e80: c2204cf0 c1d32000 c22b40b0 0e4a4000 c2076d80 c2203d00 c022bc70 c1d33e9c [ 10.479365] 3ea0: c1d33e9c c1d33ea4 c1d33ea4 00976ec2 c02f65a0 c220308c 00000003 c1d32000 [ 10.487561] 3ec0: c22b07e4 00000100 d1003000 00000008 c1d33f44 c1d33ee0 c020238c c0a0c6cc [ 10.495755] 3ee0: c1d33f14 c1d33ef0 00000001 00400000 c2203d00 fffb942f c206b2e4 c2076040 [ 10.503950] 3f00: c2076040 0000000a c2203080 c206b358 c1d33ee0 00000004 c90c9500 ffffe000 [ 10.512145] 3f20: 00000000 00000000 00000001 c9019000 d1003000 00000000 c1d33f5c c1d33f48 [ 10.520339] 3f40: c022bc70 c0202264 c2075fbc 00000000 c1d33f84 c1d33f60 c027feb4 c022bb88 [ 10.528535] 3f60: c226d668 c220565c d100200c c1d33fb0 d1002000 d1003000 c1d33fac c1d33f88 [ 10.536730] 3f80: c0202214 c027fe50 b6f1e0b6 000f0030 ffffffff 30c5387d 30c5387d 00000000 [ 10.544924] 3fa0: 00000000 c1d33fb0 c0201d8c c02021c4 b6ec1778 b6f32094 62632e73 62740000 [ 10.553120] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 00000000 00000000 00000006 [ 10.561316] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 ffffffff 00000000 00000000 [ 10.569508] Backtrace: [ 10.571970] [<c0a57998>] (gro_cells_receive) from [<c0b4a65c>] (dsa_switch_rcv+0x120/0x1d4) [ 10.580338] r5:c4927000 r4:c8adc300 [ 10.583929] [<c0b4a53c>] (dsa_switch_rcv) from [<c0a0b38c>] (__netif_receive_skb_list_core+0x218/0x238) [ 10.593343] r7:00000000 r6:c1d33d24 r5:c2208920 r4:c1d33d24 [ 10.599017] [<c0a0b174>] (__netif_receive_skb_list_core) from [<c0a0b5c0>] (netif_receive_skb_list_internal+0x214/0x2dc) [ 10.609909] r10:c230742c r9:00000000 r8:c1d32000 r7:c1d33d74 r6:00000000 r5:c89015c0 [ 10.617755] r4:c8adc300 [ 10.620300] [<c0a0b3ac>] (netif_receive_skb_list_internal) from [<c0a0b7fc>] (gro_normal_list.part.42+0x24/0x38) [ 10.630496] r10:00000162 r9:0000016c r8:00000001 r7:00000000 r6:c8901540 r5:c89015c0 [ 10.638342] r4:c8901540 [ 10.640885] [<c0a0b7d8>] (gro_normal_list.part.42) from [<c0a0c660>] (napi_complete_done+0x170/0x1d0) [ 10.650123] r5:c8adc300 r4:00000000 [ 10.653712] [<c0a0c4f0>] (napi_complete_done) from [<c083df00>] (bcm_sysport_poll+0x410/0x4b4) [ 10.662343] r9:0000016c r8:00000001 r7:c8901000 r6:0000015e r5:c8adc300 r4:c8901540 [ 10.670104] [<c083daf0>] (bcm_sysport_poll) from [<c0a0c94c>] (net_rx_action+0x28c/0x44c) [ 10.678301] r10:fffb942f r9:d1003000 r8:0000012c r7:c22b07e4 r6:00000040 r5:00000001 [ 10.686146] r4:c8901540 [ 10.688693] [<c0a0c6c0>] (net_rx_action) from [<c020238c>] (__do_softirq+0x134/0x414) [ 10.696542] r10:00000008 r9:d1003000 r8:00000100 r7:c22b07e4 r6:c1d32000 r5:00000003 [ 10.704387] r4:c220308c [ 10.706931] [<c0202258>] (__do_softirq) from [<c022bc70>] (irq_exit+0xf4/0x100) [ 10.714257] r10:00000000 r9:d1003000 r8:c9019000 r7:00000001 r6:00000000 r5:00000000 [ 10.722104] r4:ffffe000 [ 10.724650] [<c022bb7c>] (irq_exit) from [<c027feb4>] (__handle_domain_irq+0x70/0xc4) [ 10.732497] r5:00000000 r4:c2075fbc [ 10.736083] [<c027fe44>] (__handle_domain_irq) from [<c0202214>] (gic_handle_irq+0x5c/0xa0) [ 10.744453] r9:d1003000 r8:d1002000 r7:c1d33fb0 r6:d100200c r5:c220565c r4:c226d668 [ 10.752215] [<c02021b8>] (gic_handle_irq) from [<c0201d8c>] (__irq_usr+0x4c/0x60) [ 10.759713] Exception stack(0xc1d33fb0 to 0xc1d33ff8) [ 10.764776] 3fa0: b6ec1778 b6f32094 62632e73 62740000 [ 10.772973] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 00000000 00000000 00000006 [ 10.781167] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 ffffffff [ 10.787797] r9:00000000 r8:30c5387d r7:30c5387d r6:ffffffff r5:000f0030 r4:b6f1e0b6 [ 10.795559] Code: e30609dc e34c0220 e083300c e590c000 (e5930008) [ 10.801670] ---[ end trace 97c3942fa73eff4c ]--- [ 10.806300] Kernel panic - not syncing: Fatal exception in interrupt [ 10.812678] CPU2: stopping [ 10.815403] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 10.824641] Hardware name: Broadcom STB (Flattened Device Tree) [ 10.830573] Backtrace: [ 10.833036] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 10.840624] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 [ 10.846302] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 10.853546] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 10.860960] r10:00000000 r9:c2205400 r8:00000000 r7:00000002 r6:c22b0744 r5:00000004 [ 10.868807] r4:c22ce308 r3:00976ec2 [ 10.872395] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 10.879983] r10:00000000 r9:d1003000 r8:d1002000 r7:c9131f18 r6:d100200c r5:c220565c [ 10.887829] r4:c226d668 [ 10.890373] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 10.897871] Exception stack(0xc9131f18 to 0xc9131f60) [ 10.902934] 1f00: c020a47c 00000000 [ 10.911131] 1f20: 0e4ca000 600c0093 c9130000 c2204cf0 c2204d34 00000004 00000000 c20757b0 [ 10.919327] 1f40: 00000000 c9131f74 c9130000 c9131f68 00000000 c020a480 600c0013 ffffffff [ 10.927523] r9:c9130000 r8:00000000 r7:c9131f4c r6:ffffffff r5:600c0013 r4:c020a480 [ 10.935291] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 10.943403] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 10.951164] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 10.958752] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000002 [ 10.966598] r4:0000008a [ 10.969142] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 10.978210] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 10.985447] r5:00000000 r4:090eaa40 [ 10.989032] CPU3: stopping [ 10.991754] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 11.000992] Hardware name: Broadcom STB (Flattened Device Tree) [ 11.006924] Backtrace: [ 11.009384] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 11.016972] r7:c22a86d0 r6:00000000 r5:600f0193 r4:c22a86d0 [ 11.022649] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 11.029892] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 11.037307] r10:00000000 r9:c2205400 r8:00000000 r7:00000003 r6:c22b0744 r5:00000004 [ 11.045154] r4:c22ce308 r3:00976ec2 [ 11.048741] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 11.056329] r10:00000000 r9:d1003000 r8:d1002000 r7:c9133f18 r6:d100200c r5:c220565c [ 11.064176] r4:c226d668 [ 11.066719] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 11.074216] Exception stack(0xc9133f18 to 0xc9133f60) [ 11.079280] 3f00: c020a47c 00000000 [ 11.087476] 3f20: 0e4dd000 600f0093 c9132000 c2204cf0 c2204d34 00000008 00000000 c20757b0 [ 11.095671] 3f40: 00000000 c9133f74 c9132000 c9133f68 00000000 c020a480 600f0013 ffffffff [ 11.103867] r9:c9132000 r8:00000000 r7:c9133f4c r6:ffffffff r5:600f0013 r4:c020a480 [ 11.111633] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 11.119744] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 11.127506] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 11.135093] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000003 [ 11.142939] r4:0000008a [ 11.145484] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 11.154550] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 11.161788] r5:00000000 r4:090eaa40 [ 11.165372] CPU1: stopping [ 11.168094] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 11.177332] Hardware name: Broadcom STB (Flattened Device Tree) [ 11.183264] Backtrace: [ 11.185723] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 11.193311] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 [ 11.198988] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 11.206230] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 11.213644] r10:00000000 r9:c2205400 r8:00000000 r7:00000001 r6:c22b0744 r5:00000004 [ 11.221491] r4:c22ce308 r3:00976ec2 [ 11.225078] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 11.232666] r10:00000000 r9:d1003000 r8:d1002000 r7:c912ff18 r6:d100200c r5:c220565c [ 11.240512] r4:c226d668 [ 11.243054] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 11.250553] Exception stack(0xc912ff18 to 0xc912ff60) [ 11.255617] ff00: c020a47c 00000000 [ 11.263814] ff20: 0e4b7000 600c0093 c912e000 c2204cf0 c2204d34 00000002 00000000 c20757b0 [ 11.272010] ff40: 00000000 c912ff74 c912e000 c912ff68 00000000 c020a480 600c0013 ffffffff [ 11.280206] r9:c912e000 r8:00000000 r7:c912ff4c r6:ffffffff r5:600c0013 r4:c020a480 [ 11.287970] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 11.296081] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 11.303842] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 11.311430] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000001 [ 11.319275] r4:0000008a [ 11.321819] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 11.330886] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 11.338124] r5:00000000 r4:090eaa40 [ 11.341729] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---
it is not marked as fixing anything so I wonder how it landed in stable?
It was requested to be merged. Do you have the same crash on newer kernels with this commit in it (like Linus's tree?)
thanks,
greg k-h
On 3/11/21 5:08 AM, Greg KH wrote:
On Wed, Mar 10, 2021 at 08:19:45PM -0800, Florian Fainelli wrote:
+Alex,
On 3/10/2021 5:24 AM, gregkh@linuxfoundation.org wrote:
From: Greg Kroah-Hartman gregkh@linuxfoundation.org
This is the start of the stable review cycle for the 5.4.105 release. There are 24 patches in this series, all will be posted as a response to this one. If anyone has any issues with these being applied, please let me know.
Responses should be made by Fri, 12 Mar 2021 13:23:09 +0000. Anything received after that time might be too late.
The whole patch series can be found in one patch at: https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/patch-5.4.105-rc1... or in the git tree and branch at: git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-5.4.y and the diffstat can be found below.
thanks,
greg k-h
I believe you need to drop "net: dsa: add GRO support via gro_cells" as it causes the following kernel panic on a DSA-enabled platform:
Configuring rgmii_2 interface [ 10.170527] brcm-sf2 f0b00000.ethernet_switch rgmii_2: configuring for fixed/rgmii-txid link mode [ 10.179597] 8021q: adding VLAN 0 to HW filter on device rgmii_2 [ 10.185608] brcm-sf2 f0b00000.ethernet_switch rgmii_2: Link is Up - 1Gbps/Full - flow control off [ 10.198631] IPv6: ADDRCONF(NETDEV_CHANGE): rgmii_2: link becomes ready Configuring sit0 interface [ 10.254346] 8<--- cut here --- [ 10.257438] Unable to handle kernel paging request at virtual address d6df6190 [ 10.264685] pgd = (ptrval) [ 10.267411] [d6df6190] *pgd=80000000007003, *pmd=00000000 [ 10.272846] Internal error: Oops: 206 [#1] SMP ARM [ 10.277661] Modules linked in: [ 10.280739] CPU: 0 PID: 1886 Comm: sed Not tainted 5.4.105-1.0pre-geff642e2af2b #4 [ 10.288337] Hardware name: Broadcom STB (Flattened Device Tree) [ 10.294292] PC is at gro_cells_receive+0x90/0x11c [ 10.299020] LR is at dsa_switch_rcv+0x120/0x1d4 [ 10.303562] pc : [<c0a57a28>] lr : [<c0b4a65c>] psr: 600f0113 [ 10.309841] sp : c1d33cd0 ip : 000003e8 fp : c1d33ce4 [ 10.315078] r10: c8901000 r9 : c8901000 r8 : c0b4a53c [ 10.320314] r7 : c2208920 r6 : 00000000 r5 : 00000000 r4 : 00004000 [ 10.326855] r3 : d6df6188 r2 : c4927000 r1 : c8adc300 r0 : c22069dc [ 10.333398] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user [ 10.340547] Control: 30c5387d Table: 04ac4c80 DAC: fffffffd [ 10.346307] Process sed (pid: 1886, stack limit = 0x(ptrval)) [ 10.352066] Stack: (0xc1d33cd0 to 0xc1d34000) [ 10.356434] 3cc0: c8adc300 c4927000 c1d33d04 c1d33ce8 [ 10.364631] 3ce0: c0b4a65c c0a579a4 c1d33d24 c2208920 c1d33d24 00000000 c1d33d5c c1d33d08 [ 10.372827] 3d00: c0a0b38c c0b4a548 c021e070 c2204cc8 00000000 c89015c0 04b87700 c89015c0 [ 10.381023] 3d20: c2208920 c1d33d24 c1d33d24 00976ec2 04b87700 c8adc300 c89015c0 00000000 [ 10.389218] 3d40: c1d33d74 c1d32000 00000000 c230742c c1d33dac c1d33d60 c0a0b5c0 c0a0b180 [ 10.397414] 3d60: 00000000 c2204cc8 00000000 c1d33d6c c1d33d6c c1d33d80 c029daf8 00976ec2 [ 10.405610] 3d80: 00000800 c8901540 c89015c0 c8901540 00000000 00000001 0000016c 00000162 [ 10.413805] 3da0: c1d33dc4 c1d33db0 c0a0b7fc c0a0b3b8 00000000 c8adc300 c1d33dfc c1d33dc8 [ 10.422001] 3dc0: c0a0c660 c0a0b7e4 c8901540 c8adc300 c1d33dfc c1d33de0 c8901540 c8adc300 [ 10.430196] 3de0: 0000015e c8901000 00000001 0000016c c1d33e74 c1d33e00 c083df00 c0a0c4fc [ 10.438391] 3e00: 0000012c c22b0f14 c1d33e4c c1d33e18 c0fbd9b8 c0fbd9cc c0fbd9e0 c0fbd98c [ 10.446586] 3e20: 00000001 00000040 c8901500 00000001 00000000 00000000 00000000 00000000 [ 10.454780] 3e40: 00000000 00000000 c02f65a0 c8901540 00000001 00000040 c22b07e4 0000012c [ 10.462975] 3e60: d1003000 fffb942f c1d33edc c1d33e78 c0a0c94c c083dafc d051ad80 c2204cc8 [ 10.471170] 3e80: c2204cf0 c1d32000 c22b40b0 0e4a4000 c2076d80 c2203d00 c022bc70 c1d33e9c [ 10.479365] 3ea0: c1d33e9c c1d33ea4 c1d33ea4 00976ec2 c02f65a0 c220308c 00000003 c1d32000 [ 10.487561] 3ec0: c22b07e4 00000100 d1003000 00000008 c1d33f44 c1d33ee0 c020238c c0a0c6cc [ 10.495755] 3ee0: c1d33f14 c1d33ef0 00000001 00400000 c2203d00 fffb942f c206b2e4 c2076040 [ 10.503950] 3f00: c2076040 0000000a c2203080 c206b358 c1d33ee0 00000004 c90c9500 ffffe000 [ 10.512145] 3f20: 00000000 00000000 00000001 c9019000 d1003000 00000000 c1d33f5c c1d33f48 [ 10.520339] 3f40: c022bc70 c0202264 c2075fbc 00000000 c1d33f84 c1d33f60 c027feb4 c022bb88 [ 10.528535] 3f60: c226d668 c220565c d100200c c1d33fb0 d1002000 d1003000 c1d33fac c1d33f88 [ 10.536730] 3f80: c0202214 c027fe50 b6f1e0b6 000f0030 ffffffff 30c5387d 30c5387d 00000000 [ 10.544924] 3fa0: 00000000 c1d33fb0 c0201d8c c02021c4 b6ec1778 b6f32094 62632e73 62740000 [ 10.553120] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 00000000 00000000 00000006 [ 10.561316] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 ffffffff 00000000 00000000 [ 10.569508] Backtrace: [ 10.571970] [<c0a57998>] (gro_cells_receive) from [<c0b4a65c>] (dsa_switch_rcv+0x120/0x1d4) [ 10.580338] r5:c4927000 r4:c8adc300 [ 10.583929] [<c0b4a53c>] (dsa_switch_rcv) from [<c0a0b38c>] (__netif_receive_skb_list_core+0x218/0x238) [ 10.593343] r7:00000000 r6:c1d33d24 r5:c2208920 r4:c1d33d24 [ 10.599017] [<c0a0b174>] (__netif_receive_skb_list_core) from [<c0a0b5c0>] (netif_receive_skb_list_internal+0x214/0x2dc) [ 10.609909] r10:c230742c r9:00000000 r8:c1d32000 r7:c1d33d74 r6:00000000 r5:c89015c0 [ 10.617755] r4:c8adc300 [ 10.620300] [<c0a0b3ac>] (netif_receive_skb_list_internal) from [<c0a0b7fc>] (gro_normal_list.part.42+0x24/0x38) [ 10.630496] r10:00000162 r9:0000016c r8:00000001 r7:00000000 r6:c8901540 r5:c89015c0 [ 10.638342] r4:c8901540 [ 10.640885] [<c0a0b7d8>] (gro_normal_list.part.42) from [<c0a0c660>] (napi_complete_done+0x170/0x1d0) [ 10.650123] r5:c8adc300 r4:00000000 [ 10.653712] [<c0a0c4f0>] (napi_complete_done) from [<c083df00>] (bcm_sysport_poll+0x410/0x4b4) [ 10.662343] r9:0000016c r8:00000001 r7:c8901000 r6:0000015e r5:c8adc300 r4:c8901540 [ 10.670104] [<c083daf0>] (bcm_sysport_poll) from [<c0a0c94c>] (net_rx_action+0x28c/0x44c) [ 10.678301] r10:fffb942f r9:d1003000 r8:0000012c r7:c22b07e4 r6:00000040 r5:00000001 [ 10.686146] r4:c8901540 [ 10.688693] [<c0a0c6c0>] (net_rx_action) from [<c020238c>] (__do_softirq+0x134/0x414) [ 10.696542] r10:00000008 r9:d1003000 r8:00000100 r7:c22b07e4 r6:c1d32000 r5:00000003 [ 10.704387] r4:c220308c [ 10.706931] [<c0202258>] (__do_softirq) from [<c022bc70>] (irq_exit+0xf4/0x100) [ 10.714257] r10:00000000 r9:d1003000 r8:c9019000 r7:00000001 r6:00000000 r5:00000000 [ 10.722104] r4:ffffe000 [ 10.724650] [<c022bb7c>] (irq_exit) from [<c027feb4>] (__handle_domain_irq+0x70/0xc4) [ 10.732497] r5:00000000 r4:c2075fbc [ 10.736083] [<c027fe44>] (__handle_domain_irq) from [<c0202214>] (gic_handle_irq+0x5c/0xa0) [ 10.744453] r9:d1003000 r8:d1002000 r7:c1d33fb0 r6:d100200c r5:c220565c r4:c226d668 [ 10.752215] [<c02021b8>] (gic_handle_irq) from [<c0201d8c>] (__irq_usr+0x4c/0x60) [ 10.759713] Exception stack(0xc1d33fb0 to 0xc1d33ff8) [ 10.764776] 3fa0: b6ec1778 b6f32094 62632e73 62740000 [ 10.772973] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 00000000 00000000 00000006 [ 10.781167] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 ffffffff [ 10.787797] r9:00000000 r8:30c5387d r7:30c5387d r6:ffffffff r5:000f0030 r4:b6f1e0b6 [ 10.795559] Code: e30609dc e34c0220 e083300c e590c000 (e5930008) [ 10.801670] ---[ end trace 97c3942fa73eff4c ]--- [ 10.806300] Kernel panic - not syncing: Fatal exception in interrupt [ 10.812678] CPU2: stopping [ 10.815403] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 10.824641] Hardware name: Broadcom STB (Flattened Device Tree) [ 10.830573] Backtrace: [ 10.833036] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 10.840624] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 [ 10.846302] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 10.853546] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 10.860960] r10:00000000 r9:c2205400 r8:00000000 r7:00000002 r6:c22b0744 r5:00000004 [ 10.868807] r4:c22ce308 r3:00976ec2 [ 10.872395] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 10.879983] r10:00000000 r9:d1003000 r8:d1002000 r7:c9131f18 r6:d100200c r5:c220565c [ 10.887829] r4:c226d668 [ 10.890373] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 10.897871] Exception stack(0xc9131f18 to 0xc9131f60) [ 10.902934] 1f00: c020a47c 00000000 [ 10.911131] 1f20: 0e4ca000 600c0093 c9130000 c2204cf0 c2204d34 00000004 00000000 c20757b0 [ 10.919327] 1f40: 00000000 c9131f74 c9130000 c9131f68 00000000 c020a480 600c0013 ffffffff [ 10.927523] r9:c9130000 r8:00000000 r7:c9131f4c r6:ffffffff r5:600c0013 r4:c020a480 [ 10.935291] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 10.943403] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 10.951164] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 10.958752] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000002 [ 10.966598] r4:0000008a [ 10.969142] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 10.978210] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 10.985447] r5:00000000 r4:090eaa40 [ 10.989032] CPU3: stopping [ 10.991754] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 11.000992] Hardware name: Broadcom STB (Flattened Device Tree) [ 11.006924] Backtrace: [ 11.009384] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 11.016972] r7:c22a86d0 r6:00000000 r5:600f0193 r4:c22a86d0 [ 11.022649] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 11.029892] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 11.037307] r10:00000000 r9:c2205400 r8:00000000 r7:00000003 r6:c22b0744 r5:00000004 [ 11.045154] r4:c22ce308 r3:00976ec2 [ 11.048741] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 11.056329] r10:00000000 r9:d1003000 r8:d1002000 r7:c9133f18 r6:d100200c r5:c220565c [ 11.064176] r4:c226d668 [ 11.066719] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 11.074216] Exception stack(0xc9133f18 to 0xc9133f60) [ 11.079280] 3f00: c020a47c 00000000 [ 11.087476] 3f20: 0e4dd000 600f0093 c9132000 c2204cf0 c2204d34 00000008 00000000 c20757b0 [ 11.095671] 3f40: 00000000 c9133f74 c9132000 c9133f68 00000000 c020a480 600f0013 ffffffff [ 11.103867] r9:c9132000 r8:00000000 r7:c9133f4c r6:ffffffff r5:600f0013 r4:c020a480 [ 11.111633] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 11.119744] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 11.127506] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 11.135093] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000003 [ 11.142939] r4:0000008a [ 11.145484] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 11.154550] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 11.161788] r5:00000000 r4:090eaa40 [ 11.165372] CPU1: stopping [ 11.168094] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 11.177332] Hardware name: Broadcom STB (Flattened Device Tree) [ 11.183264] Backtrace: [ 11.185723] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 11.193311] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 [ 11.198988] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 11.206230] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 11.213644] r10:00000000 r9:c2205400 r8:00000000 r7:00000001 r6:c22b0744 r5:00000004 [ 11.221491] r4:c22ce308 r3:00976ec2 [ 11.225078] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 11.232666] r10:00000000 r9:d1003000 r8:d1002000 r7:c912ff18 r6:d100200c r5:c220565c [ 11.240512] r4:c226d668 [ 11.243054] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 11.250553] Exception stack(0xc912ff18 to 0xc912ff60) [ 11.255617] ff00: c020a47c 00000000 [ 11.263814] ff20: 0e4b7000 600c0093 c912e000 c2204cf0 c2204d34 00000002 00000000 c20757b0 [ 11.272010] ff40: 00000000 c912ff74 c912e000 c912ff68 00000000 c020a480 600c0013 ffffffff [ 11.280206] r9:c912e000 r8:00000000 r7:c912ff4c r6:ffffffff r5:600c0013 r4:c020a480 [ 11.287970] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 11.296081] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 11.303842] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 11.311430] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000001 [ 11.319275] r4:0000008a [ 11.321819] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 11.330886] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 11.338124] r5:00000000 r4:090eaa40 [ 11.341729] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---
it is not marked as fixing anything so I wonder how it landed in stable?
It was requested to be merged.
OK.
Do you have the same crash on newer kernels with this commit in it (like Linus's tree?)
There are no crashes with 5.10 or upstream otherwise it would have been noticed earlier, I have not kept track of which additional changes/fixes we may need but I would suggest we drop this one for now.
Alexander, do you want me to test additional patches if your change somehow must be included in an upcoming 5.4? I thought the platform you are working is still not upstream, so who is going to benefit from this performance improvement? -- Florian
On Thu, Mar 11, 2021 at 09:23:56AM -0800, Florian Fainelli wrote:
On 3/11/21 5:08 AM, Greg KH wrote:
On Wed, Mar 10, 2021 at 08:19:45PM -0800, Florian Fainelli wrote:
+Alex,
On 3/10/2021 5:24 AM, gregkh@linuxfoundation.org wrote:
From: Greg Kroah-Hartman gregkh@linuxfoundation.org
This is the start of the stable review cycle for the 5.4.105 release. There are 24 patches in this series, all will be posted as a response to this one. If anyone has any issues with these being applied, please let me know.
Responses should be made by Fri, 12 Mar 2021 13:23:09 +0000. Anything received after that time might be too late.
The whole patch series can be found in one patch at: https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/patch-5.4.105-rc1... or in the git tree and branch at: git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-5.4.y and the diffstat can be found below.
thanks,
greg k-h
I believe you need to drop "net: dsa: add GRO support via gro_cells" as it causes the following kernel panic on a DSA-enabled platform:
Configuring rgmii_2 interface [ 10.170527] brcm-sf2 f0b00000.ethernet_switch rgmii_2: configuring for fixed/rgmii-txid link mode [ 10.179597] 8021q: adding VLAN 0 to HW filter on device rgmii_2 [ 10.185608] brcm-sf2 f0b00000.ethernet_switch rgmii_2: Link is Up - 1Gbps/Full - flow control off [ 10.198631] IPv6: ADDRCONF(NETDEV_CHANGE): rgmii_2: link becomes ready Configuring sit0 interface [ 10.254346] 8<--- cut here --- [ 10.257438] Unable to handle kernel paging request at virtual address d6df6190 [ 10.264685] pgd = (ptrval) [ 10.267411] [d6df6190] *pgd=80000000007003, *pmd=00000000 [ 10.272846] Internal error: Oops: 206 [#1] SMP ARM [ 10.277661] Modules linked in: [ 10.280739] CPU: 0 PID: 1886 Comm: sed Not tainted 5.4.105-1.0pre-geff642e2af2b #4 [ 10.288337] Hardware name: Broadcom STB (Flattened Device Tree) [ 10.294292] PC is at gro_cells_receive+0x90/0x11c [ 10.299020] LR is at dsa_switch_rcv+0x120/0x1d4 [ 10.303562] pc : [<c0a57a28>] lr : [<c0b4a65c>] psr: 600f0113 [ 10.309841] sp : c1d33cd0 ip : 000003e8 fp : c1d33ce4 [ 10.315078] r10: c8901000 r9 : c8901000 r8 : c0b4a53c [ 10.320314] r7 : c2208920 r6 : 00000000 r5 : 00000000 r4 : 00004000 [ 10.326855] r3 : d6df6188 r2 : c4927000 r1 : c8adc300 r0 : c22069dc [ 10.333398] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user [ 10.340547] Control: 30c5387d Table: 04ac4c80 DAC: fffffffd [ 10.346307] Process sed (pid: 1886, stack limit = 0x(ptrval)) [ 10.352066] Stack: (0xc1d33cd0 to 0xc1d34000) [ 10.356434] 3cc0: c8adc300 c4927000 c1d33d04 c1d33ce8 [ 10.364631] 3ce0: c0b4a65c c0a579a4 c1d33d24 c2208920 c1d33d24 00000000 c1d33d5c c1d33d08 [ 10.372827] 3d00: c0a0b38c c0b4a548 c021e070 c2204cc8 00000000 c89015c0 04b87700 c89015c0 [ 10.381023] 3d20: c2208920 c1d33d24 c1d33d24 00976ec2 04b87700 c8adc300 c89015c0 00000000 [ 10.389218] 3d40: c1d33d74 c1d32000 00000000 c230742c c1d33dac c1d33d60 c0a0b5c0 c0a0b180 [ 10.397414] 3d60: 00000000 c2204cc8 00000000 c1d33d6c c1d33d6c c1d33d80 c029daf8 00976ec2 [ 10.405610] 3d80: 00000800 c8901540 c89015c0 c8901540 00000000 00000001 0000016c 00000162 [ 10.413805] 3da0: c1d33dc4 c1d33db0 c0a0b7fc c0a0b3b8 00000000 c8adc300 c1d33dfc c1d33dc8 [ 10.422001] 3dc0: c0a0c660 c0a0b7e4 c8901540 c8adc300 c1d33dfc c1d33de0 c8901540 c8adc300 [ 10.430196] 3de0: 0000015e c8901000 00000001 0000016c c1d33e74 c1d33e00 c083df00 c0a0c4fc [ 10.438391] 3e00: 0000012c c22b0f14 c1d33e4c c1d33e18 c0fbd9b8 c0fbd9cc c0fbd9e0 c0fbd98c [ 10.446586] 3e20: 00000001 00000040 c8901500 00000001 00000000 00000000 00000000 00000000 [ 10.454780] 3e40: 00000000 00000000 c02f65a0 c8901540 00000001 00000040 c22b07e4 0000012c [ 10.462975] 3e60: d1003000 fffb942f c1d33edc c1d33e78 c0a0c94c c083dafc d051ad80 c2204cc8 [ 10.471170] 3e80: c2204cf0 c1d32000 c22b40b0 0e4a4000 c2076d80 c2203d00 c022bc70 c1d33e9c [ 10.479365] 3ea0: c1d33e9c c1d33ea4 c1d33ea4 00976ec2 c02f65a0 c220308c 00000003 c1d32000 [ 10.487561] 3ec0: c22b07e4 00000100 d1003000 00000008 c1d33f44 c1d33ee0 c020238c c0a0c6cc [ 10.495755] 3ee0: c1d33f14 c1d33ef0 00000001 00400000 c2203d00 fffb942f c206b2e4 c2076040 [ 10.503950] 3f00: c2076040 0000000a c2203080 c206b358 c1d33ee0 00000004 c90c9500 ffffe000 [ 10.512145] 3f20: 00000000 00000000 00000001 c9019000 d1003000 00000000 c1d33f5c c1d33f48 [ 10.520339] 3f40: c022bc70 c0202264 c2075fbc 00000000 c1d33f84 c1d33f60 c027feb4 c022bb88 [ 10.528535] 3f60: c226d668 c220565c d100200c c1d33fb0 d1002000 d1003000 c1d33fac c1d33f88 [ 10.536730] 3f80: c0202214 c027fe50 b6f1e0b6 000f0030 ffffffff 30c5387d 30c5387d 00000000 [ 10.544924] 3fa0: 00000000 c1d33fb0 c0201d8c c02021c4 b6ec1778 b6f32094 62632e73 62740000 [ 10.553120] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 00000000 00000000 00000006 [ 10.561316] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 ffffffff 00000000 00000000 [ 10.569508] Backtrace: [ 10.571970] [<c0a57998>] (gro_cells_receive) from [<c0b4a65c>] (dsa_switch_rcv+0x120/0x1d4) [ 10.580338] r5:c4927000 r4:c8adc300 [ 10.583929] [<c0b4a53c>] (dsa_switch_rcv) from [<c0a0b38c>] (__netif_receive_skb_list_core+0x218/0x238) [ 10.593343] r7:00000000 r6:c1d33d24 r5:c2208920 r4:c1d33d24 [ 10.599017] [<c0a0b174>] (__netif_receive_skb_list_core) from [<c0a0b5c0>] (netif_receive_skb_list_internal+0x214/0x2dc) [ 10.609909] r10:c230742c r9:00000000 r8:c1d32000 r7:c1d33d74 r6:00000000 r5:c89015c0 [ 10.617755] r4:c8adc300 [ 10.620300] [<c0a0b3ac>] (netif_receive_skb_list_internal) from [<c0a0b7fc>] (gro_normal_list.part.42+0x24/0x38) [ 10.630496] r10:00000162 r9:0000016c r8:00000001 r7:00000000 r6:c8901540 r5:c89015c0 [ 10.638342] r4:c8901540 [ 10.640885] [<c0a0b7d8>] (gro_normal_list.part.42) from [<c0a0c660>] (napi_complete_done+0x170/0x1d0) [ 10.650123] r5:c8adc300 r4:00000000 [ 10.653712] [<c0a0c4f0>] (napi_complete_done) from [<c083df00>] (bcm_sysport_poll+0x410/0x4b4) [ 10.662343] r9:0000016c r8:00000001 r7:c8901000 r6:0000015e r5:c8adc300 r4:c8901540 [ 10.670104] [<c083daf0>] (bcm_sysport_poll) from [<c0a0c94c>] (net_rx_action+0x28c/0x44c) [ 10.678301] r10:fffb942f r9:d1003000 r8:0000012c r7:c22b07e4 r6:00000040 r5:00000001 [ 10.686146] r4:c8901540 [ 10.688693] [<c0a0c6c0>] (net_rx_action) from [<c020238c>] (__do_softirq+0x134/0x414) [ 10.696542] r10:00000008 r9:d1003000 r8:00000100 r7:c22b07e4 r6:c1d32000 r5:00000003 [ 10.704387] r4:c220308c [ 10.706931] [<c0202258>] (__do_softirq) from [<c022bc70>] (irq_exit+0xf4/0x100) [ 10.714257] r10:00000000 r9:d1003000 r8:c9019000 r7:00000001 r6:00000000 r5:00000000 [ 10.722104] r4:ffffe000 [ 10.724650] [<c022bb7c>] (irq_exit) from [<c027feb4>] (__handle_domain_irq+0x70/0xc4) [ 10.732497] r5:00000000 r4:c2075fbc [ 10.736083] [<c027fe44>] (__handle_domain_irq) from [<c0202214>] (gic_handle_irq+0x5c/0xa0) [ 10.744453] r9:d1003000 r8:d1002000 r7:c1d33fb0 r6:d100200c r5:c220565c r4:c226d668 [ 10.752215] [<c02021b8>] (gic_handle_irq) from [<c0201d8c>] (__irq_usr+0x4c/0x60) [ 10.759713] Exception stack(0xc1d33fb0 to 0xc1d33ff8) [ 10.764776] 3fa0: b6ec1778 b6f32094 62632e73 62740000 [ 10.772973] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 00000000 00000000 00000006 [ 10.781167] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 ffffffff [ 10.787797] r9:00000000 r8:30c5387d r7:30c5387d r6:ffffffff r5:000f0030 r4:b6f1e0b6 [ 10.795559] Code: e30609dc e34c0220 e083300c e590c000 (e5930008) [ 10.801670] ---[ end trace 97c3942fa73eff4c ]--- [ 10.806300] Kernel panic - not syncing: Fatal exception in interrupt [ 10.812678] CPU2: stopping [ 10.815403] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 10.824641] Hardware name: Broadcom STB (Flattened Device Tree) [ 10.830573] Backtrace: [ 10.833036] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 10.840624] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 [ 10.846302] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 10.853546] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 10.860960] r10:00000000 r9:c2205400 r8:00000000 r7:00000002 r6:c22b0744 r5:00000004 [ 10.868807] r4:c22ce308 r3:00976ec2 [ 10.872395] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 10.879983] r10:00000000 r9:d1003000 r8:d1002000 r7:c9131f18 r6:d100200c r5:c220565c [ 10.887829] r4:c226d668 [ 10.890373] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 10.897871] Exception stack(0xc9131f18 to 0xc9131f60) [ 10.902934] 1f00: c020a47c 00000000 [ 10.911131] 1f20: 0e4ca000 600c0093 c9130000 c2204cf0 c2204d34 00000004 00000000 c20757b0 [ 10.919327] 1f40: 00000000 c9131f74 c9130000 c9131f68 00000000 c020a480 600c0013 ffffffff [ 10.927523] r9:c9130000 r8:00000000 r7:c9131f4c r6:ffffffff r5:600c0013 r4:c020a480 [ 10.935291] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 10.943403] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 10.951164] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 10.958752] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000002 [ 10.966598] r4:0000008a [ 10.969142] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 10.978210] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 10.985447] r5:00000000 r4:090eaa40 [ 10.989032] CPU3: stopping [ 10.991754] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 11.000992] Hardware name: Broadcom STB (Flattened Device Tree) [ 11.006924] Backtrace: [ 11.009384] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 11.016972] r7:c22a86d0 r6:00000000 r5:600f0193 r4:c22a86d0 [ 11.022649] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 11.029892] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 11.037307] r10:00000000 r9:c2205400 r8:00000000 r7:00000003 r6:c22b0744 r5:00000004 [ 11.045154] r4:c22ce308 r3:00976ec2 [ 11.048741] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 11.056329] r10:00000000 r9:d1003000 r8:d1002000 r7:c9133f18 r6:d100200c r5:c220565c [ 11.064176] r4:c226d668 [ 11.066719] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 11.074216] Exception stack(0xc9133f18 to 0xc9133f60) [ 11.079280] 3f00: c020a47c 00000000 [ 11.087476] 3f20: 0e4dd000 600f0093 c9132000 c2204cf0 c2204d34 00000008 00000000 c20757b0 [ 11.095671] 3f40: 00000000 c9133f74 c9132000 c9133f68 00000000 c020a480 600f0013 ffffffff [ 11.103867] r9:c9132000 r8:00000000 r7:c9133f4c r6:ffffffff r5:600f0013 r4:c020a480 [ 11.111633] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 11.119744] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 11.127506] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 11.135093] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000003 [ 11.142939] r4:0000008a [ 11.145484] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 11.154550] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 11.161788] r5:00000000 r4:090eaa40 [ 11.165372] CPU1: stopping [ 11.168094] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 11.177332] Hardware name: Broadcom STB (Flattened Device Tree) [ 11.183264] Backtrace: [ 11.185723] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 11.193311] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 [ 11.198988] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 11.206230] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 11.213644] r10:00000000 r9:c2205400 r8:00000000 r7:00000001 r6:c22b0744 r5:00000004 [ 11.221491] r4:c22ce308 r3:00976ec2 [ 11.225078] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 11.232666] r10:00000000 r9:d1003000 r8:d1002000 r7:c912ff18 r6:d100200c r5:c220565c [ 11.240512] r4:c226d668 [ 11.243054] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 11.250553] Exception stack(0xc912ff18 to 0xc912ff60) [ 11.255617] ff00: c020a47c 00000000 [ 11.263814] ff20: 0e4b7000 600c0093 c912e000 c2204cf0 c2204d34 00000002 00000000 c20757b0 [ 11.272010] ff40: 00000000 c912ff74 c912e000 c912ff68 00000000 c020a480 600c0013 ffffffff [ 11.280206] r9:c912e000 r8:00000000 r7:c912ff4c r6:ffffffff r5:600c0013 r4:c020a480 [ 11.287970] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 11.296081] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 11.303842] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 11.311430] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000001 [ 11.319275] r4:0000008a [ 11.321819] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 11.330886] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 11.338124] r5:00000000 r4:090eaa40 [ 11.341729] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---
it is not marked as fixing anything so I wonder how it landed in stable?
It was requested to be merged.
OK.
Do you have the same crash on newer kernels with this commit in it (like Linus's tree?)
There are no crashes with 5.10 or upstream otherwise it would have been noticed earlier, I have not kept track of which additional changes/fixes we may need but I would suggest we drop this one for now.
Alexander, do you want me to test additional patches if your change somehow must be included in an upcoming 5.4? I thought the platform you are working is still not upstream, so who is going to benefit from this performance improvement?
Is 4.19 also failing for you now?
thanks,
greg k-h
On 3/11/21 9:40 AM, Greg KH wrote:
On Thu, Mar 11, 2021 at 09:23:56AM -0800, Florian Fainelli wrote:
On 3/11/21 5:08 AM, Greg KH wrote:
On Wed, Mar 10, 2021 at 08:19:45PM -0800, Florian Fainelli wrote:
+Alex,
On 3/10/2021 5:24 AM, gregkh@linuxfoundation.org wrote:
From: Greg Kroah-Hartman gregkh@linuxfoundation.org
This is the start of the stable review cycle for the 5.4.105 release. There are 24 patches in this series, all will be posted as a response to this one. If anyone has any issues with these being applied, please let me know.
Responses should be made by Fri, 12 Mar 2021 13:23:09 +0000. Anything received after that time might be too late.
The whole patch series can be found in one patch at: https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/patch-5.4.105-rc1... or in the git tree and branch at: git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-5.4.y and the diffstat can be found below.
thanks,
greg k-h
I believe you need to drop "net: dsa: add GRO support via gro_cells" as it causes the following kernel panic on a DSA-enabled platform:
Configuring rgmii_2 interface [ 10.170527] brcm-sf2 f0b00000.ethernet_switch rgmii_2: configuring for fixed/rgmii-txid link mode [ 10.179597] 8021q: adding VLAN 0 to HW filter on device rgmii_2 [ 10.185608] brcm-sf2 f0b00000.ethernet_switch rgmii_2: Link is Up - 1Gbps/Full - flow control off [ 10.198631] IPv6: ADDRCONF(NETDEV_CHANGE): rgmii_2: link becomes ready Configuring sit0 interface [ 10.254346] 8<--- cut here --- [ 10.257438] Unable to handle kernel paging request at virtual address d6df6190 [ 10.264685] pgd = (ptrval) [ 10.267411] [d6df6190] *pgd=80000000007003, *pmd=00000000 [ 10.272846] Internal error: Oops: 206 [#1] SMP ARM [ 10.277661] Modules linked in: [ 10.280739] CPU: 0 PID: 1886 Comm: sed Not tainted 5.4.105-1.0pre-geff642e2af2b #4 [ 10.288337] Hardware name: Broadcom STB (Flattened Device Tree) [ 10.294292] PC is at gro_cells_receive+0x90/0x11c [ 10.299020] LR is at dsa_switch_rcv+0x120/0x1d4 [ 10.303562] pc : [<c0a57a28>] lr : [<c0b4a65c>] psr: 600f0113 [ 10.309841] sp : c1d33cd0 ip : 000003e8 fp : c1d33ce4 [ 10.315078] r10: c8901000 r9 : c8901000 r8 : c0b4a53c [ 10.320314] r7 : c2208920 r6 : 00000000 r5 : 00000000 r4 : 00004000 [ 10.326855] r3 : d6df6188 r2 : c4927000 r1 : c8adc300 r0 : c22069dc [ 10.333398] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user [ 10.340547] Control: 30c5387d Table: 04ac4c80 DAC: fffffffd [ 10.346307] Process sed (pid: 1886, stack limit = 0x(ptrval)) [ 10.352066] Stack: (0xc1d33cd0 to 0xc1d34000) [ 10.356434] 3cc0: c8adc300 c4927000 c1d33d04 c1d33ce8 [ 10.364631] 3ce0: c0b4a65c c0a579a4 c1d33d24 c2208920 c1d33d24 00000000 c1d33d5c c1d33d08 [ 10.372827] 3d00: c0a0b38c c0b4a548 c021e070 c2204cc8 00000000 c89015c0 04b87700 c89015c0 [ 10.381023] 3d20: c2208920 c1d33d24 c1d33d24 00976ec2 04b87700 c8adc300 c89015c0 00000000 [ 10.389218] 3d40: c1d33d74 c1d32000 00000000 c230742c c1d33dac c1d33d60 c0a0b5c0 c0a0b180 [ 10.397414] 3d60: 00000000 c2204cc8 00000000 c1d33d6c c1d33d6c c1d33d80 c029daf8 00976ec2 [ 10.405610] 3d80: 00000800 c8901540 c89015c0 c8901540 00000000 00000001 0000016c 00000162 [ 10.413805] 3da0: c1d33dc4 c1d33db0 c0a0b7fc c0a0b3b8 00000000 c8adc300 c1d33dfc c1d33dc8 [ 10.422001] 3dc0: c0a0c660 c0a0b7e4 c8901540 c8adc300 c1d33dfc c1d33de0 c8901540 c8adc300 [ 10.430196] 3de0: 0000015e c8901000 00000001 0000016c c1d33e74 c1d33e00 c083df00 c0a0c4fc [ 10.438391] 3e00: 0000012c c22b0f14 c1d33e4c c1d33e18 c0fbd9b8 c0fbd9cc c0fbd9e0 c0fbd98c [ 10.446586] 3e20: 00000001 00000040 c8901500 00000001 00000000 00000000 00000000 00000000 [ 10.454780] 3e40: 00000000 00000000 c02f65a0 c8901540 00000001 00000040 c22b07e4 0000012c [ 10.462975] 3e60: d1003000 fffb942f c1d33edc c1d33e78 c0a0c94c c083dafc d051ad80 c2204cc8 [ 10.471170] 3e80: c2204cf0 c1d32000 c22b40b0 0e4a4000 c2076d80 c2203d00 c022bc70 c1d33e9c [ 10.479365] 3ea0: c1d33e9c c1d33ea4 c1d33ea4 00976ec2 c02f65a0 c220308c 00000003 c1d32000 [ 10.487561] 3ec0: c22b07e4 00000100 d1003000 00000008 c1d33f44 c1d33ee0 c020238c c0a0c6cc [ 10.495755] 3ee0: c1d33f14 c1d33ef0 00000001 00400000 c2203d00 fffb942f c206b2e4 c2076040 [ 10.503950] 3f00: c2076040 0000000a c2203080 c206b358 c1d33ee0 00000004 c90c9500 ffffe000 [ 10.512145] 3f20: 00000000 00000000 00000001 c9019000 d1003000 00000000 c1d33f5c c1d33f48 [ 10.520339] 3f40: c022bc70 c0202264 c2075fbc 00000000 c1d33f84 c1d33f60 c027feb4 c022bb88 [ 10.528535] 3f60: c226d668 c220565c d100200c c1d33fb0 d1002000 d1003000 c1d33fac c1d33f88 [ 10.536730] 3f80: c0202214 c027fe50 b6f1e0b6 000f0030 ffffffff 30c5387d 30c5387d 00000000 [ 10.544924] 3fa0: 00000000 c1d33fb0 c0201d8c c02021c4 b6ec1778 b6f32094 62632e73 62740000 [ 10.553120] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 00000000 00000000 00000006 [ 10.561316] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 ffffffff 00000000 00000000 [ 10.569508] Backtrace: [ 10.571970] [<c0a57998>] (gro_cells_receive) from [<c0b4a65c>] (dsa_switch_rcv+0x120/0x1d4) [ 10.580338] r5:c4927000 r4:c8adc300 [ 10.583929] [<c0b4a53c>] (dsa_switch_rcv) from [<c0a0b38c>] (__netif_receive_skb_list_core+0x218/0x238) [ 10.593343] r7:00000000 r6:c1d33d24 r5:c2208920 r4:c1d33d24 [ 10.599017] [<c0a0b174>] (__netif_receive_skb_list_core) from [<c0a0b5c0>] (netif_receive_skb_list_internal+0x214/0x2dc) [ 10.609909] r10:c230742c r9:00000000 r8:c1d32000 r7:c1d33d74 r6:00000000 r5:c89015c0 [ 10.617755] r4:c8adc300 [ 10.620300] [<c0a0b3ac>] (netif_receive_skb_list_internal) from [<c0a0b7fc>] (gro_normal_list.part.42+0x24/0x38) [ 10.630496] r10:00000162 r9:0000016c r8:00000001 r7:00000000 r6:c8901540 r5:c89015c0 [ 10.638342] r4:c8901540 [ 10.640885] [<c0a0b7d8>] (gro_normal_list.part.42) from [<c0a0c660>] (napi_complete_done+0x170/0x1d0) [ 10.650123] r5:c8adc300 r4:00000000 [ 10.653712] [<c0a0c4f0>] (napi_complete_done) from [<c083df00>] (bcm_sysport_poll+0x410/0x4b4) [ 10.662343] r9:0000016c r8:00000001 r7:c8901000 r6:0000015e r5:c8adc300 r4:c8901540 [ 10.670104] [<c083daf0>] (bcm_sysport_poll) from [<c0a0c94c>] (net_rx_action+0x28c/0x44c) [ 10.678301] r10:fffb942f r9:d1003000 r8:0000012c r7:c22b07e4 r6:00000040 r5:00000001 [ 10.686146] r4:c8901540 [ 10.688693] [<c0a0c6c0>] (net_rx_action) from [<c020238c>] (__do_softirq+0x134/0x414) [ 10.696542] r10:00000008 r9:d1003000 r8:00000100 r7:c22b07e4 r6:c1d32000 r5:00000003 [ 10.704387] r4:c220308c [ 10.706931] [<c0202258>] (__do_softirq) from [<c022bc70>] (irq_exit+0xf4/0x100) [ 10.714257] r10:00000000 r9:d1003000 r8:c9019000 r7:00000001 r6:00000000 r5:00000000 [ 10.722104] r4:ffffe000 [ 10.724650] [<c022bb7c>] (irq_exit) from [<c027feb4>] (__handle_domain_irq+0x70/0xc4) [ 10.732497] r5:00000000 r4:c2075fbc [ 10.736083] [<c027fe44>] (__handle_domain_irq) from [<c0202214>] (gic_handle_irq+0x5c/0xa0) [ 10.744453] r9:d1003000 r8:d1002000 r7:c1d33fb0 r6:d100200c r5:c220565c r4:c226d668 [ 10.752215] [<c02021b8>] (gic_handle_irq) from [<c0201d8c>] (__irq_usr+0x4c/0x60) [ 10.759713] Exception stack(0xc1d33fb0 to 0xc1d33ff8) [ 10.764776] 3fa0: b6ec1778 b6f32094 62632e73 62740000 [ 10.772973] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 00000000 00000000 00000006 [ 10.781167] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 ffffffff [ 10.787797] r9:00000000 r8:30c5387d r7:30c5387d r6:ffffffff r5:000f0030 r4:b6f1e0b6 [ 10.795559] Code: e30609dc e34c0220 e083300c e590c000 (e5930008) [ 10.801670] ---[ end trace 97c3942fa73eff4c ]--- [ 10.806300] Kernel panic - not syncing: Fatal exception in interrupt [ 10.812678] CPU2: stopping [ 10.815403] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 10.824641] Hardware name: Broadcom STB (Flattened Device Tree) [ 10.830573] Backtrace: [ 10.833036] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 10.840624] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 [ 10.846302] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 10.853546] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 10.860960] r10:00000000 r9:c2205400 r8:00000000 r7:00000002 r6:c22b0744 r5:00000004 [ 10.868807] r4:c22ce308 r3:00976ec2 [ 10.872395] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 10.879983] r10:00000000 r9:d1003000 r8:d1002000 r7:c9131f18 r6:d100200c r5:c220565c [ 10.887829] r4:c226d668 [ 10.890373] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 10.897871] Exception stack(0xc9131f18 to 0xc9131f60) [ 10.902934] 1f00: c020a47c 00000000 [ 10.911131] 1f20: 0e4ca000 600c0093 c9130000 c2204cf0 c2204d34 00000004 00000000 c20757b0 [ 10.919327] 1f40: 00000000 c9131f74 c9130000 c9131f68 00000000 c020a480 600c0013 ffffffff [ 10.927523] r9:c9130000 r8:00000000 r7:c9131f4c r6:ffffffff r5:600c0013 r4:c020a480 [ 10.935291] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 10.943403] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 10.951164] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 10.958752] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000002 [ 10.966598] r4:0000008a [ 10.969142] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 10.978210] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 10.985447] r5:00000000 r4:090eaa40 [ 10.989032] CPU3: stopping [ 10.991754] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 11.000992] Hardware name: Broadcom STB (Flattened Device Tree) [ 11.006924] Backtrace: [ 11.009384] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 11.016972] r7:c22a86d0 r6:00000000 r5:600f0193 r4:c22a86d0 [ 11.022649] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 11.029892] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 11.037307] r10:00000000 r9:c2205400 r8:00000000 r7:00000003 r6:c22b0744 r5:00000004 [ 11.045154] r4:c22ce308 r3:00976ec2 [ 11.048741] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 11.056329] r10:00000000 r9:d1003000 r8:d1002000 r7:c9133f18 r6:d100200c r5:c220565c [ 11.064176] r4:c226d668 [ 11.066719] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 11.074216] Exception stack(0xc9133f18 to 0xc9133f60) [ 11.079280] 3f00: c020a47c 00000000 [ 11.087476] 3f20: 0e4dd000 600f0093 c9132000 c2204cf0 c2204d34 00000008 00000000 c20757b0 [ 11.095671] 3f40: 00000000 c9133f74 c9132000 c9133f68 00000000 c020a480 600f0013 ffffffff [ 11.103867] r9:c9132000 r8:00000000 r7:c9133f4c r6:ffffffff r5:600f0013 r4:c020a480 [ 11.111633] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 11.119744] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 11.127506] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 11.135093] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000003 [ 11.142939] r4:0000008a [ 11.145484] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 11.154550] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 11.161788] r5:00000000 r4:090eaa40 [ 11.165372] CPU1: stopping [ 11.168094] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 11.177332] Hardware name: Broadcom STB (Flattened Device Tree) [ 11.183264] Backtrace: [ 11.185723] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 11.193311] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 [ 11.198988] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 11.206230] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 11.213644] r10:00000000 r9:c2205400 r8:00000000 r7:00000001 r6:c22b0744 r5:00000004 [ 11.221491] r4:c22ce308 r3:00976ec2 [ 11.225078] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 11.232666] r10:00000000 r9:d1003000 r8:d1002000 r7:c912ff18 r6:d100200c r5:c220565c [ 11.240512] r4:c226d668 [ 11.243054] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 11.250553] Exception stack(0xc912ff18 to 0xc912ff60) [ 11.255617] ff00: c020a47c 00000000 [ 11.263814] ff20: 0e4b7000 600c0093 c912e000 c2204cf0 c2204d34 00000002 00000000 c20757b0 [ 11.272010] ff40: 00000000 c912ff74 c912e000 c912ff68 00000000 c020a480 600c0013 ffffffff [ 11.280206] r9:c912e000 r8:00000000 r7:c912ff4c r6:ffffffff r5:600c0013 r4:c020a480 [ 11.287970] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 11.296081] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 11.303842] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 11.311430] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000001 [ 11.319275] r4:0000008a [ 11.321819] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 11.330886] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 11.338124] r5:00000000 r4:090eaa40 [ 11.341729] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---
it is not marked as fixing anything so I wonder how it landed in stable?
It was requested to be merged.
OK.
Do you have the same crash on newer kernels with this commit in it (like Linus's tree?)
There are no crashes with 5.10 or upstream otherwise it would have been noticed earlier, I have not kept track of which additional changes/fixes we may need but I would suggest we drop this one for now.
Alexander, do you want me to test additional patches if your change somehow must be included in an upcoming 5.4? I thought the platform you are working is still not upstream, so who is going to benefit from this performance improvement?
Is 4.19 also failing for you now?
I cannot easily test 4.19 without applying some additional patches bringing in various ARM SCMI changes, I would suspect the same to happen. Give me a few hours and I will see if I can give you a better answer with an actual test.
From: Florian Fainelli f.fainelli@gmail.com Date: Thu, 11 Mar 2021 09:41:27 -0800
Hi Florian,
On 3/11/21 9:40 AM, Greg KH wrote:
On Thu, Mar 11, 2021 at 09:23:56AM -0800, Florian Fainelli wrote:
On 3/11/21 5:08 AM, Greg KH wrote:
On Wed, Mar 10, 2021 at 08:19:45PM -0800, Florian Fainelli wrote:
+Alex,
On 3/10/2021 5:24 AM, gregkh@linuxfoundation.org wrote:
From: Greg Kroah-Hartman gregkh@linuxfoundation.org
This is the start of the stable review cycle for the 5.4.105 release. There are 24 patches in this series, all will be posted as a response to this one. If anyone has any issues with these being applied, please let me know.
Responses should be made by Fri, 12 Mar 2021 13:23:09 +0000. Anything received after that time might be too late.
The whole patch series can be found in one patch at: https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/patch-5.4.105-rc1... or in the git tree and branch at: git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-5.4.y and the diffstat can be found below.
thanks,
greg k-h
I believe you need to drop "net: dsa: add GRO support via gro_cells" as it causes the following kernel panic on a DSA-enabled platform:
Configuring rgmii_2 interface [ 10.170527] brcm-sf2 f0b00000.ethernet_switch rgmii_2: configuring for fixed/rgmii-txid link mode [ 10.179597] 8021q: adding VLAN 0 to HW filter on device rgmii_2 [ 10.185608] brcm-sf2 f0b00000.ethernet_switch rgmii_2: Link is Up - 1Gbps/Full - flow control off [ 10.198631] IPv6: ADDRCONF(NETDEV_CHANGE): rgmii_2: link becomes ready Configuring sit0 interface [ 10.254346] 8<--- cut here --- [ 10.257438] Unable to handle kernel paging request at virtual address d6df6190 [ 10.264685] pgd = (ptrval) [ 10.267411] [d6df6190] *pgd=80000000007003, *pmd=00000000 [ 10.272846] Internal error: Oops: 206 [#1] SMP ARM [ 10.277661] Modules linked in: [ 10.280739] CPU: 0 PID: 1886 Comm: sed Not tainted 5.4.105-1.0pre-geff642e2af2b #4 [ 10.288337] Hardware name: Broadcom STB (Flattened Device Tree) [ 10.294292] PC is at gro_cells_receive+0x90/0x11c [ 10.299020] LR is at dsa_switch_rcv+0x120/0x1d4 [ 10.303562] pc : [<c0a57a28>] lr : [<c0b4a65c>] psr: 600f0113 [ 10.309841] sp : c1d33cd0 ip : 000003e8 fp : c1d33ce4 [ 10.315078] r10: c8901000 r9 : c8901000 r8 : c0b4a53c [ 10.320314] r7 : c2208920 r6 : 00000000 r5 : 00000000 r4 : 00004000 [ 10.326855] r3 : d6df6188 r2 : c4927000 r1 : c8adc300 r0 : c22069dc [ 10.333398] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user [ 10.340547] Control: 30c5387d Table: 04ac4c80 DAC: fffffffd [ 10.346307] Process sed (pid: 1886, stack limit = 0x(ptrval)) [ 10.352066] Stack: (0xc1d33cd0 to 0xc1d34000) [ 10.356434] 3cc0: c8adc300 c4927000 c1d33d04 c1d33ce8 [ 10.364631] 3ce0: c0b4a65c c0a579a4 c1d33d24 c2208920 c1d33d24 00000000 c1d33d5c c1d33d08 [ 10.372827] 3d00: c0a0b38c c0b4a548 c021e070 c2204cc8 00000000 c89015c0 04b87700 c89015c0 [ 10.381023] 3d20: c2208920 c1d33d24 c1d33d24 00976ec2 04b87700 c8adc300 c89015c0 00000000 [ 10.389218] 3d40: c1d33d74 c1d32000 00000000 c230742c c1d33dac c1d33d60 c0a0b5c0 c0a0b180 [ 10.397414] 3d60: 00000000 c2204cc8 00000000 c1d33d6c c1d33d6c c1d33d80 c029daf8 00976ec2 [ 10.405610] 3d80: 00000800 c8901540 c89015c0 c8901540 00000000 00000001 0000016c 00000162 [ 10.413805] 3da0: c1d33dc4 c1d33db0 c0a0b7fc c0a0b3b8 00000000 c8adc300 c1d33dfc c1d33dc8 [ 10.422001] 3dc0: c0a0c660 c0a0b7e4 c8901540 c8adc300 c1d33dfc c1d33de0 c8901540 c8adc300 [ 10.430196] 3de0: 0000015e c8901000 00000001 0000016c c1d33e74 c1d33e00 c083df00 c0a0c4fc [ 10.438391] 3e00: 0000012c c22b0f14 c1d33e4c c1d33e18 c0fbd9b8 c0fbd9cc c0fbd9e0 c0fbd98c [ 10.446586] 3e20: 00000001 00000040 c8901500 00000001 00000000 00000000 00000000 00000000 [ 10.454780] 3e40: 00000000 00000000 c02f65a0 c8901540 00000001 00000040 c22b07e4 0000012c [ 10.462975] 3e60: d1003000 fffb942f c1d33edc c1d33e78 c0a0c94c c083dafc d051ad80 c2204cc8 [ 10.471170] 3e80: c2204cf0 c1d32000 c22b40b0 0e4a4000 c2076d80 c2203d00 c022bc70 c1d33e9c [ 10.479365] 3ea0: c1d33e9c c1d33ea4 c1d33ea4 00976ec2 c02f65a0 c220308c 00000003 c1d32000 [ 10.487561] 3ec0: c22b07e4 00000100 d1003000 00000008 c1d33f44 c1d33ee0 c020238c c0a0c6cc [ 10.495755] 3ee0: c1d33f14 c1d33ef0 00000001 00400000 c2203d00 fffb942f c206b2e4 c2076040 [ 10.503950] 3f00: c2076040 0000000a c2203080 c206b358 c1d33ee0 00000004 c90c9500 ffffe000 [ 10.512145] 3f20: 00000000 00000000 00000001 c9019000 d1003000 00000000 c1d33f5c c1d33f48 [ 10.520339] 3f40: c022bc70 c0202264 c2075fbc 00000000 c1d33f84 c1d33f60 c027feb4 c022bb88 [ 10.528535] 3f60: c226d668 c220565c d100200c c1d33fb0 d1002000 d1003000 c1d33fac c1d33f88 [ 10.536730] 3f80: c0202214 c027fe50 b6f1e0b6 000f0030 ffffffff 30c5387d 30c5387d 00000000 [ 10.544924] 3fa0: 00000000 c1d33fb0 c0201d8c c02021c4 b6ec1778 b6f32094 62632e73 62740000 [ 10.553120] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 00000000 00000000 00000006 [ 10.561316] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 ffffffff 00000000 00000000 [ 10.569508] Backtrace: [ 10.571970] [<c0a57998>] (gro_cells_receive) from [<c0b4a65c>] (dsa_switch_rcv+0x120/0x1d4) [ 10.580338] r5:c4927000 r4:c8adc300 [ 10.583929] [<c0b4a53c>] (dsa_switch_rcv) from [<c0a0b38c>] (__netif_receive_skb_list_core+0x218/0x238) [ 10.593343] r7:00000000 r6:c1d33d24 r5:c2208920 r4:c1d33d24 [ 10.599017] [<c0a0b174>] (__netif_receive_skb_list_core) from [<c0a0b5c0>] (netif_receive_skb_list_internal+0x214/0x2dc) [ 10.609909] r10:c230742c r9:00000000 r8:c1d32000 r7:c1d33d74 r6:00000000 r5:c89015c0 [ 10.617755] r4:c8adc300 [ 10.620300] [<c0a0b3ac>] (netif_receive_skb_list_internal) from [<c0a0b7fc>] (gro_normal_list.part.42+0x24/0x38) [ 10.630496] r10:00000162 r9:0000016c r8:00000001 r7:00000000 r6:c8901540 r5:c89015c0 [ 10.638342] r4:c8901540 [ 10.640885] [<c0a0b7d8>] (gro_normal_list.part.42) from [<c0a0c660>] (napi_complete_done+0x170/0x1d0) [ 10.650123] r5:c8adc300 r4:00000000 [ 10.653712] [<c0a0c4f0>] (napi_complete_done) from [<c083df00>] (bcm_sysport_poll+0x410/0x4b4) [ 10.662343] r9:0000016c r8:00000001 r7:c8901000 r6:0000015e r5:c8adc300 r4:c8901540 [ 10.670104] [<c083daf0>] (bcm_sysport_poll) from [<c0a0c94c>] (net_rx_action+0x28c/0x44c) [ 10.678301] r10:fffb942f r9:d1003000 r8:0000012c r7:c22b07e4 r6:00000040 r5:00000001 [ 10.686146] r4:c8901540 [ 10.688693] [<c0a0c6c0>] (net_rx_action) from [<c020238c>] (__do_softirq+0x134/0x414) [ 10.696542] r10:00000008 r9:d1003000 r8:00000100 r7:c22b07e4 r6:c1d32000 r5:00000003 [ 10.704387] r4:c220308c [ 10.706931] [<c0202258>] (__do_softirq) from [<c022bc70>] (irq_exit+0xf4/0x100) [ 10.714257] r10:00000000 r9:d1003000 r8:c9019000 r7:00000001 r6:00000000 r5:00000000 [ 10.722104] r4:ffffe000 [ 10.724650] [<c022bb7c>] (irq_exit) from [<c027feb4>] (__handle_domain_irq+0x70/0xc4) [ 10.732497] r5:00000000 r4:c2075fbc [ 10.736083] [<c027fe44>] (__handle_domain_irq) from [<c0202214>] (gic_handle_irq+0x5c/0xa0) [ 10.744453] r9:d1003000 r8:d1002000 r7:c1d33fb0 r6:d100200c r5:c220565c r4:c226d668 [ 10.752215] [<c02021b8>] (gic_handle_irq) from [<c0201d8c>] (__irq_usr+0x4c/0x60) [ 10.759713] Exception stack(0xc1d33fb0 to 0xc1d33ff8) [ 10.764776] 3fa0: b6ec1778 b6f32094 62632e73 62740000 [ 10.772973] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 00000000 00000000 00000006 [ 10.781167] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 ffffffff [ 10.787797] r9:00000000 r8:30c5387d r7:30c5387d r6:ffffffff r5:000f0030 r4:b6f1e0b6 [ 10.795559] Code: e30609dc e34c0220 e083300c e590c000 (e5930008) [ 10.801670] ---[ end trace 97c3942fa73eff4c ]--- [ 10.806300] Kernel panic - not syncing: Fatal exception in interrupt [ 10.812678] CPU2: stopping [ 10.815403] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 10.824641] Hardware name: Broadcom STB (Flattened Device Tree) [ 10.830573] Backtrace: [ 10.833036] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 10.840624] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 [ 10.846302] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 10.853546] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 10.860960] r10:00000000 r9:c2205400 r8:00000000 r7:00000002 r6:c22b0744 r5:00000004 [ 10.868807] r4:c22ce308 r3:00976ec2 [ 10.872395] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 10.879983] r10:00000000 r9:d1003000 r8:d1002000 r7:c9131f18 r6:d100200c r5:c220565c [ 10.887829] r4:c226d668 [ 10.890373] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 10.897871] Exception stack(0xc9131f18 to 0xc9131f60) [ 10.902934] 1f00: c020a47c 00000000 [ 10.911131] 1f20: 0e4ca000 600c0093 c9130000 c2204cf0 c2204d34 00000004 00000000 c20757b0 [ 10.919327] 1f40: 00000000 c9131f74 c9130000 c9131f68 00000000 c020a480 600c0013 ffffffff [ 10.927523] r9:c9130000 r8:00000000 r7:c9131f4c r6:ffffffff r5:600c0013 r4:c020a480 [ 10.935291] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 10.943403] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 10.951164] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 10.958752] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000002 [ 10.966598] r4:0000008a [ 10.969142] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 10.978210] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 10.985447] r5:00000000 r4:090eaa40 [ 10.989032] CPU3: stopping [ 10.991754] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 11.000992] Hardware name: Broadcom STB (Flattened Device Tree) [ 11.006924] Backtrace: [ 11.009384] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 11.016972] r7:c22a86d0 r6:00000000 r5:600f0193 r4:c22a86d0 [ 11.022649] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 11.029892] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 11.037307] r10:00000000 r9:c2205400 r8:00000000 r7:00000003 r6:c22b0744 r5:00000004 [ 11.045154] r4:c22ce308 r3:00976ec2 [ 11.048741] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 11.056329] r10:00000000 r9:d1003000 r8:d1002000 r7:c9133f18 r6:d100200c r5:c220565c [ 11.064176] r4:c226d668 [ 11.066719] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 11.074216] Exception stack(0xc9133f18 to 0xc9133f60) [ 11.079280] 3f00: c020a47c 00000000 [ 11.087476] 3f20: 0e4dd000 600f0093 c9132000 c2204cf0 c2204d34 00000008 00000000 c20757b0 [ 11.095671] 3f40: 00000000 c9133f74 c9132000 c9133f68 00000000 c020a480 600f0013 ffffffff [ 11.103867] r9:c9132000 r8:00000000 r7:c9133f4c r6:ffffffff r5:600f0013 r4:c020a480 [ 11.111633] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 11.119744] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 11.127506] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 11.135093] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000003 [ 11.142939] r4:0000008a [ 11.145484] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 11.154550] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 11.161788] r5:00000000 r4:090eaa40 [ 11.165372] CPU1: stopping [ 11.168094] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 11.177332] Hardware name: Broadcom STB (Flattened Device Tree) [ 11.183264] Backtrace: [ 11.185723] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 11.193311] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 [ 11.198988] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 11.206230] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 11.213644] r10:00000000 r9:c2205400 r8:00000000 r7:00000001 r6:c22b0744 r5:00000004 [ 11.221491] r4:c22ce308 r3:00976ec2 [ 11.225078] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 11.232666] r10:00000000 r9:d1003000 r8:d1002000 r7:c912ff18 r6:d100200c r5:c220565c [ 11.240512] r4:c226d668 [ 11.243054] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 11.250553] Exception stack(0xc912ff18 to 0xc912ff60) [ 11.255617] ff00: c020a47c 00000000 [ 11.263814] ff20: 0e4b7000 600c0093 c912e000 c2204cf0 c2204d34 00000002 00000000 c20757b0 [ 11.272010] ff40: 00000000 c912ff74 c912e000 c912ff68 00000000 c020a480 600c0013 ffffffff [ 11.280206] r9:c912e000 r8:00000000 r7:c912ff4c r6:ffffffff r5:600c0013 r4:c020a480 [ 11.287970] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 11.296081] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 11.303842] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 11.311430] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000001 [ 11.319275] r4:0000008a [ 11.321819] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 11.330886] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 11.338124] r5:00000000 r4:090eaa40 [ 11.341729] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---
it is not marked as fixing anything so I wonder how it landed in stable?
It was requested to be merged.
OK.
Do you have the same crash on newer kernels with this commit in it (like Linus's tree?)
There are no crashes with 5.10 or upstream otherwise it would have been noticed earlier, I have not kept track of which additional changes/fixes we may need but I would suggest we drop this one for now.
Alexander, do you want me to test additional patches if your change somehow must be included in an upcoming 5.4? I thought the platform you are working is still not upstream, so who is going to benefit from this performance improvement?
This patch was added to stable trees by Pali, not me (link: [0]). I added him to Ccs, so you can ask him. As you noted correctly, I don't work on/with stable releases.
Is 4.19 also failing for you now?
I cannot easily test 4.19 without applying some additional patches bringing in various ARM SCMI changes, I would suspect the same to happen. Give me a few hours and I will see if I can give you a better answer with an actual test. -- Florian
[0] https://lore.kernel.org/stable/20210308175757.8373-1-pali@kernel.org
Thanks, Al
On Friday 12 March 2021 12:54:18 Alexander Lobakin wrote:
From: Florian Fainelli f.fainelli@gmail.com Date: Thu, 11 Mar 2021 09:41:27 -0800
Hi Florian,
On 3/11/21 9:40 AM, Greg KH wrote:
On Thu, Mar 11, 2021 at 09:23:56AM -0800, Florian Fainelli wrote:
On 3/11/21 5:08 AM, Greg KH wrote:
On Wed, Mar 10, 2021 at 08:19:45PM -0800, Florian Fainelli wrote:
+Alex,
On 3/10/2021 5:24 AM, gregkh@linuxfoundation.org wrote: > From: Greg Kroah-Hartman gregkh@linuxfoundation.org > > This is the start of the stable review cycle for the 5.4.105 release. > There are 24 patches in this series, all will be posted as a response > to this one. If anyone has any issues with these being applied, please > let me know. > > Responses should be made by Fri, 12 Mar 2021 13:23:09 +0000. > Anything received after that time might be too late. > > The whole patch series can be found in one patch at: > https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/patch-5.4.105-rc1... > or in the git tree and branch at: > git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-5.4.y > and the diffstat can be found below. > > thanks, > > greg k-h
I believe you need to drop "net: dsa: add GRO support via gro_cells" as it causes the following kernel panic on a DSA-enabled platform:
Configuring rgmii_2 interface [ 10.170527] brcm-sf2 f0b00000.ethernet_switch rgmii_2: configuring for fixed/rgmii-txid link mode [ 10.179597] 8021q: adding VLAN 0 to HW filter on device rgmii_2 [ 10.185608] brcm-sf2 f0b00000.ethernet_switch rgmii_2: Link is Up - 1Gbps/Full - flow control off [ 10.198631] IPv6: ADDRCONF(NETDEV_CHANGE): rgmii_2: link becomes ready Configuring sit0 interface [ 10.254346] 8<--- cut here --- [ 10.257438] Unable to handle kernel paging request at virtual address d6df6190 [ 10.264685] pgd = (ptrval) [ 10.267411] [d6df6190] *pgd=80000000007003, *pmd=00000000 [ 10.272846] Internal error: Oops: 206 [#1] SMP ARM [ 10.277661] Modules linked in: [ 10.280739] CPU: 0 PID: 1886 Comm: sed Not tainted 5.4.105-1.0pre-geff642e2af2b #4 [ 10.288337] Hardware name: Broadcom STB (Flattened Device Tree) [ 10.294292] PC is at gro_cells_receive+0x90/0x11c [ 10.299020] LR is at dsa_switch_rcv+0x120/0x1d4 [ 10.303562] pc : [<c0a57a28>] lr : [<c0b4a65c>] psr: 600f0113 [ 10.309841] sp : c1d33cd0 ip : 000003e8 fp : c1d33ce4 [ 10.315078] r10: c8901000 r9 : c8901000 r8 : c0b4a53c [ 10.320314] r7 : c2208920 r6 : 00000000 r5 : 00000000 r4 : 00004000 [ 10.326855] r3 : d6df6188 r2 : c4927000 r1 : c8adc300 r0 : c22069dc [ 10.333398] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user [ 10.340547] Control: 30c5387d Table: 04ac4c80 DAC: fffffffd [ 10.346307] Process sed (pid: 1886, stack limit = 0x(ptrval)) [ 10.352066] Stack: (0xc1d33cd0 to 0xc1d34000) [ 10.356434] 3cc0: c8adc300 c4927000 c1d33d04 c1d33ce8 [ 10.364631] 3ce0: c0b4a65c c0a579a4 c1d33d24 c2208920 c1d33d24 00000000 c1d33d5c c1d33d08 [ 10.372827] 3d00: c0a0b38c c0b4a548 c021e070 c2204cc8 00000000 c89015c0 04b87700 c89015c0 [ 10.381023] 3d20: c2208920 c1d33d24 c1d33d24 00976ec2 04b87700 c8adc300 c89015c0 00000000 [ 10.389218] 3d40: c1d33d74 c1d32000 00000000 c230742c c1d33dac c1d33d60 c0a0b5c0 c0a0b180 [ 10.397414] 3d60: 00000000 c2204cc8 00000000 c1d33d6c c1d33d6c c1d33d80 c029daf8 00976ec2 [ 10.405610] 3d80: 00000800 c8901540 c89015c0 c8901540 00000000 00000001 0000016c 00000162 [ 10.413805] 3da0: c1d33dc4 c1d33db0 c0a0b7fc c0a0b3b8 00000000 c8adc300 c1d33dfc c1d33dc8 [ 10.422001] 3dc0: c0a0c660 c0a0b7e4 c8901540 c8adc300 c1d33dfc c1d33de0 c8901540 c8adc300 [ 10.430196] 3de0: 0000015e c8901000 00000001 0000016c c1d33e74 c1d33e00 c083df00 c0a0c4fc [ 10.438391] 3e00: 0000012c c22b0f14 c1d33e4c c1d33e18 c0fbd9b8 c0fbd9cc c0fbd9e0 c0fbd98c [ 10.446586] 3e20: 00000001 00000040 c8901500 00000001 00000000 00000000 00000000 00000000 [ 10.454780] 3e40: 00000000 00000000 c02f65a0 c8901540 00000001 00000040 c22b07e4 0000012c [ 10.462975] 3e60: d1003000 fffb942f c1d33edc c1d33e78 c0a0c94c c083dafc d051ad80 c2204cc8 [ 10.471170] 3e80: c2204cf0 c1d32000 c22b40b0 0e4a4000 c2076d80 c2203d00 c022bc70 c1d33e9c [ 10.479365] 3ea0: c1d33e9c c1d33ea4 c1d33ea4 00976ec2 c02f65a0 c220308c 00000003 c1d32000 [ 10.487561] 3ec0: c22b07e4 00000100 d1003000 00000008 c1d33f44 c1d33ee0 c020238c c0a0c6cc [ 10.495755] 3ee0: c1d33f14 c1d33ef0 00000001 00400000 c2203d00 fffb942f c206b2e4 c2076040 [ 10.503950] 3f00: c2076040 0000000a c2203080 c206b358 c1d33ee0 00000004 c90c9500 ffffe000 [ 10.512145] 3f20: 00000000 00000000 00000001 c9019000 d1003000 00000000 c1d33f5c c1d33f48 [ 10.520339] 3f40: c022bc70 c0202264 c2075fbc 00000000 c1d33f84 c1d33f60 c027feb4 c022bb88 [ 10.528535] 3f60: c226d668 c220565c d100200c c1d33fb0 d1002000 d1003000 c1d33fac c1d33f88 [ 10.536730] 3f80: c0202214 c027fe50 b6f1e0b6 000f0030 ffffffff 30c5387d 30c5387d 00000000 [ 10.544924] 3fa0: 00000000 c1d33fb0 c0201d8c c02021c4 b6ec1778 b6f32094 62632e73 62740000 [ 10.553120] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 00000000 00000000 00000006 [ 10.561316] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 ffffffff 00000000 00000000 [ 10.569508] Backtrace: [ 10.571970] [<c0a57998>] (gro_cells_receive) from [<c0b4a65c>] (dsa_switch_rcv+0x120/0x1d4) [ 10.580338] r5:c4927000 r4:c8adc300 [ 10.583929] [<c0b4a53c>] (dsa_switch_rcv) from [<c0a0b38c>] (__netif_receive_skb_list_core+0x218/0x238) [ 10.593343] r7:00000000 r6:c1d33d24 r5:c2208920 r4:c1d33d24 [ 10.599017] [<c0a0b174>] (__netif_receive_skb_list_core) from [<c0a0b5c0>] (netif_receive_skb_list_internal+0x214/0x2dc) [ 10.609909] r10:c230742c r9:00000000 r8:c1d32000 r7:c1d33d74 r6:00000000 r5:c89015c0 [ 10.617755] r4:c8adc300 [ 10.620300] [<c0a0b3ac>] (netif_receive_skb_list_internal) from [<c0a0b7fc>] (gro_normal_list.part.42+0x24/0x38) [ 10.630496] r10:00000162 r9:0000016c r8:00000001 r7:00000000 r6:c8901540 r5:c89015c0 [ 10.638342] r4:c8901540 [ 10.640885] [<c0a0b7d8>] (gro_normal_list.part.42) from [<c0a0c660>] (napi_complete_done+0x170/0x1d0) [ 10.650123] r5:c8adc300 r4:00000000 [ 10.653712] [<c0a0c4f0>] (napi_complete_done) from [<c083df00>] (bcm_sysport_poll+0x410/0x4b4) [ 10.662343] r9:0000016c r8:00000001 r7:c8901000 r6:0000015e r5:c8adc300 r4:c8901540 [ 10.670104] [<c083daf0>] (bcm_sysport_poll) from [<c0a0c94c>] (net_rx_action+0x28c/0x44c) [ 10.678301] r10:fffb942f r9:d1003000 r8:0000012c r7:c22b07e4 r6:00000040 r5:00000001 [ 10.686146] r4:c8901540 [ 10.688693] [<c0a0c6c0>] (net_rx_action) from [<c020238c>] (__do_softirq+0x134/0x414) [ 10.696542] r10:00000008 r9:d1003000 r8:00000100 r7:c22b07e4 r6:c1d32000 r5:00000003 [ 10.704387] r4:c220308c [ 10.706931] [<c0202258>] (__do_softirq) from [<c022bc70>] (irq_exit+0xf4/0x100) [ 10.714257] r10:00000000 r9:d1003000 r8:c9019000 r7:00000001 r6:00000000 r5:00000000 [ 10.722104] r4:ffffe000 [ 10.724650] [<c022bb7c>] (irq_exit) from [<c027feb4>] (__handle_domain_irq+0x70/0xc4) [ 10.732497] r5:00000000 r4:c2075fbc [ 10.736083] [<c027fe44>] (__handle_domain_irq) from [<c0202214>] (gic_handle_irq+0x5c/0xa0) [ 10.744453] r9:d1003000 r8:d1002000 r7:c1d33fb0 r6:d100200c r5:c220565c r4:c226d668 [ 10.752215] [<c02021b8>] (gic_handle_irq) from [<c0201d8c>] (__irq_usr+0x4c/0x60) [ 10.759713] Exception stack(0xc1d33fb0 to 0xc1d33ff8) [ 10.764776] 3fa0: b6ec1778 b6f32094 62632e73 62740000 [ 10.772973] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 00000000 00000000 00000006 [ 10.781167] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 ffffffff [ 10.787797] r9:00000000 r8:30c5387d r7:30c5387d r6:ffffffff r5:000f0030 r4:b6f1e0b6 [ 10.795559] Code: e30609dc e34c0220 e083300c e590c000 (e5930008) [ 10.801670] ---[ end trace 97c3942fa73eff4c ]--- [ 10.806300] Kernel panic - not syncing: Fatal exception in interrupt [ 10.812678] CPU2: stopping [ 10.815403] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 10.824641] Hardware name: Broadcom STB (Flattened Device Tree) [ 10.830573] Backtrace: [ 10.833036] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 10.840624] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 [ 10.846302] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 10.853546] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 10.860960] r10:00000000 r9:c2205400 r8:00000000 r7:00000002 r6:c22b0744 r5:00000004 [ 10.868807] r4:c22ce308 r3:00976ec2 [ 10.872395] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 10.879983] r10:00000000 r9:d1003000 r8:d1002000 r7:c9131f18 r6:d100200c r5:c220565c [ 10.887829] r4:c226d668 [ 10.890373] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 10.897871] Exception stack(0xc9131f18 to 0xc9131f60) [ 10.902934] 1f00: c020a47c 00000000 [ 10.911131] 1f20: 0e4ca000 600c0093 c9130000 c2204cf0 c2204d34 00000004 00000000 c20757b0 [ 10.919327] 1f40: 00000000 c9131f74 c9130000 c9131f68 00000000 c020a480 600c0013 ffffffff [ 10.927523] r9:c9130000 r8:00000000 r7:c9131f4c r6:ffffffff r5:600c0013 r4:c020a480 [ 10.935291] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 10.943403] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 10.951164] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 10.958752] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000002 [ 10.966598] r4:0000008a [ 10.969142] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 10.978210] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 10.985447] r5:00000000 r4:090eaa40 [ 10.989032] CPU3: stopping [ 10.991754] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 11.000992] Hardware name: Broadcom STB (Flattened Device Tree) [ 11.006924] Backtrace: [ 11.009384] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 11.016972] r7:c22a86d0 r6:00000000 r5:600f0193 r4:c22a86d0 [ 11.022649] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 11.029892] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 11.037307] r10:00000000 r9:c2205400 r8:00000000 r7:00000003 r6:c22b0744 r5:00000004 [ 11.045154] r4:c22ce308 r3:00976ec2 [ 11.048741] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 11.056329] r10:00000000 r9:d1003000 r8:d1002000 r7:c9133f18 r6:d100200c r5:c220565c [ 11.064176] r4:c226d668 [ 11.066719] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 11.074216] Exception stack(0xc9133f18 to 0xc9133f60) [ 11.079280] 3f00: c020a47c 00000000 [ 11.087476] 3f20: 0e4dd000 600f0093 c9132000 c2204cf0 c2204d34 00000008 00000000 c20757b0 [ 11.095671] 3f40: 00000000 c9133f74 c9132000 c9133f68 00000000 c020a480 600f0013 ffffffff [ 11.103867] r9:c9132000 r8:00000000 r7:c9133f4c r6:ffffffff r5:600f0013 r4:c020a480 [ 11.111633] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 11.119744] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 11.127506] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 11.135093] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000003 [ 11.142939] r4:0000008a [ 11.145484] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 11.154550] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 11.161788] r5:00000000 r4:090eaa40 [ 11.165372] CPU1: stopping [ 11.168094] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 11.177332] Hardware name: Broadcom STB (Flattened Device Tree) [ 11.183264] Backtrace: [ 11.185723] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 11.193311] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 [ 11.198988] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 11.206230] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 11.213644] r10:00000000 r9:c2205400 r8:00000000 r7:00000001 r6:c22b0744 r5:00000004 [ 11.221491] r4:c22ce308 r3:00976ec2 [ 11.225078] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 11.232666] r10:00000000 r9:d1003000 r8:d1002000 r7:c912ff18 r6:d100200c r5:c220565c [ 11.240512] r4:c226d668 [ 11.243054] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 11.250553] Exception stack(0xc912ff18 to 0xc912ff60) [ 11.255617] ff00: c020a47c 00000000 [ 11.263814] ff20: 0e4b7000 600c0093 c912e000 c2204cf0 c2204d34 00000002 00000000 c20757b0 [ 11.272010] ff40: 00000000 c912ff74 c912e000 c912ff68 00000000 c020a480 600c0013 ffffffff [ 11.280206] r9:c912e000 r8:00000000 r7:c912ff4c r6:ffffffff r5:600c0013 r4:c020a480 [ 11.287970] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 11.296081] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 11.303842] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 11.311430] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000001 [ 11.319275] r4:0000008a [ 11.321819] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 11.330886] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 11.338124] r5:00000000 r4:090eaa40 [ 11.341729] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---
it is not marked as fixing anything so I wonder how it landed in stable?
It was requested to be merged.
OK.
Do you have the same crash on newer kernels with this commit in it (like Linus's tree?)
There are no crashes with 5.10 or upstream otherwise it would have been noticed earlier, I have not kept track of which additional changes/fixes we may need but I would suggest we drop this one for now.
Alexander, do you want me to test additional patches if your change somehow must be included in an upcoming 5.4? I thought the platform you are working is still not upstream, so who is going to benefit from this performance improvement?
Hello Florian!
I have heavily tested this change backported to 4.19 and 5.4 kernels and it radically increased network performance on DSA setup with mv88e6xxx switch. I have not seen any stability issues. So based on tests I have sent backport request as patch is relatively small, is already in mainline kernel and can be cleanly applied to 4.19 and 5.4 kernel.
But if this patch really cause issues, I think based on 'no regression' rule we have no other options, just to drop this patch from stable queue.
Greg, please drop this patch (for now).
But I would be really interested why this patch does not cause any regression with mainline kernel and cause with 5.4...
Florian, which DSA driver and hardware is causing this issue?
Could you checkout kernel at e131a5634830047923c694b4ce0c3b31745ff01b commit and test if this issue happen too?
This patch was added to stable trees by Pali, not me (link: [0]). I added him to Ccs, so you can ask him. As you noted correctly, I don't work on/with stable releases.
Is 4.19 also failing for you now?
I cannot easily test 4.19 without applying some additional patches bringing in various ARM SCMI changes, I would suspect the same to happen. Give me a few hours and I will see if I can give you a better answer with an actual test.
Ok, if you have some results let me know.
-- Florian
[0] https://lore.kernel.org/stable/20210308175757.8373-1-pali@kernel.org
Thanks, Al
On 3/15/2021 2:50 AM, Pali Rohár wrote:
On Friday 12 March 2021 12:54:18 Alexander Lobakin wrote:
From: Florian Fainelli f.fainelli@gmail.com Date: Thu, 11 Mar 2021 09:41:27 -0800
Hi Florian,
On 3/11/21 9:40 AM, Greg KH wrote:
On Thu, Mar 11, 2021 at 09:23:56AM -0800, Florian Fainelli wrote:
On 3/11/21 5:08 AM, Greg KH wrote:
On Wed, Mar 10, 2021 at 08:19:45PM -0800, Florian Fainelli wrote: > +Alex, > > On 3/10/2021 5:24 AM, gregkh@linuxfoundation.org wrote: >> From: Greg Kroah-Hartman gregkh@linuxfoundation.org >> >> This is the start of the stable review cycle for the 5.4.105 release. >> There are 24 patches in this series, all will be posted as a response >> to this one. If anyone has any issues with these being applied, please >> let me know. >> >> Responses should be made by Fri, 12 Mar 2021 13:23:09 +0000. >> Anything received after that time might be too late. >> >> The whole patch series can be found in one patch at: >> https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/patch-5.4.105-rc1... >> or in the git tree and branch at: >> git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-5.4.y >> and the diffstat can be found below. >> >> thanks, >> >> greg k-h > > I believe you need to drop "net: dsa: add GRO support via gro_cells" as > it causes the following kernel panic on a DSA-enabled platform: > > Configuring rgmii_2 interface > [ 10.170527] brcm-sf2 f0b00000.ethernet_switch rgmii_2: configuring > for fixed/rgmii-txid link mode > [ 10.179597] 8021q: adding VLAN 0 to HW filter on device rgmii_2 > [ 10.185608] brcm-sf2 f0b00000.ethernet_switch rgmii_2: Link is Up - > 1Gbps/Full - flow control off > [ 10.198631] IPv6: ADDRCONF(NETDEV_CHANGE): rgmii_2: link becomes ready > Configuring sit0 interface > [ 10.254346] 8<--- cut here --- > [ 10.257438] Unable to handle kernel paging request at virtual address > d6df6190 > [ 10.264685] pgd = (ptrval) > [ 10.267411] [d6df6190] *pgd=80000000007003, *pmd=00000000 > [ 10.272846] Internal error: Oops: 206 [#1] SMP ARM > [ 10.277661] Modules linked in: > [ 10.280739] CPU: 0 PID: 1886 Comm: sed Not tainted > 5.4.105-1.0pre-geff642e2af2b #4 > [ 10.288337] Hardware name: Broadcom STB (Flattened Device Tree) > [ 10.294292] PC is at gro_cells_receive+0x90/0x11c > [ 10.299020] LR is at dsa_switch_rcv+0x120/0x1d4 > [ 10.303562] pc : [<c0a57a28>] lr : [<c0b4a65c>] psr: 600f0113 > [ 10.309841] sp : c1d33cd0 ip : 000003e8 fp : c1d33ce4 > [ 10.315078] r10: c8901000 r9 : c8901000 r8 : c0b4a53c > [ 10.320314] r7 : c2208920 r6 : 00000000 r5 : 00000000 r4 : 00004000 > [ 10.326855] r3 : d6df6188 r2 : c4927000 r1 : c8adc300 r0 : c22069dc > [ 10.333398] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM > Segment user > [ 10.340547] Control: 30c5387d Table: 04ac4c80 DAC: fffffffd > [ 10.346307] Process sed (pid: 1886, stack limit = 0x(ptrval)) > [ 10.352066] Stack: (0xc1d33cd0 to 0xc1d34000) > [ 10.356434] 3cc0: c8adc300 > c4927000 c1d33d04 c1d33ce8 > [ 10.364631] 3ce0: c0b4a65c c0a579a4 c1d33d24 c2208920 c1d33d24 > 00000000 c1d33d5c c1d33d08 > [ 10.372827] 3d00: c0a0b38c c0b4a548 c021e070 c2204cc8 00000000 > c89015c0 04b87700 c89015c0 > [ 10.381023] 3d20: c2208920 c1d33d24 c1d33d24 00976ec2 04b87700 > c8adc300 c89015c0 00000000 > [ 10.389218] 3d40: c1d33d74 c1d32000 00000000 c230742c c1d33dac > c1d33d60 c0a0b5c0 c0a0b180 > [ 10.397414] 3d60: 00000000 c2204cc8 00000000 c1d33d6c c1d33d6c > c1d33d80 c029daf8 00976ec2 > [ 10.405610] 3d80: 00000800 c8901540 c89015c0 c8901540 00000000 > 00000001 0000016c 00000162 > [ 10.413805] 3da0: c1d33dc4 c1d33db0 c0a0b7fc c0a0b3b8 00000000 > c8adc300 c1d33dfc c1d33dc8 > [ 10.422001] 3dc0: c0a0c660 c0a0b7e4 c8901540 c8adc300 c1d33dfc > c1d33de0 c8901540 c8adc300 > [ 10.430196] 3de0: 0000015e c8901000 00000001 0000016c c1d33e74 > c1d33e00 c083df00 c0a0c4fc > [ 10.438391] 3e00: 0000012c c22b0f14 c1d33e4c c1d33e18 c0fbd9b8 > c0fbd9cc c0fbd9e0 c0fbd98c > [ 10.446586] 3e20: 00000001 00000040 c8901500 00000001 00000000 > 00000000 00000000 00000000 > [ 10.454780] 3e40: 00000000 00000000 c02f65a0 c8901540 00000001 > 00000040 c22b07e4 0000012c > [ 10.462975] 3e60: d1003000 fffb942f c1d33edc c1d33e78 c0a0c94c > c083dafc d051ad80 c2204cc8 > [ 10.471170] 3e80: c2204cf0 c1d32000 c22b40b0 0e4a4000 c2076d80 > c2203d00 c022bc70 c1d33e9c > [ 10.479365] 3ea0: c1d33e9c c1d33ea4 c1d33ea4 00976ec2 c02f65a0 > c220308c 00000003 c1d32000 > [ 10.487561] 3ec0: c22b07e4 00000100 d1003000 00000008 c1d33f44 > c1d33ee0 c020238c c0a0c6cc > [ 10.495755] 3ee0: c1d33f14 c1d33ef0 00000001 00400000 c2203d00 > fffb942f c206b2e4 c2076040 > [ 10.503950] 3f00: c2076040 0000000a c2203080 c206b358 c1d33ee0 > 00000004 c90c9500 ffffe000 > [ 10.512145] 3f20: 00000000 00000000 00000001 c9019000 d1003000 > 00000000 c1d33f5c c1d33f48 > [ 10.520339] 3f40: c022bc70 c0202264 c2075fbc 00000000 c1d33f84 > c1d33f60 c027feb4 c022bb88 > [ 10.528535] 3f60: c226d668 c220565c d100200c c1d33fb0 d1002000 > d1003000 c1d33fac c1d33f88 > [ 10.536730] 3f80: c0202214 c027fe50 b6f1e0b6 000f0030 ffffffff > 30c5387d 30c5387d 00000000 > [ 10.544924] 3fa0: 00000000 c1d33fb0 c0201d8c c02021c4 b6ec1778 > b6f32094 62632e73 62740000 > [ 10.553120] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 > 00000000 00000000 00000006 > [ 10.561316] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 > ffffffff 00000000 00000000 > [ 10.569508] Backtrace: > [ 10.571970] [<c0a57998>] (gro_cells_receive) from [<c0b4a65c>] > (dsa_switch_rcv+0x120/0x1d4) > [ 10.580338] r5:c4927000 r4:c8adc300 > [ 10.583929] [<c0b4a53c>] (dsa_switch_rcv) from [<c0a0b38c>] > (__netif_receive_skb_list_core+0x218/0x238) > [ 10.593343] r7:00000000 r6:c1d33d24 r5:c2208920 r4:c1d33d24 > [ 10.599017] [<c0a0b174>] (__netif_receive_skb_list_core) from > [<c0a0b5c0>] (netif_receive_skb_list_internal+0x214/0x2dc) > [ 10.609909] r10:c230742c r9:00000000 r8:c1d32000 r7:c1d33d74 > r6:00000000 r5:c89015c0 > [ 10.617755] r4:c8adc300 > [ 10.620300] [<c0a0b3ac>] (netif_receive_skb_list_internal) from > [<c0a0b7fc>] (gro_normal_list.part.42+0x24/0x38) > [ 10.630496] r10:00000162 r9:0000016c r8:00000001 r7:00000000 > r6:c8901540 r5:c89015c0 > [ 10.638342] r4:c8901540 > [ 10.640885] [<c0a0b7d8>] (gro_normal_list.part.42) from [<c0a0c660>] > (napi_complete_done+0x170/0x1d0) > [ 10.650123] r5:c8adc300 r4:00000000 > [ 10.653712] [<c0a0c4f0>] (napi_complete_done) from [<c083df00>] > (bcm_sysport_poll+0x410/0x4b4) > [ 10.662343] r9:0000016c r8:00000001 r7:c8901000 r6:0000015e > r5:c8adc300 r4:c8901540 > [ 10.670104] [<c083daf0>] (bcm_sysport_poll) from [<c0a0c94c>] > (net_rx_action+0x28c/0x44c) > [ 10.678301] r10:fffb942f r9:d1003000 r8:0000012c r7:c22b07e4 > r6:00000040 r5:00000001 > [ 10.686146] r4:c8901540 > [ 10.688693] [<c0a0c6c0>] (net_rx_action) from [<c020238c>] > (__do_softirq+0x134/0x414) > [ 10.696542] r10:00000008 r9:d1003000 r8:00000100 r7:c22b07e4 > r6:c1d32000 r5:00000003 > [ 10.704387] r4:c220308c > [ 10.706931] [<c0202258>] (__do_softirq) from [<c022bc70>] > (irq_exit+0xf4/0x100) > [ 10.714257] r10:00000000 r9:d1003000 r8:c9019000 r7:00000001 > r6:00000000 r5:00000000 > [ 10.722104] r4:ffffe000 > [ 10.724650] [<c022bb7c>] (irq_exit) from [<c027feb4>] > (__handle_domain_irq+0x70/0xc4) > [ 10.732497] r5:00000000 r4:c2075fbc > [ 10.736083] [<c027fe44>] (__handle_domain_irq) from [<c0202214>] > (gic_handle_irq+0x5c/0xa0) > [ 10.744453] r9:d1003000 r8:d1002000 r7:c1d33fb0 r6:d100200c > r5:c220565c r4:c226d668 > [ 10.752215] [<c02021b8>] (gic_handle_irq) from [<c0201d8c>] > (__irq_usr+0x4c/0x60) > [ 10.759713] Exception stack(0xc1d33fb0 to 0xc1d33ff8) > [ 10.764776] 3fa0: b6ec1778 > b6f32094 62632e73 62740000 > [ 10.772973] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 > 00000000 00000000 00000006 > [ 10.781167] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 ffffffff > [ 10.787797] r9:00000000 r8:30c5387d r7:30c5387d r6:ffffffff > r5:000f0030 r4:b6f1e0b6 > [ 10.795559] Code: e30609dc e34c0220 e083300c e590c000 (e5930008) > [ 10.801670] ---[ end trace 97c3942fa73eff4c ]--- > [ 10.806300] Kernel panic - not syncing: Fatal exception in interrupt > [ 10.812678] CPU2: stopping > [ 10.815403] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G D > 5.4.105-1.0pre-geff642e2af2b #4 > [ 10.824641] Hardware name: Broadcom STB (Flattened Device Tree) > [ 10.830573] Backtrace: > [ 10.833036] [<c020dd30>] (dump_backtrace) from [<c020e04c>] > (show_stack+0x20/0x24) > [ 10.840624] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 > [ 10.846302] [<c020e02c>] (show_stack) from [<c0c09924>] > (dump_stack+0xb8/0xe4) > [ 10.853546] [<c0c0986c>] (dump_stack) from [<c0212160>] > (handle_IPI+0x344/0x3cc) > [ 10.860960] r10:00000000 r9:c2205400 r8:00000000 r7:00000002 > r6:c22b0744 r5:00000004 > [ 10.868807] r4:c22ce308 r3:00976ec2 > [ 10.872395] [<c0211e1c>] (handle_IPI) from [<c0202254>] > (gic_handle_irq+0x9c/0xa0) > [ 10.879983] r10:00000000 r9:d1003000 r8:d1002000 r7:c9131f18 > r6:d100200c r5:c220565c > [ 10.887829] r4:c226d668 > [ 10.890373] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] > (__irq_svc+0x5c/0x7c) > [ 10.897871] Exception stack(0xc9131f18 to 0xc9131f60) > [ 10.902934] 1f00: > c020a47c 00000000 > [ 10.911131] 1f20: 0e4ca000 600c0093 c9130000 c2204cf0 c2204d34 > 00000004 00000000 c20757b0 > [ 10.919327] 1f40: 00000000 c9131f74 c9130000 c9131f68 00000000 > c020a480 600c0013 ffffffff > [ 10.927523] r9:c9130000 r8:00000000 r7:c9131f4c r6:ffffffff > r5:600c0013 r4:c020a480 > [ 10.935291] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] > (default_idle_call+0x34/0x48) > [ 10.943403] [<c0c13138>] (default_idle_call) from [<c02580c0>] > (do_idle+0x1d4/0x2c0) > [ 10.951164] [<c0257eec>] (do_idle) from [<c0258494>] > (cpu_startup_entry+0x28/0x2c) > [ 10.958752] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 > r6:30c0387d r5:00000002 > [ 10.966598] r4:0000008a > [ 10.969142] [<c025846c>] (cpu_startup_entry) from [<c0211644>] > (secondary_start_kernel+0x17c/0x1a0) > [ 10.978210] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] > (0x20270c) > [ 10.985447] r5:00000000 r4:090eaa40 > [ 10.989032] CPU3: stopping > [ 10.991754] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G D > 5.4.105-1.0pre-geff642e2af2b #4 > [ 11.000992] Hardware name: Broadcom STB (Flattened Device Tree) > [ 11.006924] Backtrace: > [ 11.009384] [<c020dd30>] (dump_backtrace) from [<c020e04c>] > (show_stack+0x20/0x24) > [ 11.016972] r7:c22a86d0 r6:00000000 r5:600f0193 r4:c22a86d0 > [ 11.022649] [<c020e02c>] (show_stack) from [<c0c09924>] > (dump_stack+0xb8/0xe4) > [ 11.029892] [<c0c0986c>] (dump_stack) from [<c0212160>] > (handle_IPI+0x344/0x3cc) > [ 11.037307] r10:00000000 r9:c2205400 r8:00000000 r7:00000003 > r6:c22b0744 r5:00000004 > [ 11.045154] r4:c22ce308 r3:00976ec2 > [ 11.048741] [<c0211e1c>] (handle_IPI) from [<c0202254>] > (gic_handle_irq+0x9c/0xa0) > [ 11.056329] r10:00000000 r9:d1003000 r8:d1002000 r7:c9133f18 > r6:d100200c r5:c220565c > [ 11.064176] r4:c226d668 > [ 11.066719] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] > (__irq_svc+0x5c/0x7c) > [ 11.074216] Exception stack(0xc9133f18 to 0xc9133f60) > [ 11.079280] 3f00: > c020a47c 00000000 > [ 11.087476] 3f20: 0e4dd000 600f0093 c9132000 c2204cf0 c2204d34 > 00000008 00000000 c20757b0 > [ 11.095671] 3f40: 00000000 c9133f74 c9132000 c9133f68 00000000 > c020a480 600f0013 ffffffff > [ 11.103867] r9:c9132000 r8:00000000 r7:c9133f4c r6:ffffffff > r5:600f0013 r4:c020a480 > [ 11.111633] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] > (default_idle_call+0x34/0x48) > [ 11.119744] [<c0c13138>] (default_idle_call) from [<c02580c0>] > (do_idle+0x1d4/0x2c0) > [ 11.127506] [<c0257eec>] (do_idle) from [<c0258494>] > (cpu_startup_entry+0x28/0x2c) > [ 11.135093] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 > r6:30c0387d r5:00000003 > [ 11.142939] r4:0000008a > [ 11.145484] [<c025846c>] (cpu_startup_entry) from [<c0211644>] > (secondary_start_kernel+0x17c/0x1a0) > [ 11.154550] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] > (0x20270c) > [ 11.161788] r5:00000000 r4:090eaa40 > [ 11.165372] CPU1: stopping > [ 11.168094] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G D > 5.4.105-1.0pre-geff642e2af2b #4 > [ 11.177332] Hardware name: Broadcom STB (Flattened Device Tree) > [ 11.183264] Backtrace: > [ 11.185723] [<c020dd30>] (dump_backtrace) from [<c020e04c>] > (show_stack+0x20/0x24) > [ 11.193311] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 > [ 11.198988] [<c020e02c>] (show_stack) from [<c0c09924>] > (dump_stack+0xb8/0xe4) > [ 11.206230] [<c0c0986c>] (dump_stack) from [<c0212160>] > (handle_IPI+0x344/0x3cc) > [ 11.213644] r10:00000000 r9:c2205400 r8:00000000 r7:00000001 > r6:c22b0744 r5:00000004 > [ 11.221491] r4:c22ce308 r3:00976ec2 > [ 11.225078] [<c0211e1c>] (handle_IPI) from [<c0202254>] > (gic_handle_irq+0x9c/0xa0) > [ 11.232666] r10:00000000 r9:d1003000 r8:d1002000 r7:c912ff18 > r6:d100200c r5:c220565c > [ 11.240512] r4:c226d668 > [ 11.243054] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] > (__irq_svc+0x5c/0x7c) > [ 11.250553] Exception stack(0xc912ff18 to 0xc912ff60) > [ 11.255617] ff00: > c020a47c 00000000 > [ 11.263814] ff20: 0e4b7000 600c0093 c912e000 c2204cf0 c2204d34 > 00000002 00000000 c20757b0 > [ 11.272010] ff40: 00000000 c912ff74 c912e000 c912ff68 00000000 > c020a480 600c0013 ffffffff > [ 11.280206] r9:c912e000 r8:00000000 r7:c912ff4c r6:ffffffff > r5:600c0013 r4:c020a480 > [ 11.287970] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] > (default_idle_call+0x34/0x48) > [ 11.296081] [<c0c13138>] (default_idle_call) from [<c02580c0>] > (do_idle+0x1d4/0x2c0) > [ 11.303842] [<c0257eec>] (do_idle) from [<c0258494>] > (cpu_startup_entry+0x28/0x2c) > [ 11.311430] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 > r6:30c0387d r5:00000001 > [ 11.319275] r4:0000008a > [ 11.321819] [<c025846c>] (cpu_startup_entry) from [<c0211644>] > (secondary_start_kernel+0x17c/0x1a0) > [ 11.330886] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] > (0x20270c) > [ 11.338124] r5:00000000 r4:090eaa40 > [ 11.341729] ---[ end Kernel panic - not syncing: Fatal exception in > interrupt ]--- > > it is not marked as fixing anything so I wonder how it landed in stable?
It was requested to be merged.
OK.
Do you have the same crash on newer kernels with this commit in it (like Linus's tree?)
There are no crashes with 5.10 or upstream otherwise it would have been noticed earlier, I have not kept track of which additional changes/fixes we may need but I would suggest we drop this one for now.
Alexander, do you want me to test additional patches if your change somehow must be included in an upcoming 5.4? I thought the platform you are working is still not upstream, so who is going to benefit from this performance improvement?
Hello Florian!
I have heavily tested this change backported to 4.19 and 5.4 kernels and it radically increased network performance on DSA setup with mv88e6xxx switch. I have not seen any stability issues. So based on tests I have sent backport request as patch is relatively small, is already in mainline kernel and can be cleanly applied to 4.19 and 5.4 kernel.
But if this patch really cause issues, I think based on 'no regression' rule we have no other options, just to drop this patch from stable queue.
Greg, please drop this patch (for now).
But I would be really interested why this patch does not cause any regression with mainline kernel and cause with 5.4...
Florian, which DSA driver and hardware is causing this issue?
This is using drivers/net/dsa/bcm_sf2.c for the switch driver (which uses b53 as the library for switch operations) and drivers/net/ethernet/broadcom/bcmsysport.c for the DSA "master".
Could you checkout kernel at e131a5634830047923c694b4ce0c3b31745ff01b commit and test if this issue happen too?
This patch was added to stable trees by Pali, not me (link: [0]). I added him to Ccs, so you can ask him. As you noted correctly, I don't work on/with stable releases.
Is 4.19 also failing for you now?
I cannot easily test 4.19 without applying some additional patches bringing in various ARM SCMI changes, I would suspect the same to happen. Give me a few hours and I will see if I can give you a better answer with an actual test.
Ok, if you have some results let me know.
I did test 4.19 eventually and was not able to reproduce the problem, and have not been able to reproduce the problem on 5.4 a second time around and ever since merging v5.4.105 in the past couple weeks.
I would consider this a non-issue and just a fluke. The board that exposed the problem is known for having power distribution issues when an aggressive CPU frequency scaling governor like schedutil is used. And it makes sense that the CPU frequency would be changed when traffic picks up. The boards have since been hard coded to the "performance" governor, that is the problematic ones at least.
On Tuesday 23 March 2021 10:20:27 Florian Fainelli wrote:
On 3/15/2021 2:50 AM, Pali Rohár wrote:
On Friday 12 March 2021 12:54:18 Alexander Lobakin wrote:
From: Florian Fainelli f.fainelli@gmail.com Date: Thu, 11 Mar 2021 09:41:27 -0800
Hi Florian,
On 3/11/21 9:40 AM, Greg KH wrote:
On Thu, Mar 11, 2021 at 09:23:56AM -0800, Florian Fainelli wrote:
On 3/11/21 5:08 AM, Greg KH wrote: > On Wed, Mar 10, 2021 at 08:19:45PM -0800, Florian Fainelli wrote: >> +Alex, >> >> On 3/10/2021 5:24 AM, gregkh@linuxfoundation.org wrote: >>> From: Greg Kroah-Hartman gregkh@linuxfoundation.org >>> >>> This is the start of the stable review cycle for the 5.4.105 release. >>> There are 24 patches in this series, all will be posted as a response >>> to this one. If anyone has any issues with these being applied, please >>> let me know. >>> >>> Responses should be made by Fri, 12 Mar 2021 13:23:09 +0000. >>> Anything received after that time might be too late. >>> >>> The whole patch series can be found in one patch at: >>> https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/patch-5.4.105-rc1... >>> or in the git tree and branch at: >>> git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-5.4.y >>> and the diffstat can be found below. >>> >>> thanks, >>> >>> greg k-h >> >> I believe you need to drop "net: dsa: add GRO support via gro_cells" as >> it causes the following kernel panic on a DSA-enabled platform: >> >> Configuring rgmii_2 interface >> [ 10.170527] brcm-sf2 f0b00000.ethernet_switch rgmii_2: configuring >> for fixed/rgmii-txid link mode >> [ 10.179597] 8021q: adding VLAN 0 to HW filter on device rgmii_2 >> [ 10.185608] brcm-sf2 f0b00000.ethernet_switch rgmii_2: Link is Up - >> 1Gbps/Full - flow control off >> [ 10.198631] IPv6: ADDRCONF(NETDEV_CHANGE): rgmii_2: link becomes ready >> Configuring sit0 interface >> [ 10.254346] 8<--- cut here --- >> [ 10.257438] Unable to handle kernel paging request at virtual address >> d6df6190 >> [ 10.264685] pgd = (ptrval) >> [ 10.267411] [d6df6190] *pgd=80000000007003, *pmd=00000000 >> [ 10.272846] Internal error: Oops: 206 [#1] SMP ARM >> [ 10.277661] Modules linked in: >> [ 10.280739] CPU: 0 PID: 1886 Comm: sed Not tainted >> 5.4.105-1.0pre-geff642e2af2b #4 >> [ 10.288337] Hardware name: Broadcom STB (Flattened Device Tree) >> [ 10.294292] PC is at gro_cells_receive+0x90/0x11c >> [ 10.299020] LR is at dsa_switch_rcv+0x120/0x1d4 >> [ 10.303562] pc : [<c0a57a28>] lr : [<c0b4a65c>] psr: 600f0113 >> [ 10.309841] sp : c1d33cd0 ip : 000003e8 fp : c1d33ce4 >> [ 10.315078] r10: c8901000 r9 : c8901000 r8 : c0b4a53c >> [ 10.320314] r7 : c2208920 r6 : 00000000 r5 : 00000000 r4 : 00004000 >> [ 10.326855] r3 : d6df6188 r2 : c4927000 r1 : c8adc300 r0 : c22069dc >> [ 10.333398] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM >> Segment user >> [ 10.340547] Control: 30c5387d Table: 04ac4c80 DAC: fffffffd >> [ 10.346307] Process sed (pid: 1886, stack limit = 0x(ptrval)) >> [ 10.352066] Stack: (0xc1d33cd0 to 0xc1d34000) >> [ 10.356434] 3cc0: c8adc300 >> c4927000 c1d33d04 c1d33ce8 >> [ 10.364631] 3ce0: c0b4a65c c0a579a4 c1d33d24 c2208920 c1d33d24 >> 00000000 c1d33d5c c1d33d08 >> [ 10.372827] 3d00: c0a0b38c c0b4a548 c021e070 c2204cc8 00000000 >> c89015c0 04b87700 c89015c0 >> [ 10.381023] 3d20: c2208920 c1d33d24 c1d33d24 00976ec2 04b87700 >> c8adc300 c89015c0 00000000 >> [ 10.389218] 3d40: c1d33d74 c1d32000 00000000 c230742c c1d33dac >> c1d33d60 c0a0b5c0 c0a0b180 >> [ 10.397414] 3d60: 00000000 c2204cc8 00000000 c1d33d6c c1d33d6c >> c1d33d80 c029daf8 00976ec2 >> [ 10.405610] 3d80: 00000800 c8901540 c89015c0 c8901540 00000000 >> 00000001 0000016c 00000162 >> [ 10.413805] 3da0: c1d33dc4 c1d33db0 c0a0b7fc c0a0b3b8 00000000 >> c8adc300 c1d33dfc c1d33dc8 >> [ 10.422001] 3dc0: c0a0c660 c0a0b7e4 c8901540 c8adc300 c1d33dfc >> c1d33de0 c8901540 c8adc300 >> [ 10.430196] 3de0: 0000015e c8901000 00000001 0000016c c1d33e74 >> c1d33e00 c083df00 c0a0c4fc >> [ 10.438391] 3e00: 0000012c c22b0f14 c1d33e4c c1d33e18 c0fbd9b8 >> c0fbd9cc c0fbd9e0 c0fbd98c >> [ 10.446586] 3e20: 00000001 00000040 c8901500 00000001 00000000 >> 00000000 00000000 00000000 >> [ 10.454780] 3e40: 00000000 00000000 c02f65a0 c8901540 00000001 >> 00000040 c22b07e4 0000012c >> [ 10.462975] 3e60: d1003000 fffb942f c1d33edc c1d33e78 c0a0c94c >> c083dafc d051ad80 c2204cc8 >> [ 10.471170] 3e80: c2204cf0 c1d32000 c22b40b0 0e4a4000 c2076d80 >> c2203d00 c022bc70 c1d33e9c >> [ 10.479365] 3ea0: c1d33e9c c1d33ea4 c1d33ea4 00976ec2 c02f65a0 >> c220308c 00000003 c1d32000 >> [ 10.487561] 3ec0: c22b07e4 00000100 d1003000 00000008 c1d33f44 >> c1d33ee0 c020238c c0a0c6cc >> [ 10.495755] 3ee0: c1d33f14 c1d33ef0 00000001 00400000 c2203d00 >> fffb942f c206b2e4 c2076040 >> [ 10.503950] 3f00: c2076040 0000000a c2203080 c206b358 c1d33ee0 >> 00000004 c90c9500 ffffe000 >> [ 10.512145] 3f20: 00000000 00000000 00000001 c9019000 d1003000 >> 00000000 c1d33f5c c1d33f48 >> [ 10.520339] 3f40: c022bc70 c0202264 c2075fbc 00000000 c1d33f84 >> c1d33f60 c027feb4 c022bb88 >> [ 10.528535] 3f60: c226d668 c220565c d100200c c1d33fb0 d1002000 >> d1003000 c1d33fac c1d33f88 >> [ 10.536730] 3f80: c0202214 c027fe50 b6f1e0b6 000f0030 ffffffff >> 30c5387d 30c5387d 00000000 >> [ 10.544924] 3fa0: 00000000 c1d33fb0 c0201d8c c02021c4 b6ec1778 >> b6f32094 62632e73 62740000 >> [ 10.553120] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 >> 00000000 00000000 00000006 >> [ 10.561316] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 >> ffffffff 00000000 00000000 >> [ 10.569508] Backtrace: >> [ 10.571970] [<c0a57998>] (gro_cells_receive) from [<c0b4a65c>] >> (dsa_switch_rcv+0x120/0x1d4) >> [ 10.580338] r5:c4927000 r4:c8adc300 >> [ 10.583929] [<c0b4a53c>] (dsa_switch_rcv) from [<c0a0b38c>] >> (__netif_receive_skb_list_core+0x218/0x238) >> [ 10.593343] r7:00000000 r6:c1d33d24 r5:c2208920 r4:c1d33d24 >> [ 10.599017] [<c0a0b174>] (__netif_receive_skb_list_core) from >> [<c0a0b5c0>] (netif_receive_skb_list_internal+0x214/0x2dc) >> [ 10.609909] r10:c230742c r9:00000000 r8:c1d32000 r7:c1d33d74 >> r6:00000000 r5:c89015c0 >> [ 10.617755] r4:c8adc300 >> [ 10.620300] [<c0a0b3ac>] (netif_receive_skb_list_internal) from >> [<c0a0b7fc>] (gro_normal_list.part.42+0x24/0x38) >> [ 10.630496] r10:00000162 r9:0000016c r8:00000001 r7:00000000 >> r6:c8901540 r5:c89015c0 >> [ 10.638342] r4:c8901540 >> [ 10.640885] [<c0a0b7d8>] (gro_normal_list.part.42) from [<c0a0c660>] >> (napi_complete_done+0x170/0x1d0) >> [ 10.650123] r5:c8adc300 r4:00000000 >> [ 10.653712] [<c0a0c4f0>] (napi_complete_done) from [<c083df00>] >> (bcm_sysport_poll+0x410/0x4b4) >> [ 10.662343] r9:0000016c r8:00000001 r7:c8901000 r6:0000015e >> r5:c8adc300 r4:c8901540 >> [ 10.670104] [<c083daf0>] (bcm_sysport_poll) from [<c0a0c94c>] >> (net_rx_action+0x28c/0x44c) >> [ 10.678301] r10:fffb942f r9:d1003000 r8:0000012c r7:c22b07e4 >> r6:00000040 r5:00000001 >> [ 10.686146] r4:c8901540 >> [ 10.688693] [<c0a0c6c0>] (net_rx_action) from [<c020238c>] >> (__do_softirq+0x134/0x414) >> [ 10.696542] r10:00000008 r9:d1003000 r8:00000100 r7:c22b07e4 >> r6:c1d32000 r5:00000003 >> [ 10.704387] r4:c220308c >> [ 10.706931] [<c0202258>] (__do_softirq) from [<c022bc70>] >> (irq_exit+0xf4/0x100) >> [ 10.714257] r10:00000000 r9:d1003000 r8:c9019000 r7:00000001 >> r6:00000000 r5:00000000 >> [ 10.722104] r4:ffffe000 >> [ 10.724650] [<c022bb7c>] (irq_exit) from [<c027feb4>] >> (__handle_domain_irq+0x70/0xc4) >> [ 10.732497] r5:00000000 r4:c2075fbc >> [ 10.736083] [<c027fe44>] (__handle_domain_irq) from [<c0202214>] >> (gic_handle_irq+0x5c/0xa0) >> [ 10.744453] r9:d1003000 r8:d1002000 r7:c1d33fb0 r6:d100200c >> r5:c220565c r4:c226d668 >> [ 10.752215] [<c02021b8>] (gic_handle_irq) from [<c0201d8c>] >> (__irq_usr+0x4c/0x60) >> [ 10.759713] Exception stack(0xc1d33fb0 to 0xc1d33ff8) >> [ 10.764776] 3fa0: b6ec1778 >> b6f32094 62632e73 62740000 >> [ 10.772973] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 >> 00000000 00000000 00000006 >> [ 10.781167] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 ffffffff >> [ 10.787797] r9:00000000 r8:30c5387d r7:30c5387d r6:ffffffff >> r5:000f0030 r4:b6f1e0b6 >> [ 10.795559] Code: e30609dc e34c0220 e083300c e590c000 (e5930008) >> [ 10.801670] ---[ end trace 97c3942fa73eff4c ]--- >> [ 10.806300] Kernel panic - not syncing: Fatal exception in interrupt >> [ 10.812678] CPU2: stopping >> [ 10.815403] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G D >> 5.4.105-1.0pre-geff642e2af2b #4 >> [ 10.824641] Hardware name: Broadcom STB (Flattened Device Tree) >> [ 10.830573] Backtrace: >> [ 10.833036] [<c020dd30>] (dump_backtrace) from [<c020e04c>] >> (show_stack+0x20/0x24) >> [ 10.840624] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 >> [ 10.846302] [<c020e02c>] (show_stack) from [<c0c09924>] >> (dump_stack+0xb8/0xe4) >> [ 10.853546] [<c0c0986c>] (dump_stack) from [<c0212160>] >> (handle_IPI+0x344/0x3cc) >> [ 10.860960] r10:00000000 r9:c2205400 r8:00000000 r7:00000002 >> r6:c22b0744 r5:00000004 >> [ 10.868807] r4:c22ce308 r3:00976ec2 >> [ 10.872395] [<c0211e1c>] (handle_IPI) from [<c0202254>] >> (gic_handle_irq+0x9c/0xa0) >> [ 10.879983] r10:00000000 r9:d1003000 r8:d1002000 r7:c9131f18 >> r6:d100200c r5:c220565c >> [ 10.887829] r4:c226d668 >> [ 10.890373] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] >> (__irq_svc+0x5c/0x7c) >> [ 10.897871] Exception stack(0xc9131f18 to 0xc9131f60) >> [ 10.902934] 1f00: >> c020a47c 00000000 >> [ 10.911131] 1f20: 0e4ca000 600c0093 c9130000 c2204cf0 c2204d34 >> 00000004 00000000 c20757b0 >> [ 10.919327] 1f40: 00000000 c9131f74 c9130000 c9131f68 00000000 >> c020a480 600c0013 ffffffff >> [ 10.927523] r9:c9130000 r8:00000000 r7:c9131f4c r6:ffffffff >> r5:600c0013 r4:c020a480 >> [ 10.935291] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] >> (default_idle_call+0x34/0x48) >> [ 10.943403] [<c0c13138>] (default_idle_call) from [<c02580c0>] >> (do_idle+0x1d4/0x2c0) >> [ 10.951164] [<c0257eec>] (do_idle) from [<c0258494>] >> (cpu_startup_entry+0x28/0x2c) >> [ 10.958752] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 >> r6:30c0387d r5:00000002 >> [ 10.966598] r4:0000008a >> [ 10.969142] [<c025846c>] (cpu_startup_entry) from [<c0211644>] >> (secondary_start_kernel+0x17c/0x1a0) >> [ 10.978210] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] >> (0x20270c) >> [ 10.985447] r5:00000000 r4:090eaa40 >> [ 10.989032] CPU3: stopping >> [ 10.991754] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G D >> 5.4.105-1.0pre-geff642e2af2b #4 >> [ 11.000992] Hardware name: Broadcom STB (Flattened Device Tree) >> [ 11.006924] Backtrace: >> [ 11.009384] [<c020dd30>] (dump_backtrace) from [<c020e04c>] >> (show_stack+0x20/0x24) >> [ 11.016972] r7:c22a86d0 r6:00000000 r5:600f0193 r4:c22a86d0 >> [ 11.022649] [<c020e02c>] (show_stack) from [<c0c09924>] >> (dump_stack+0xb8/0xe4) >> [ 11.029892] [<c0c0986c>] (dump_stack) from [<c0212160>] >> (handle_IPI+0x344/0x3cc) >> [ 11.037307] r10:00000000 r9:c2205400 r8:00000000 r7:00000003 >> r6:c22b0744 r5:00000004 >> [ 11.045154] r4:c22ce308 r3:00976ec2 >> [ 11.048741] [<c0211e1c>] (handle_IPI) from [<c0202254>] >> (gic_handle_irq+0x9c/0xa0) >> [ 11.056329] r10:00000000 r9:d1003000 r8:d1002000 r7:c9133f18 >> r6:d100200c r5:c220565c >> [ 11.064176] r4:c226d668 >> [ 11.066719] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] >> (__irq_svc+0x5c/0x7c) >> [ 11.074216] Exception stack(0xc9133f18 to 0xc9133f60) >> [ 11.079280] 3f00: >> c020a47c 00000000 >> [ 11.087476] 3f20: 0e4dd000 600f0093 c9132000 c2204cf0 c2204d34 >> 00000008 00000000 c20757b0 >> [ 11.095671] 3f40: 00000000 c9133f74 c9132000 c9133f68 00000000 >> c020a480 600f0013 ffffffff >> [ 11.103867] r9:c9132000 r8:00000000 r7:c9133f4c r6:ffffffff >> r5:600f0013 r4:c020a480 >> [ 11.111633] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] >> (default_idle_call+0x34/0x48) >> [ 11.119744] [<c0c13138>] (default_idle_call) from [<c02580c0>] >> (do_idle+0x1d4/0x2c0) >> [ 11.127506] [<c0257eec>] (do_idle) from [<c0258494>] >> (cpu_startup_entry+0x28/0x2c) >> [ 11.135093] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 >> r6:30c0387d r5:00000003 >> [ 11.142939] r4:0000008a >> [ 11.145484] [<c025846c>] (cpu_startup_entry) from [<c0211644>] >> (secondary_start_kernel+0x17c/0x1a0) >> [ 11.154550] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] >> (0x20270c) >> [ 11.161788] r5:00000000 r4:090eaa40 >> [ 11.165372] CPU1: stopping >> [ 11.168094] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G D >> 5.4.105-1.0pre-geff642e2af2b #4 >> [ 11.177332] Hardware name: Broadcom STB (Flattened Device Tree) >> [ 11.183264] Backtrace: >> [ 11.185723] [<c020dd30>] (dump_backtrace) from [<c020e04c>] >> (show_stack+0x20/0x24) >> [ 11.193311] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 >> [ 11.198988] [<c020e02c>] (show_stack) from [<c0c09924>] >> (dump_stack+0xb8/0xe4) >> [ 11.206230] [<c0c0986c>] (dump_stack) from [<c0212160>] >> (handle_IPI+0x344/0x3cc) >> [ 11.213644] r10:00000000 r9:c2205400 r8:00000000 r7:00000001 >> r6:c22b0744 r5:00000004 >> [ 11.221491] r4:c22ce308 r3:00976ec2 >> [ 11.225078] [<c0211e1c>] (handle_IPI) from [<c0202254>] >> (gic_handle_irq+0x9c/0xa0) >> [ 11.232666] r10:00000000 r9:d1003000 r8:d1002000 r7:c912ff18 >> r6:d100200c r5:c220565c >> [ 11.240512] r4:c226d668 >> [ 11.243054] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] >> (__irq_svc+0x5c/0x7c) >> [ 11.250553] Exception stack(0xc912ff18 to 0xc912ff60) >> [ 11.255617] ff00: >> c020a47c 00000000 >> [ 11.263814] ff20: 0e4b7000 600c0093 c912e000 c2204cf0 c2204d34 >> 00000002 00000000 c20757b0 >> [ 11.272010] ff40: 00000000 c912ff74 c912e000 c912ff68 00000000 >> c020a480 600c0013 ffffffff >> [ 11.280206] r9:c912e000 r8:00000000 r7:c912ff4c r6:ffffffff >> r5:600c0013 r4:c020a480 >> [ 11.287970] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] >> (default_idle_call+0x34/0x48) >> [ 11.296081] [<c0c13138>] (default_idle_call) from [<c02580c0>] >> (do_idle+0x1d4/0x2c0) >> [ 11.303842] [<c0257eec>] (do_idle) from [<c0258494>] >> (cpu_startup_entry+0x28/0x2c) >> [ 11.311430] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 >> r6:30c0387d r5:00000001 >> [ 11.319275] r4:0000008a >> [ 11.321819] [<c025846c>] (cpu_startup_entry) from [<c0211644>] >> (secondary_start_kernel+0x17c/0x1a0) >> [ 11.330886] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] >> (0x20270c) >> [ 11.338124] r5:00000000 r4:090eaa40 >> [ 11.341729] ---[ end Kernel panic - not syncing: Fatal exception in >> interrupt ]--- >> >> it is not marked as fixing anything so I wonder how it landed in stable? > > It was requested to be merged.
OK.
> Do you have the same crash on newer kernels with this commit in it (like Linus's tree?)
There are no crashes with 5.10 or upstream otherwise it would have been noticed earlier, I have not kept track of which additional changes/fixes we may need but I would suggest we drop this one for now.
Alexander, do you want me to test additional patches if your change somehow must be included in an upcoming 5.4? I thought the platform you are working is still not upstream, so who is going to benefit from this performance improvement?
Hello Florian!
I have heavily tested this change backported to 4.19 and 5.4 kernels and it radically increased network performance on DSA setup with mv88e6xxx switch. I have not seen any stability issues. So based on tests I have sent backport request as patch is relatively small, is already in mainline kernel and can be cleanly applied to 4.19 and 5.4 kernel.
But if this patch really cause issues, I think based on 'no regression' rule we have no other options, just to drop this patch from stable queue.
Greg, please drop this patch (for now).
But I would be really interested why this patch does not cause any regression with mainline kernel and cause with 5.4...
Florian, which DSA driver and hardware is causing this issue?
This is using drivers/net/dsa/bcm_sf2.c for the switch driver (which uses b53 as the library for switch operations) and drivers/net/ethernet/broadcom/bcmsysport.c for the DSA "master".
Could you checkout kernel at e131a5634830047923c694b4ce0c3b31745ff01b commit and test if this issue happen too?
This patch was added to stable trees by Pali, not me (link: [0]). I added him to Ccs, so you can ask him. As you noted correctly, I don't work on/with stable releases.
Is 4.19 also failing for you now?
I cannot easily test 4.19 without applying some additional patches bringing in various ARM SCMI changes, I would suspect the same to happen. Give me a few hours and I will see if I can give you a better answer with an actual test.
Ok, if you have some results let me know.
I did test 4.19 eventually and was not able to reproduce the problem, and have not been able to reproduce the problem on 5.4 a second time around and ever since merging v5.4.105 in the past couple weeks.
Ok, fine! Nice to have this information.
I would consider this a non-issue and just a fluke. The board that exposed the problem is known for having power distribution issues when an aggressive CPU frequency scaling governor like schedutil is used. And it makes sense that the CPU frequency would be changed when traffic picks up. The boards have since been hard coded to the "performance" governor, that is the problematic ones at least. -- Florian
Makes sense. This patch increase bandwidth throw DSA ports so it can also increase CPU usage or frequency. And if board has CPU instability issues then increased CPU usage can hit this issue with higher probability. But in this case patch itself should be safe-
Thank you for sharing results of your investigation. Original DSA patch in this case can be backported into stable kernels and boards with DSA switches can benefit from better performance.
On Tue, Mar 23, 2021 at 10:20:27AM -0700, Florian Fainelli wrote:
On 3/15/2021 2:50 AM, Pali Rohár wrote:
On Friday 12 March 2021 12:54:18 Alexander Lobakin wrote:
From: Florian Fainelli f.fainelli@gmail.com Date: Thu, 11 Mar 2021 09:41:27 -0800
Hi Florian,
On 3/11/21 9:40 AM, Greg KH wrote:
On Thu, Mar 11, 2021 at 09:23:56AM -0800, Florian Fainelli wrote:
On 3/11/21 5:08 AM, Greg KH wrote: > On Wed, Mar 10, 2021 at 08:19:45PM -0800, Florian Fainelli wrote: >> +Alex, >> >> On 3/10/2021 5:24 AM, gregkh@linuxfoundation.org wrote: >>> From: Greg Kroah-Hartman gregkh@linuxfoundation.org >>> >>> This is the start of the stable review cycle for the 5.4.105 release. >>> There are 24 patches in this series, all will be posted as a response >>> to this one. If anyone has any issues with these being applied, please >>> let me know. >>> >>> Responses should be made by Fri, 12 Mar 2021 13:23:09 +0000. >>> Anything received after that time might be too late. >>> >>> The whole patch series can be found in one patch at: >>> https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/patch-5.4.105-rc1... >>> or in the git tree and branch at: >>> git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-5.4.y >>> and the diffstat can be found below. >>> >>> thanks, >>> >>> greg k-h >> >> I believe you need to drop "net: dsa: add GRO support via gro_cells" as >> it causes the following kernel panic on a DSA-enabled platform: >> >> Configuring rgmii_2 interface >> [ 10.170527] brcm-sf2 f0b00000.ethernet_switch rgmii_2: configuring >> for fixed/rgmii-txid link mode >> [ 10.179597] 8021q: adding VLAN 0 to HW filter on device rgmii_2 >> [ 10.185608] brcm-sf2 f0b00000.ethernet_switch rgmii_2: Link is Up - >> 1Gbps/Full - flow control off >> [ 10.198631] IPv6: ADDRCONF(NETDEV_CHANGE): rgmii_2: link becomes ready >> Configuring sit0 interface >> [ 10.254346] 8<--- cut here --- >> [ 10.257438] Unable to handle kernel paging request at virtual address >> d6df6190 >> [ 10.264685] pgd = (ptrval) >> [ 10.267411] [d6df6190] *pgd=80000000007003, *pmd=00000000 >> [ 10.272846] Internal error: Oops: 206 [#1] SMP ARM >> [ 10.277661] Modules linked in: >> [ 10.280739] CPU: 0 PID: 1886 Comm: sed Not tainted >> 5.4.105-1.0pre-geff642e2af2b #4 >> [ 10.288337] Hardware name: Broadcom STB (Flattened Device Tree) >> [ 10.294292] PC is at gro_cells_receive+0x90/0x11c >> [ 10.299020] LR is at dsa_switch_rcv+0x120/0x1d4 >> [ 10.303562] pc : [<c0a57a28>] lr : [<c0b4a65c>] psr: 600f0113 >> [ 10.309841] sp : c1d33cd0 ip : 000003e8 fp : c1d33ce4 >> [ 10.315078] r10: c8901000 r9 : c8901000 r8 : c0b4a53c >> [ 10.320314] r7 : c2208920 r6 : 00000000 r5 : 00000000 r4 : 00004000 >> [ 10.326855] r3 : d6df6188 r2 : c4927000 r1 : c8adc300 r0 : c22069dc >> [ 10.333398] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM >> Segment user >> [ 10.340547] Control: 30c5387d Table: 04ac4c80 DAC: fffffffd >> [ 10.346307] Process sed (pid: 1886, stack limit = 0x(ptrval)) >> [ 10.352066] Stack: (0xc1d33cd0 to 0xc1d34000) >> [ 10.356434] 3cc0: c8adc300 >> c4927000 c1d33d04 c1d33ce8 >> [ 10.364631] 3ce0: c0b4a65c c0a579a4 c1d33d24 c2208920 c1d33d24 >> 00000000 c1d33d5c c1d33d08 >> [ 10.372827] 3d00: c0a0b38c c0b4a548 c021e070 c2204cc8 00000000 >> c89015c0 04b87700 c89015c0 >> [ 10.381023] 3d20: c2208920 c1d33d24 c1d33d24 00976ec2 04b87700 >> c8adc300 c89015c0 00000000 >> [ 10.389218] 3d40: c1d33d74 c1d32000 00000000 c230742c c1d33dac >> c1d33d60 c0a0b5c0 c0a0b180 >> [ 10.397414] 3d60: 00000000 c2204cc8 00000000 c1d33d6c c1d33d6c >> c1d33d80 c029daf8 00976ec2 >> [ 10.405610] 3d80: 00000800 c8901540 c89015c0 c8901540 00000000 >> 00000001 0000016c 00000162 >> [ 10.413805] 3da0: c1d33dc4 c1d33db0 c0a0b7fc c0a0b3b8 00000000 >> c8adc300 c1d33dfc c1d33dc8 >> [ 10.422001] 3dc0: c0a0c660 c0a0b7e4 c8901540 c8adc300 c1d33dfc >> c1d33de0 c8901540 c8adc300 >> [ 10.430196] 3de0: 0000015e c8901000 00000001 0000016c c1d33e74 >> c1d33e00 c083df00 c0a0c4fc >> [ 10.438391] 3e00: 0000012c c22b0f14 c1d33e4c c1d33e18 c0fbd9b8 >> c0fbd9cc c0fbd9e0 c0fbd98c >> [ 10.446586] 3e20: 00000001 00000040 c8901500 00000001 00000000 >> 00000000 00000000 00000000 >> [ 10.454780] 3e40: 00000000 00000000 c02f65a0 c8901540 00000001 >> 00000040 c22b07e4 0000012c >> [ 10.462975] 3e60: d1003000 fffb942f c1d33edc c1d33e78 c0a0c94c >> c083dafc d051ad80 c2204cc8 >> [ 10.471170] 3e80: c2204cf0 c1d32000 c22b40b0 0e4a4000 c2076d80 >> c2203d00 c022bc70 c1d33e9c >> [ 10.479365] 3ea0: c1d33e9c c1d33ea4 c1d33ea4 00976ec2 c02f65a0 >> c220308c 00000003 c1d32000 >> [ 10.487561] 3ec0: c22b07e4 00000100 d1003000 00000008 c1d33f44 >> c1d33ee0 c020238c c0a0c6cc >> [ 10.495755] 3ee0: c1d33f14 c1d33ef0 00000001 00400000 c2203d00 >> fffb942f c206b2e4 c2076040 >> [ 10.503950] 3f00: c2076040 0000000a c2203080 c206b358 c1d33ee0 >> 00000004 c90c9500 ffffe000 >> [ 10.512145] 3f20: 00000000 00000000 00000001 c9019000 d1003000 >> 00000000 c1d33f5c c1d33f48 >> [ 10.520339] 3f40: c022bc70 c0202264 c2075fbc 00000000 c1d33f84 >> c1d33f60 c027feb4 c022bb88 >> [ 10.528535] 3f60: c226d668 c220565c d100200c c1d33fb0 d1002000 >> d1003000 c1d33fac c1d33f88 >> [ 10.536730] 3f80: c0202214 c027fe50 b6f1e0b6 000f0030 ffffffff >> 30c5387d 30c5387d 00000000 >> [ 10.544924] 3fa0: 00000000 c1d33fb0 c0201d8c c02021c4 b6ec1778 >> b6f32094 62632e73 62740000 >> [ 10.553120] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 >> 00000000 00000000 00000006 >> [ 10.561316] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 >> ffffffff 00000000 00000000 >> [ 10.569508] Backtrace: >> [ 10.571970] [<c0a57998>] (gro_cells_receive) from [<c0b4a65c>] >> (dsa_switch_rcv+0x120/0x1d4) >> [ 10.580338] r5:c4927000 r4:c8adc300 >> [ 10.583929] [<c0b4a53c>] (dsa_switch_rcv) from [<c0a0b38c>] >> (__netif_receive_skb_list_core+0x218/0x238) >> [ 10.593343] r7:00000000 r6:c1d33d24 r5:c2208920 r4:c1d33d24 >> [ 10.599017] [<c0a0b174>] (__netif_receive_skb_list_core) from >> [<c0a0b5c0>] (netif_receive_skb_list_internal+0x214/0x2dc) >> [ 10.609909] r10:c230742c r9:00000000 r8:c1d32000 r7:c1d33d74 >> r6:00000000 r5:c89015c0 >> [ 10.617755] r4:c8adc300 >> [ 10.620300] [<c0a0b3ac>] (netif_receive_skb_list_internal) from >> [<c0a0b7fc>] (gro_normal_list.part.42+0x24/0x38) >> [ 10.630496] r10:00000162 r9:0000016c r8:00000001 r7:00000000 >> r6:c8901540 r5:c89015c0 >> [ 10.638342] r4:c8901540 >> [ 10.640885] [<c0a0b7d8>] (gro_normal_list.part.42) from [<c0a0c660>] >> (napi_complete_done+0x170/0x1d0) >> [ 10.650123] r5:c8adc300 r4:00000000 >> [ 10.653712] [<c0a0c4f0>] (napi_complete_done) from [<c083df00>] >> (bcm_sysport_poll+0x410/0x4b4) >> [ 10.662343] r9:0000016c r8:00000001 r7:c8901000 r6:0000015e >> r5:c8adc300 r4:c8901540 >> [ 10.670104] [<c083daf0>] (bcm_sysport_poll) from [<c0a0c94c>] >> (net_rx_action+0x28c/0x44c) >> [ 10.678301] r10:fffb942f r9:d1003000 r8:0000012c r7:c22b07e4 >> r6:00000040 r5:00000001 >> [ 10.686146] r4:c8901540 >> [ 10.688693] [<c0a0c6c0>] (net_rx_action) from [<c020238c>] >> (__do_softirq+0x134/0x414) >> [ 10.696542] r10:00000008 r9:d1003000 r8:00000100 r7:c22b07e4 >> r6:c1d32000 r5:00000003 >> [ 10.704387] r4:c220308c >> [ 10.706931] [<c0202258>] (__do_softirq) from [<c022bc70>] >> (irq_exit+0xf4/0x100) >> [ 10.714257] r10:00000000 r9:d1003000 r8:c9019000 r7:00000001 >> r6:00000000 r5:00000000 >> [ 10.722104] r4:ffffe000 >> [ 10.724650] [<c022bb7c>] (irq_exit) from [<c027feb4>] >> (__handle_domain_irq+0x70/0xc4) >> [ 10.732497] r5:00000000 r4:c2075fbc >> [ 10.736083] [<c027fe44>] (__handle_domain_irq) from [<c0202214>] >> (gic_handle_irq+0x5c/0xa0) >> [ 10.744453] r9:d1003000 r8:d1002000 r7:c1d33fb0 r6:d100200c >> r5:c220565c r4:c226d668 >> [ 10.752215] [<c02021b8>] (gic_handle_irq) from [<c0201d8c>] >> (__irq_usr+0x4c/0x60) >> [ 10.759713] Exception stack(0xc1d33fb0 to 0xc1d33ff8) >> [ 10.764776] 3fa0: b6ec1778 >> b6f32094 62632e73 62740000 >> [ 10.772973] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 >> 00000000 00000000 00000006 >> [ 10.781167] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 ffffffff >> [ 10.787797] r9:00000000 r8:30c5387d r7:30c5387d r6:ffffffff >> r5:000f0030 r4:b6f1e0b6 >> [ 10.795559] Code: e30609dc e34c0220 e083300c e590c000 (e5930008) >> [ 10.801670] ---[ end trace 97c3942fa73eff4c ]--- >> [ 10.806300] Kernel panic - not syncing: Fatal exception in interrupt >> [ 10.812678] CPU2: stopping >> [ 10.815403] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G D >> 5.4.105-1.0pre-geff642e2af2b #4 >> [ 10.824641] Hardware name: Broadcom STB (Flattened Device Tree) >> [ 10.830573] Backtrace: >> [ 10.833036] [<c020dd30>] (dump_backtrace) from [<c020e04c>] >> (show_stack+0x20/0x24) >> [ 10.840624] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 >> [ 10.846302] [<c020e02c>] (show_stack) from [<c0c09924>] >> (dump_stack+0xb8/0xe4) >> [ 10.853546] [<c0c0986c>] (dump_stack) from [<c0212160>] >> (handle_IPI+0x344/0x3cc) >> [ 10.860960] r10:00000000 r9:c2205400 r8:00000000 r7:00000002 >> r6:c22b0744 r5:00000004 >> [ 10.868807] r4:c22ce308 r3:00976ec2 >> [ 10.872395] [<c0211e1c>] (handle_IPI) from [<c0202254>] >> (gic_handle_irq+0x9c/0xa0) >> [ 10.879983] r10:00000000 r9:d1003000 r8:d1002000 r7:c9131f18 >> r6:d100200c r5:c220565c >> [ 10.887829] r4:c226d668 >> [ 10.890373] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] >> (__irq_svc+0x5c/0x7c) >> [ 10.897871] Exception stack(0xc9131f18 to 0xc9131f60) >> [ 10.902934] 1f00: >> c020a47c 00000000 >> [ 10.911131] 1f20: 0e4ca000 600c0093 c9130000 c2204cf0 c2204d34 >> 00000004 00000000 c20757b0 >> [ 10.919327] 1f40: 00000000 c9131f74 c9130000 c9131f68 00000000 >> c020a480 600c0013 ffffffff >> [ 10.927523] r9:c9130000 r8:00000000 r7:c9131f4c r6:ffffffff >> r5:600c0013 r4:c020a480 >> [ 10.935291] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] >> (default_idle_call+0x34/0x48) >> [ 10.943403] [<c0c13138>] (default_idle_call) from [<c02580c0>] >> (do_idle+0x1d4/0x2c0) >> [ 10.951164] [<c0257eec>] (do_idle) from [<c0258494>] >> (cpu_startup_entry+0x28/0x2c) >> [ 10.958752] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 >> r6:30c0387d r5:00000002 >> [ 10.966598] r4:0000008a >> [ 10.969142] [<c025846c>] (cpu_startup_entry) from [<c0211644>] >> (secondary_start_kernel+0x17c/0x1a0) >> [ 10.978210] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] >> (0x20270c) >> [ 10.985447] r5:00000000 r4:090eaa40 >> [ 10.989032] CPU3: stopping >> [ 10.991754] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G D >> 5.4.105-1.0pre-geff642e2af2b #4 >> [ 11.000992] Hardware name: Broadcom STB (Flattened Device Tree) >> [ 11.006924] Backtrace: >> [ 11.009384] [<c020dd30>] (dump_backtrace) from [<c020e04c>] >> (show_stack+0x20/0x24) >> [ 11.016972] r7:c22a86d0 r6:00000000 r5:600f0193 r4:c22a86d0 >> [ 11.022649] [<c020e02c>] (show_stack) from [<c0c09924>] >> (dump_stack+0xb8/0xe4) >> [ 11.029892] [<c0c0986c>] (dump_stack) from [<c0212160>] >> (handle_IPI+0x344/0x3cc) >> [ 11.037307] r10:00000000 r9:c2205400 r8:00000000 r7:00000003 >> r6:c22b0744 r5:00000004 >> [ 11.045154] r4:c22ce308 r3:00976ec2 >> [ 11.048741] [<c0211e1c>] (handle_IPI) from [<c0202254>] >> (gic_handle_irq+0x9c/0xa0) >> [ 11.056329] r10:00000000 r9:d1003000 r8:d1002000 r7:c9133f18 >> r6:d100200c r5:c220565c >> [ 11.064176] r4:c226d668 >> [ 11.066719] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] >> (__irq_svc+0x5c/0x7c) >> [ 11.074216] Exception stack(0xc9133f18 to 0xc9133f60) >> [ 11.079280] 3f00: >> c020a47c 00000000 >> [ 11.087476] 3f20: 0e4dd000 600f0093 c9132000 c2204cf0 c2204d34 >> 00000008 00000000 c20757b0 >> [ 11.095671] 3f40: 00000000 c9133f74 c9132000 c9133f68 00000000 >> c020a480 600f0013 ffffffff >> [ 11.103867] r9:c9132000 r8:00000000 r7:c9133f4c r6:ffffffff >> r5:600f0013 r4:c020a480 >> [ 11.111633] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] >> (default_idle_call+0x34/0x48) >> [ 11.119744] [<c0c13138>] (default_idle_call) from [<c02580c0>] >> (do_idle+0x1d4/0x2c0) >> [ 11.127506] [<c0257eec>] (do_idle) from [<c0258494>] >> (cpu_startup_entry+0x28/0x2c) >> [ 11.135093] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 >> r6:30c0387d r5:00000003 >> [ 11.142939] r4:0000008a >> [ 11.145484] [<c025846c>] (cpu_startup_entry) from [<c0211644>] >> (secondary_start_kernel+0x17c/0x1a0) >> [ 11.154550] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] >> (0x20270c) >> [ 11.161788] r5:00000000 r4:090eaa40 >> [ 11.165372] CPU1: stopping >> [ 11.168094] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G D >> 5.4.105-1.0pre-geff642e2af2b #4 >> [ 11.177332] Hardware name: Broadcom STB (Flattened Device Tree) >> [ 11.183264] Backtrace: >> [ 11.185723] [<c020dd30>] (dump_backtrace) from [<c020e04c>] >> (show_stack+0x20/0x24) >> [ 11.193311] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 >> [ 11.198988] [<c020e02c>] (show_stack) from [<c0c09924>] >> (dump_stack+0xb8/0xe4) >> [ 11.206230] [<c0c0986c>] (dump_stack) from [<c0212160>] >> (handle_IPI+0x344/0x3cc) >> [ 11.213644] r10:00000000 r9:c2205400 r8:00000000 r7:00000001 >> r6:c22b0744 r5:00000004 >> [ 11.221491] r4:c22ce308 r3:00976ec2 >> [ 11.225078] [<c0211e1c>] (handle_IPI) from [<c0202254>] >> (gic_handle_irq+0x9c/0xa0) >> [ 11.232666] r10:00000000 r9:d1003000 r8:d1002000 r7:c912ff18 >> r6:d100200c r5:c220565c >> [ 11.240512] r4:c226d668 >> [ 11.243054] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] >> (__irq_svc+0x5c/0x7c) >> [ 11.250553] Exception stack(0xc912ff18 to 0xc912ff60) >> [ 11.255617] ff00: >> c020a47c 00000000 >> [ 11.263814] ff20: 0e4b7000 600c0093 c912e000 c2204cf0 c2204d34 >> 00000002 00000000 c20757b0 >> [ 11.272010] ff40: 00000000 c912ff74 c912e000 c912ff68 00000000 >> c020a480 600c0013 ffffffff >> [ 11.280206] r9:c912e000 r8:00000000 r7:c912ff4c r6:ffffffff >> r5:600c0013 r4:c020a480 >> [ 11.287970] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] >> (default_idle_call+0x34/0x48) >> [ 11.296081] [<c0c13138>] (default_idle_call) from [<c02580c0>] >> (do_idle+0x1d4/0x2c0) >> [ 11.303842] [<c0257eec>] (do_idle) from [<c0258494>] >> (cpu_startup_entry+0x28/0x2c) >> [ 11.311430] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 >> r6:30c0387d r5:00000001 >> [ 11.319275] r4:0000008a >> [ 11.321819] [<c025846c>] (cpu_startup_entry) from [<c0211644>] >> (secondary_start_kernel+0x17c/0x1a0) >> [ 11.330886] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] >> (0x20270c) >> [ 11.338124] r5:00000000 r4:090eaa40 >> [ 11.341729] ---[ end Kernel panic - not syncing: Fatal exception in >> interrupt ]--- >> >> it is not marked as fixing anything so I wonder how it landed in stable? > > It was requested to be merged.
OK.
> Do you have the same crash on newer kernels with this commit in it (like Linus's tree?)
There are no crashes with 5.10 or upstream otherwise it would have been noticed earlier, I have not kept track of which additional changes/fixes we may need but I would suggest we drop this one for now.
Alexander, do you want me to test additional patches if your change somehow must be included in an upcoming 5.4? I thought the platform you are working is still not upstream, so who is going to benefit from this performance improvement?
Hello Florian!
I have heavily tested this change backported to 4.19 and 5.4 kernels and it radically increased network performance on DSA setup with mv88e6xxx switch. I have not seen any stability issues. So based on tests I have sent backport request as patch is relatively small, is already in mainline kernel and can be cleanly applied to 4.19 and 5.4 kernel.
But if this patch really cause issues, I think based on 'no regression' rule we have no other options, just to drop this patch from stable queue.
Greg, please drop this patch (for now).
But I would be really interested why this patch does not cause any regression with mainline kernel and cause with 5.4...
Florian, which DSA driver and hardware is causing this issue?
This is using drivers/net/dsa/bcm_sf2.c for the switch driver (which uses b53 as the library for switch operations) and drivers/net/ethernet/broadcom/bcmsysport.c for the DSA "master".
Could you checkout kernel at e131a5634830047923c694b4ce0c3b31745ff01b commit and test if this issue happen too?
This patch was added to stable trees by Pali, not me (link: [0]). I added him to Ccs, so you can ask him. As you noted correctly, I don't work on/with stable releases.
Is 4.19 also failing for you now?
I cannot easily test 4.19 without applying some additional patches bringing in various ARM SCMI changes, I would suspect the same to happen. Give me a few hours and I will see if I can give you a better answer with an actual test.
Ok, if you have some results let me know.
I did test 4.19 eventually and was not able to reproduce the problem, and have not been able to reproduce the problem on 5.4 a second time around and ever since merging v5.4.105 in the past couple weeks.
I would consider this a non-issue and just a fluke. The board that exposed the problem is known for having power distribution issues when an aggressive CPU frequency scaling governor like schedutil is used. And it makes sense that the CPU frequency would be changed when traffic picks up. The boards have since been hard coded to the "performance" governor, that is the problematic ones at least.
THanks for tracking this down and letting us know.
greg k-h
On 3/11/21 9:41 AM, Florian Fainelli wrote:
On 3/11/21 9:40 AM, Greg KH wrote:
On Thu, Mar 11, 2021 at 09:23:56AM -0800, Florian Fainelli wrote:
On 3/11/21 5:08 AM, Greg KH wrote:
On Wed, Mar 10, 2021 at 08:19:45PM -0800, Florian Fainelli wrote:
+Alex,
On 3/10/2021 5:24 AM, gregkh@linuxfoundation.org wrote:
From: Greg Kroah-Hartman gregkh@linuxfoundation.org
This is the start of the stable review cycle for the 5.4.105 release. There are 24 patches in this series, all will be posted as a response to this one. If anyone has any issues with these being applied, please let me know.
Responses should be made by Fri, 12 Mar 2021 13:23:09 +0000. Anything received after that time might be too late.
The whole patch series can be found in one patch at: https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/patch-5.4.105-rc1... or in the git tree and branch at: git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-5.4.y and the diffstat can be found below.
thanks,
greg k-h
I believe you need to drop "net: dsa: add GRO support via gro_cells" as it causes the following kernel panic on a DSA-enabled platform:
Configuring rgmii_2 interface [ 10.170527] brcm-sf2 f0b00000.ethernet_switch rgmii_2: configuring for fixed/rgmii-txid link mode [ 10.179597] 8021q: adding VLAN 0 to HW filter on device rgmii_2 [ 10.185608] brcm-sf2 f0b00000.ethernet_switch rgmii_2: Link is Up - 1Gbps/Full - flow control off [ 10.198631] IPv6: ADDRCONF(NETDEV_CHANGE): rgmii_2: link becomes ready Configuring sit0 interface [ 10.254346] 8<--- cut here --- [ 10.257438] Unable to handle kernel paging request at virtual address d6df6190 [ 10.264685] pgd = (ptrval) [ 10.267411] [d6df6190] *pgd=80000000007003, *pmd=00000000 [ 10.272846] Internal error: Oops: 206 [#1] SMP ARM [ 10.277661] Modules linked in: [ 10.280739] CPU: 0 PID: 1886 Comm: sed Not tainted 5.4.105-1.0pre-geff642e2af2b #4 [ 10.288337] Hardware name: Broadcom STB (Flattened Device Tree) [ 10.294292] PC is at gro_cells_receive+0x90/0x11c [ 10.299020] LR is at dsa_switch_rcv+0x120/0x1d4 [ 10.303562] pc : [<c0a57a28>] lr : [<c0b4a65c>] psr: 600f0113 [ 10.309841] sp : c1d33cd0 ip : 000003e8 fp : c1d33ce4 [ 10.315078] r10: c8901000 r9 : c8901000 r8 : c0b4a53c [ 10.320314] r7 : c2208920 r6 : 00000000 r5 : 00000000 r4 : 00004000 [ 10.326855] r3 : d6df6188 r2 : c4927000 r1 : c8adc300 r0 : c22069dc [ 10.333398] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user [ 10.340547] Control: 30c5387d Table: 04ac4c80 DAC: fffffffd [ 10.346307] Process sed (pid: 1886, stack limit = 0x(ptrval)) [ 10.352066] Stack: (0xc1d33cd0 to 0xc1d34000) [ 10.356434] 3cc0: c8adc300 c4927000 c1d33d04 c1d33ce8 [ 10.364631] 3ce0: c0b4a65c c0a579a4 c1d33d24 c2208920 c1d33d24 00000000 c1d33d5c c1d33d08 [ 10.372827] 3d00: c0a0b38c c0b4a548 c021e070 c2204cc8 00000000 c89015c0 04b87700 c89015c0 [ 10.381023] 3d20: c2208920 c1d33d24 c1d33d24 00976ec2 04b87700 c8adc300 c89015c0 00000000 [ 10.389218] 3d40: c1d33d74 c1d32000 00000000 c230742c c1d33dac c1d33d60 c0a0b5c0 c0a0b180 [ 10.397414] 3d60: 00000000 c2204cc8 00000000 c1d33d6c c1d33d6c c1d33d80 c029daf8 00976ec2 [ 10.405610] 3d80: 00000800 c8901540 c89015c0 c8901540 00000000 00000001 0000016c 00000162 [ 10.413805] 3da0: c1d33dc4 c1d33db0 c0a0b7fc c0a0b3b8 00000000 c8adc300 c1d33dfc c1d33dc8 [ 10.422001] 3dc0: c0a0c660 c0a0b7e4 c8901540 c8adc300 c1d33dfc c1d33de0 c8901540 c8adc300 [ 10.430196] 3de0: 0000015e c8901000 00000001 0000016c c1d33e74 c1d33e00 c083df00 c0a0c4fc [ 10.438391] 3e00: 0000012c c22b0f14 c1d33e4c c1d33e18 c0fbd9b8 c0fbd9cc c0fbd9e0 c0fbd98c [ 10.446586] 3e20: 00000001 00000040 c8901500 00000001 00000000 00000000 00000000 00000000 [ 10.454780] 3e40: 00000000 00000000 c02f65a0 c8901540 00000001 00000040 c22b07e4 0000012c [ 10.462975] 3e60: d1003000 fffb942f c1d33edc c1d33e78 c0a0c94c c083dafc d051ad80 c2204cc8 [ 10.471170] 3e80: c2204cf0 c1d32000 c22b40b0 0e4a4000 c2076d80 c2203d00 c022bc70 c1d33e9c [ 10.479365] 3ea0: c1d33e9c c1d33ea4 c1d33ea4 00976ec2 c02f65a0 c220308c 00000003 c1d32000 [ 10.487561] 3ec0: c22b07e4 00000100 d1003000 00000008 c1d33f44 c1d33ee0 c020238c c0a0c6cc [ 10.495755] 3ee0: c1d33f14 c1d33ef0 00000001 00400000 c2203d00 fffb942f c206b2e4 c2076040 [ 10.503950] 3f00: c2076040 0000000a c2203080 c206b358 c1d33ee0 00000004 c90c9500 ffffe000 [ 10.512145] 3f20: 00000000 00000000 00000001 c9019000 d1003000 00000000 c1d33f5c c1d33f48 [ 10.520339] 3f40: c022bc70 c0202264 c2075fbc 00000000 c1d33f84 c1d33f60 c027feb4 c022bb88 [ 10.528535] 3f60: c226d668 c220565c d100200c c1d33fb0 d1002000 d1003000 c1d33fac c1d33f88 [ 10.536730] 3f80: c0202214 c027fe50 b6f1e0b6 000f0030 ffffffff 30c5387d 30c5387d 00000000 [ 10.544924] 3fa0: 00000000 c1d33fb0 c0201d8c c02021c4 b6ec1778 b6f32094 62632e73 62740000 [ 10.553120] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 00000000 00000000 00000006 [ 10.561316] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 ffffffff 00000000 00000000 [ 10.569508] Backtrace: [ 10.571970] [<c0a57998>] (gro_cells_receive) from [<c0b4a65c>] (dsa_switch_rcv+0x120/0x1d4) [ 10.580338] r5:c4927000 r4:c8adc300 [ 10.583929] [<c0b4a53c>] (dsa_switch_rcv) from [<c0a0b38c>] (__netif_receive_skb_list_core+0x218/0x238) [ 10.593343] r7:00000000 r6:c1d33d24 r5:c2208920 r4:c1d33d24 [ 10.599017] [<c0a0b174>] (__netif_receive_skb_list_core) from [<c0a0b5c0>] (netif_receive_skb_list_internal+0x214/0x2dc) [ 10.609909] r10:c230742c r9:00000000 r8:c1d32000 r7:c1d33d74 r6:00000000 r5:c89015c0 [ 10.617755] r4:c8adc300 [ 10.620300] [<c0a0b3ac>] (netif_receive_skb_list_internal) from [<c0a0b7fc>] (gro_normal_list.part.42+0x24/0x38) [ 10.630496] r10:00000162 r9:0000016c r8:00000001 r7:00000000 r6:c8901540 r5:c89015c0 [ 10.638342] r4:c8901540 [ 10.640885] [<c0a0b7d8>] (gro_normal_list.part.42) from [<c0a0c660>] (napi_complete_done+0x170/0x1d0) [ 10.650123] r5:c8adc300 r4:00000000 [ 10.653712] [<c0a0c4f0>] (napi_complete_done) from [<c083df00>] (bcm_sysport_poll+0x410/0x4b4) [ 10.662343] r9:0000016c r8:00000001 r7:c8901000 r6:0000015e r5:c8adc300 r4:c8901540 [ 10.670104] [<c083daf0>] (bcm_sysport_poll) from [<c0a0c94c>] (net_rx_action+0x28c/0x44c) [ 10.678301] r10:fffb942f r9:d1003000 r8:0000012c r7:c22b07e4 r6:00000040 r5:00000001 [ 10.686146] r4:c8901540 [ 10.688693] [<c0a0c6c0>] (net_rx_action) from [<c020238c>] (__do_softirq+0x134/0x414) [ 10.696542] r10:00000008 r9:d1003000 r8:00000100 r7:c22b07e4 r6:c1d32000 r5:00000003 [ 10.704387] r4:c220308c [ 10.706931] [<c0202258>] (__do_softirq) from [<c022bc70>] (irq_exit+0xf4/0x100) [ 10.714257] r10:00000000 r9:d1003000 r8:c9019000 r7:00000001 r6:00000000 r5:00000000 [ 10.722104] r4:ffffe000 [ 10.724650] [<c022bb7c>] (irq_exit) from [<c027feb4>] (__handle_domain_irq+0x70/0xc4) [ 10.732497] r5:00000000 r4:c2075fbc [ 10.736083] [<c027fe44>] (__handle_domain_irq) from [<c0202214>] (gic_handle_irq+0x5c/0xa0) [ 10.744453] r9:d1003000 r8:d1002000 r7:c1d33fb0 r6:d100200c r5:c220565c r4:c226d668 [ 10.752215] [<c02021b8>] (gic_handle_irq) from [<c0201d8c>] (__irq_usr+0x4c/0x60) [ 10.759713] Exception stack(0xc1d33fb0 to 0xc1d33ff8) [ 10.764776] 3fa0: b6ec1778 b6f32094 62632e73 62740000 [ 10.772973] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 00000000 00000000 00000006 [ 10.781167] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 ffffffff [ 10.787797] r9:00000000 r8:30c5387d r7:30c5387d r6:ffffffff r5:000f0030 r4:b6f1e0b6 [ 10.795559] Code: e30609dc e34c0220 e083300c e590c000 (e5930008) [ 10.801670] ---[ end trace 97c3942fa73eff4c ]--- [ 10.806300] Kernel panic - not syncing: Fatal exception in interrupt [ 10.812678] CPU2: stopping [ 10.815403] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 10.824641] Hardware name: Broadcom STB (Flattened Device Tree) [ 10.830573] Backtrace: [ 10.833036] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 10.840624] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 [ 10.846302] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 10.853546] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 10.860960] r10:00000000 r9:c2205400 r8:00000000 r7:00000002 r6:c22b0744 r5:00000004 [ 10.868807] r4:c22ce308 r3:00976ec2 [ 10.872395] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 10.879983] r10:00000000 r9:d1003000 r8:d1002000 r7:c9131f18 r6:d100200c r5:c220565c [ 10.887829] r4:c226d668 [ 10.890373] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 10.897871] Exception stack(0xc9131f18 to 0xc9131f60) [ 10.902934] 1f00: c020a47c 00000000 [ 10.911131] 1f20: 0e4ca000 600c0093 c9130000 c2204cf0 c2204d34 00000004 00000000 c20757b0 [ 10.919327] 1f40: 00000000 c9131f74 c9130000 c9131f68 00000000 c020a480 600c0013 ffffffff [ 10.927523] r9:c9130000 r8:00000000 r7:c9131f4c r6:ffffffff r5:600c0013 r4:c020a480 [ 10.935291] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 10.943403] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 10.951164] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 10.958752] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000002 [ 10.966598] r4:0000008a [ 10.969142] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 10.978210] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 10.985447] r5:00000000 r4:090eaa40 [ 10.989032] CPU3: stopping [ 10.991754] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 11.000992] Hardware name: Broadcom STB (Flattened Device Tree) [ 11.006924] Backtrace: [ 11.009384] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 11.016972] r7:c22a86d0 r6:00000000 r5:600f0193 r4:c22a86d0 [ 11.022649] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 11.029892] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 11.037307] r10:00000000 r9:c2205400 r8:00000000 r7:00000003 r6:c22b0744 r5:00000004 [ 11.045154] r4:c22ce308 r3:00976ec2 [ 11.048741] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 11.056329] r10:00000000 r9:d1003000 r8:d1002000 r7:c9133f18 r6:d100200c r5:c220565c [ 11.064176] r4:c226d668 [ 11.066719] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 11.074216] Exception stack(0xc9133f18 to 0xc9133f60) [ 11.079280] 3f00: c020a47c 00000000 [ 11.087476] 3f20: 0e4dd000 600f0093 c9132000 c2204cf0 c2204d34 00000008 00000000 c20757b0 [ 11.095671] 3f40: 00000000 c9133f74 c9132000 c9133f68 00000000 c020a480 600f0013 ffffffff [ 11.103867] r9:c9132000 r8:00000000 r7:c9133f4c r6:ffffffff r5:600f0013 r4:c020a480 [ 11.111633] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 11.119744] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 11.127506] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 11.135093] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000003 [ 11.142939] r4:0000008a [ 11.145484] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 11.154550] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 11.161788] r5:00000000 r4:090eaa40 [ 11.165372] CPU1: stopping [ 11.168094] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 11.177332] Hardware name: Broadcom STB (Flattened Device Tree) [ 11.183264] Backtrace: [ 11.185723] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 11.193311] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 [ 11.198988] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 11.206230] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 11.213644] r10:00000000 r9:c2205400 r8:00000000 r7:00000001 r6:c22b0744 r5:00000004 [ 11.221491] r4:c22ce308 r3:00976ec2 [ 11.225078] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 11.232666] r10:00000000 r9:d1003000 r8:d1002000 r7:c912ff18 r6:d100200c r5:c220565c [ 11.240512] r4:c226d668 [ 11.243054] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 11.250553] Exception stack(0xc912ff18 to 0xc912ff60) [ 11.255617] ff00: c020a47c 00000000 [ 11.263814] ff20: 0e4b7000 600c0093 c912e000 c2204cf0 c2204d34 00000002 00000000 c20757b0 [ 11.272010] ff40: 00000000 c912ff74 c912e000 c912ff68 00000000 c020a480 600c0013 ffffffff [ 11.280206] r9:c912e000 r8:00000000 r7:c912ff4c r6:ffffffff r5:600c0013 r4:c020a480 [ 11.287970] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 11.296081] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 11.303842] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 11.311430] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000001 [ 11.319275] r4:0000008a [ 11.321819] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 11.330886] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 11.338124] r5:00000000 r4:090eaa40 [ 11.341729] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---
it is not marked as fixing anything so I wonder how it landed in stable?
It was requested to be merged.
OK.
Do you have the same crash on newer kernels with this commit in it (like Linus's tree?)
There are no crashes with 5.10 or upstream otherwise it would have been noticed earlier, I have not kept track of which additional changes/fixes we may need but I would suggest we drop this one for now.
Alexander, do you want me to test additional patches if your change somehow must be included in an upcoming 5.4? I thought the platform you are working is still not upstream, so who is going to benefit from this performance improvement?
Is 4.19 also failing for you now?
I cannot easily test 4.19 without applying some additional patches bringing in various ARM SCMI changes, I would suspect the same to happen. Give me a few hours and I will see if I can give you a better answer with an actual test.
I have not been able to reproduce this failure since merging the official v5.4.105 tag (more on that below), and 4.19 did not seem to trigger the problem either after applying the necessary patches to bring up the Ethernet PHY, switch and controller's clocks.
So maybe this was just a fluke, I know our CPU frequency scaling driver may be too aggressive and the boards may suffer from power distribution problems leading to crashes, although I have never seen anything like that before, and the call trace was pretty obvious as to what had changed compared to v5.4.104 which had no such issues.
So I guess we are good, until we are not. It concerns me however that this (latent at the time) issue was reported at Wed, 10 Mar 2021 20:19:48 -0800 which is well before the deadline of Fri, 12 Mar 2021 13:23:09 +0000, and yet, the v5.4.105 was announced on Thu, 11 Mar 2021 05:33:31 -0800 (PST) and it went through with that patch nonetheless.
On Fri, Mar 12, 2021 at 02:55:49PM -0800, Florian Fainelli wrote:
On 3/11/21 9:41 AM, Florian Fainelli wrote:
On 3/11/21 9:40 AM, Greg KH wrote:
On Thu, Mar 11, 2021 at 09:23:56AM -0800, Florian Fainelli wrote:
On 3/11/21 5:08 AM, Greg KH wrote:
On Wed, Mar 10, 2021 at 08:19:45PM -0800, Florian Fainelli wrote:
+Alex,
On 3/10/2021 5:24 AM, gregkh@linuxfoundation.org wrote: > From: Greg Kroah-Hartman gregkh@linuxfoundation.org > > This is the start of the stable review cycle for the 5.4.105 release. > There are 24 patches in this series, all will be posted as a response > to this one. If anyone has any issues with these being applied, please > let me know. > > Responses should be made by Fri, 12 Mar 2021 13:23:09 +0000. > Anything received after that time might be too late. > > The whole patch series can be found in one patch at: > https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/patch-5.4.105-rc1... > or in the git tree and branch at: > git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-5.4.y > and the diffstat can be found below. > > thanks, > > greg k-h
I believe you need to drop "net: dsa: add GRO support via gro_cells" as it causes the following kernel panic on a DSA-enabled platform:
Configuring rgmii_2 interface [ 10.170527] brcm-sf2 f0b00000.ethernet_switch rgmii_2: configuring for fixed/rgmii-txid link mode [ 10.179597] 8021q: adding VLAN 0 to HW filter on device rgmii_2 [ 10.185608] brcm-sf2 f0b00000.ethernet_switch rgmii_2: Link is Up - 1Gbps/Full - flow control off [ 10.198631] IPv6: ADDRCONF(NETDEV_CHANGE): rgmii_2: link becomes ready Configuring sit0 interface [ 10.254346] 8<--- cut here --- [ 10.257438] Unable to handle kernel paging request at virtual address d6df6190 [ 10.264685] pgd = (ptrval) [ 10.267411] [d6df6190] *pgd=80000000007003, *pmd=00000000 [ 10.272846] Internal error: Oops: 206 [#1] SMP ARM [ 10.277661] Modules linked in: [ 10.280739] CPU: 0 PID: 1886 Comm: sed Not tainted 5.4.105-1.0pre-geff642e2af2b #4 [ 10.288337] Hardware name: Broadcom STB (Flattened Device Tree) [ 10.294292] PC is at gro_cells_receive+0x90/0x11c [ 10.299020] LR is at dsa_switch_rcv+0x120/0x1d4 [ 10.303562] pc : [<c0a57a28>] lr : [<c0b4a65c>] psr: 600f0113 [ 10.309841] sp : c1d33cd0 ip : 000003e8 fp : c1d33ce4 [ 10.315078] r10: c8901000 r9 : c8901000 r8 : c0b4a53c [ 10.320314] r7 : c2208920 r6 : 00000000 r5 : 00000000 r4 : 00004000 [ 10.326855] r3 : d6df6188 r2 : c4927000 r1 : c8adc300 r0 : c22069dc [ 10.333398] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user [ 10.340547] Control: 30c5387d Table: 04ac4c80 DAC: fffffffd [ 10.346307] Process sed (pid: 1886, stack limit = 0x(ptrval)) [ 10.352066] Stack: (0xc1d33cd0 to 0xc1d34000) [ 10.356434] 3cc0: c8adc300 c4927000 c1d33d04 c1d33ce8 [ 10.364631] 3ce0: c0b4a65c c0a579a4 c1d33d24 c2208920 c1d33d24 00000000 c1d33d5c c1d33d08 [ 10.372827] 3d00: c0a0b38c c0b4a548 c021e070 c2204cc8 00000000 c89015c0 04b87700 c89015c0 [ 10.381023] 3d20: c2208920 c1d33d24 c1d33d24 00976ec2 04b87700 c8adc300 c89015c0 00000000 [ 10.389218] 3d40: c1d33d74 c1d32000 00000000 c230742c c1d33dac c1d33d60 c0a0b5c0 c0a0b180 [ 10.397414] 3d60: 00000000 c2204cc8 00000000 c1d33d6c c1d33d6c c1d33d80 c029daf8 00976ec2 [ 10.405610] 3d80: 00000800 c8901540 c89015c0 c8901540 00000000 00000001 0000016c 00000162 [ 10.413805] 3da0: c1d33dc4 c1d33db0 c0a0b7fc c0a0b3b8 00000000 c8adc300 c1d33dfc c1d33dc8 [ 10.422001] 3dc0: c0a0c660 c0a0b7e4 c8901540 c8adc300 c1d33dfc c1d33de0 c8901540 c8adc300 [ 10.430196] 3de0: 0000015e c8901000 00000001 0000016c c1d33e74 c1d33e00 c083df00 c0a0c4fc [ 10.438391] 3e00: 0000012c c22b0f14 c1d33e4c c1d33e18 c0fbd9b8 c0fbd9cc c0fbd9e0 c0fbd98c [ 10.446586] 3e20: 00000001 00000040 c8901500 00000001 00000000 00000000 00000000 00000000 [ 10.454780] 3e40: 00000000 00000000 c02f65a0 c8901540 00000001 00000040 c22b07e4 0000012c [ 10.462975] 3e60: d1003000 fffb942f c1d33edc c1d33e78 c0a0c94c c083dafc d051ad80 c2204cc8 [ 10.471170] 3e80: c2204cf0 c1d32000 c22b40b0 0e4a4000 c2076d80 c2203d00 c022bc70 c1d33e9c [ 10.479365] 3ea0: c1d33e9c c1d33ea4 c1d33ea4 00976ec2 c02f65a0 c220308c 00000003 c1d32000 [ 10.487561] 3ec0: c22b07e4 00000100 d1003000 00000008 c1d33f44 c1d33ee0 c020238c c0a0c6cc [ 10.495755] 3ee0: c1d33f14 c1d33ef0 00000001 00400000 c2203d00 fffb942f c206b2e4 c2076040 [ 10.503950] 3f00: c2076040 0000000a c2203080 c206b358 c1d33ee0 00000004 c90c9500 ffffe000 [ 10.512145] 3f20: 00000000 00000000 00000001 c9019000 d1003000 00000000 c1d33f5c c1d33f48 [ 10.520339] 3f40: c022bc70 c0202264 c2075fbc 00000000 c1d33f84 c1d33f60 c027feb4 c022bb88 [ 10.528535] 3f60: c226d668 c220565c d100200c c1d33fb0 d1002000 d1003000 c1d33fac c1d33f88 [ 10.536730] 3f80: c0202214 c027fe50 b6f1e0b6 000f0030 ffffffff 30c5387d 30c5387d 00000000 [ 10.544924] 3fa0: 00000000 c1d33fb0 c0201d8c c02021c4 b6ec1778 b6f32094 62632e73 62740000 [ 10.553120] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 00000000 00000000 00000006 [ 10.561316] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 ffffffff 00000000 00000000 [ 10.569508] Backtrace: [ 10.571970] [<c0a57998>] (gro_cells_receive) from [<c0b4a65c>] (dsa_switch_rcv+0x120/0x1d4) [ 10.580338] r5:c4927000 r4:c8adc300 [ 10.583929] [<c0b4a53c>] (dsa_switch_rcv) from [<c0a0b38c>] (__netif_receive_skb_list_core+0x218/0x238) [ 10.593343] r7:00000000 r6:c1d33d24 r5:c2208920 r4:c1d33d24 [ 10.599017] [<c0a0b174>] (__netif_receive_skb_list_core) from [<c0a0b5c0>] (netif_receive_skb_list_internal+0x214/0x2dc) [ 10.609909] r10:c230742c r9:00000000 r8:c1d32000 r7:c1d33d74 r6:00000000 r5:c89015c0 [ 10.617755] r4:c8adc300 [ 10.620300] [<c0a0b3ac>] (netif_receive_skb_list_internal) from [<c0a0b7fc>] (gro_normal_list.part.42+0x24/0x38) [ 10.630496] r10:00000162 r9:0000016c r8:00000001 r7:00000000 r6:c8901540 r5:c89015c0 [ 10.638342] r4:c8901540 [ 10.640885] [<c0a0b7d8>] (gro_normal_list.part.42) from [<c0a0c660>] (napi_complete_done+0x170/0x1d0) [ 10.650123] r5:c8adc300 r4:00000000 [ 10.653712] [<c0a0c4f0>] (napi_complete_done) from [<c083df00>] (bcm_sysport_poll+0x410/0x4b4) [ 10.662343] r9:0000016c r8:00000001 r7:c8901000 r6:0000015e r5:c8adc300 r4:c8901540 [ 10.670104] [<c083daf0>] (bcm_sysport_poll) from [<c0a0c94c>] (net_rx_action+0x28c/0x44c) [ 10.678301] r10:fffb942f r9:d1003000 r8:0000012c r7:c22b07e4 r6:00000040 r5:00000001 [ 10.686146] r4:c8901540 [ 10.688693] [<c0a0c6c0>] (net_rx_action) from [<c020238c>] (__do_softirq+0x134/0x414) [ 10.696542] r10:00000008 r9:d1003000 r8:00000100 r7:c22b07e4 r6:c1d32000 r5:00000003 [ 10.704387] r4:c220308c [ 10.706931] [<c0202258>] (__do_softirq) from [<c022bc70>] (irq_exit+0xf4/0x100) [ 10.714257] r10:00000000 r9:d1003000 r8:c9019000 r7:00000001 r6:00000000 r5:00000000 [ 10.722104] r4:ffffe000 [ 10.724650] [<c022bb7c>] (irq_exit) from [<c027feb4>] (__handle_domain_irq+0x70/0xc4) [ 10.732497] r5:00000000 r4:c2075fbc [ 10.736083] [<c027fe44>] (__handle_domain_irq) from [<c0202214>] (gic_handle_irq+0x5c/0xa0) [ 10.744453] r9:d1003000 r8:d1002000 r7:c1d33fb0 r6:d100200c r5:c220565c r4:c226d668 [ 10.752215] [<c02021b8>] (gic_handle_irq) from [<c0201d8c>] (__irq_usr+0x4c/0x60) [ 10.759713] Exception stack(0xc1d33fb0 to 0xc1d33ff8) [ 10.764776] 3fa0: b6ec1778 b6f32094 62632e73 62740000 [ 10.772973] 3fc0: 00172e73 00001700 b6f3293c 00000001 00000001 00000000 00000000 00000006 [ 10.781167] 3fe0: ffffffff bef37ac8 b6f18151 b6f1e0b6 000f0030 ffffffff [ 10.787797] r9:00000000 r8:30c5387d r7:30c5387d r6:ffffffff r5:000f0030 r4:b6f1e0b6 [ 10.795559] Code: e30609dc e34c0220 e083300c e590c000 (e5930008) [ 10.801670] ---[ end trace 97c3942fa73eff4c ]--- [ 10.806300] Kernel panic - not syncing: Fatal exception in interrupt [ 10.812678] CPU2: stopping [ 10.815403] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 10.824641] Hardware name: Broadcom STB (Flattened Device Tree) [ 10.830573] Backtrace: [ 10.833036] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 10.840624] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 [ 10.846302] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 10.853546] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 10.860960] r10:00000000 r9:c2205400 r8:00000000 r7:00000002 r6:c22b0744 r5:00000004 [ 10.868807] r4:c22ce308 r3:00976ec2 [ 10.872395] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 10.879983] r10:00000000 r9:d1003000 r8:d1002000 r7:c9131f18 r6:d100200c r5:c220565c [ 10.887829] r4:c226d668 [ 10.890373] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 10.897871] Exception stack(0xc9131f18 to 0xc9131f60) [ 10.902934] 1f00: c020a47c 00000000 [ 10.911131] 1f20: 0e4ca000 600c0093 c9130000 c2204cf0 c2204d34 00000004 00000000 c20757b0 [ 10.919327] 1f40: 00000000 c9131f74 c9130000 c9131f68 00000000 c020a480 600c0013 ffffffff [ 10.927523] r9:c9130000 r8:00000000 r7:c9131f4c r6:ffffffff r5:600c0013 r4:c020a480 [ 10.935291] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 10.943403] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 10.951164] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 10.958752] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000002 [ 10.966598] r4:0000008a [ 10.969142] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 10.978210] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 10.985447] r5:00000000 r4:090eaa40 [ 10.989032] CPU3: stopping [ 10.991754] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 11.000992] Hardware name: Broadcom STB (Flattened Device Tree) [ 11.006924] Backtrace: [ 11.009384] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 11.016972] r7:c22a86d0 r6:00000000 r5:600f0193 r4:c22a86d0 [ 11.022649] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 11.029892] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 11.037307] r10:00000000 r9:c2205400 r8:00000000 r7:00000003 r6:c22b0744 r5:00000004 [ 11.045154] r4:c22ce308 r3:00976ec2 [ 11.048741] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 11.056329] r10:00000000 r9:d1003000 r8:d1002000 r7:c9133f18 r6:d100200c r5:c220565c [ 11.064176] r4:c226d668 [ 11.066719] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 11.074216] Exception stack(0xc9133f18 to 0xc9133f60) [ 11.079280] 3f00: c020a47c 00000000 [ 11.087476] 3f20: 0e4dd000 600f0093 c9132000 c2204cf0 c2204d34 00000008 00000000 c20757b0 [ 11.095671] 3f40: 00000000 c9133f74 c9132000 c9133f68 00000000 c020a480 600f0013 ffffffff [ 11.103867] r9:c9132000 r8:00000000 r7:c9133f4c r6:ffffffff r5:600f0013 r4:c020a480 [ 11.111633] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 11.119744] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 11.127506] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 11.135093] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000003 [ 11.142939] r4:0000008a [ 11.145484] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 11.154550] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 11.161788] r5:00000000 r4:090eaa40 [ 11.165372] CPU1: stopping [ 11.168094] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G D 5.4.105-1.0pre-geff642e2af2b #4 [ 11.177332] Hardware name: Broadcom STB (Flattened Device Tree) [ 11.183264] Backtrace: [ 11.185723] [<c020dd30>] (dump_backtrace) from [<c020e04c>] (show_stack+0x20/0x24) [ 11.193311] r7:c22a86d0 r6:00000000 r5:600c0193 r4:c22a86d0 [ 11.198988] [<c020e02c>] (show_stack) from [<c0c09924>] (dump_stack+0xb8/0xe4) [ 11.206230] [<c0c0986c>] (dump_stack) from [<c0212160>] (handle_IPI+0x344/0x3cc) [ 11.213644] r10:00000000 r9:c2205400 r8:00000000 r7:00000001 r6:c22b0744 r5:00000004 [ 11.221491] r4:c22ce308 r3:00976ec2 [ 11.225078] [<c0211e1c>] (handle_IPI) from [<c0202254>] (gic_handle_irq+0x9c/0xa0) [ 11.232666] r10:00000000 r9:d1003000 r8:d1002000 r7:c912ff18 r6:d100200c r5:c220565c [ 11.240512] r4:c226d668 [ 11.243054] [<c02021b8>] (gic_handle_irq) from [<c0201a3c>] (__irq_svc+0x5c/0x7c) [ 11.250553] Exception stack(0xc912ff18 to 0xc912ff60) [ 11.255617] ff00: c020a47c 00000000 [ 11.263814] ff20: 0e4b7000 600c0093 c912e000 c2204cf0 c2204d34 00000002 00000000 c20757b0 [ 11.272010] ff40: 00000000 c912ff74 c912e000 c912ff68 00000000 c020a480 600c0013 ffffffff [ 11.280206] r9:c912e000 r8:00000000 r7:c912ff4c r6:ffffffff r5:600c0013 r4:c020a480 [ 11.287970] [<c020a44c>] (arch_cpu_idle) from [<c0c1316c>] (default_idle_call+0x34/0x48) [ 11.296081] [<c0c13138>] (default_idle_call) from [<c02580c0>] (do_idle+0x1d4/0x2c0) [ 11.303842] [<c0257eec>] (do_idle) from [<c0258494>] (cpu_startup_entry+0x28/0x2c) [ 11.311430] r10:00000000 r9:420f00f3 r8:00007000 r7:c22ce318 r6:30c0387d r5:00000001 [ 11.319275] r4:0000008a [ 11.321819] [<c025846c>] (cpu_startup_entry) from [<c0211644>] (secondary_start_kernel+0x17c/0x1a0) [ 11.330886] [<c02114c8>] (secondary_start_kernel) from [<0020270c>] (0x20270c) [ 11.338124] r5:00000000 r4:090eaa40 [ 11.341729] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---
it is not marked as fixing anything so I wonder how it landed in stable?
It was requested to be merged.
OK.
Do you have the same crash on newer kernels with this commit in it (like Linus's tree?)
There are no crashes with 5.10 or upstream otherwise it would have been noticed earlier, I have not kept track of which additional changes/fixes we may need but I would suggest we drop this one for now.
Alexander, do you want me to test additional patches if your change somehow must be included in an upcoming 5.4? I thought the platform you are working is still not upstream, so who is going to benefit from this performance improvement?
Is 4.19 also failing for you now?
I cannot easily test 4.19 without applying some additional patches bringing in various ARM SCMI changes, I would suspect the same to happen. Give me a few hours and I will see if I can give you a better answer with an actual test.
I have not been able to reproduce this failure since merging the official v5.4.105 tag (more on that below), and 4.19 did not seem to trigger the problem either after applying the necessary patches to bring up the Ethernet PHY, switch and controller's clocks.
Great, thanks for working to figure this out.
So maybe this was just a fluke, I know our CPU frequency scaling driver may be too aggressive and the boards may suffer from power distribution problems leading to crashes, although I have never seen anything like that before, and the call trace was pretty obvious as to what had changed compared to v5.4.104 which had no such issues.
So I guess we are good, until we are not. It concerns me however that this (latent at the time) issue was reported at Wed, 10 Mar 2021 20:19:48 -0800 which is well before the deadline of Fri, 12 Mar 2021 13:23:09 +0000, and yet, the v5.4.105 was announced on Thu, 11 Mar 2021 05:33:31 -0800 (PST) and it went through with that patch nonetheless.
It's a judgement call on my side as to when to do the release, based on the testing that has happened, any reports, and my knowledge of what is in the patches themselves. For this patchset, all of the expected testers came back with no problems, except for your report.
And if your report turned out to be real (the fact that it was a backport of an "old" patch made it much less likely to be real), I can always instantly revert it and push out a new release quickly for the tiny subset of those that have problems with this.
So I took a guess based on all of this and decided it was more important to get the release out early, so that it can start to make its way to the huge majority of systems that did report testing worked fine, than to delay it to wait for your single system report. Because again, if this turned out to be a real issue, a quick release for any affected systems would have been trivial to create.
thanks,
greg k-h
Hi!
So I guess we are good, until we are not. It concerns me however that this (latent at the time) issue was reported at Wed, 10 Mar 2021 20:19:48 -0800 which is well before the deadline of Fri, 12 Mar 2021 13:23:09 +0000, and yet, the v5.4.105 was announced on Thu, 11 Mar 2021 05:33:31 -0800 (PST) and it went through with that patch nonetheless.
It's a judgement call on my side as to when to do the release, based on the testing that has happened, any reports, and my knowledge of what is in the patches themselves. For this patchset, all of the expected testers came back with no problems, except for your report.
And if your report turned out to be real (the fact that it was a backport of an "old" patch made it much less likely to be real), I can always instantly revert it and push out a new release quickly for the tiny subset of those that have problems with this.
So I took a guess based on all of this and decided it was more important to get the release out early, so that it can start to make its way to the huge majority of systems that did report testing worked fine, than to delay it to wait for your single system report. Because again, if this turned out to be a real issue, a quick release for any affected systems would have been trivial to create.
You are setting yourself (and testers) a deadline... and then you ignore it.
People are not only testing the release, they are also reviewing the patches, and having at least two days for that is useful.
You clearly disagree, but in any case you should not mention deadline in the initial if you don't intend to keep them. Thats confusing, and clearly it is not only confusing to me.
Best regards, Pavel
On Wed, 10 Mar 2021 at 18:55, gregkh@linuxfoundation.org wrote:
From: Greg Kroah-Hartman gregkh@linuxfoundation.org
This is the start of the stable review cycle for the 5.4.105 release. There are 24 patches in this series, all will be posted as a response to this one. If anyone has any issues with these being applied, please let me know.
Responses should be made by Fri, 12 Mar 2021 13:23:09 +0000. Anything received after that time might be too late.
The whole patch series can be found in one patch at: https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/patch-5.4.105-rc1... or in the git tree and branch at: git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-5.4.y and the diffstat can be found below.
thanks,
greg k-h
Results from Linaro’s test farm. No regressions on arm64, arm, x86_64, and i386.
Tested-by: Linux Kernel Functional Testing lkft@linaro.org
Summary ------------------------------------------------------------------------
kernel: 5.4.105-rc1 git repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git git branch: linux-5.4.y git commit: 62f8f08c9d2fbc6c5692d90f64dd70e3a8edd986 git describe: v5.4.104-25-g62f8f08c9d2f Test details: https://qa-reports.linaro.org/lkft/linux-stable-rc-linux-5.4.y/build/v5.4.10...
No regressions (compared to build v5.4.104)
No fixes (compared to build v5.4.104)
Ran 53992 total tests in the following environments and test suites.
Environments -------------- - arc - arm - arm64 - dragonboard-410c - hi6220-hikey - i386 - juno-r2 - juno-r2-compat - juno-r2-kasan - mips - nxp-ls2088 - nxp-ls2088-64k_page_size - parisc - powerpc - qemu-arm-clang - qemu-arm64-clang - qemu-arm64-kasan - qemu-x86_64-clang - qemu-x86_64-kasan - qemu-x86_64-kcsan - qemu_arm - qemu_arm64 - qemu_arm64-compat - qemu_i386 - qemu_x86_64 - qemu_x86_64-compat - riscv - s390 - sh - sparc - x15 - x86 - x86-kasan - x86_64
Test Suites ----------- * build * linux-log-parser * igt-gpu-tools * install-android-platform-tools-r2600 * kselftest- * kselftest-android * kselftest-bpf * kselftest-capabilities * kselftest-cgroup * kselftest-clone3 * kselftest-core * kselftest-cpu-hotplug * kselftest-cpufreq * kselftest-efivarfs * kselftest-filesystems * kselftest-firmware * kselftest-fpu * kselftest-futex * kselftest-gpio * kselftest-intel_pstate * kselftest-ipc * kselftest-ir * kselftest-kcmp * kselftest-kvm * kselftest-livepatch * kselftest-lkdtm * kselftest-net * kselftest-netfilter * kselftest-nsfs * kselftest-ptrace * kselftest-rseq * kselftest-rtc * kselftest-seccomp * kselftest-sigaltstack * kselftest-size * kselftest-splice * kselftest-static_keys * kselftest-sync * kselftest-sysctl * kselftest-tc-testing * ltp-cap_bounds-tests * ltp-containers-tests * ltp-cpuhotplug-tests * ltp-crypto-tests * ltp-cve-tests * ltp-dio-tests * ltp-fs-tests * ltp-hugetlb-tests * ltp-io-tests * ltp-mm-tests * ltp-nptl-tests * ltp-pty-tests * ltp-sched-tests * ltp-securebits-tests * ltp-syscalls-tests * ltp-tracing-tests * perf * fwts * kselftest-lib * kselftest-membarrier * kselftest-memfd * kselftest-memory-hotplug * kselftest-mincore * kselftest-mount * kselftest-mqueue * kselftest-openat2 * kselftest-pid_namespace * kselftest-pidfd * kselftest-proc * kselftest-pstore * kselftest-timens * kselftest-timers * kselftest-tmpfs * kselftest-tpm2 * kselftest-user * kselftest-zram * libhugetlbfs * ltp-commands-tests * ltp-controllers-tests * ltp-fcntl-locktests-tests * ltp-filecaps-tests * ltp-fs_bind-tests * ltp-fs_perms_simple-tests * ltp-fsx-tests * ltp-ipc-tests * ltp-math-tests * network-basic-tests * v4l2-compliance * kselftest-kexec * kselftest-vm * kselftest-x86 * ltp-open-posix-tests * kvm-unit-tests * rcutorture * kselftest-vsyscall-mode-native- * ssuite
On Wed, 10 Mar 2021 14:24:12 +0100, gregkh@linuxfoundation.org wrote:
From: Greg Kroah-Hartman gregkh@linuxfoundation.org
This is the start of the stable review cycle for the 5.4.105 release. There are 24 patches in this series, all will be posted as a response to this one. If anyone has any issues with these being applied, please let me know.
Responses should be made by Fri, 12 Mar 2021 13:23:09 +0000. Anything received after that time might be too late.
The whole patch series can be found in one patch at: https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/patch-5.4.105-rc1... or in the git tree and branch at: git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-5.4.y and the diffstat can be found below.
thanks,
greg k-h
All tests passing for Tegra ...
Test results for stable-v5.4: 12 builds: 12 pass, 0 fail 26 boots: 26 pass, 0 fail 57 tests: 57 pass, 0 fail
Linux version: 5.4.105-rc1-g62f8f08c9d2f Boards tested: tegra124-jetson-tk1, tegra186-p2771-0000, tegra194-p2972-0000, tegra20-ventana, tegra210-p2371-2180, tegra210-p3450-0000, tegra30-cardhu-a04
Tested-by: Jon Hunter jonathanh@nvidia.com
Jon
On 2021/3/10 21:24, gregkh@linuxfoundation.org wrote:
From: Greg Kroah-Hartman gregkh@linuxfoundation.org
This is the start of the stable review cycle for the 5.4.105 release. There are 24 patches in this series, all will be posted as a response to this one. If anyone has any issues with these being applied, please let me know.
Responses should be made by Fri, 12 Mar 2021 13:23:09 +0000. Anything received after that time might be too late.
The whole patch series can be found in one patch at: https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/patch-5.4.105-rc1... or in the git tree and branch at: git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-5.4.y and the diffstat can be found below.
thanks,
greg k-h
Tested on arm64 and x86 for 5.4.105-rc1,
Kernel repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git Branch: linux-5.4.y Version: 5.4.105-rc1+ Commit: 62f8f08c9d2fbc6c5692d90f64dd70e3a8edd986 Compiler: gcc version 7.3.0 (GCC)
arm64: -------------------------------------------------------------------- Testcase Result Summary: total_num: 4703 succeed_num: 4703 failed_num: 0 timeout_num: 0
x86: -------------------------------------------------------------------- Testcase Result Summary: total_num: 4703 succeed_num: 4703 failed_num: 0 timeout_num: 0
Tested-by: Hulk Robot hulkrobot@huawei.com
-
Alexander Lobakin -
Florian Fainelli -
Greg KH -
gregkh＠linuxfoundation.org
-
Guenter Roeck -
Jon Hunter -
Naresh Kamboju -
Pali Rohár -
Pavel Machek -
Ross Schmidt -
Samuel Zou -
Shuah Khan