Hello Marc,
since 773c05f417fa1 ("irqchip/gic-v3: Work around insecure GIC integrations") landed in stable v6.12.7 as 0bf32f482887, here the rk3399 fails to boot (~4 out of 10 times) because OP-TEE panics (gets a secure interrupt that it cannot handle).
Setup is: - BL31 proprietary (since mainline TF-A has no DMA) - OP-TEE mainline version: 3.20 - Kernel v6.12.7
<snip> [ 0.000000] GICv3: Broken GIC integration, security disabled <snip> E/TC:4 0 Panic 'Secure interrupt handler not defined' at core/kernel/interrupt.c:139 <itr_core_handler> E/TC:4 0 TEE load address @ 0x30000000 E/TC:4 0 Call stack: E/TC:4 0 0x300091f8 E/TC:4 0 0x30016664 E/TC:4 0 0x30015710 E/TC:4 0 0x30005714 [ 26.087363] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: [ 26.087925] rcu: (detected by 2, t=21002 jiffies, g=2233, q=2416 ncpus=6) [ 26.088530] rcu: All QSes seen, last rcu_preempt kthread activity 21002 (4294693363-4294672361), jiffies_till_next_fqs=3, root ->qsmask 0x0 [ 26.089623] rcu: rcu_preempt kthread timer wakeup didn't happen for 20999 jiffies! g2233 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x200 [ 26.090617] rcu: Possible timer handling issue on cpu=4 timer-softirq=293 [ 26.091218] rcu: rcu_preempt kthread starved for 21002 jiffies! g2233 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x200 ->cpu=4 [ 26.092131] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [ 26.092926] rcu: RCU grace-period kthread stack dump: [ 26.093369] task:rcu_preempt state:R stack:0 pid:16 tgid:16 ppid:2 flags:0x00000008 [ 26.094192] Call trace: [ 26.094409] __switch_to+0xf0/0x14c [ 26.094728] __schedule+0x264/0xa90 [ 26.095040] schedule+0x34/0x104 [ 26.095329] schedule_timeout+0x80/0xf4 [ 26.095672] rcu_gp_fqs_loop+0x14c/0x4a4 [ 26.096027] rcu_gp_kthread+0x138/0x164 [ 26.096369] kthread+0x114/0x118 [ 26.096661] ret_from_fork+0x10/0x20 [ 26.096982] rcu: Stack dump where RCU GP kthread last ran: [ 26.097463] Sending NMI from CPU 2 to CPUs 4: [ 46.276364] sched: DL replenish lagged too much
Is it too late for the kernel to disable the "security" since OP-TEE assumes it is enabled?
Any ideas?
Thanks -- Christoph
Hi Christoph,
Thanks for reporting this.
On Wed, 29 Jan 2025 21:31:53 +0000, Christoph Fritz chf.fritz@googlemail.com wrote:
Hello Marc,
since 773c05f417fa1 ("irqchip/gic-v3: Work around insecure GIC integrations") landed in stable v6.12.7 as 0bf32f482887, here the rk3399 fails to boot (~4 out of 10 times) because OP-TEE panics (gets a secure interrupt that it cannot handle).
I think it may actually get a *non-secure* interrupt.
Setup is:
- BL31 proprietary (since mainline TF-A has no DMA)
- OP-TEE mainline version: 3.20
- Kernel v6.12.7
<snip> [ 0.000000] GICv3: Broken GIC integration, security disabled <snip> E/TC:4 0 Panic 'Secure interrupt handler not defined' at core/kernel/interrupt.c:139 <itr_core_handler> E/TC:4 0 TEE load address @ 0x30000000 E/TC:4 0 Call stack: E/TC:4 0 0x300091f8 E/TC:4 0 0x30016664 E/TC:4 0 0x30015710 E/TC:4 0 0x30005714 [ 26.087363] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: [ 26.087925] rcu: (detected by 2, t=21002 jiffies, g=2233, q=2416 ncpus=6) [ 26.088530] rcu: All QSes seen, last rcu_preempt kthread activity 21002 (4294693363-4294672361), jiffies_till_next_fqs=3, root ->qsmask 0x0 [ 26.089623] rcu: rcu_preempt kthread timer wakeup didn't happen for 20999 jiffies! g2233 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x200 [ 26.090617] rcu: Possible timer handling issue on cpu=4 timer-softirq=293 [ 26.091218] rcu: rcu_preempt kthread starved for 21002 jiffies! g2233 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x200 ->cpu=4 [ 26.092131] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [ 26.092926] rcu: RCU grace-period kthread stack dump: [ 26.093369] task:rcu_preempt state:R stack:0 pid:16 tgid:16 ppid:2 flags:0x00000008 [ 26.094192] Call trace: [ 26.094409] __switch_to+0xf0/0x14c [ 26.094728] __schedule+0x264/0xa90 [ 26.095040] schedule+0x34/0x104 [ 26.095329] schedule_timeout+0x80/0xf4 [ 26.095672] rcu_gp_fqs_loop+0x14c/0x4a4 [ 26.096027] rcu_gp_kthread+0x138/0x164 [ 26.096369] kthread+0x114/0x118 [ 26.096661] ret_from_fork+0x10/0x20 [ 26.096982] rcu: Stack dump where RCU GP kthread last ran: [ 26.097463] Sending NMI from CPU 2 to CPUs 4: [ 46.276364] sched: DL replenish lagged too much
Is it too late for the kernel to disable the "security" since OP-TEE assumes it is enabled?
Well, it was never effective security the first place, but clearly this effect is not expected (my own machine doesn't have anything running on the secure side).
Any ideas?
I think this calls for a revert of this patch, potentially at the expense if NMI support on this machine. Could you show how SCR_EL3.FIQ is configured on this machine? Mine shows:
[ 0.000000] GICv3: GICD_CTRL.DS=0, SCR_EL3.FIQ=0
and I suspect yours has FIQ=1. If that's the case, we could use that as the discriminant.
However, this machine has a much bigger issues. For things to work as expected, the GIC driver must preserve all the secure configuration, and nothing does that today.
So even before this patch, your secure payload won't get any interrupt, as we blindly configure everything to be Group-1NS.
Thanks,
M.
Any ideas?
I think this calls for a revert of this patch, potentially at the expense if NMI support on this machine. Could you show how SCR_EL3.FIQ is configured on this machine? Mine shows:
[ 0.000000] GICv3: GICD_CTRL.DS=0, SCR_EL3.FIQ=0
and I suspect yours has FIQ=1.
yes it's 1:
[ 0.000000] GICv3: GICv3 features: 16 PPIs [ 0.000000] GICv3: Broken GIC integration, security disabled [ 0.000000] GICv3: GICD_CTRL.DS=1, SCR_EL3.FIQ=1 [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000fef00000
The device is not closed (there is no docu) and OP-TEE is actually here not used for anything. But I think I had the OP-TEE xtests running successfully before.
Here is the full boot log:
DDR Version 1.27 20211018 In soft reset SRX channel 0 CS = 0 MR0=0xB8 MR4=0x2 MR5=0xFF MR8=0x8 MR12=0x72 MR14=0x72 MR18=0x0 MR19=0x0 MR24=0x8 MR25=0x0 channel 1 CS = 0 MR0=0x38 MR4=0x2 MR5=0xFF MR8=0x8 MR12=0x72 MR14=0x72 MR18=0x0 MR19=0x0 MR24=0x8 MR25=0x0 channel 0 training pass! channel 1 training pass! change freq to 416MHz 0,1 Channel 0: LPDDR4,416MHz Bus Width=32 Col=10 Bank=8 Row=15 CS=1 Die Bus-Width=16 Size=1024MB Channel 1: LPDDR4,416MHz Bus Width=32 Col=10 Bank=8 Row=15 CS=1 Die Bus-Width=16 Size=1024MB 256B stride channel 0 CS = 0 MR0=0xB8 MR4=0x2 MR5=0xFF MR8=0x8 MR12=0x72 MR14=0x72 MR18=0x0 MR19=0x0 MR24=0x8 MR25=0x0 channel 1 CS = 0 MR0=0x38 MR4=0x2 MR5=0xFF MR8=0x8 MR12=0x72 MR14=0x72 MR18=0x0 MR19=0x0 MR24=0x8 MR25=0x0 channel 0 training pass! channel 1 training pass! channel 0, cs 0, advanced training done channel 1, cs 0, advanced training done channel 0, cs 0, dq 3 RISK!!! TdiVW_total violate spec channel 0, cs 0, dq 4 RISK!!! TdiVW_total violate spec channel 0, cs 0, dq 6 RISK!!! TdiVW_total violate spec channel 0, cs 0, dq 8 RISK!!! TdiVW_total violate spec channel 0, cs 0, dq 10 RISK!!! TdiVW_total violate spec channel 0, cs 0, dq 11 RISK!!! TdiVW_total violate spec channel 0, cs 0, dq 12 RISK!!! TdiVW_total violate spec channel 0, cs 0, dq 13 RISK!!! TdiVW_total violate spec channel 0, cs 0, dq 14 RISK!!! TdiVW_total violate spec channel 0, cs 0, dq 15 RISK!!! TdiVW_total violate spec channel 0, cs 0, dq 17 RISK!!! TdiVW_total violate spec channel 0, cs 0, dq 18 RISK!!! TdiVW_total violate spec channel 0, cs 0, dq 19 RISK!!! TdiVW_total violate spec channel 0, cs 0, dq 20 RISK!!! TdiVW_total violate spec channel 0, cs 0, dq 23 RISK!!! TdiVW_total violate spec channel 0, cs 0, dq 24 RISK!!! read vref 27% no find pass-eye channel 0, cs 0, dq 25 RISK!!! TdiVW_total violate spec channel 0, cs 0, dq 26 RISK!!! TdiVW_total violate spec channel 0, cs 0, dq 27 RISK!!! TdiVW_total violate spec channel 0, cs 0, dq 28 RISK!!! TdiVW_total violate spec channel 1, cs 0, dq 1 RISK!!! TdiVW_total violate spec channel 1, cs 0, dq 2 RISK!!! TdiVW_total violate spec channel 1, cs 0, dq 8 RISK!!! TdiVW_total violate spec channel 1, cs 0, dq 12 RISK!!! TdiVW_total violate spec channel 1, cs 0, dq 13 RISK!!! TdiVW_total violate spec channel 1, cs 0, dq 14 RISK!!! TdiVW_total violate spec channel 1, cs 0, dq 16 RISK!!! TdiVW_total violate spec channel 1, cs 0, dq 17 RISK!!! TdiVW_total violate spec channel 1, cs 0, dq 18 RISK!!! TdiVW_total violate spec channel 1, cs 0, dq 19 RISK!!! read vref 27% no find pass-eye channel 1, cs 0, dq 20 RISK!!! TdiVW_total violate spec channel 1, cs 0, dq 21 RISK!!! TdiVW_total violate spec channel 1, cs 0, dq 22 RISK!!! read vref 27% no find pass-eye channel 1, cs 0, dq 23 RISK!!! read vref 27% no find pass-eye channel 1, cs 0, dq 24 RISK!!! read vref 27% no find pass-eye channel 1, cs 0, dq 25 RISK!!! read vref 27% no find pass-eye channel 1, cs 0, dq 26 RISK!!! TdiVW_total violate spec channel 1, cs 0, dq 27 RISK!!! read vref 27% no find pass-eye channel 1, cs 0, dq 28 RISK!!! TdiVW_total violate spec channel 1, cs 0, dq 29 RISK!!! TdiVW_total violate spec channel 1, cs 0, dq 30 RISK!!! read vref 27% no find pass-eye channel 1, cs 0, dq 31 RISK!!! TdiVW_total violate spec change freq to 856MHz 1,0 ch 0 ddrconfig = 0x101, ddrsize = 0x20 ch 1 ddrconfig = 0x101, ddrsize = 0x20 pmugrf_os_reg[2] = 0x3281F281, stride = 0x9 ddr_set_rate to 328MHZ ddr_set_rate to 666MHZ ddr_set_rate to 416MHZ, ctl_index 0 ddr_set_rate to 856MHZ, ctl_index 1 support 416 856 328 666 MHz, current 856MHz OUT Boot1 Release Time: May 29 2020 17:36:36, version: 1.26 CPUId = 0x0 ChipType = 0x10, 438 SdmmcInit=2 0 BootCapSize=100000 UserCapSize=29820MB FwPartOffset=2000 , 100000 mmc0:cmd8,20 mmc0:cmd5,20 mmc0:cmd55,20 mmc0:cmd1,20 mmc0:cmd8,20 mmc0:cmd5,20 mmc0:cmd55,20 mmc0:cmd1,20 mmc0:cmd8,20 mmc0:cmd5,20 mmc0:cmd55,20 mmc0:cmd1,20 SdmmcInit=0 1 StorageInit ok = 71316 SecureMode = 0 SecureInit read PBA: 0x4 SecureInit read PBA: 0x404 SecureInit read PBA: 0x804 SecureInit read PBA: 0xc04 SecureInit read PBA: 0x1004 SecureInit read PBA: 0x1404 SecureInit read PBA: 0x1804 SecureInit read PBA: 0x1c04 SecureInit ret = 0, SecureMode = 0 atags_set_bootdev: ret:(0) Trust Addr:0x4000, 0x58334c42 No find bl30.bin Load uboot, ReadLba = 2000 Load OK, addr=0x200000, size=0xf33b0 RunBL31 0x40000 @ 122275 us NOTICE: BL31: v1.3(release):845ee93 NOTICE: BL31: Built : 15:51:11, Jul 22 2020 NOTICE: BL31: Rockchip release version: v1.1 INFO: GICv3 with legacy support detected. ARM GICV3 driver initialized in EL3 INFO: Using opteed sec cpu_context! INFO: boot cpu mask: 0 INFO: plat_rockchip_pmu_init(1196): pd status 3e INFO: BL31: Initializing runtime services INFO: BL31: Initializing BL32 I/TC: I/TC: No non-secure external DT I/TC: OP-TEE version: 3.20.0-3-g7067d1c29 (gcc version 12.2.0 (Debian 12.2.0-14)) #1 Sat Jan 6 14:14:21 UTC 2024 aarch64 I/TC: WARNING: This OP-TEE configuration might be insecure! I/TC: WARNING: Please check https://optee.readthedocs.io/en/latest/architecture/porting_guidelines.html I/TC: Primary CPU initializing I/TC: Primary CPU switching to normal world boot INFO: BL31: Preparing for EL3 exit to normal world INFO: Entry point address = 0x200000 INFO: SPSR = 0x3c9 ns16550_serial serial@ff1a0000: pinctrl_select_state_full: uclass_get_device_by_phandle_id: err=-19
U-Boot 2023.04-00022-gb92cd75aa58-dirty (Jan 06 2024 - 15:42:35 +0100)
SoC: Rockchip rk3399 Reset cause: RST Model: tcirk DRAM: 2 GiB optee optee: OP-TEE: revision 3.20 (7067d1c2) I/TC: Reserved shared memory is enabled I/TC: Dynamic shared memory is enabled I/TC: Normal World virtualization support is disabled I/TC: Asynchronous notifications are disabled PMIC: RK808 Core: 290 devices, 26 uclasses, devicetree: separate MMC: mmc@fe320000: 1, mmc@fe330000: 0 Loading Environment from MMC... OK In: serial Out: serial Err: serial Model: tcirk Net: No ethernet found. Hit any key to stop autoboot: 0 switch to partitions #0, OK mmc0(part 0) is current device Booting from mmc ... 18937052 bytes read in 1297 ms (13.9 MiB/s) ## Loading kernel from FIT Image at 00800800 ... Using 'conf-rockchip_rk3399-tcirk.dtb' configuration Verifying Hash Integrity ... OK Trying 'kernel-1' kernel subimage Description: Linux kernel Created: 2025-01-29 20:53:29 UTC Type: Kernel Image Compression: gzip compressed Data Start: 0x0080090c Data Size: 9619255 Bytes = 9.2 MiB Architecture: AArch64 OS: Linux Load Address: 0x02200000 Entry Point: 0x02200000 Hash algo: sha256 Hash value: 4e4a1ce89d766d2b34ecfd7a61327ec4f26e42f02f69c999fc8b905c8c98a663 Verifying Hash Integrity ... sha256+ OK Decrypting Data ... OK ## Loading ramdisk from FIT Image at 00800800 ... Using 'conf-rockchip_rk3399-tcirk.dtb' configuration Verifying Hash Integrity ... OK Trying 'ramdisk-1' ramdisk subimage Description: initramfs Created: 2025-01-29 20:53:29 UTC Type: RAMDisk Image Compression: uncompressed Data Start: 0x0113d7dc Data Size: 9248611 Bytes = 8.8 MiB Architecture: AArch64 OS: Linux Load Address: unavailable Entry Point: unavailable Hash algo: sha256 Hash value: 7cb5aaab6a280284d8f6809daaec2e208fb7c0a1b1eff918371925f04ff8e2ea Verifying Hash Integrity ... sha256+ OK Decrypting Data ... OK ## Loading fdt from FIT Image at 00800800 ... Using 'conf-rockchip_rk3399-tcirk.dtb' configuration Verifying Hash Integrity ... OK Trying 'fdt-rockchip_rk3399-tcirk.dtb' fdt subimage Description: Flattened Device Tree blob Created: 2025-01-29 20:53:29 UTC Type: Flat Device Tree Compression: uncompressed Data Start: 0x0112d160 Data Size: 66993 Bytes = 65.4 KiB Architecture: AArch64 Hash algo: sha256 Hash value: 3fe8d11246291479c1bdd95fd1367caee7a15c0de5a81a7ac5aca40b9c4240ba Verifying Hash Integrity ... sha256+ OK Decrypting Data ... OK Booting using the fdt blob at 0x112d160 Working FDT set to 112d160 Uncompressing Kernel Image Loading Ramdisk to 5e5cc000, end 5ee9df63 ... OK Loading Device Tree to 000000005e5b8000, end 000000005e5cb5b0 ... OK Working FDT set to 5e5b8000
Starting kernel ...
[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034] [ 0.000000] Linux version 6.12.6-00065-g20bf32f48288-dirty (x@x) (aarch64-linux-gnu-gcc (Debian 12.2.0-14) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #1 SMP PREEMPT Wed Jan 29 21:53:22 CET 2025 [ 0.000000] KASLR disabled due to lack of seed [ 0.000000] Machine model: tcirk-vir4-hw [ 0.000000] earlycon: uart8250 at MMIO32 0x00000000ff1a0000 (options '') [ 0.000000] printk: legacy bootconsole [uart8250] enabled [ 0.000000] efi: UEFI not found. [ 0.000000] OF: reserved mem: 0x0000000000000000..0x00000000001fffff (2048 KiB) nomap non-reusable secmon@0 [ 0.000000] OF: reserved mem: 0x0000000030000000..0x0000000031ffffff (32768 KiB) nomap non-reusable optee@43200000 [ 0.000000] NUMA: Faking a node at [mem 0x0000000000200000-0x000000007fffffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x7fbab600-0x7fbadc3f] [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000200000-0x000000007fffffff] [ 0.000000] DMA32 empty [ 0.000000] Normal empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000200000-0x000000002fffffff] [ 0.000000] node 0: [mem 0x0000000030000000-0x0000000031ffffff] [ 0.000000] node 0: [mem 0x0000000032000000-0x000000007fffffff] [ 0.000000] Initmem setup node 0 [mem 0x0000000000200000-0x000000007fffffff] [ 0.000000] On node 0, zone DMA: 512 pages in unavailable ranges [ 0.000000] cma: Reserved 32 MiB at 0x000000007ba00000 on node -1 [ 0.000000] psci: probing for conduit method from DT. [ 0.000000] psci: PSCIv1.0 detected in firmware. [ 0.000000] psci: Using standard PSCI v0.2 function IDs [ 0.000000] psci: Trusted OS migration not required [ 0.000000] psci: SMC Calling Convention v1.0 [ 0.000000] percpu: Embedded 23 pages/cpu s53272 r8192 d32744 u94208 [ 0.000000] Detected VIPT I-cache on CPU0 [ 0.000000] CPU features: detected: GIC system register CPU interface [ 0.000000] CPU features: detected: ARM erratum 845719 [ 0.000000] alternatives: applying boot alternatives [ 0.000000] Kernel command line: panic=3 console=ttyS2,1500000 earlycon=uart8250,mmio32,0xff1a0000 root=PARTUUID=14ac38b7-213e-19fa-9ba2-2147628lbaa3 rootwait fsck.repair=yes [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) [ 0.000000] Fallback order for Node 0: 0 [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 523776 [ 0.000000] Policy zone: DMA [ 0.000000] mem auto-init: stack:all(zero), heap alloc:off, heap free:off [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB [ 0.000000] software IO TLB: area num 8. [ 0.000000] software IO TLB: mapped [mem 0x000000007b600000-0x000000007b800000] (2MB) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=6, Nodes=1 [ 0.000000] rcu: Preemptible hierarchical RCU implementation. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=512 to nr_cpu_ids=6. [ 0.000000] Trampoline variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=6 [ 0.000000] RCU Tasks: Setting shift to 3 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=6. [ 0.000000] RCU Tasks Trace: Setting shift to 3 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=6. [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 [ 0.000000] GICv3: GIC: Using split EOI/Deactivate mode [ 0.000000] GICv3: 256 SPIs implemented [ 0.000000] GICv3: 0 Extended SPIs implemented [ 0.000000] Root IRQ handler: gic_handle_irq [ 0.000000] GICv3: GICv3 features: 16 PPIs [ 0.000000] GICv3: Broken GIC integration, security disabled [ 0.000000] GICv3: GICD_CTRL.DS=1, SCR_EL3.FIQ=1 [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000fef00000 [ 0.000000] ITS [mem 0xfee20000-0xfee3ffff] [ 0.000000] ITS@0x00000000fee20000: allocated 65536 Devices @480000 (flat, esz 8, psz 64K, shr 0) [ 0.000000] ITS: using cache flushing for cmd queue [ 0.000000] GICv3: using LPI property table @0x0000000000460000 [ 0.000000] GIC: using cache flushing for LPI property table [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000000470000 [ 0.000000] GICv3: GIC: PPI partition interrupt-partition-0[0] { /cpus/cpu@0[0] /cpus/cpu@1[1] /cpus/cpu@2[2] /cpus/cpu@3[3] } [ 0.000000] GICv3: GIC: PPI partition interrupt-partition-1[1] { /cpus/cpu@100[4] /cpus/cpu@101[5] } [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. [ 0.000000] arch_timer: cp15 timer(s) running at 24.00MHz (phys). [ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x588fe9dc0, max_idle_ns: 440795202592 ns [ 0.000003] sched_clock: 56 bits at 24MHz, resolution 41ns, wraps every 4398046511097ns [ 0.004487] Console: colour dummy device 80x25 [ 0.005311] Calibrating delay loop (skipped), value calculated using timer frequency.. 48.00 BogoMIPS (lpj=24000) [ 0.006425] pid_max: default: 32768 minimum: 301 [ 0.007248] LSM: initializing lsm=capability [ 0.008339] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) [ 0.009195] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) [ 0.022860] rcu: Hierarchical SRCU implementation. [ 0.023390] rcu: Max phase no-delay instances is 400. [ 0.025119] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level [ 0.028650] EFI services will not be available. [ 0.030776] smp: Bringing up secondary CPUs ... I/TC: Secondary CPU 1 initializing I/TC: Secondary CPU 1 switching to normal world boot I/TC: Secondary CPU 2 initializing I/TC: Secondary CPU 2 switching to normal world boot I/TC: Secondary CPU 3 initializing I/TC: Secondary CPU 3 switching to normal world boot I/TC: Secondary CPU 4 initializing I/TC: Secondary CPU 4 switching to normal world boot I/TC: Secondary CPU 5 initializing I/TC: Secondary CPU 5 switching to normal world boot [ 0.036689] Detected VIPT I-cache on CPU1 [ 0.036956] GICv3: CPU1: found redistributor 1 region 0:0x00000000fef20000 [ 0.037013] GICv3: CPU1: using allocated LPI pending table @0x0000000000500000 [ 0.037146] CPU1: Booted secondary processor 0x0000000001 [0x410fd034] [ 0.042458] Detected VIPT I-cache on CPU2 [ 0.042733] GICv3: CPU2: found redistributor 2 region 0:0x00000000fef40000 [ 0.042791] GICv3: CPU2: using allocated LPI pending table @0x0000000000510000 [ 0.042924] CPU2: Booted secondary processor 0x0000000002 [0x410fd034] [ 0.048145] Detected VIPT I-cache on CPU3 [ 0.048436] GICv3: CPU3: found redistributor 3 region 0:0x00000000fef60000 [ 0.048494] GICv3: CPU3: using allocated LPI pending table @0x0000000000520000 [ 0.048625] CPU3: Booted secondary processor 0x0000000003 [0x410fd034] [ 0.064979] CPU features: detected: Spectre-v2 [ 0.065590] CPU features: detected: Spectre-v3a [ 0.065933] CPU features: detected: Spectre-v4 [ 0.066243] CPU features: detected: Spectre-BHB [ 0.066641] CPU features: detected: ARM erratum 1742098 [ 0.067027] CPU features: detected: ARM errata 1165522, 1319367, or 1530923 [ 0.067364] Detected PIPT I-cache on CPU4 [ 0.071008] GICv3: CPU4: found redistributor 100 region 0:0x00000000fef80000 [ 0.071769] GICv3: CPU4: using allocated LPI pending table @0x0000000000530000 [ 0.073446] CPU4: Booted secondary processor 0x0000000100 [0x410fd082] [ 0.102961] Detected PIPT I-cache on CPU5 [ 0.107087] GICv3: CPU5: found redistributor 101 region 0:0x00000000fefa0000 [ 0.107886] GICv3: CPU5: using allocated LPI pending table @0x0000000000540000 [ 0.109552] CPU5: Booted secondary processor 0x0000000101 [0x410fd082] [ 0.121026] smp: Brought up 1 node, 6 CPUs [ 0.137438] SMP: Total of 6 processors activated. [ 0.137994] CPU: All CPU(s) started at EL2 [ 0.138462] CPU features: detected: 32-bit EL0 Support [ 0.139081] CPU features: detected: CRC32 instructions [ 0.142315] alternatives: applying system-wide alternatives [ 0.152865] Memory: 1946248K/2095104K available (13824K kernel code, 1346K rwdata, 4528K rodata, 3520K init, 603K bss, 108856K reserved, 32768K cma-reserved) [ 0.157602] devtmpfs: initialized [ 0.226157] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.227568] futex hash table entries: 2048 (order: 5, 131072 bytes, linear) [ 0.232219] 26752 pages in range for non-PLT usage [ 0.232591] 518272 pages in range for PLT usage [ 0.234879] pinctrl core: initialized pinctrl subsystem [ 0.238462] DMI not present or invalid. [ 0.262606] NET: Registered PF_NETLINK/PF_ROUTE protocol family [ 0.274891] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations [ 0.277008] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.279510] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.287254] thermal_sys: Registered thermal governor 'step_wise' [ 0.287334] thermal_sys: Registered thermal governor 'power_allocator' [ 0.288498] cpuidle: using governor menu [ 0.291457] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers. [ 0.295740] ASID allocator initialised with 65536 entries [ 0.374178] /syscon@ff770000/phy@f780: Fixed dependency cycle(s) with /mmc@fe330000 [ 0.378621] /mmc@fe330000: Fixed dependency cycle(s) with /syscon@ff770000/phy@f780 [ 0.379769] /syscon@ff770000/phy@f780: Fixed dependency cycle(s) with /mmc@fe330000 [ 0.400094] /vop@ff900000: Fixed dependency cycle(s) with /dsi@ff960000 [ 0.401196] /vop@ff8f0000: Fixed dependency cycle(s) with /dsi@ff960000 [ 0.402205] /dsi@ff960000: Fixed dependency cycle(s) with /dsi@ff960000/bridge-dsi@0 [ 0.403080] /dsi@ff960000: dependency cycle(s) with /vop@ff8f0000 [ 0.404041] /dsi@ff960000: Fixed dependency cycle(s) with /vop@ff900000 [ 0.405380] /dsi@ff960000/bridge-dsi@0: Fixed dependency cycle(s) with /dsi@ff960000 [ 0.431355] gpio gpiochip0: Static allocation of GPIO base is deprecated, use dynamic allocation. [ 0.434094] rockchip-gpio ff720000.gpio: probed /pinctrl/gpio@ff720000 [ 0.436711] gpio gpiochip1: Static allocation of GPIO base is deprecated, use dynamic allocation. [ 0.438580] rockchip-gpio ff730000.gpio: probed /pinctrl/gpio@ff730000 [ 0.440577] gpio gpiochip2: Static allocation of GPIO base is deprecated, use dynamic allocation. [ 0.442324] rockchip-gpio ff780000.gpio: probed /pinctrl/gpio@ff780000 [ 0.444303] gpio gpiochip3: Static allocation of GPIO base is deprecated, use dynamic allocation. [ 0.446154] rockchip-gpio ff788000.gpio: probed /pinctrl/gpio@ff788000 [ 0.448175] gpio gpiochip4: Static allocation of GPIO base is deprecated, use dynamic allocation. [ 0.449955] rockchip-gpio ff790000.gpio: probed /pinctrl/gpio@ff790000 [ 0.461437] /dsi@ff960000/bridge-dsi@0: Fixed dependency cycle(s) with /panel-dpi [ 0.462719] /panel-dpi: Fixed dependency cycle(s) with /dsi@ff960000/bridge-dsi@0 [ 0.490476] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages [ 0.491515] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page [ 0.492218] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages [ 0.492922] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page [ 0.493619] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages [ 0.494320] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page [ 0.495014] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages [ 0.495713] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page [ 0.525614] iommu: Default domain type: Passthrough [ 0.537266] SCSI subsystem initialized [ 0.540585] usbcore: registered new interface driver usbfs [ 0.541351] usbcore: registered new interface driver hub [ 0.542062] usbcore: registered new device driver usb [ 0.543936] mc: Linux media interface: v0.10 [ 0.544562] videodev: Linux video capture interface: v2.00 [ 0.545400] pps_core: LinuxPPS API ver. 1 registered [ 0.545928] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti giometti@linux.it [ 0.546946] PTP clock support registered [ 0.547523] EDAC MC: Ver: 3.0.0 [ 0.550022] scmi_core: SCMI protocol bus registered [ 0.553236] Advanced Linux Sound Architecture Driver Initialized. [ 0.566115] clocksource: Switched to clocksource arch_sys_counter [ 0.568672] VFS: Disk quotas dquot_6.6.0 [ 0.569218] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.613752] NET: Registered PF_INET protocol family [ 0.615404] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) [ 0.625739] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384bytes, linear) [ 0.626896] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) [ 0.629804] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) [ 0.631401] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) [ 0.634324] TCP: Hash tables configured (established 16384 bind 16384) [ 0.635926] UDP hash table entries: 1024 (order: 3, 32768 bytes, linear) [ 0.637024] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes, linear) [ 0.638943] NET: Registered PF_UNIX/PF_LOCAL protocol family [ 0.644047] RPC: Registered named UNIX socket transport module. [ 0.644806] RPC: Registered udp transport module. [ 0.645354] RPC: Registered tcp transport module. [ 0.645852] RPC: Registered tcp-with-tls transport module. [ 0.646469] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.654176] Unpacking initramfs... [ 0.669172] Initialise system trusted keyrings [ 0.672849] workingset: timestamp_bits=42 max_order=19 bucket_order=0 [ 0.675780] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ 0.679691] NFS: Registering the id_resolver key type [ 0.680711] Key type id_resolver registered [ 0.681176] Key type id_legacy registered [ 0.681793] nfs4filelayout_init: NFSv4 File Layout Driver Registering... [ 0.682558] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering.. . [ 0.684806] 9p: Installing v9fs 9p2000 file system support [ 1.014409] Key type asymmetric registered [ 1.015335] Asymmetric key parser 'x509' registered [ 1.016907] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 245) [ 1.017753] io scheduler mq-deadline registered [ 1.018287] io scheduler kyber registered [ 1.018886] io scheduler bfq registered [ 1.103497] ledtrig-cpu: registered to indicate activity on CPUs [ 1.121682] dma-pl330 ff6d0000.dma-controller: Loaded driver for PL330 DMAC-241330 [ 1.122828] dma-pl330 ff6d0000.dma-controller: DBUFF-32x8bytes Num_Chans-6 Num_Peri-12 Num_Events-12 [ 1.130971] dma-pl330 ff6e0000.dma-controller: Loaded driver for PL330 DMAC-241330 [ 1.132002] dma-pl330 ff6e0000.dma-controller: DBUFF-128x8bytes Num_Chans-8 Num_Peri-20 Num_Events-16 [ 1.165185] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled [ 1.179214] printk: legacy console [ttyS2] disabled [ 1.182047] ff1a0000.serial: ttyS2 at MMIO 0xff1a0000 (irq = 37, base_baud = 1500000) is a 16550A [ 1.184023] printk: legacy console [ttyS2] enabled [ 1.184023] printk: legacy console [ttyS2] enabled [ 1.185074] printk: legacy bootconsole [uart8250] disabled [ 1.185074] printk: legacy bootconsole [uart8250] disabled [ 1.196793] ff1b0000.serial: ttyS3 at MMIO 0xff1b0000 (irq = 38, base_baud = 1500000) is a 16550A [ 1.224035] rockchip-vop ff8f0000.vop: Adding to iommu group 0 [ 1.227910] rockchip-vop ff900000.vop: Adding to iommu group 1 [ 1.238640] /panel-dpi: Fixed dependency cycle(s) with /dsi@ff960000/bridge-dsi@0 [ 1.240012] /dsi@ff960000: Fixed dependency cycle(s) with /dsi@ff960000/bridge-dsi@0 [ 1.241498] /dsi@ff960000/bridge-dsi@0: Fixed dependency cycle(s) with /panel-dpi [ 1.242550] /dsi@ff960000/bridge-dsi@0: Fixed dependency cycle(s) with /dsi@ff960000 [ 1.359770] loop: module loaded [ 1.391983] tun: Universal TUN/TAP device driver, 1.6 [ 1.398847] usbcore: registered new device driver r8152-cfgselector [ 1.399799] usbcore: registered new interface driver r8152 [ 1.401916] VFIO - User Level meta-driver version: 0.3 [ 1.439447] ehci-platform fe3c0000.usb: EHCI Host Controller [ 1.439552] xhci-hcd xhci-hcd.0.auto: xHCI Host Controller [ 1.439684] ohci-platform fe3e0000.usb: Generic Platform OHCI controller [ 1.439811] ehci-platform fe3c0000.usb: new USB bus registered, assigned bus number 1 [ 1.439814] ohci-platform fe3e0000.usb: new USB bus registered, assigned bus number 2 [ 1.440401] xhci-hcd xhci-hcd.0.auto: new USB bus registered, assigned bus number 3 [ 1.440455] ohci-platform fe3e0000.usb: irq 47, io mem 0xfe3e0000 [ 1.441218] ehci-platform fe3c0000.usb: irq 46, io mem 0xfe3c0000 [ 1.441889] xhci-hcd xhci-hcd.0.auto: hcc params 0x0220fe64 hci version 0x110 quirks 0x0000808002000010 [ 1.445802] xhci-hcd xhci-hcd.0.auto: irq 45, io mem 0xfe900000 [ 1.447205] xhci-hcd xhci-hcd.0.auto: xHCI Host Controller [ 1.447954] xhci-hcd xhci-hcd.0.auto: new USB bus registered, assigned bus number 4 [ 1.448611] ehci-platform fe3c0000.usb: USB 2.0 started, EHCI 1.00 [ 1.448781] xhci-hcd xhci-hcd.0.auto: Host supports USB 3.0 SuperSpeed [ 1.453899] hub 1-0:1.0: USB hub found [ 1.454581] hub 1-0:1.0: 1 port detected [ 1.460692] hub 3-0:1.0: USB hub found [ 1.461448] hub 3-0:1.0: 1 port detected [ 1.463918] usb usb4: We don't know the algorithms for LPM for this host, disabling LPM. [ 1.468035] hub 4-0:1.0: USB hub found [ 1.468626] hub 4-0:1.0: 1 port detected [ 1.471219] usbcore: registered new interface driver usb-storage [ 1.483730] UDC core: g_ether: couldn't find an available UDC [ 1.489488] i2c_dev: i2c /dev entries driver [ 1.515561] hub 2-0:1.0: USB hub found [ 1.516545] hub 2-0:1.0: 1 port detected [ 1.532183] rtc-s35390a 5-0030: registered as rtc0 [ 1.534539] rtc-s35390a 5-0030: setting system clock to 2025-01-29T20:53:51 UTC (1738184031) [ 1.543848] /i2c@ff3c0000/pmic@1b: Fixed dependency cycle(s) with /i2c@ff3c0000 /pmic@1b/regulators/DCDC_REG4 [ 1.609013] rk808-rtc rk808-rtc.3.auto: registered as rtc1 [ 1.616450] fan53555-regulator 0-0040: FAN53555 Option[8] Rev[1] Detected! [ 1.620451] fan53555-regulator 0-0041: FAN53555 Option[8] Rev[1] Detected! [ 1.622577] hantro-vpu ff650000.video-codec: Adding to iommu group 2 [ 1.628648] hantro-vpu ff650000.video-codec: registered rockchip,rk3399-vpu-enc as /dev/video0 [ 1.631489] hantro-vpu ff650000.video-codec: registered rockchip,rk3399-vpu-dec as /dev/video1 [ 1.660392] dw_wdt ff848000.watchdog: No valid TOPs array specified [ 1.679904] cpufreq: cpufreq_online: CPU0: Running at unlisted initial frequency: 200000 KHz, changing to: 408000 KHz [ 1.682388] cpu cpu0: EM: created perf domain [ 1.691575] cpufreq: cpufreq_online: CPU4: Running at unlisted initial frequency: 12000 KHz, changing to: 408000 KHz [ 1.692804] cpu cpu4: EM: created perf domain [ 1.695948] sdhci: Secure Digital Host Controller Interface driver [ 1.696503] sdhci: Copyright(c) Pierre Ossman [ 1.697090] Synopsys Designware Multimedia Card Interface Driver [ 1.698013] sdhci-pltfm: SDHCI platform and OF driver helper [ 1.699318] dwmmc_rockchip fe320000.mmc: IDMAC supports 32-bit address mode. [ 1.699665] usbcore: registered new interface driver usbhid [ 1.699974] dwmmc_rockchip fe320000.mmc: Using internal DMA controller. [ 1.700062] mmc1: CQHCI version 5.10 [ 1.700444] usbhid: USB HID core driver [ 1.701709] dwmmc_rockchip fe320000.mmc: Version ID is 270a [ 1.702284] dwmmc_rockchip fe320000.mmc: DW MMC controller at irq 66,32 bit host data width,256 deep fifo [ 1.703364] dwmmc_rockchip fe320000.mmc: Got CD GPIO [ 1.705422] hw perfevents: enabled with armv8_cortex_a53 PMU driver, 7 (0,8000003f) counters available [ 1.706582] hw perfevents: enabled with armv8_cortex_a72 PMU driver, 7 (0,8000003f) counters available [ 1.708117] optee: probing for conduit method. I/TC: Reserved shared memory is enabled I/TC: Dynamic shared memory is enabled I/TC: Normal World virtualization support is disabled I/TC: Asynchronous notifications are disabled [ 1.708530] optee: revision 3.20 (7067d1c2) [ 1.710428] optee: dynamic shared memory is enabled E/TC:4 0 Panic 'Secure interrupt handler not defined' at core/kernel/interrupt.c:139 <itr_core_handler> E/TC:4 0 TEE load address @ 0x30000000 E/TC:4 0 Call stack: E/TC:4 0 0x300091f8 E/TC:4 0 0x30016664 E/TC:4 0 0x30015710 [ 1.716572] mmc_host mmc0: Bus speed (slot 0) = 400000Hz (slot req 400000Hz, actual 400000HZ div = 0) [ 1.795412] usb 4-1: new SuperSpeed USB device number 2 using xhci-hcd [ 1.843025] Freeing initrd memory: 9028K [ 1.962283] r8152-cfgselector 4-1: reset SuperSpeed USB device number 2 using xhci-hcd [ 1.983718] r8152 4-1:1.0 eth0: v1.12.13 [ 22.722238] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: [ 22.722774] rcu: 4-...0: (1 GPs behind) idle=11cc/1/0x4000000000000000 softirq=98/99 fqs=4193 [ 22.723528] rcu: (detected by 5, t=21005 jiffies, g=-743, q=988 ncpus=6) [ 22.724124] Sending NMI from CPU 5 to CPUs 4: [ 32.725383] rcu: rcu_preempt kthread starved for 9995 jiffies! g-743 f0x0 RCU_GP_DOING_FQS(6) ->state=0x0 ->cpu=1 [ 32.725442] rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { [ 32.725767] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [ 32.725770] rcu: RCU grace-period kthread stack dump: [ 32.725772] task:rcu_preempt state:I stack:0 pid:16 tgid:16 ppid:2 flags:0x00000008 [ 32.725780] Call trace: [ 32.725782] __switch_to+0xf0/0x14c [ 32.726713] 4-...D [ 32.727301] __schedule+0x264/0xa90 [ 32.727305] schedule+0x34/0x104 [ 32.727308] schedule_timeout+0x80/0xf4 [ 32.727314] rcu_gp_fqs_loop+0x14c/0x4a4 [ 32.727323] rcu_gp_kthread+0x138/0x164 [ 32.727327] kthread+0x114/0x118 [ 32.728139] } 30878 jiffies s: 37 root: 0x10/. [ 32.728564] ret_from_fork+0x10/0x20 [ 32.728570] rcu: Stack dump where RCU GP kthread last ran: [ 32.728572] Sending NMI from CPU 5 to CPUs 1: [ 32.728581] NMI backtrace for cpu 1 [ 32.729421] rcu: blocking rcu_node structures (internal RCU debug): [ 32.729619] CPU: 1 UID: 0 PID: 0 Comm: swapper/1 Not tainted 6.12.6-00065-g20bf32f48288-dirty #1 [ 32.729922] [ 32.730108] Hardware name: tcirk-vir4-hw (DT) [ 32.735680] pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 32.736292] pc : cpuidle_enter_state+0xac/0x2fc [ 32.736699] lr : cpuidle_enter_state+0xa4/0x2fc [ 32.737101] sp : ffff80008196bd90 [ 32.737394] x29: ffff80008196bd90 x28: 0000000000000000 x27: 0000000000000000 [ 32.738027] x26: 0000000000000000 x25: 000000079ec1bd5c x24: 0000000000000000 [ 32.738659] x23: 0000000000000000 x22: ffff000001c58880 x21: ffff00007fb2c138 [ 32.739292] x20: 0000000000000000 x19: 000000079ec66d11 x18: 0000000000000001 [ 32.739924] x17: 000000040044ffff x16: 00500072b5503510 x15: 0000000000000000 [ 32.740556] x14: ffff80008133d900 x13: 0000000000000400 x12: 0000000000000000 [ 32.741188] x11: ffff80008133bf00 x10: 071c71c71c71c71c x9 : 0000000000000350 [ 32.741820] x8 : 00000000000cf080 x7 : 00000000000000c0 x6 : 000000001521f5e1 [ 32.742452] x5 : 00ffffffffffffff x4 : 0000000000000015 x3 : 000000000050d8f0 [ 32.743084] x2 : ffff7ffffe7f0000 x1 : ffff00007fb2d900 x0 : 0000000000000000 [ 32.743717] Call trace: [ 32.743934] cpuidle_enter_state+0xac/0x2fc [ 32.744307] cpuidle_enter+0x38/0x50 [ 32.744630] do_idle+0x1e8/0x258 [ 32.744920] cpu_startup_entry+0x38/0x3c [ 32.745268] secondary_start_kernel+0x124/0x144 [ 32.745671] __secondary_switched+0xb8/0xbc
On Thu, 30 Jan 2025 18:14:50 +0000, Christoph Fritz chf.fritz@googlemail.com wrote:
Any ideas?
I think this calls for a revert of this patch, potentially at the expense if NMI support on this machine. Could you show how SCR_EL3.FIQ is configured on this machine? Mine shows:
[ 0.000000] GICv3: GICD_CTRL.DS=0, SCR_EL3.FIQ=0
and I suspect yours has FIQ=1.
yes it's 1:
[ 0.000000] GICv3: GICv3 features: 16 PPIs [ 0.000000] GICv3: Broken GIC integration, security disabled [ 0.000000] GICv3: GICD_CTRL.DS=1, SCR_EL3.FIQ=1 [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000fef00000
The device is not closed (there is no docu) and OP-TEE is actually here not used for anything. But I think I had the OP-TEE xtests running successfully before.
I don't think OP-TEE *really* works, as it cannot take any interrupt after the kernel has booted. We blindly reconfigure everything to be non-secure, so unless you solely rely on services that do not interact with devices, it is absolutely dead.
Anyway, can you try the hack below? it works on my own rk3399, but I don't have anything on the secure side. Please also try it by passing "irqchip.gicv3_pseudo_nmi=1" on the command-line.
Thanks,
M.
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index 79d8cc80693c3..f60d4d7e87639 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -44,6 +44,7 @@ static u8 dist_prio_nmi __ro_after_init = GICV3_PRIO_NMI; #define FLAGS_WORKAROUND_GICR_WAKER_MSM8996 (1ULL << 0) #define FLAGS_WORKAROUND_CAVIUM_ERRATUM_38539 (1ULL << 1) #define FLAGS_WORKAROUND_ASR_ERRATUM_8601001 (1ULL << 2) +#define FLAGS_WORKAROUND_INSECURE (1ULL << 3)
#define GIC_IRQ_TYPE_PARTITION (GIC_IRQ_TYPE_LPI + 1)
@@ -83,6 +84,8 @@ static DEFINE_STATIC_KEY_TRUE(supports_deactivate_key); #define GIC_LINE_NR min(GICD_TYPER_SPIS(gic_data.rdists.gicd_typer), 1020U) #define GIC_ESPI_NR GICD_TYPER_ESPIS(gic_data.rdists.gicd_typer)
+static bool nmi_support_forbidden; + /* * There are 16 SGIs, though we only actually use 8 in Linux. The other 8 SGIs * are potentially stolen by the secure side. Some code, especially code dealing @@ -163,21 +166,27 @@ static void __init gic_prio_init(void) { bool ds;
- ds = gic_dist_security_disabled(); - if (!ds) { - u32 val; - - val = readl_relaxed(gic_data.dist_base + GICD_CTLR); - val |= GICD_CTLR_DS; - writel_relaxed(val, gic_data.dist_base + GICD_CTLR); + cpus_have_group0 = gic_has_group0();
- ds = gic_dist_security_disabled(); - if (ds) - pr_warn("Broken GIC integration, security disabled"); + ds = gic_dist_security_disabled(); + if ((gic_data.flags & FLAGS_WORKAROUND_INSECURE) && !ds) { + if (cpus_have_group0) { + u32 val; + + val = readl_relaxed(gic_data.dist_base + GICD_CTLR); + val |= GICD_CTLR_DS; + writel_relaxed(val, gic_data.dist_base + GICD_CTLR); + + ds = gic_dist_security_disabled(); + if (ds) + pr_warn("Broken GIC integration, security disabled\n"); + } else { + pr_warn("Broken GIC integration, pNMI forbidden\n"); + nmi_support_forbidden = true; + } }
cpus_have_security_disabled = ds; - cpus_have_group0 = gic_has_group0();
/* * How priority values are used by the GIC depends on two things: @@ -209,7 +218,7 @@ static void __init gic_prio_init(void) * be in the non-secure range, we program the non-secure values into * the distributor to match the PMR values we want. */ - if (cpus_have_group0 & !cpus_have_security_disabled) { + if (cpus_have_group0 && !cpus_have_security_disabled) { dist_prio_irq = __gicv3_prio_to_ns(dist_prio_irq); dist_prio_nmi = __gicv3_prio_to_ns(dist_prio_nmi); } @@ -1922,6 +1931,18 @@ static bool gic_enable_quirk_arm64_2941627(void *data) return true; }
+static bool gic_enable_quirk_rk3399(void *data) +{ + struct gic_chip_data *d = data; + + if (of_machine_is_compatible("rockchip,rk3399")) { + d->flags |= FLAGS_WORKAROUND_INSECURE; + return true; + } + + return false; +} + static bool rd_set_non_coherent(void *data) { struct gic_chip_data *d = data; @@ -1996,6 +2017,12 @@ static const struct gic_quirk gic_quirks[] = { .property = "dma-noncoherent", .init = rd_set_non_coherent, }, + { + .desc = "GICv3: Insecure RK3399 integration", + .iidr = 0x0000043b, + .mask = 0xff000fff, + .init = gic_enable_quirk_rk3399, + }, { } }; @@ -2004,7 +2031,7 @@ static void gic_enable_nmi_support(void) { int i;
- if (!gic_prio_masking_enabled()) + if (!gic_prio_masking_enabled() || nmi_support_forbidden) return;
rdist_nmi_refs = kcalloc(gic_data.ppi_nr + SGI_NR,
Anyway, can you try the hack below? it works on my own rk3399, but I don't have anything on the secure side. Please also try it by passing "irqchip.gicv3_pseudo_nmi=1" on the command-line.
Done, works in both cases now for me. If you want you can add my Tested-by and/or Reported-by.
Thanks -- Christoph
linux-stable-mirror@lists.linaro.org
-
Christoph Fritz -
Marc Zyngier