Hi Vikram, Joonwoo,
[ + EAS mailing list ]
On Hikey960 with EASv1.3, I encountered many times for WALT panic, it reports the bug from below two functions; you also could see log in the below. Before I dig into this, could you give some suggestion for this? Or if there have some existed fixing for this? Thanks in advance.
void walt_dec_cumulative_runnable_avg(struct rq *rq, struct task_struct *p) { rq->cumulative_runnable_avg -= p->ravg.demand; BUG_ON((s64)rq->cumulative_runnable_avg < 0); }
static void fixup_cumulative_runnable_avg(struct rq *rq, struct task_struct *p, u64 new_task_load) { s64 task_load_delta = (s64)new_task_load - task_load(p);
rq->cumulative_runnable_avg += task_load_delta; if ((s64)rq->cumulative_runnable_avg < 0) panic("cra less than zero: tld: %lld, task_load(p) = %u\n", task_load_delta, task_load(p)); }
--- Panic Log ---
[ 1108.441865] init: Untracked pid 15425 exited with status 0 [ 1108.657107] ------------[ cut here ]------------ [ 1108.661746] kernel BUG at kernel/sched/walt.c:109! [ 1108.666538] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP [ 1108.672026] CPU: 1 PID: 1248 Comm: kschedfreq:0 Not tainted 4.4.78-07635-g0255026 #45 [ 1108.679851] Hardware name: HiKey960 (DT) [ 1108.683770] task: ffffffc0b166c080 ti: ffffffc0b0e64000 task.ti: ffffffc0b0e64000 [ 1108.691261] PC is at walt_dec_cumulative_runnable_avg+0x40/0x44 [ 1108.697179] LR is at dequeue_task_rt+0x40/0x8c [ 1108.701617] pc : [<ffffff8008112428>] lr : [<ffffff800810c82c>] pstate: 60000185 [ 1108.709007] sp : ffffffc0b0e67b90 [ 1108.712315] x29: ffffffc0b0e67b90 x28: 0000000000000001 [ 1108.717633] x27: ffffff8008bc4fc4 x26: ffffffc0bff13400 [ 1108.722948] x25: ffffffc0b166c6c8 x24: 0000000000000000 [ 1108.728263] x23: ffffff8009095000 x22: ffffffc0b166c080 [ 1108.733579] x21: ffffffc0bff13be8 x20: ffffffc0b166c080 [ 1108.738895] x19: ffffffc0bff13400 x18: 0000000000000000 [ 1108.744209] x17: 0000000000000000 x16: 0000000000000000 [ 1108.749524] x15: 0000000000000000 x14: 0000000000000000 [ 1108.754839] x13: 0000000000000000 x12: 0000000034d5d91d [ 1108.760156] x11: ffffff8008be13cc x10: 00000000000009d0 [ 1108.765471] x9 : ffffffc0b0e64000 x8 : ffffffc0b0e67ce0 [ 1108.770786] x7 : ffffffc0ae6cfe30 x6 : ffffff8009095000 [ 1108.776101] x5 : 0000000000000001 x4 : 00000040b6ea8000 [ 1108.781415] x3 : 0000000000000002 x2 : 0000000000000000 [ 1108.786730] x1 : fffffffffffedce2 x0 : 00000000000bc75e [ 1108.792047] [ 1108.792047] SP: 0xffffffc0b0e67b10: [ 1108.797006] 7b10 b166c080 ffffffc0 09095000 ffffff80 00000000 00000000 b166c6c8 ffffffc0 [ 1108.805223] 7b30 bff13400 ffffffc0 08bc4fc4 ffffff80 00000001 00000000 b0e67b90 ffffffc0 [ 1108.813440] 7b50 0810c82c ffffff80 b0e67b90 ffffffc0 08112428 ffffff80 60000185 00000000 [ 1108.821656] 7b70 b0e67ba0 ffffffc0 0810c538 ffffff80 ffffffff ffffffff 0810c574 ffffff80 [ 1108.829872] 7b90 b0e67bc0 ffffffc0 0810c82c ffffff80 bff13400 ffffffc0 0810c820 ffffff80 [ 1108.838090] 7bb0 bff13400 ffffffc0 b166c080 ffffffc0 b0e67bf0 ffffffc0 080eead8 ffffff80 [ 1108.846306] 7bd0 bff13400 ffffffc0 0906b000 ffffff80 bff13400 ffffffc0 0906b000 ffffff80 [ 1108.854522] 7bf0 b0e67c20 ffffffc0 08bc4b80 ffffff80 bff13400 ffffffc0 08bc47e4 ffffff80 [ 1108.862741] [ 1108.862741] X1: 0xfffffffffffedc62: [ 1108.867700] dc60 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.875924] dc80 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.884140] dca0 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.892358] dcc0 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.900576] dce0 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.908793] dd00 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.917009] dd20 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.925227] dd40 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.933446] dd60 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.941666] [ 1108.941666] X7: 0xffffffc0ae6cfdb0: [ 1108.946625] fdb0 0000c350 00000000 00000001 00000000 00000000 00000000 ae6cfeb0 ffffffc0 [ 1108.954840] fdd0 00000000 00000000 00000001 00000000 00000000 00000000 0000c350 00000001 [ 1108.963055] fdf0 ae6cfe90 ffffffc0 0813dd98 ffffff80 b55c0418 0000007f 00000000 00000000 [ 1108.971271] fe10 ffffffff ffffffff b76b299c 0000007f ae6cfe60 ffffffc0 080efae0 00000001 [ 1108.979489] fe30 b0e67ce0 ffffffc0 00000000 00000000 00000000 00000000 0c0b8de9 00000102 [ 1108.987707] fe50 0c0aca99 00000102 0813c598 ffffff80 bff0ed40 ffffffc0 00000001 00000825 [ 1108.995923] fe70 08bc80e8 ffffff80 696c616d 6d656d2d 7275702d 00006567 b11f3100 ffffffc0 [ 1109.004140] fe90 00000000 00000000 08085f30 ffffff80 00000000 00000000 b6ee1020 0000007f [ 1109.012359] [ 1109.012359] X8: 0xffffffc0b0e67c60: [ 1109.017318] 7c60 b1092800 ffffffc0 091a8000 ffffff80 00000000 00000000 00009f4c 00000000 [ 1109.025536] 7c80 b0e67ca0 ffffffc0 08bc82a8 ffffff80 b0e67d98 ffffffc0 00000100 00000000 [ 1109.033752] 7ca0 b0e67d40 ffffffc0 08bc8348 ffffff80 b0e67d98 ffffffc0 00000064 00000000 [ 1109.041970] 7cc0 b11bb580 ffffffc0 b0e67d30 ffffffc0 0808e7e4 ffffff80 b166c080 00000001 [ 1109.050186] 7ce0 bff0f2d1 ffffffc0 ae6cfe30 ffffffc0 bff0f170 ffffffc0 09828528 00000102 [ 1109.058405] 7d00 0980fe88 00000102 0813c598 ffffff80 bff0ed40 ffffffc0 00000001 000004e0 [ 1109.066621] 7d20 08bc829c ffffff80 6863736b 72666465 303a7165 00000000 b166c080 ffffffc0 [ 1109.074839] 7d40 b0e67d70 ffffffc0 08bc8068 ffffff80 026e40e0 00000000 001a13c8 00000000 [ 1109.083058] [ 1109.083058] X9: 0xffffffc0b0e63f80: [ 1109.088015] 3f80 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.096232] 3fa0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.104448] 3fc0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.112665] 3fe0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.120883] 4000 00000000 00000000 ffffffff ffffffff b166c080 ffffffc0 00000003 00000001 [ 1109.129100] 4020 57ac6e9d 00000000 32273028 0d0f1c33 10233816 201d111a 3b013e3c 532f2f26 [ 1109.137317] 4040 ae7a3648 ffffffc0 ae354db8 ffffffc0 ae7a36c0 ffffffc0 ae7a36c0 ffffffc0 [ 1109.145535] 4060 00000001 00000000 aca585e0 ffffffc0 aca584e0 ffffffc0 07fb7c71 00000000 [ 1109.153753] [ 1109.153753] X19: 0xffffffc0bff13380: [ 1109.158799] 3380 00000000[ 1109.160774] mali e82c0000.mali: Reset interrupt didn't reach CPU. Check interrupt assignments.
[ 1109.169934] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.177103] 33a0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.185319] 33c0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.193537] 33e0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.201755] 3400 fab6faaf 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.209971] 3420 00000000 00000000 00000000 00000000 00031548 00000001 00000000 00000000 [ 1109.218187] 3440 00000000 00000000 00000001 00000000 00000000 00000000 00000000 00000000 [ 1109.226402] 3460 00017d53 00000000 0002db81 00000000 00000000 00000000 00000000 00000000 [ 1109.234619] [ 1109.234619] X20: 0xffffffc0b166c000: [ 1109.239664] c000 0b030d00 08010211 100f0d05 07091214 040e0a0c ffffff00 00000001 00000000 [ 1109.247881] c020 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.256097] c040 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.264313] c060 00000000 00000010 00000000 00000000 ffffffff 0000003f ffffffff 0000003f [ 1109.272528] c080 00000002 00000000 b0e64000 ffffffc0 00000003 04208040 00000000 00000000 [ 1109.280745] c0a0 00000000 00000000 00000001 00000000 00031435 00000001 84b47180 ffffffc0 [ 1109.288960] c0c0 00000001 00000001 00000031 00000078 00000031 00000032 08be15c8 ffffff80 [ 1109.297178] c0e0 00000400 00000000 00400000 00000000 00000001 00000000 00000000 00000000 [ 1109.305395] [ 1109.305395] X21: 0xffffffc0bff13b68: [ 1109.310440] 3b68 bff13b60 ffffffc0 bff13b70 ffffffc0 bff13b70 ffffffc0 bff13b80 ffffffc0 [ 1109.318658] 3b88 bff13b80 ffffffc0 bff13b90 ffffffc0 bff13b90 ffffffc0 bff13ba0 ffffffc0 [ 1109.326875] 3ba8 bff13ba0 ffffffc0 bff13bb0 ffffffc0 bff13bb0 ffffffc0 00000000 00000064 [ 1109.335092] 3bc8 00000064 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.343309] 3be8 bff13be8 ffffffc0 bff13be8 ffffffc0 00000000 00000000 00000000 00000000 [ 1109.351526] 3c08 00000000 00000000 0810cb88 ffffff80 00020002 00000000 00000000 00000000 [ 1109.359742] 3c28 006303e4 00000000 389fd980 00000000 f80df80d 00000000 00000000 00000000 [ 1109.367959] 3c48 bff13400 ffffffc0 091c42a0 ffffff80 00000000 00000000 00000000 00000000 [ 1109.376176] [ 1109.376176] X22: 0xffffffc0b166c000: [ 1109.381221] c000 0b030d00 08010211 100f0d05 07091214 040e0a0c ffffff00 00000001 00000000 [ 1109.389439] c020 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.397656] c040 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.405873] c060 00000000 00000010 00000000 00000000 ffffffff 0000003f ffffffff 0000003f [ 1109.414090] c080 00000002 00000000 b0e64000 ffffffc0 00000003 04208040 00000000 00000000 [ 1109.422307] c0a0 00000000 00000000 00000001 00000000 00031435 00000001 84b47180 ffffffc0 [ 1109.430524] c0c0 00000001 00000001 00000031 00000078 00000031 00000032 08be15c8 ffffff80 [ 1109.438741] c0e0 00000400 00000000 00400000 00000000 00000001 00000000 00000000 00000000 [ 1109.446959] [ 1109.446959] X25: 0xffffffc0b166c648: [ 1109.452004] c648 b166c648 ffffffc0 b166c648 ffffffc0 b10c8910 ffffffc0 b10c8910 ffffffc0 [ 1109.460220] c668 b0e67ea0 ffffffc0 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.468436] c688 00000073 00000000 00000000 00000000 00000073 00000000 00000000 00000000 [ 1109.476653] c6a8 00000000 00000000 00000000 00000000 00000000 00000000 00009814 00000000 [ 1109.484869] c6c8 00000004 00000000 09ec2856 00000001 09ec2856 00000001 00000000 00000000 [ 1109.493087] c6e8 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.501305] c708 b166c708 ffffffc0 b166c708 ffffffc0 b166c718 ffffffc0 b166c718 ffffffc0 [ 1109.509521] c728 b166c728 ffffffc0 b166c728 ffffffc0 00000000 00000000 b0e0b880 ffffffc0 [ 1109.517740] [ 1109.517740] X26: 0xffffffc0bff13380: [ 1109.522785] 3380 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.531001] 33a0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.539217] 33c0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.547434] 33e0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.555651] 3400 fab6faaf 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.563868] 3420 00000000 00000000 00000000 00000000 00031548 00000001 00000000 00000000 [ 1109.572085] 3440 00000000 00000000 00000001 00000000 00000000 00000000 00000000 00000000 [ 1109.580301] 3460 00017d53 00000000 0002db81 00000000 00000000 00000000 00000000 00000000 [ 1109.588521] [ 1109.588521] X29: 0xffffffc0b0e67b10: [ 1109.593567] 7b10 b166c080 ffffffc0 09095000 ffffff80 00000000 00000000 b166c6c8 ffffffc0 [ 1109.601784] 7b30 bff13400 ffffffc0 08bc4fc4 ffffff80 00000001 00000000 b0e67b90 ffffffc0 [ 1109.610001] 7b50 0810c82c ffffff80 b0e67b90 ffffffc0 08112428 ffffff80 60000185 00000000 [ 1109.618218] 7b70 b0e67ba0 ffffffc0 0810c538 ffffff80 ffffffff ffffffff 0810c574 ffffff80 [ 1109.626436] 7b90 b0e67bc0 ffffffc0 0810c82c ffffff80 bff13400 ffffffc0 0810c820 ffffff80 [ 1109.634653] 7bb0 bff13400 ffffffc0 b166c080 ffffffc0 b0e67bf0 ffffffc0 080eead8 ffffff80 [ 1109.642869] 7bd0 bff13400 ffffffc0 0906b000 ffffff80 bff13400 ffffffc0 0906b000 ffffff80 [ 1109.651086] 7bf0 b0e67c20 ffffffc0 08bc4b80 ffffff80 bff13400 ffffffc0 08bc47e4 ffffff80 [ 1109.659303] [ 1109.660788] Process kschedfreq:0 (pid: 1248, stack limit = 0xffffffc0b0e64020) [ 1109.668006] Stack: (0xffffffc0b0e67b90 to 0xffffffc0b0e68000) [ 1109.673748] 7b80: ffffffc0b0e67bc0 ffffff800810c82c [ 1109.681574] 7ba0: ffffffc0bff13400 ffffff800810c820 ffffffc0bff13400 ffffffc0b166c080 [ 1109.689399] 7bc0: ffffffc0b0e67bf0 ffffff80080eead8 ffffffc0bff13400 ffffff800906b000 [ 1109.697225] 7be0: ffffffc0bff13400 ffffff800906b000 ffffffc0b0e67c20 ffffff8008bc4b80 [ 1109.705052] 7c00: ffffffc0bff13400 ffffff8008bc47e4 ffffffc000000001 ffffffc0b166c080 [ 1109.712877] 7c20: ffffffc0b0e67c80 ffffff8008bc4fc4 ffffffc0b0e64000 0000000000000001 [ 1109.720703] 7c40: 00000000000186a0 ffffffc0b0e64000 ffffff8008be0000 00000000001a13c8 [ 1109.728529] 7c60: ffffffc0b1092800 ffffff80091a8000 0000000000000000 0000000000009f4c [ 1109.736355] 7c80: ffffffc0b0e67ca0 ffffff8008bc82a8 ffffffc0b0e67d98 0000000000000100 [ 1109.744181] 7ca0: ffffffc0b0e67d40 ffffff8008bc8348 ffffffc0b0e67d98 0000000000000064 [ 1109.752006] 7cc0: ffffffc0b11bb580 ffffffc0b0e67d30 ffffff800808e7e4 00000001b166c080 [ 1109.759832] 7ce0: ffffffc0bff0f2d1 ffffffc0ae6cfe30 ffffffc0bff0f170 0000010209828528 [ 1109.767658] 7d00: 000001020980fe88 ffffff800813c598 ffffffc0bff0ed40 000004e000000001 [ 1109.775483] 7d20: ffffff8008bc829c 726664656863736b 00000000303a7165 ffffffc0b166c080 [ 1109.783309] 7d40: ffffffc0b0e67d70 ffffff8008bc8068 00000000026e40e0 00000000001a13c8 [ 1109.791135] 7d60: 00000000000186a0 0000000108142a54 ffffffc0b0e67da0 ffffff800811a770 [ 1109.798961] 7d80: 000001020980f324 ffffffc0b0e64000 ffffffc0b11bb580 00000000026e40e0 [ 1109.806787] 7da0: ffffffc0b0e67e20 ffffff80080e1a14 ffffffc0b11bb400 ffffffc0b0e64000 [ 1109.814613] 7dc0: ffffff80091c41c8 ffffffc0b1092800 ffffff800811a6b0 0000000000000000 [ 1109.822438] 7de0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.830263] 7e00: ffffffc0b11bb400 ffffffc0b0e64000 ffffff80091c41c8 ffffffc000000032 [ 1109.838089] 7e20: 0000000000000000 ffffff8008085ed0 ffffff80080e192c ffffffc0b11bb400 [ 1109.845914] 7e40: 0000000000000000 0000000000000000 0000000000000000 ffffff80080efe18 [ 1109.853740] 7e60: 0000000000000000 0000000000000000 0000000000000000 ffffffc0b1092800 [ 1109.861565] 7e80: ffffffc000000000 ffffff8000000000 ffffffc0b0e67e90 ffffffc0b0e67e90 [ 1109.869392] 7ea0: 0000000000000000 ffffff8000000000 ffffffc0b0e67eb0 ffffffc0b0e67eb0 [ 1109.877217] 7ec0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.885042] 7ee0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.892867] 7f00: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.900692] 7f20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.908518] 7f40: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.916344] 7f60: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.924169] 7f80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.931994] 7fa0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.939820] 7fc0: 0000000000000000 0000000000000005 0000000000000000 0000000000000000 [ 1109.947646] 7fe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.955470] Call trace: [ 1109.957911] Exception stack(0xffffffc0b0e679c0 to 0xffffffc0b0e67af0) [ 1109.964348] 79c0: ffffffc0bff13400 0000008000000000 ffffffc0b0e67b90 ffffff8008112428 [ 1109.972173] 79e0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.979999] 7a00: 0000000000000000 0000000000000000 0000000000000000 0000000000000009 [ 1109.987824] 7a20: 0000000000000010 0000000000000010 0000000000000000 000000000000068a [ 1109.995650] 7a40: ffffffc0b0e67a90 ffffff8008bc8bac 0000000000000180 ffffff800928c688 [ 1110.003476] 7a60: 00000000000bc75e fffffffffffedce2 0000000000000000 0000000000000002 [ 1110.011301] 7a80: 00000040b6ea8000 0000000000000001 ffffff8009095000 ffffffc0ae6cfe30 [ 1110.019127] 7aa0: ffffffc0b0e67ce0 ffffffc0b0e64000 00000000000009d0 ffffff8008be13cc [ 1110.026952] 7ac0: 0000000034d5d91d 0000000000000000 0000000000000000 0000000000000000 [ 1110.034777] 7ae0: 0000000000000000 0000000000000000 [ 1110.039651] [<ffffff8008112428>] walt_dec_cumulative_runnable_avg+0x40/0x44 [ 1110.046609] [<ffffff800810c82c>] dequeue_task_rt+0x40/0x8c [ 1110.052093] [<ffffff80080eead8>] deactivate_task+0x98/0xbc [ 1110.057580] [<ffffff8008bc4b80>] __schedule+0x44c/0x7c0 [ 1110.062800] [<ffffff8008bc4fc4>] schedule+0x40/0xa0 [ 1110.067674] [<ffffff8008bc82a8>] schedule_hrtimeout_range_clock+0x94/0x100 [ 1110.074544] [<ffffff8008bc8348>] schedule_hrtimeout_range+0x34/0x40 [ 1110.080806] [<ffffff8008bc8068>] usleep_range+0x4c/0x58 [ 1110.086028] [<ffffff800811a770>] cpufreq_sched_thread+0xc0/0x1e4 [ 1110.092032] [<ffffff80080e1a14>] kthread+0xe8/0xfc [ 1110.096821] [<ffffff8008085ed0>] ret_from_fork+0x10/0x40 [ 1110.102129] Code: b7f80081 f9400bf3 a8c37bfd d65f03c0 (d4210000)
Hi Leo,
The first thing I would like you to check is whether your hardware has sane clock source. I encountered the problem mostly when hw clock is buggy so sched_clock() returns value less than previously returned. Can you check if that's the case?
Having said, there were couple of legitimate bugs we fixed as well. I will check if that's the case too.
Thanks, Joonwoo
On 08/23/2017 01:00 AM, Leo Yan wrote:
Hi Vikram, Joonwoo,
[ + EAS mailing list ]
On Hikey960 with EASv1.3, I encountered many times for WALT panic, it reports the bug from below two functions; you also could see log in the below. Before I dig into this, could you give some suggestion for this? Or if there have some existed fixing for this? Thanks in advance.
void walt_dec_cumulative_runnable_avg(struct rq *rq, struct task_struct *p) { rq->cumulative_runnable_avg -= p->ravg.demand; BUG_ON((s64)rq->cumulative_runnable_avg < 0); }
static void fixup_cumulative_runnable_avg(struct rq *rq, struct task_struct *p, u64 new_task_load) { s64 task_load_delta = (s64)new_task_load - task_load(p);
rq->cumulative_runnable_avg += task_load_delta; if ((s64)rq->cumulative_runnable_avg < 0) panic("cra less than zero: tld: %lld, task_load(p) = %u\n", task_load_delta, task_load(p));}
--- Panic Log ---
[ 1108.441865] init: Untracked pid 15425 exited with status 0 [ 1108.657107] ------------[ cut here ]------------ [ 1108.661746] kernel BUG at kernel/sched/walt.c:109! [ 1108.666538] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP [ 1108.672026] CPU: 1 PID: 1248 Comm: kschedfreq:0 Not tainted 4.4.78-07635-g0255026 #45 [ 1108.679851] Hardware name: HiKey960 (DT) [ 1108.683770] task: ffffffc0b166c080 ti: ffffffc0b0e64000 task.ti: ffffffc0b0e64000 [ 1108.691261] PC is at walt_dec_cumulative_runnable_avg+0x40/0x44 [ 1108.697179] LR is at dequeue_task_rt+0x40/0x8c [ 1108.701617] pc : [<ffffff8008112428>] lr : [<ffffff800810c82c>] pstate: 60000185 [ 1108.709007] sp : ffffffc0b0e67b90 [ 1108.712315] x29: ffffffc0b0e67b90 x28: 0000000000000001 [ 1108.717633] x27: ffffff8008bc4fc4 x26: ffffffc0bff13400 [ 1108.722948] x25: ffffffc0b166c6c8 x24: 0000000000000000 [ 1108.728263] x23: ffffff8009095000 x22: ffffffc0b166c080 [ 1108.733579] x21: ffffffc0bff13be8 x20: ffffffc0b166c080 [ 1108.738895] x19: ffffffc0bff13400 x18: 0000000000000000 [ 1108.744209] x17: 0000000000000000 x16: 0000000000000000 [ 1108.749524] x15: 0000000000000000 x14: 0000000000000000 [ 1108.754839] x13: 0000000000000000 x12: 0000000034d5d91d [ 1108.760156] x11: ffffff8008be13cc x10: 00000000000009d0 [ 1108.765471] x9 : ffffffc0b0e64000 x8 : ffffffc0b0e67ce0 [ 1108.770786] x7 : ffffffc0ae6cfe30 x6 : ffffff8009095000 [ 1108.776101] x5 : 0000000000000001 x4 : 00000040b6ea8000 [ 1108.781415] x3 : 0000000000000002 x2 : 0000000000000000 [ 1108.786730] x1 : fffffffffffedce2 x0 : 00000000000bc75e [ 1108.792047] [ 1108.792047] SP: 0xffffffc0b0e67b10: [ 1108.797006] 7b10 b166c080 ffffffc0 09095000 ffffff80 00000000 00000000 b166c6c8 ffffffc0 [ 1108.805223] 7b30 bff13400 ffffffc0 08bc4fc4 ffffff80 00000001 00000000 b0e67b90 ffffffc0 [ 1108.813440] 7b50 0810c82c ffffff80 b0e67b90 ffffffc0 08112428 ffffff80 60000185 00000000 [ 1108.821656] 7b70 b0e67ba0 ffffffc0 0810c538 ffffff80 ffffffff ffffffff 0810c574 ffffff80 [ 1108.829872] 7b90 b0e67bc0 ffffffc0 0810c82c ffffff80 bff13400 ffffffc0 0810c820 ffffff80 [ 1108.838090] 7bb0 bff13400 ffffffc0 b166c080 ffffffc0 b0e67bf0 ffffffc0 080eead8 ffffff80 [ 1108.846306] 7bd0 bff13400 ffffffc0 0906b000 ffffff80 bff13400 ffffffc0 0906b000 ffffff80 [ 1108.854522] 7bf0 b0e67c20 ffffffc0 08bc4b80 ffffff80 bff13400 ffffffc0 08bc47e4 ffffff80 [ 1108.862741] [ 1108.862741] X1: 0xfffffffffffedc62: [ 1108.867700] dc60 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.875924] dc80 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.884140] dca0 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.892358] dcc0 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.900576] dce0 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.908793] dd00 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.917009] dd20 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.925227] dd40 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.933446] dd60 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.941666] [ 1108.941666] X7: 0xffffffc0ae6cfdb0: [ 1108.946625] fdb0 0000c350 00000000 00000001 00000000 00000000 00000000 ae6cfeb0 ffffffc0 [ 1108.954840] fdd0 00000000 00000000 00000001 00000000 00000000 00000000 0000c350 00000001 [ 1108.963055] fdf0 ae6cfe90 ffffffc0 0813dd98 ffffff80 b55c0418 0000007f 00000000 00000000 [ 1108.971271] fe10 ffffffff ffffffff b76b299c 0000007f ae6cfe60 ffffffc0 080efae0 00000001 [ 1108.979489] fe30 b0e67ce0 ffffffc0 00000000 00000000 00000000 00000000 0c0b8de9 00000102 [ 1108.987707] fe50 0c0aca99 00000102 0813c598 ffffff80 bff0ed40 ffffffc0 00000001 00000825 [ 1108.995923] fe70 08bc80e8 ffffff80 696c616d 6d656d2d 7275702d 00006567 b11f3100 ffffffc0 [ 1109.004140] fe90 00000000 00000000 08085f30 ffffff80 00000000 00000000 b6ee1020 0000007f [ 1109.012359] [ 1109.012359] X8: 0xffffffc0b0e67c60: [ 1109.017318] 7c60 b1092800 ffffffc0 091a8000 ffffff80 00000000 00000000 00009f4c 00000000 [ 1109.025536] 7c80 b0e67ca0 ffffffc0 08bc82a8 ffffff80 b0e67d98 ffffffc0 00000100 00000000 [ 1109.033752] 7ca0 b0e67d40 ffffffc0 08bc8348 ffffff80 b0e67d98 ffffffc0 00000064 00000000 [ 1109.041970] 7cc0 b11bb580 ffffffc0 b0e67d30 ffffffc0 0808e7e4 ffffff80 b166c080 00000001 [ 1109.050186] 7ce0 bff0f2d1 ffffffc0 ae6cfe30 ffffffc0 bff0f170 ffffffc0 09828528 00000102 [ 1109.058405] 7d00 0980fe88 00000102 0813c598 ffffff80 bff0ed40 ffffffc0 00000001 000004e0 [ 1109.066621] 7d20 08bc829c ffffff80 6863736b 72666465 303a7165 00000000 b166c080 ffffffc0 [ 1109.074839] 7d40 b0e67d70 ffffffc0 08bc8068 ffffff80 026e40e0 00000000 001a13c8 00000000 [ 1109.083058] [ 1109.083058] X9: 0xffffffc0b0e63f80: [ 1109.088015] 3f80 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.096232] 3fa0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.104448] 3fc0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.112665] 3fe0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.120883] 4000 00000000 00000000 ffffffff ffffffff b166c080 ffffffc0 00000003 00000001 [ 1109.129100] 4020 57ac6e9d 00000000 32273028 0d0f1c33 10233816 201d111a 3b013e3c 532f2f26 [ 1109.137317] 4040 ae7a3648 ffffffc0 ae354db8 ffffffc0 ae7a36c0 ffffffc0 ae7a36c0 ffffffc0 [ 1109.145535] 4060 00000001 00000000 aca585e0 ffffffc0 aca584e0 ffffffc0 07fb7c71 00000000 [ 1109.153753] [ 1109.153753] X19: 0xffffffc0bff13380: [ 1109.158799] 3380 00000000[ 1109.160774] mali e82c0000.mali: Reset interrupt didn't reach CPU. Check interrupt assignments.
[ 1109.169934] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.177103] 33a0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.185319] 33c0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.193537] 33e0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.201755] 3400 fab6faaf 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.209971] 3420 00000000 00000000 00000000 00000000 00031548 00000001 00000000 00000000 [ 1109.218187] 3440 00000000 00000000 00000001 00000000 00000000 00000000 00000000 00000000 [ 1109.226402] 3460 00017d53 00000000 0002db81 00000000 00000000 00000000 00000000 00000000 [ 1109.234619] [ 1109.234619] X20: 0xffffffc0b166c000: [ 1109.239664] c000 0b030d00 08010211 100f0d05 07091214 040e0a0c ffffff00 00000001 00000000 [ 1109.247881] c020 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.256097] c040 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.264313] c060 00000000 00000010 00000000 00000000 ffffffff 0000003f ffffffff 0000003f [ 1109.272528] c080 00000002 00000000 b0e64000 ffffffc0 00000003 04208040 00000000 00000000 [ 1109.280745] c0a0 00000000 00000000 00000001 00000000 00031435 00000001 84b47180 ffffffc0 [ 1109.288960] c0c0 00000001 00000001 00000031 00000078 00000031 00000032 08be15c8 ffffff80 [ 1109.297178] c0e0 00000400 00000000 00400000 00000000 00000001 00000000 00000000 00000000 [ 1109.305395] [ 1109.305395] X21: 0xffffffc0bff13b68: [ 1109.310440] 3b68 bff13b60 ffffffc0 bff13b70 ffffffc0 bff13b70 ffffffc0 bff13b80 ffffffc0 [ 1109.318658] 3b88 bff13b80 ffffffc0 bff13b90 ffffffc0 bff13b90 ffffffc0 bff13ba0 ffffffc0 [ 1109.326875] 3ba8 bff13ba0 ffffffc0 bff13bb0 ffffffc0 bff13bb0 ffffffc0 00000000 00000064 [ 1109.335092] 3bc8 00000064 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.343309] 3be8 bff13be8 ffffffc0 bff13be8 ffffffc0 00000000 00000000 00000000 00000000 [ 1109.351526] 3c08 00000000 00000000 0810cb88 ffffff80 00020002 00000000 00000000 00000000 [ 1109.359742] 3c28 006303e4 00000000 389fd980 00000000 f80df80d 00000000 00000000 00000000 [ 1109.367959] 3c48 bff13400 ffffffc0 091c42a0 ffffff80 00000000 00000000 00000000 00000000 [ 1109.376176] [ 1109.376176] X22: 0xffffffc0b166c000: [ 1109.381221] c000 0b030d00 08010211 100f0d05 07091214 040e0a0c ffffff00 00000001 00000000 [ 1109.389439] c020 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.397656] c040 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.405873] c060 00000000 00000010 00000000 00000000 ffffffff 0000003f ffffffff 0000003f [ 1109.414090] c080 00000002 00000000 b0e64000 ffffffc0 00000003 04208040 00000000 00000000 [ 1109.422307] c0a0 00000000 00000000 00000001 00000000 00031435 00000001 84b47180 ffffffc0 [ 1109.430524] c0c0 00000001 00000001 00000031 00000078 00000031 00000032 08be15c8 ffffff80 [ 1109.438741] c0e0 00000400 00000000 00400000 00000000 00000001 00000000 00000000 00000000 [ 1109.446959] [ 1109.446959] X25: 0xffffffc0b166c648: [ 1109.452004] c648 b166c648 ffffffc0 b166c648 ffffffc0 b10c8910 ffffffc0 b10c8910 ffffffc0 [ 1109.460220] c668 b0e67ea0 ffffffc0 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.468436] c688 00000073 00000000 00000000 00000000 00000073 00000000 00000000 00000000 [ 1109.476653] c6a8 00000000 00000000 00000000 00000000 00000000 00000000 00009814 00000000 [ 1109.484869] c6c8 00000004 00000000 09ec2856 00000001 09ec2856 00000001 00000000 00000000 [ 1109.493087] c6e8 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.501305] c708 b166c708 ffffffc0 b166c708 ffffffc0 b166c718 ffffffc0 b166c718 ffffffc0 [ 1109.509521] c728 b166c728 ffffffc0 b166c728 ffffffc0 00000000 00000000 b0e0b880 ffffffc0 [ 1109.517740] [ 1109.517740] X26: 0xffffffc0bff13380: [ 1109.522785] 3380 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.531001] 33a0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.539217] 33c0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.547434] 33e0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.555651] 3400 fab6faaf 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.563868] 3420 00000000 00000000 00000000 00000000 00031548 00000001 00000000 00000000 [ 1109.572085] 3440 00000000 00000000 00000001 00000000 00000000 00000000 00000000 00000000 [ 1109.580301] 3460 00017d53 00000000 0002db81 00000000 00000000 00000000 00000000 00000000 [ 1109.588521] [ 1109.588521] X29: 0xffffffc0b0e67b10: [ 1109.593567] 7b10 b166c080 ffffffc0 09095000 ffffff80 00000000 00000000 b166c6c8 ffffffc0 [ 1109.601784] 7b30 bff13400 ffffffc0 08bc4fc4 ffffff80 00000001 00000000 b0e67b90 ffffffc0 [ 1109.610001] 7b50 0810c82c ffffff80 b0e67b90 ffffffc0 08112428 ffffff80 60000185 00000000 [ 1109.618218] 7b70 b0e67ba0 ffffffc0 0810c538 ffffff80 ffffffff ffffffff 0810c574 ffffff80 [ 1109.626436] 7b90 b0e67bc0 ffffffc0 0810c82c ffffff80 bff13400 ffffffc0 0810c820 ffffff80 [ 1109.634653] 7bb0 bff13400 ffffffc0 b166c080 ffffffc0 b0e67bf0 ffffffc0 080eead8 ffffff80 [ 1109.642869] 7bd0 bff13400 ffffffc0 0906b000 ffffff80 bff13400 ffffffc0 0906b000 ffffff80 [ 1109.651086] 7bf0 b0e67c20 ffffffc0 08bc4b80 ffffff80 bff13400 ffffffc0 08bc47e4 ffffff80 [ 1109.659303] [ 1109.660788] Process kschedfreq:0 (pid: 1248, stack limit = 0xffffffc0b0e64020) [ 1109.668006] Stack: (0xffffffc0b0e67b90 to 0xffffffc0b0e68000) [ 1109.673748] 7b80: ffffffc0b0e67bc0 ffffff800810c82c [ 1109.681574] 7ba0: ffffffc0bff13400 ffffff800810c820 ffffffc0bff13400 ffffffc0b166c080 [ 1109.689399] 7bc0: ffffffc0b0e67bf0 ffffff80080eead8 ffffffc0bff13400 ffffff800906b000 [ 1109.697225] 7be0: ffffffc0bff13400 ffffff800906b000 ffffffc0b0e67c20 ffffff8008bc4b80 [ 1109.705052] 7c00: ffffffc0bff13400 ffffff8008bc47e4 ffffffc000000001 ffffffc0b166c080 [ 1109.712877] 7c20: ffffffc0b0e67c80 ffffff8008bc4fc4 ffffffc0b0e64000 0000000000000001 [ 1109.720703] 7c40: 00000000000186a0 ffffffc0b0e64000 ffffff8008be0000 00000000001a13c8 [ 1109.728529] 7c60: ffffffc0b1092800 ffffff80091a8000 0000000000000000 0000000000009f4c [ 1109.736355] 7c80: ffffffc0b0e67ca0 ffffff8008bc82a8 ffffffc0b0e67d98 0000000000000100 [ 1109.744181] 7ca0: ffffffc0b0e67d40 ffffff8008bc8348 ffffffc0b0e67d98 0000000000000064 [ 1109.752006] 7cc0: ffffffc0b11bb580 ffffffc0b0e67d30 ffffff800808e7e4 00000001b166c080 [ 1109.759832] 7ce0: ffffffc0bff0f2d1 ffffffc0ae6cfe30 ffffffc0bff0f170 0000010209828528 [ 1109.767658] 7d00: 000001020980fe88 ffffff800813c598 ffffffc0bff0ed40 000004e000000001 [ 1109.775483] 7d20: ffffff8008bc829c 726664656863736b 00000000303a7165 ffffffc0b166c080 [ 1109.783309] 7d40: ffffffc0b0e67d70 ffffff8008bc8068 00000000026e40e0 00000000001a13c8 [ 1109.791135] 7d60: 00000000000186a0 0000000108142a54 ffffffc0b0e67da0 ffffff800811a770 [ 1109.798961] 7d80: 000001020980f324 ffffffc0b0e64000 ffffffc0b11bb580 00000000026e40e0 [ 1109.806787] 7da0: ffffffc0b0e67e20 ffffff80080e1a14 ffffffc0b11bb400 ffffffc0b0e64000 [ 1109.814613] 7dc0: ffffff80091c41c8 ffffffc0b1092800 ffffff800811a6b0 0000000000000000 [ 1109.822438] 7de0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.830263] 7e00: ffffffc0b11bb400 ffffffc0b0e64000 ffffff80091c41c8 ffffffc000000032 [ 1109.838089] 7e20: 0000000000000000 ffffff8008085ed0 ffffff80080e192c ffffffc0b11bb400 [ 1109.845914] 7e40: 0000000000000000 0000000000000000 0000000000000000 ffffff80080efe18 [ 1109.853740] 7e60: 0000000000000000 0000000000000000 0000000000000000 ffffffc0b1092800 [ 1109.861565] 7e80: ffffffc000000000 ffffff8000000000 ffffffc0b0e67e90 ffffffc0b0e67e90 [ 1109.869392] 7ea0: 0000000000000000 ffffff8000000000 ffffffc0b0e67eb0 ffffffc0b0e67eb0 [ 1109.877217] 7ec0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.885042] 7ee0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.892867] 7f00: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.900692] 7f20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.908518] 7f40: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.916344] 7f60: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.924169] 7f80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.931994] 7fa0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.939820] 7fc0: 0000000000000000 0000000000000005 0000000000000000 0000000000000000 [ 1109.947646] 7fe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.955470] Call trace: [ 1109.957911] Exception stack(0xffffffc0b0e679c0 to 0xffffffc0b0e67af0) [ 1109.964348] 79c0: ffffffc0bff13400 0000008000000000 ffffffc0b0e67b90 ffffff8008112428 [ 1109.972173] 79e0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.979999] 7a00: 0000000000000000 0000000000000000 0000000000000000 0000000000000009 [ 1109.987824] 7a20: 0000000000000010 0000000000000010 0000000000000000 000000000000068a [ 1109.995650] 7a40: ffffffc0b0e67a90 ffffff8008bc8bac 0000000000000180 ffffff800928c688 [ 1110.003476] 7a60: 00000000000bc75e fffffffffffedce2 0000000000000000 0000000000000002 [ 1110.011301] 7a80: 00000040b6ea8000 0000000000000001 ffffff8009095000 ffffffc0ae6cfe30 [ 1110.019127] 7aa0: ffffffc0b0e67ce0 ffffffc0b0e64000 00000000000009d0 ffffff8008be13cc [ 1110.026952] 7ac0: 0000000034d5d91d 0000000000000000 0000000000000000 0000000000000000 [ 1110.034777] 7ae0: 0000000000000000 0000000000000000 [ 1110.039651] [<ffffff8008112428>] walt_dec_cumulative_runnable_avg+0x40/0x44 [ 1110.046609] [<ffffff800810c82c>] dequeue_task_rt+0x40/0x8c [ 1110.052093] [<ffffff80080eead8>] deactivate_task+0x98/0xbc [ 1110.057580] [<ffffff8008bc4b80>] __schedule+0x44c/0x7c0 [ 1110.062800] [<ffffff8008bc4fc4>] schedule+0x40/0xa0 [ 1110.067674] [<ffffff8008bc82a8>] schedule_hrtimeout_range_clock+0x94/0x100 [ 1110.074544] [<ffffff8008bc8348>] schedule_hrtimeout_range+0x34/0x40 [ 1110.080806] [<ffffff8008bc8068>] usleep_range+0x4c/0x58 [ 1110.086028] [<ffffff800811a770>] cpufreq_sched_thread+0xc0/0x1e4 [ 1110.092032] [<ffffff80080e1a14>] kthread+0xe8/0xfc [ 1110.096821] [<ffffff8008085ed0>] ret_from_fork+0x10/0x40 [ 1110.102129] Code: b7f80081 f9400bf3 a8c37bfd d65f03c0 (d4210000)
eas-dev mailing list eas-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/eas-dev
Hi Joonwoo,
On Wed, Aug 23, 2017 at 03:10:26PM -0700, Joonwoo Park wrote:
Hi Leo,
The first thing I would like you to check is whether your hardware has sane clock source. I encountered the problem mostly when hw clock is buggy so sched_clock() returns value less than previously returned. Can you check if that's the case?
Sure, thanks for the suggestion.
I have noticed CA73 has ERRATUM 858921, and I ported the fix in my local code, but this fix doesn't work. I will continue to check if sched_clock() has rollback issue.
diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h index fbe0ca3..92388f6 100644 --- a/arch/arm64/include/asm/arch_timer.h +++ b/arch/arm64/include/asm/arch_timer.h @@ -116,12 +116,11 @@ static inline u64 arch_counter_get_cntpct(void)
static inline u64 arch_counter_get_cntvct(void) { - u64 cval; + u64 old, new;
- isb(); - asm volatile("mrs %0, cntvct_el0" : "=r" (cval)); - - return cval; + old = read_sysreg(cntvct_el0); + new = read_sysreg(cntvct_el0); + return (((old ^ new) >> 32) & 1) ? old : new; }
static inline int arch_timer_arch_init(void)
Having said, there were couple of legitimate bugs we fixed as well. I will check if that's the case too.
Thanks a lot.
Thanks, Joonwoo
On 08/23/2017 01:00 AM, Leo Yan wrote:
Hi Vikram, Joonwoo,
[ + EAS mailing list ]
On Hikey960 with EASv1.3, I encountered many times for WALT panic, it reports the bug from below two functions; you also could see log in the below. Before I dig into this, could you give some suggestion for this? Or if there have some existed fixing for this? Thanks in advance.
void walt_dec_cumulative_runnable_avg(struct rq *rq, struct task_struct *p) { rq->cumulative_runnable_avg -= p->ravg.demand; BUG_ON((s64)rq->cumulative_runnable_avg < 0); }
static void fixup_cumulative_runnable_avg(struct rq *rq, struct task_struct *p, u64 new_task_load) { s64 task_load_delta = (s64)new_task_load - task_load(p);
rq->cumulative_runnable_avg += task_load_delta; if ((s64)rq->cumulative_runnable_avg < 0) panic("cra less than zero: tld: %lld, task_load(p) = %u\n", task_load_delta, task_load(p));}
--- Panic Log ---
[ 1108.441865] init: Untracked pid 15425 exited with status 0 [ 1108.657107] ------------[ cut here ]------------ [ 1108.661746] kernel BUG at kernel/sched/walt.c:109! [ 1108.666538] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP [ 1108.672026] CPU: 1 PID: 1248 Comm: kschedfreq:0 Not tainted 4.4.78-07635-g0255026 #45 [ 1108.679851] Hardware name: HiKey960 (DT) [ 1108.683770] task: ffffffc0b166c080 ti: ffffffc0b0e64000 task.ti: ffffffc0b0e64000 [ 1108.691261] PC is at walt_dec_cumulative_runnable_avg+0x40/0x44 [ 1108.697179] LR is at dequeue_task_rt+0x40/0x8c [ 1108.701617] pc : [<ffffff8008112428>] lr : [<ffffff800810c82c>] pstate: 60000185 [ 1108.709007] sp : ffffffc0b0e67b90 [ 1108.712315] x29: ffffffc0b0e67b90 x28: 0000000000000001 [ 1108.717633] x27: ffffff8008bc4fc4 x26: ffffffc0bff13400 [ 1108.722948] x25: ffffffc0b166c6c8 x24: 0000000000000000 [ 1108.728263] x23: ffffff8009095000 x22: ffffffc0b166c080 [ 1108.733579] x21: ffffffc0bff13be8 x20: ffffffc0b166c080 [ 1108.738895] x19: ffffffc0bff13400 x18: 0000000000000000 [ 1108.744209] x17: 0000000000000000 x16: 0000000000000000 [ 1108.749524] x15: 0000000000000000 x14: 0000000000000000 [ 1108.754839] x13: 0000000000000000 x12: 0000000034d5d91d [ 1108.760156] x11: ffffff8008be13cc x10: 00000000000009d0 [ 1108.765471] x9 : ffffffc0b0e64000 x8 : ffffffc0b0e67ce0 [ 1108.770786] x7 : ffffffc0ae6cfe30 x6 : ffffff8009095000 [ 1108.776101] x5 : 0000000000000001 x4 : 00000040b6ea8000 [ 1108.781415] x3 : 0000000000000002 x2 : 0000000000000000 [ 1108.786730] x1 : fffffffffffedce2 x0 : 00000000000bc75e [ 1108.792047] [ 1108.792047] SP: 0xffffffc0b0e67b10: [ 1108.797006] 7b10 b166c080 ffffffc0 09095000 ffffff80 00000000 00000000 b166c6c8 ffffffc0 [ 1108.805223] 7b30 bff13400 ffffffc0 08bc4fc4 ffffff80 00000001 00000000 b0e67b90 ffffffc0 [ 1108.813440] 7b50 0810c82c ffffff80 b0e67b90 ffffffc0 08112428 ffffff80 60000185 00000000 [ 1108.821656] 7b70 b0e67ba0 ffffffc0 0810c538 ffffff80 ffffffff ffffffff 0810c574 ffffff80 [ 1108.829872] 7b90 b0e67bc0 ffffffc0 0810c82c ffffff80 bff13400 ffffffc0 0810c820 ffffff80 [ 1108.838090] 7bb0 bff13400 ffffffc0 b166c080 ffffffc0 b0e67bf0 ffffffc0 080eead8 ffffff80 [ 1108.846306] 7bd0 bff13400 ffffffc0 0906b000 ffffff80 bff13400 ffffffc0 0906b000 ffffff80 [ 1108.854522] 7bf0 b0e67c20 ffffffc0 08bc4b80 ffffff80 bff13400 ffffffc0 08bc47e4 ffffff80 [ 1108.862741] [ 1108.862741] X1: 0xfffffffffffedc62: [ 1108.867700] dc60 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.875924] dc80 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.884140] dca0 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.892358] dcc0 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.900576] dce0 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.908793] dd00 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.917009] dd20 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.925227] dd40 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.933446] dd60 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.941666] [ 1108.941666] X7: 0xffffffc0ae6cfdb0: [ 1108.946625] fdb0 0000c350 00000000 00000001 00000000 00000000 00000000 ae6cfeb0 ffffffc0 [ 1108.954840] fdd0 00000000 00000000 00000001 00000000 00000000 00000000 0000c350 00000001 [ 1108.963055] fdf0 ae6cfe90 ffffffc0 0813dd98 ffffff80 b55c0418 0000007f 00000000 00000000 [ 1108.971271] fe10 ffffffff ffffffff b76b299c 0000007f ae6cfe60 ffffffc0 080efae0 00000001 [ 1108.979489] fe30 b0e67ce0 ffffffc0 00000000 00000000 00000000 00000000 0c0b8de9 00000102 [ 1108.987707] fe50 0c0aca99 00000102 0813c598 ffffff80 bff0ed40 ffffffc0 00000001 00000825 [ 1108.995923] fe70 08bc80e8 ffffff80 696c616d 6d656d2d 7275702d 00006567 b11f3100 ffffffc0 [ 1109.004140] fe90 00000000 00000000 08085f30 ffffff80 00000000 00000000 b6ee1020 0000007f [ 1109.012359] [ 1109.012359] X8: 0xffffffc0b0e67c60: [ 1109.017318] 7c60 b1092800 ffffffc0 091a8000 ffffff80 00000000 00000000 00009f4c 00000000 [ 1109.025536] 7c80 b0e67ca0 ffffffc0 08bc82a8 ffffff80 b0e67d98 ffffffc0 00000100 00000000 [ 1109.033752] 7ca0 b0e67d40 ffffffc0 08bc8348 ffffff80 b0e67d98 ffffffc0 00000064 00000000 [ 1109.041970] 7cc0 b11bb580 ffffffc0 b0e67d30 ffffffc0 0808e7e4 ffffff80 b166c080 00000001 [ 1109.050186] 7ce0 bff0f2d1 ffffffc0 ae6cfe30 ffffffc0 bff0f170 ffffffc0 09828528 00000102 [ 1109.058405] 7d00 0980fe88 00000102 0813c598 ffffff80 bff0ed40 ffffffc0 00000001 000004e0 [ 1109.066621] 7d20 08bc829c ffffff80 6863736b 72666465 303a7165 00000000 b166c080 ffffffc0 [ 1109.074839] 7d40 b0e67d70 ffffffc0 08bc8068 ffffff80 026e40e0 00000000 001a13c8 00000000 [ 1109.083058] [ 1109.083058] X9: 0xffffffc0b0e63f80: [ 1109.088015] 3f80 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.096232] 3fa0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.104448] 3fc0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.112665] 3fe0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.120883] 4000 00000000 00000000 ffffffff ffffffff b166c080 ffffffc0 00000003 00000001 [ 1109.129100] 4020 57ac6e9d 00000000 32273028 0d0f1c33 10233816 201d111a 3b013e3c 532f2f26 [ 1109.137317] 4040 ae7a3648 ffffffc0 ae354db8 ffffffc0 ae7a36c0 ffffffc0 ae7a36c0 ffffffc0 [ 1109.145535] 4060 00000001 00000000 aca585e0 ffffffc0 aca584e0 ffffffc0 07fb7c71 00000000 [ 1109.153753] [ 1109.153753] X19: 0xffffffc0bff13380: [ 1109.158799] 3380 00000000[ 1109.160774] mali e82c0000.mali: Reset interrupt didn't reach CPU. Check interrupt assignments.
[ 1109.169934] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.177103] 33a0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.185319] 33c0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.193537] 33e0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.201755] 3400 fab6faaf 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.209971] 3420 00000000 00000000 00000000 00000000 00031548 00000001 00000000 00000000 [ 1109.218187] 3440 00000000 00000000 00000001 00000000 00000000 00000000 00000000 00000000 [ 1109.226402] 3460 00017d53 00000000 0002db81 00000000 00000000 00000000 00000000 00000000 [ 1109.234619] [ 1109.234619] X20: 0xffffffc0b166c000: [ 1109.239664] c000 0b030d00 08010211 100f0d05 07091214 040e0a0c ffffff00 00000001 00000000 [ 1109.247881] c020 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.256097] c040 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.264313] c060 00000000 00000010 00000000 00000000 ffffffff 0000003f ffffffff 0000003f [ 1109.272528] c080 00000002 00000000 b0e64000 ffffffc0 00000003 04208040 00000000 00000000 [ 1109.280745] c0a0 00000000 00000000 00000001 00000000 00031435 00000001 84b47180 ffffffc0 [ 1109.288960] c0c0 00000001 00000001 00000031 00000078 00000031 00000032 08be15c8 ffffff80 [ 1109.297178] c0e0 00000400 00000000 00400000 00000000 00000001 00000000 00000000 00000000 [ 1109.305395] [ 1109.305395] X21: 0xffffffc0bff13b68: [ 1109.310440] 3b68 bff13b60 ffffffc0 bff13b70 ffffffc0 bff13b70 ffffffc0 bff13b80 ffffffc0 [ 1109.318658] 3b88 bff13b80 ffffffc0 bff13b90 ffffffc0 bff13b90 ffffffc0 bff13ba0 ffffffc0 [ 1109.326875] 3ba8 bff13ba0 ffffffc0 bff13bb0 ffffffc0 bff13bb0 ffffffc0 00000000 00000064 [ 1109.335092] 3bc8 00000064 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.343309] 3be8 bff13be8 ffffffc0 bff13be8 ffffffc0 00000000 00000000 00000000 00000000 [ 1109.351526] 3c08 00000000 00000000 0810cb88 ffffff80 00020002 00000000 00000000 00000000 [ 1109.359742] 3c28 006303e4 00000000 389fd980 00000000 f80df80d 00000000 00000000 00000000 [ 1109.367959] 3c48 bff13400 ffffffc0 091c42a0 ffffff80 00000000 00000000 00000000 00000000 [ 1109.376176] [ 1109.376176] X22: 0xffffffc0b166c000: [ 1109.381221] c000 0b030d00 08010211 100f0d05 07091214 040e0a0c ffffff00 00000001 00000000 [ 1109.389439] c020 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.397656] c040 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.405873] c060 00000000 00000010 00000000 00000000 ffffffff 0000003f ffffffff 0000003f [ 1109.414090] c080 00000002 00000000 b0e64000 ffffffc0 00000003 04208040 00000000 00000000 [ 1109.422307] c0a0 00000000 00000000 00000001 00000000 00031435 00000001 84b47180 ffffffc0 [ 1109.430524] c0c0 00000001 00000001 00000031 00000078 00000031 00000032 08be15c8 ffffff80 [ 1109.438741] c0e0 00000400 00000000 00400000 00000000 00000001 00000000 00000000 00000000 [ 1109.446959] [ 1109.446959] X25: 0xffffffc0b166c648: [ 1109.452004] c648 b166c648 ffffffc0 b166c648 ffffffc0 b10c8910 ffffffc0 b10c8910 ffffffc0 [ 1109.460220] c668 b0e67ea0 ffffffc0 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.468436] c688 00000073 00000000 00000000 00000000 00000073 00000000 00000000 00000000 [ 1109.476653] c6a8 00000000 00000000 00000000 00000000 00000000 00000000 00009814 00000000 [ 1109.484869] c6c8 00000004 00000000 09ec2856 00000001 09ec2856 00000001 00000000 00000000 [ 1109.493087] c6e8 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.501305] c708 b166c708 ffffffc0 b166c708 ffffffc0 b166c718 ffffffc0 b166c718 ffffffc0 [ 1109.509521] c728 b166c728 ffffffc0 b166c728 ffffffc0 00000000 00000000 b0e0b880 ffffffc0 [ 1109.517740] [ 1109.517740] X26: 0xffffffc0bff13380: [ 1109.522785] 3380 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.531001] 33a0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.539217] 33c0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.547434] 33e0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.555651] 3400 fab6faaf 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.563868] 3420 00000000 00000000 00000000 00000000 00031548 00000001 00000000 00000000 [ 1109.572085] 3440 00000000 00000000 00000001 00000000 00000000 00000000 00000000 00000000 [ 1109.580301] 3460 00017d53 00000000 0002db81 00000000 00000000 00000000 00000000 00000000 [ 1109.588521] [ 1109.588521] X29: 0xffffffc0b0e67b10: [ 1109.593567] 7b10 b166c080 ffffffc0 09095000 ffffff80 00000000 00000000 b166c6c8 ffffffc0 [ 1109.601784] 7b30 bff13400 ffffffc0 08bc4fc4 ffffff80 00000001 00000000 b0e67b90 ffffffc0 [ 1109.610001] 7b50 0810c82c ffffff80 b0e67b90 ffffffc0 08112428 ffffff80 60000185 00000000 [ 1109.618218] 7b70 b0e67ba0 ffffffc0 0810c538 ffffff80 ffffffff ffffffff 0810c574 ffffff80 [ 1109.626436] 7b90 b0e67bc0 ffffffc0 0810c82c ffffff80 bff13400 ffffffc0 0810c820 ffffff80 [ 1109.634653] 7bb0 bff13400 ffffffc0 b166c080 ffffffc0 b0e67bf0 ffffffc0 080eead8 ffffff80 [ 1109.642869] 7bd0 bff13400 ffffffc0 0906b000 ffffff80 bff13400 ffffffc0 0906b000 ffffff80 [ 1109.651086] 7bf0 b0e67c20 ffffffc0 08bc4b80 ffffff80 bff13400 ffffffc0 08bc47e4 ffffff80 [ 1109.659303] [ 1109.660788] Process kschedfreq:0 (pid: 1248, stack limit = 0xffffffc0b0e64020) [ 1109.668006] Stack: (0xffffffc0b0e67b90 to 0xffffffc0b0e68000) [ 1109.673748] 7b80: ffffffc0b0e67bc0 ffffff800810c82c [ 1109.681574] 7ba0: ffffffc0bff13400 ffffff800810c820 ffffffc0bff13400 ffffffc0b166c080 [ 1109.689399] 7bc0: ffffffc0b0e67bf0 ffffff80080eead8 ffffffc0bff13400 ffffff800906b000 [ 1109.697225] 7be0: ffffffc0bff13400 ffffff800906b000 ffffffc0b0e67c20 ffffff8008bc4b80 [ 1109.705052] 7c00: ffffffc0bff13400 ffffff8008bc47e4 ffffffc000000001 ffffffc0b166c080 [ 1109.712877] 7c20: ffffffc0b0e67c80 ffffff8008bc4fc4 ffffffc0b0e64000 0000000000000001 [ 1109.720703] 7c40: 00000000000186a0 ffffffc0b0e64000 ffffff8008be0000 00000000001a13c8 [ 1109.728529] 7c60: ffffffc0b1092800 ffffff80091a8000 0000000000000000 0000000000009f4c [ 1109.736355] 7c80: ffffffc0b0e67ca0 ffffff8008bc82a8 ffffffc0b0e67d98 0000000000000100 [ 1109.744181] 7ca0: ffffffc0b0e67d40 ffffff8008bc8348 ffffffc0b0e67d98 0000000000000064 [ 1109.752006] 7cc0: ffffffc0b11bb580 ffffffc0b0e67d30 ffffff800808e7e4 00000001b166c080 [ 1109.759832] 7ce0: ffffffc0bff0f2d1 ffffffc0ae6cfe30 ffffffc0bff0f170 0000010209828528 [ 1109.767658] 7d00: 000001020980fe88 ffffff800813c598 ffffffc0bff0ed40 000004e000000001 [ 1109.775483] 7d20: ffffff8008bc829c 726664656863736b 00000000303a7165 ffffffc0b166c080 [ 1109.783309] 7d40: ffffffc0b0e67d70 ffffff8008bc8068 00000000026e40e0 00000000001a13c8 [ 1109.791135] 7d60: 00000000000186a0 0000000108142a54 ffffffc0b0e67da0 ffffff800811a770 [ 1109.798961] 7d80: 000001020980f324 ffffffc0b0e64000 ffffffc0b11bb580 00000000026e40e0 [ 1109.806787] 7da0: ffffffc0b0e67e20 ffffff80080e1a14 ffffffc0b11bb400 ffffffc0b0e64000 [ 1109.814613] 7dc0: ffffff80091c41c8 ffffffc0b1092800 ffffff800811a6b0 0000000000000000 [ 1109.822438] 7de0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.830263] 7e00: ffffffc0b11bb400 ffffffc0b0e64000 ffffff80091c41c8 ffffffc000000032 [ 1109.838089] 7e20: 0000000000000000 ffffff8008085ed0 ffffff80080e192c ffffffc0b11bb400 [ 1109.845914] 7e40: 0000000000000000 0000000000000000 0000000000000000 ffffff80080efe18 [ 1109.853740] 7e60: 0000000000000000 0000000000000000 0000000000000000 ffffffc0b1092800 [ 1109.861565] 7e80: ffffffc000000000 ffffff8000000000 ffffffc0b0e67e90 ffffffc0b0e67e90 [ 1109.869392] 7ea0: 0000000000000000 ffffff8000000000 ffffffc0b0e67eb0 ffffffc0b0e67eb0 [ 1109.877217] 7ec0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.885042] 7ee0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.892867] 7f00: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.900692] 7f20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.908518] 7f40: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.916344] 7f60: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.924169] 7f80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.931994] 7fa0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.939820] 7fc0: 0000000000000000 0000000000000005 0000000000000000 0000000000000000 [ 1109.947646] 7fe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.955470] Call trace: [ 1109.957911] Exception stack(0xffffffc0b0e679c0 to 0xffffffc0b0e67af0) [ 1109.964348] 79c0: ffffffc0bff13400 0000008000000000 ffffffc0b0e67b90 ffffff8008112428 [ 1109.972173] 79e0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.979999] 7a00: 0000000000000000 0000000000000000 0000000000000000 0000000000000009 [ 1109.987824] 7a20: 0000000000000010 0000000000000010 0000000000000000 000000000000068a [ 1109.995650] 7a40: ffffffc0b0e67a90 ffffff8008bc8bac 0000000000000180 ffffff800928c688 [ 1110.003476] 7a60: 00000000000bc75e fffffffffffedce2 0000000000000000 0000000000000002 [ 1110.011301] 7a80: 00000040b6ea8000 0000000000000001 ffffff8009095000 ffffffc0ae6cfe30 [ 1110.019127] 7aa0: ffffffc0b0e67ce0 ffffffc0b0e64000 00000000000009d0 ffffff8008be13cc [ 1110.026952] 7ac0: 0000000034d5d91d 0000000000000000 0000000000000000 0000000000000000 [ 1110.034777] 7ae0: 0000000000000000 0000000000000000 [ 1110.039651] [<ffffff8008112428>] walt_dec_cumulative_runnable_avg+0x40/0x44 [ 1110.046609] [<ffffff800810c82c>] dequeue_task_rt+0x40/0x8c [ 1110.052093] [<ffffff80080eead8>] deactivate_task+0x98/0xbc [ 1110.057580] [<ffffff8008bc4b80>] __schedule+0x44c/0x7c0 [ 1110.062800] [<ffffff8008bc4fc4>] schedule+0x40/0xa0 [ 1110.067674] [<ffffff8008bc82a8>] schedule_hrtimeout_range_clock+0x94/0x100 [ 1110.074544] [<ffffff8008bc8348>] schedule_hrtimeout_range+0x34/0x40 [ 1110.080806] [<ffffff8008bc8068>] usleep_range+0x4c/0x58 [ 1110.086028] [<ffffff800811a770>] cpufreq_sched_thread+0xc0/0x1e4 [ 1110.092032] [<ffffff80080e1a14>] kthread+0xe8/0xfc [ 1110.096821] [<ffffff8008085ed0>] ret_from_fork+0x10/0x40 [ 1110.102129] Code: b7f80081 f9400bf3 a8c37bfd d65f03c0 (d4210000)
eas-dev mailing list eas-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/eas-dev
Hi Leo,
On 08/23/2017 10:00 AM, Leo Yan wrote:
Hi Vikram, Joonwoo,
[ + EAS mailing list ]
On Hikey960 with EASv1.3, I encountered many times for WALT panic, it reports the bug from below two functions; you also could see log in the below. Before I dig into this, could you give some suggestion for this? Or if there have some existed fixing for this? Thanks in advance.
void walt_dec_cumulative_runnable_avg(struct rq *rq, struct task_struct *p) { rq->cumulative_runnable_avg -= p->ravg.demand; BUG_ON((s64)rq->cumulative_runnable_avg < 0); }
static void fixup_cumulative_runnable_avg(struct rq *rq, struct task_struct *p, u64 new_task_load) { s64 task_load_delta = (s64)new_task_load - task_load(p);
rq->cumulative_runnable_avg += task_load_delta; if ((s64)rq->cumulative_runnable_avg < 0) panic("cra less than zero: tld: %lld, task_load(p) = %u\n", task_load_delta, task_load(p));}
I'm running EAS v1.3 (4.4.80-02913-gc40a9a77daf7) with the default hikey960_defconfig on my hikey960 as well.
Haven't seen this WALT panic yet.
Let's compare the boards, maybe you use a different setup:
(1) kernel:
https://android.googlesource.com/kernel/hikey-linaro c40a9a77daf7
top commit is:
commit c40a9a77daf770cd8ca75572235b015ef6ff6589 Author: Brendan Jackman brendan.jackman@arm.com Date: Fri Aug 4 10:58:20 2017 +0100
hikey960:/ # zcat /proc/config.gz | grep _SCHED_
CONFIG_SCHED_WALT=y CONFIG_GENERIC_SCHED_CLOCK=y CONFIG_SCHED_AUTOGROUP=y CONFIG_SCHED_TUNE=y CONFIG_SCHED_MC=y # CONFIG_SCHED_SMT is not set CONFIG_SCHED_HRTICK=y CONFIG_SCHED_DEBUG=y CONFIG_SCHED_INFO=y # CONFIG_SCHED_STACK_END_CHECK is not set CONFIG_SCHED_TRACER=y
hikey960:/ # cat /sys/kernel/debug/sched_features GENTLE_FAIR_SLEEPERS START_DEBIT NO_NEXT_BUDDY LAST_BUDDY CACHE_HOT_BUDDY WAKEUP_PREEMPTION NO_HRTICK NO_DOUBLE_TICK LB_BIAS NONTASK_CAPACITY TTWU_QUEUE RT_PUSH_IPI NO_FORCE_SD_OVERLAP RT_RUNTIME_SHARE NO_LB_MIN ATTACH_AGE_LOAD ENERGY_AWARE
(2) Firmware:
https://github.com/96boards-hikey/tools-images-hikey960 master
top commit is:
ccb401f72634 - recovery-flash: add '-e' flag for bash (2017-07-18 Leo Yan)
[...]
-- Dietmar
On 08/24/2017 10:55 AM, Dietmar Eggemann wrote:
Hi Leo,
On 08/23/2017 10:00 AM, Leo Yan wrote:
Hi Leo, Can you please try this patch and let me know if it helps? There seems indeed missing change in google's tree I'm on since it's reverted. The tree Dietmar gave never had this change. You will still need ERRATUM 858921 to avoid panic(curr_run_sum < 0).
Hi Dietmar, do you have https://android-review.googlesource.com/#/c/kernel/common/+/426442/? If not, the bug that 426442 aims to fix can mask the problem so that it will be really hard to hit the cr_avg < 0.
Thanks, Joonwoo
From 2730faecb26de0969df5fc39c4014bc5000a5ca8 Mon Sep 17 00:00:00 2001 From: Olav Haugan ohaugan@codeaurora.org Date: Wed, 5 Aug 2015 08:45:21 -0700 Subject: [PATCH] sched: Update task->on_rq when tasks are moving between runqueues
Task->on_rq has three states: 0 - Task is not on runqueue (rq) 1 (TASK_ON_RQ_QUEUED) - Task is on rq 2 (TASK_ON_RQ_MIGRATING) - Task is on rq but in the process of being migrated to another rq
When a task is moving between rqs task->on_rq state should be TASK_ON_RQ_MIGRATING in order for WALT to account rq's cumulative runnable average correctly. Without such state marking for all the classes, WALT's update_history() would try to fixup task's demand which was never contributed to any of CPUs during migration.
Signed-off-by: Olav Haugan ohaugan@codeaurora.org [joonwoop: Reinforced changelog to explain why this is needed by WALT. Fixed conflicts in deadline.c] Signed-off-by: Joonwoo Park joonwoop@codeaurora.org --- kernel/sched/core.c | 2 ++ kernel/sched/deadline.c | 4 ++++ kernel/sched/rt.c | 4 ++++ 3 files changed, 10 insertions(+)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 312f894..843e440 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1332,7 +1332,9 @@ static void __migrate_swap_task(struct task_struct *p, int cpu) dst_rq = cpu_rq(cpu);
deactivate_task(src_rq, p, 0); + p->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(p, cpu); + p->on_rq = TASK_ON_RQ_QUEUED; activate_task(dst_rq, p, 0); check_preempt_curr(dst_rq, p, 0); } else { diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 2aae8b8..ab1a9a9 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1587,7 +1587,9 @@ retry:
deactivate_task(rq, next_task, 0); clear_average_bw(&next_task->dl, &rq->dl); + next_task->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(next_task, later_rq->cpu); + next_task->on_rq = TASK_ON_RQ_QUEUED; add_average_bw(&next_task->dl, &later_rq->dl); activate_task(later_rq, next_task, 0); ret = 1; @@ -1677,7 +1679,9 @@ static void pull_dl_task(struct rq *this_rq)
deactivate_task(src_rq, p, 0); clear_average_bw(&p->dl, &src_rq->dl); + p->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(p, this_cpu); + p->on_rq = TASK_ON_RQ_QUEUED; add_average_bw(&p->dl, &this_rq->dl); activate_task(this_rq, p, 0); dmin = p->dl.deadline; diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 25499a3..963d6d6 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -2000,7 +2000,9 @@ retry: }
deactivate_task(rq, next_task, 0); + next_task->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(next_task, lowest_rq->cpu); + next_task->on_rq = TASK_ON_RQ_QUEUED; activate_task(lowest_rq, next_task, 0); ret = 1;
@@ -2254,7 +2256,9 @@ static void pull_rt_task(struct rq *this_rq) resched = true;
deactivate_task(src_rq, p, 0); + p->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(p, this_cpu); + p->on_rq = TASK_ON_RQ_QUEUED; activate_task(this_rq, p, 0); /* * We continue with the search, just in -- The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation
On Thu, Aug 24, 2017 at 03:06:40PM -0700, Joonwoo Park wrote:
On 08/24/2017 10:55 AM, Dietmar Eggemann wrote:
Hi Leo,
On 08/23/2017 10:00 AM, Leo Yan wrote:
Hi Leo, Can you please try this patch and let me know if it helps? There seems indeed missing change in google's tree I'm on since it's reverted. The tree Dietmar gave never had this change. You will still need ERRATUM 858921 to avoid panic(curr_run_sum < 0).
Hi Dietmar, do you have https://android-review.googlesource.com/#/c/kernel/common/+/426442/? If not, the bug that 426442 aims to fix can mask the problem so that it will be really hard to hit the cr_avg < 0.
Yes, I have applied 426442 patch.
I did two testing at yersterday but no lucky to root cause the issue:
- I captured sched_clock() value for every CPU but has not found issue; - I tried to chnage 'panic()' and 'BUG_ON' to trace logs in walt.c and run 30 times for testing and it doesn't hit the failure.
Today will verify below patch and will let you know the result. Thanks for the suggestion.
Thanks, Joonwoo
From 2730faecb26de0969df5fc39c4014bc5000a5ca8 Mon Sep 17 00:00:00 2001 From: Olav Haugan ohaugan@codeaurora.org Date: Wed, 5 Aug 2015 08:45:21 -0700 Subject: [PATCH] sched: Update task->on_rq when tasks are moving between runqueues
Task->on_rq has three states: 0 - Task is not on runqueue (rq) 1 (TASK_ON_RQ_QUEUED) - Task is on rq 2 (TASK_ON_RQ_MIGRATING) - Task is on rq but in the process of being migrated to another rq
When a task is moving between rqs task->on_rq state should be TASK_ON_RQ_MIGRATING in order for WALT to account rq's cumulative runnable average correctly. Without such state marking for all the classes, WALT's update_history() would try to fixup task's demand which was never contributed to any of CPUs during migration.
Signed-off-by: Olav Haugan ohaugan@codeaurora.org [joonwoop: Reinforced changelog to explain why this is needed by WALT. Fixed conflicts in deadline.c] Signed-off-by: Joonwoo Park joonwoop@codeaurora.org
kernel/sched/core.c | 2 ++ kernel/sched/deadline.c | 4 ++++ kernel/sched/rt.c | 4 ++++ 3 files changed, 10 insertions(+)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 312f894..843e440 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1332,7 +1332,9 @@ static void __migrate_swap_task(struct task_struct *p, int cpu) dst_rq = cpu_rq(cpu);
deactivate_task(src_rq, p, 0);
p->on_rq = TASK_ON_RQ_MIGRATING;p->on_rq = TASK_ON_RQ_QUEUED;diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 2aae8b8..ab1a9a9 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1587,7 +1587,9 @@ retry:
deactivate_task(rq, next_task, 0); clear_average_bw(&next_task->dl, &rq->dl);
- next_task->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(next_task, later_rq->cpu);
- next_task->on_rq = TASK_ON_RQ_QUEUED; add_average_bw(&next_task->dl, &later_rq->dl); activate_task(later_rq, next_task, 0); ret = 1;
@@ -1677,7 +1679,9 @@ static void pull_dl_task(struct rq *this_rq)
deactivate_task(src_rq, p, 0); clear_average_bw(&p->dl, &src_rq->dl);
p->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(p, this_cpu);p->on_rq = TASK_ON_RQ_QUEUED; add_average_bw(&p->dl, &this_rq->dl); activate_task(this_rq, p, 0); dmin = p->dl.deadline;diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 25499a3..963d6d6 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -2000,7 +2000,9 @@ retry: }
deactivate_task(rq, next_task, 0);
- next_task->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(next_task, lowest_rq->cpu);
- next_task->on_rq = TASK_ON_RQ_QUEUED; activate_task(lowest_rq, next_task, 0); ret = 1;
@@ -2254,7 +2256,9 @@ static void pull_rt_task(struct rq *this_rq) resched = true;
deactivate_task(src_rq, p, 0);
p->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(p, this_cpu);p->on_rq = TASK_ON_RQ_QUEUED; activate_task(this_rq, p, 0); /* * We continue with the search, just in-- The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation
Hi Joonwoo,
On 08/25/2017 12:06 AM, Joonwoo Park wrote:
On 08/24/2017 10:55 AM, Dietmar Eggemann wrote:
Hi Leo,
On 08/23/2017 10:00 AM, Leo Yan wrote:
Hi Leo, Can you please try this patch and let me know if it helps? There seems indeed missing change in google's tree I'm on since it's reverted. The tree Dietmar gave never had this change. You will still need ERRATUM 858921 to avoid panic(curr_run_sum < 0).
Hi Dietmar, do you have https://android-review.googlesource.com/#/c/kernel/common/+/426442/? If not, the bug that 426442 aims to fix can mask the problem so that it will be really hard to hit the cr_avg < 0.
Ah OK. No, my tree doesn't have change 426442 'sched: WALT: fix broken cumulative runnable average accounting'.
e107985-lin:/opt/git/msm_build/kernel:__hikey-linaro/android-hikey-linaro-4.4$ glg2 kernel/sched/walt.c d368c6faa19b - sched: walt: fix window misalignment when HZ=300 (2017-07-12 Joonwoo Park) e487a24793bb - sched/walt: kill {min,max}_capacity (2016-12-08 Juri Lelli) cf43809d7aa0 - sched/walt: Drop arch-specific timer access (2016-09-20 Chris Redpath) 1c81f45725d2 - sched/walt: include missing header for arm_timer_read_counter() (2016-09-09 Amit Pundir) 23ed57dbcc14 - arch_timer: add error handling when the MPM global timer is cleared (2016-08-11 Chris Redpath) 519c62750eb6 - sched/walt: Accounting for number of irqs pending on each core (2016-08-11 Srinath Sridharan) efb86bd08a2e - sched: Introduce Window Assisted Load Tracking (WALT) (2016-08-11 Srivatsa Vaddagiri)
So you're saying that I will need change 426442 and then the patch below so that this BUG_ON((s64)rq->cumulative_runnable_avg < 0) doesn't fire?
Thanks,
-- Dietmar
Thanks, Joonwoo
From 2730faecb26de0969df5fc39c4014bc5000a5ca8 Mon Sep 17 00:00:00 2001 From: Olav Haugan ohaugan@codeaurora.org Date: Wed, 5 Aug 2015 08:45:21 -0700 Subject: [PATCH] sched: Update task->on_rq when tasks are moving between runqueues
[...]
On 08/24/2017 04:33 PM, Dietmar Eggemann wrote:
Hi Joonwoo,
On 08/25/2017 12:06 AM, Joonwoo Park wrote:
On 08/24/2017 10:55 AM, Dietmar Eggemann wrote:
Hi Leo,
On 08/23/2017 10:00 AM, Leo Yan wrote:
Hi Leo, Can you please try this patch and let me know if it helps? There seems indeed missing change in google's tree I'm on since it's reverted. The tree Dietmar gave never had this change. You will still need ERRATUM 858921 to avoid panic(curr_run_sum < 0).
Hi Dietmar, do you have https://android-review.googlesource.com/#/c/kernel/common/+/426442/? If not, the bug that 426442 aims to fix can mask the problem so that it will be really hard to hit the cr_avg < 0.
Ah OK. No, my tree doesn't have change 426442 'sched: WALT: fix broken cumulative runnable average accounting'.
e107985-lin:/opt/git/msm_build/kernel:__hikey-linaro/android-hikey-linaro-4.4$ glg2 kernel/sched/walt.c d368c6faa19b - sched: walt: fix window misalignment when HZ=300 (2017-07-12 Joonwoo Park) e487a24793bb - sched/walt: kill {min,max}_capacity (2016-12-08 Juri Lelli) cf43809d7aa0 - sched/walt: Drop arch-specific timer access (2016-09-20 Chris Redpath) 1c81f45725d2 - sched/walt: include missing header for arm_timer_read_counter() (2016-09-09 Amit Pundir) 23ed57dbcc14 - arch_timer: add error handling when the MPM global timer is cleared (2016-08-11 Chris Redpath) 519c62750eb6 - sched/walt: Accounting for number of irqs pending on each core (2016-08-11 Srinath Sridharan) efb86bd08a2e - sched: Introduce Window Assisted Load Tracking (WALT) (2016-08-11 Srivatsa Vaddagiri)
So you're saying that I will need change 426442 and then the patch below so that this BUG_ON((s64)rq->cumulative_runnable_avg < 0) doesn't fire?
Hi Dietmar,
Yes. That's what I'm hoping :)
The log Leo provided showed that BUG_ON(cr_avg < 0) triggered while doing rt task migration. The earlier patch I provided fixes rt/stop/deadline classes migration fixup correctness. So I think the patch should fix Leo's case.
Thanks, Joonwoo
Hi Joonwoo,
Hi Leo, Can you please try this patch and let me know if it helps? There seems indeed missing change in google's tree I'm on since it's reverted. The tree Dietmar gave never had this change. You will still need ERRATUM 858921 to avoid panic(curr_run_sum < 0).
I have tested below patch for two days, after applied this patch so far I cannot the WALT error anymore. So I can confirm below patch can fix the WALT panic issue. So I think this patch should be merged into AOSP common kernel.
Thanks a lot for the help!
From 2730faecb26de0969df5fc39c4014bc5000a5ca8 Mon Sep 17 00:00:00 2001 From: Olav Haugan ohaugan@codeaurora.org Date: Wed, 5 Aug 2015 08:45:21 -0700 Subject: [PATCH] sched: Update task->on_rq when tasks are moving between runqueues
Task->on_rq has three states: 0 - Task is not on runqueue (rq) 1 (TASK_ON_RQ_QUEUED) - Task is on rq 2 (TASK_ON_RQ_MIGRATING) - Task is on rq but in the process of being migrated to another rq
When a task is moving between rqs task->on_rq state should be TASK_ON_RQ_MIGRATING in order for WALT to account rq's cumulative runnable average correctly. Without such state marking for all the classes, WALT's update_history() would try to fixup task's demand which was never contributed to any of CPUs during migration.
Signed-off-by: Olav Haugan ohaugan@codeaurora.org [joonwoop: Reinforced changelog to explain why this is needed by WALT. Fixed conflicts in deadline.c] Signed-off-by: Joonwoo Park joonwoop@codeaurora.org
kernel/sched/core.c | 2 ++ kernel/sched/deadline.c | 4 ++++ kernel/sched/rt.c | 4 ++++ 3 files changed, 10 insertions(+)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 312f894..843e440 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1332,7 +1332,9 @@ static void __migrate_swap_task(struct task_struct *p, int cpu) dst_rq = cpu_rq(cpu);
deactivate_task(src_rq, p, 0);
p->on_rq = TASK_ON_RQ_MIGRATING;p->on_rq = TASK_ON_RQ_QUEUED;diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 2aae8b8..ab1a9a9 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1587,7 +1587,9 @@ retry:
deactivate_task(rq, next_task, 0); clear_average_bw(&next_task->dl, &rq->dl);
- next_task->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(next_task, later_rq->cpu);
- next_task->on_rq = TASK_ON_RQ_QUEUED; add_average_bw(&next_task->dl, &later_rq->dl); activate_task(later_rq, next_task, 0); ret = 1;
@@ -1677,7 +1679,9 @@ static void pull_dl_task(struct rq *this_rq)
deactivate_task(src_rq, p, 0); clear_average_bw(&p->dl, &src_rq->dl);
p->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(p, this_cpu);p->on_rq = TASK_ON_RQ_QUEUED; add_average_bw(&p->dl, &this_rq->dl); activate_task(this_rq, p, 0); dmin = p->dl.deadline;diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 25499a3..963d6d6 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -2000,7 +2000,9 @@ retry: }
deactivate_task(rq, next_task, 0);
- next_task->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(next_task, lowest_rq->cpu);
- next_task->on_rq = TASK_ON_RQ_QUEUED; activate_task(lowest_rq, next_task, 0); ret = 1;
@@ -2254,7 +2256,9 @@ static void pull_rt_task(struct rq *this_rq) resched = true;
deactivate_task(src_rq, p, 0);
p->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(p, this_cpu);p->on_rq = TASK_ON_RQ_QUEUED; activate_task(this_rq, p, 0); /* * We continue with the search, just in-- The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation
On 08/28/2017 07:37 AM, Leo Yan wrote:
Hi Joonwoo,
Hi Leo, Can you please try this patch and let me know if it helps? There seems indeed missing change in google's tree I'm on since it's reverted. The tree Dietmar gave never had this change. You will still need ERRATUM 858921 to avoid panic(curr_run_sum < 0).
I have tested below patch for two days, after applied this patch so far I cannot the WALT error anymore. So I can confirm below patch can fix the WALT panic issue. So I think this patch should be merged into AOSP common kernel.
Thanks a lot for the help!
Great!
Thank you so much for reporting, testing and confirmation!
Joonwoo
From 2730faecb26de0969df5fc39c4014bc5000a5ca8 Mon Sep 17 00:00:00 2001 From: Olav Haugan ohaugan@codeaurora.org Date: Wed, 5 Aug 2015 08:45:21 -0700 Subject: [PATCH] sched: Update task->on_rq when tasks are moving between runqueues
Task->on_rq has three states: 0 - Task is not on runqueue (rq) 1 (TASK_ON_RQ_QUEUED) - Task is on rq 2 (TASK_ON_RQ_MIGRATING) - Task is on rq but in the process of being migrated to another rq
When a task is moving between rqs task->on_rq state should be TASK_ON_RQ_MIGRATING in order for WALT to account rq's cumulative runnable average correctly. Without such state marking for all the classes, WALT's update_history() would try to fixup task's demand which was never contributed to any of CPUs during migration.
Signed-off-by: Olav Haugan ohaugan@codeaurora.org [joonwoop: Reinforced changelog to explain why this is needed by WALT. Fixed conflicts in deadline.c] Signed-off-by: Joonwoo Park joonwoop@codeaurora.org
kernel/sched/core.c | 2 ++ kernel/sched/deadline.c | 4 ++++ kernel/sched/rt.c | 4 ++++ 3 files changed, 10 insertions(+)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 312f894..843e440 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1332,7 +1332,9 @@ static void __migrate_swap_task(struct task_struct *p, int cpu) dst_rq = cpu_rq(cpu);
deactivate_task(src_rq, p, 0);
p->on_rq = TASK_ON_RQ_MIGRATING;p->on_rq = TASK_ON_RQ_QUEUED;diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 2aae8b8..ab1a9a9 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1587,7 +1587,9 @@ retry:
deactivate_task(rq, next_task, 0); clear_average_bw(&next_task->dl, &rq->dl);
- next_task->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(next_task, later_rq->cpu);
- next_task->on_rq = TASK_ON_RQ_QUEUED; add_average_bw(&next_task->dl, &later_rq->dl); activate_task(later_rq, next_task, 0); ret = 1;
@@ -1677,7 +1679,9 @@ static void pull_dl_task(struct rq *this_rq)
deactivate_task(src_rq, p, 0); clear_average_bw(&p->dl, &src_rq->dl);
p->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(p, this_cpu);p->on_rq = TASK_ON_RQ_QUEUED; add_average_bw(&p->dl, &this_rq->dl); activate_task(this_rq, p, 0); dmin = p->dl.deadline;diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 25499a3..963d6d6 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -2000,7 +2000,9 @@ retry: }
deactivate_task(rq, next_task, 0);
- next_task->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(next_task, lowest_rq->cpu);
- next_task->on_rq = TASK_ON_RQ_QUEUED; activate_task(lowest_rq, next_task, 0); ret = 1;
@@ -2254,7 +2256,9 @@ static void pull_rt_task(struct rq *this_rq) resched = true;
deactivate_task(src_rq, p, 0);
p->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(p, this_cpu);p->on_rq = TASK_ON_RQ_QUEUED; activate_task(this_rq, p, 0); /* * We continue with the search, just in-- The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation
eas-dev mailing list eas-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/eas-dev
Hi Joonwoo,
On Tue, Aug 29, 2017 at 11:22:55AM -0700, Joonwoo Park wrote:
On 08/28/2017 07:37 AM, Leo Yan wrote:
Hi Joonwoo,
Hi Leo, Can you please try this patch and let me know if it helps? There seems indeed missing change in google's tree I'm on since it's reverted. The tree Dietmar gave never had this change. You will still need ERRATUM 858921 to avoid panic(curr_run_sum < 0).
I have tested below patch for two days, after applied this patch so far I cannot the WALT error anymore. So I can confirm below patch can fix the WALT panic issue. So I think this patch should be merged into AOSP common kernel.
Thanks a lot for the help!
Great!
Thank you so much for reporting, testing and confirmation!
Loop in John Stultz, he also reports this same issue on Hikey960. So do you have plan to commit below patch onto Android common kernel?
Thanks, Leo Yan
Joonwoo
From 2730faecb26de0969df5fc39c4014bc5000a5ca8 Mon Sep 17 00:00:00 2001 From: Olav Haugan ohaugan@codeaurora.org Date: Wed, 5 Aug 2015 08:45:21 -0700 Subject: [PATCH] sched: Update task->on_rq when tasks are moving between runqueues
Task->on_rq has three states: 0 - Task is not on runqueue (rq) 1 (TASK_ON_RQ_QUEUED) - Task is on rq 2 (TASK_ON_RQ_MIGRATING) - Task is on rq but in the process of being migrated to another rq
When a task is moving between rqs task->on_rq state should be TASK_ON_RQ_MIGRATING in order for WALT to account rq's cumulative runnable average correctly. Without such state marking for all the classes, WALT's update_history() would try to fixup task's demand which was never contributed to any of CPUs during migration.
Signed-off-by: Olav Haugan ohaugan@codeaurora.org [joonwoop: Reinforced changelog to explain why this is needed by WALT. Fixed conflicts in deadline.c] Signed-off-by: Joonwoo Park joonwoop@codeaurora.org
kernel/sched/core.c | 2 ++ kernel/sched/deadline.c | 4 ++++ kernel/sched/rt.c | 4 ++++ 3 files changed, 10 insertions(+)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 312f894..843e440 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1332,7 +1332,9 @@ static void __migrate_swap_task(struct task_struct *p, int cpu) dst_rq = cpu_rq(cpu);
deactivate_task(src_rq, p, 0);
p->on_rq = TASK_ON_RQ_MIGRATING;p->on_rq = TASK_ON_RQ_QUEUED;diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 2aae8b8..ab1a9a9 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1587,7 +1587,9 @@ retry:
deactivate_task(rq, next_task, 0); clear_average_bw(&next_task->dl, &rq->dl);
- next_task->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(next_task, later_rq->cpu);
- next_task->on_rq = TASK_ON_RQ_QUEUED; add_average_bw(&next_task->dl, &later_rq->dl); activate_task(later_rq, next_task, 0); ret = 1;
@@ -1677,7 +1679,9 @@ static void pull_dl_task(struct rq *this_rq)
deactivate_task(src_rq, p, 0); clear_average_bw(&p->dl, &src_rq->dl);
p->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(p, this_cpu);p->on_rq = TASK_ON_RQ_QUEUED; add_average_bw(&p->dl, &this_rq->dl); activate_task(this_rq, p, 0); dmin = p->dl.deadline;diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 25499a3..963d6d6 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -2000,7 +2000,9 @@ retry: }
deactivate_task(rq, next_task, 0);
- next_task->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(next_task, lowest_rq->cpu);
- next_task->on_rq = TASK_ON_RQ_QUEUED; activate_task(lowest_rq, next_task, 0); ret = 1;
@@ -2254,7 +2256,9 @@ static void pull_rt_task(struct rq *this_rq) resched = true;
deactivate_task(src_rq, p, 0);
p->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(p, this_cpu);p->on_rq = TASK_ON_RQ_QUEUED; activate_task(this_rq, p, 0); /* * We continue with the search, just in-- The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation
eas-dev mailing list eas-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/eas-dev
Hi Leo, Sorry for delay.
On 2017-09-20 11:52, Leo Yan wrote:
Loop in John Stultz, he also reports this same issue on Hikey960. So do you have plan to commit below patch onto Android common kernel?
Yes I have already uploaded patch.
https://android-review.googlesource.com/#/c/kernel/common/+/469011/
Thanks, Joonwoo
Thanks, Leo Yan
Joonwoo
From 2730faecb26de0969df5fc39c4014bc5000a5ca8 Mon Sep 17 00:00:00 2001 From: Olav Haugan ohaugan@codeaurora.org Date: Wed, 5 Aug 2015 08:45:21 -0700 Subject: [PATCH] sched: Update task->on_rq when tasks are moving between runqueues
Task->on_rq has three states: 0 - Task is not on runqueue (rq) 1 (TASK_ON_RQ_QUEUED) - Task is on rq 2 (TASK_ON_RQ_MIGRATING) - Task is on rq but in the process of being migrated to another rq
When a task is moving between rqs task->on_rq state should be TASK_ON_RQ_MIGRATING in order for WALT to account rq's cumulative runnable average correctly. Without such state marking for all the classes, WALT's update_history() would try to fixup task's demand which was never contributed to any of CPUs during migration.
Signed-off-by: Olav Haugan ohaugan@codeaurora.org [joonwoop: Reinforced changelog to explain why this is needed by WALT. Fixed conflicts in deadline.c] Signed-off-by: Joonwoo Park joonwoop@codeaurora.org
kernel/sched/core.c | 2 ++ kernel/sched/deadline.c | 4 ++++ kernel/sched/rt.c | 4 ++++ 3 files changed, 10 insertions(+)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 312f894..843e440 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1332,7 +1332,9 @@ static void __migrate_swap_task(struct task_struct *p, int cpu) dst_rq = cpu_rq(cpu);
deactivate_task(src_rq, p, 0);
p->on_rq = TASK_ON_RQ_MIGRATING;p->on_rq = TASK_ON_RQ_QUEUED;diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 2aae8b8..ab1a9a9 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -1587,7 +1587,9 @@ retry:
deactivate_task(rq, next_task, 0); clear_average_bw(&next_task->dl, &rq->dl);
- next_task->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(next_task, later_rq->cpu);
- next_task->on_rq = TASK_ON_RQ_QUEUED; add_average_bw(&next_task->dl, &later_rq->dl); activate_task(later_rq, next_task, 0); ret = 1;
@@ -1677,7 +1679,9 @@ static void pull_dl_task(struct rq *this_rq)
deactivate_task(src_rq, p, 0); clear_average_bw(&p->dl, &src_rq->dl);
p->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(p, this_cpu);p->on_rq = TASK_ON_RQ_QUEUED; add_average_bw(&p->dl, &this_rq->dl); activate_task(this_rq, p, 0); dmin = p->dl.deadline;diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 25499a3..963d6d6 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -2000,7 +2000,9 @@ retry: }
deactivate_task(rq, next_task, 0);
- next_task->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(next_task, lowest_rq->cpu);
- next_task->on_rq = TASK_ON_RQ_QUEUED; activate_task(lowest_rq, next_task, 0); ret = 1;
@@ -2254,7 +2256,9 @@ static void pull_rt_task(struct rq *this_rq) resched = true;
deactivate_task(src_rq, p, 0);
p->on_rq = TASK_ON_RQ_MIGRATING; set_task_cpu(p, this_cpu);p->on_rq = TASK_ON_RQ_QUEUED; activate_task(this_rq, p, 0); /* * We continue with the search, just in-- The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, hosted by The Linux Foundation
eas-dev mailing list eas-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/eas-dev
eas-dev mailing list eas-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/eas-dev
On Wed, Sep 27, 2017 at 10:35:19AM -0700, joonwoop@codeaurora.org wrote:
Hi Leo, Sorry for delay.
On 2017-09-20 11:52, Leo Yan wrote:
Loop in John Stultz, he also reports this same issue on Hikey960. So do you have plan to commit below patch onto Android common kernel?
Yes I have already uploaded patch.
https://android-review.googlesource.com/#/c/kernel/common/+/469011/
Thanks a lot.
Thanks, Joonwoo
Hi Dietmar,
On Thu, Aug 24, 2017 at 07:55:44PM +0200, Dietmar Eggemann wrote:
[...]
I'm running EAS v1.3 (4.4.80-02913-gc40a9a77daf7) with the default hikey960_defconfig on my hikey960 as well.
Haven't seen this WALT panic yet.
Let's compare the boards, maybe you use a different setup:
(1) kernel:
https://android.googlesource.com/kernel/hikey-linaro c40a9a77daf7
top commit is:
commit c40a9a77daf770cd8ca75572235b015ef6ff6589 Author: Brendan Jackman brendan.jackman@arm.com Date: Fri Aug 4 10:58:20 2017 +0100
hikey960:/ # zcat /proc/config.gz | grep _SCHED_
CONFIG_SCHED_WALT=y CONFIG_GENERIC_SCHED_CLOCK=y CONFIG_SCHED_AUTOGROUP=y CONFIG_SCHED_TUNE=y CONFIG_SCHED_MC=y # CONFIG_SCHED_SMT is not set CONFIG_SCHED_HRTICK=y CONFIG_SCHED_DEBUG=y CONFIG_SCHED_INFO=y # CONFIG_SCHED_STACK_END_CHECK is not set CONFIG_SCHED_TRACER=y
hikey960:/ # cat /sys/kernel/debug/sched_features GENTLE_FAIR_SLEEPERS START_DEBIT NO_NEXT_BUDDY LAST_BUDDY CACHE_HOT_BUDDY WAKEUP_PREEMPTION NO_HRTICK NO_DOUBLE_TICK LB_BIAS NONTASK_CAPACITY TTWU_QUEUE RT_PUSH_IPI NO_FORCE_SD_OVERLAP RT_RUNTIME_SHARE NO_LB_MIN ATTACH_AGE_LOAD ENERGY_AWARE
For kernel, I am using EASv1.3 for mass testing, so the last commit is: f8873f63f9b79 ('Merge remote-tracking branch 'common/android-4.4' into hikey-4.4')
Based on I applied some WALT patches when I review them on gerrit:
0255026 Revert "sched: EAS: schedfreq: fix CPU util over estimation" '-> This patch is revered by me, it introduces perfermance regression.
8bb1dcb arm64: topology: change cpu_power to cpu_capacity to match existing use 9ce9c6b arm64: topology: Tell the scheduler about the relative power of cores 027a65b arm64: topology: Export arch_get_cpu_efficiency API 5e69299 sched: WALT: fix window mis-alignment f46bcb1 sched: EAS: Round up in cap divide for cpu util 566ee8f sched: EAS: fix incorrect energy delta calculation due to rounding error 7545b2d sched: EAS/WALT: take into account of waking task's load dea0a53 cpufreq: sched: WALT: don't apply capacity margin twice 9c3eb22 sched: WALT: fix potential overflow ce7d00d cpufreq/sched: Consider max cpu capacity when choosing frequencies 06125d6 cpufreq/sched: Use cpu max freq rather than policy max 97dc38d sched: EAS: schedfreq: fix CPU util over estimation 6a1c225 sched: EAS/WALT: use cr_avg instead of prev_runnable_sum b03aeb0 sched: EAS/WALT: fix deflated energy cost of previous CPU 6826628 sched: WALT: fix broken cumulative runnable average accounting 2b69325 sched: deadline: WALT: account cumulative runnable avg
(2) Firmware:
https://github.com/96boards-hikey/tools-images-hikey960 master
Just reminding, are you using Hisilicon propietary firmwares or UEFI+ARM-TF? I am using UEFI+ARM-TF.
top commit is:
ccb401f72634 - recovery-flash: add '-e' flag for bash (2017-07-18 Leo Yan)
Yeah, this is same with me.
Thanks, Leo Yan
Hi leo,
On Wed, Aug 23, 2017 at 4:00 PM, Leo Yan leo.yan@linaro.org wrote:
Hi Vikram, Joonwoo,
[ + EAS mailing list ]
On Hikey960 with EASv1.3, I encountered many times for WALT panic, it reports the bug from below two functions; you also could see log in the below. Before I dig into this, could you give some suggestion for this? Or if there have some existed fixing for this? Thanks in advance.
I've been running EAS r1.3 kernel for 3 weeks with the default hikey960 defconfig on several hikey960 board, haven't seen this panic yet.
kernel verison: 4a80fc6 ARM64: hikey960: Enable uart3 c5ad7df HACK: clk-hi3660: Fix clk_factor_uart3 to be 100000000, not 200000000 43c93f7 Merge remote-tracking branch 'common/android-4.4' into android-hikey-linaro-4.4-aosp 7eeebce6 ANDROID: binder: Don't BUG_ON(!spin_is_locked()). 78ba0ad Merge 4.4.79 into android-4.4 ...
Does this issue appear frequently?
-steven
void walt_dec_cumulative_runnable_avg(struct rq *rq, struct task_struct *p) { rq->cumulative_runnable_avg -= p->ravg.demand; BUG_ON((s64)rq->cumulative_runnable_avg < 0); }
static void fixup_cumulative_runnable_avg(struct rq *rq, struct task_struct *p, u64 new_task_load) { s64 task_load_delta = (s64)new_task_load - task_load(p);
rq->cumulative_runnable_avg += task_load_delta; if ((s64)rq->cumulative_runnable_avg < 0) panic("cra less than zero: tld: %lld, task_load(p) = %u\n", task_load_delta, task_load(p));}
--- Panic Log ---
[ 1108.441865] init: Untracked pid 15425 exited with status 0 [ 1108.657107] ------------[ cut here ]------------ [ 1108.661746] kernel BUG at kernel/sched/walt.c:109! [ 1108.666538] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP [ 1108.672026] CPU: 1 PID: 1248 Comm: kschedfreq:0 Not tainted 4.4.78-07635-g0255026 #45 [ 1108.679851] Hardware name: HiKey960 (DT) [ 1108.683770] task: ffffffc0b166c080 ti: ffffffc0b0e64000 task.ti: ffffffc0b0e64000 [ 1108.691261] PC is at walt_dec_cumulative_runnable_avg+0x40/0x44 [ 1108.697179] LR is at dequeue_task_rt+0x40/0x8c [ 1108.701617] pc : [<ffffff8008112428>] lr : [<ffffff800810c82c>] pstate: 60000185 [ 1108.709007] sp : ffffffc0b0e67b90 [ 1108.712315] x29: ffffffc0b0e67b90 x28: 0000000000000001 [ 1108.717633] x27: ffffff8008bc4fc4 x26: ffffffc0bff13400 [ 1108.722948] x25: ffffffc0b166c6c8 x24: 0000000000000000 [ 1108.728263] x23: ffffff8009095000 x22: ffffffc0b166c080 [ 1108.733579] x21: ffffffc0bff13be8 x20: ffffffc0b166c080 [ 1108.738895] x19: ffffffc0bff13400 x18: 0000000000000000 [ 1108.744209] x17: 0000000000000000 x16: 0000000000000000 [ 1108.749524] x15: 0000000000000000 x14: 0000000000000000 [ 1108.754839] x13: 0000000000000000 x12: 0000000034d5d91d [ 1108.760156] x11: ffffff8008be13cc x10: 00000000000009d0 [ 1108.765471] x9 : ffffffc0b0e64000 x8 : ffffffc0b0e67ce0 [ 1108.770786] x7 : ffffffc0ae6cfe30 x6 : ffffff8009095000 [ 1108.776101] x5 : 0000000000000001 x4 : 00000040b6ea8000 [ 1108.781415] x3 : 0000000000000002 x2 : 0000000000000000 [ 1108.786730] x1 : fffffffffffedce2 x0 : 00000000000bc75e [ 1108.792047] [ 1108.792047] SP: 0xffffffc0b0e67b10: [ 1108.797006] 7b10 b166c080 ffffffc0 09095000 ffffff80 00000000 00000000 b166c6c8 ffffffc0 [ 1108.805223] 7b30 bff13400 ffffffc0 08bc4fc4 ffffff80 00000001 00000000 b0e67b90 ffffffc0 [ 1108.813440] 7b50 0810c82c ffffff80 b0e67b90 ffffffc0 08112428 ffffff80 60000185 00000000 [ 1108.821656] 7b70 b0e67ba0 ffffffc0 0810c538 ffffff80 ffffffff ffffffff 0810c574 ffffff80 [ 1108.829872] 7b90 b0e67bc0 ffffffc0 0810c82c ffffff80 bff13400 ffffffc0 0810c820 ffffff80 [ 1108.838090] 7bb0 bff13400 ffffffc0 b166c080 ffffffc0 b0e67bf0 ffffffc0 080eead8 ffffff80 [ 1108.846306] 7bd0 bff13400 ffffffc0 0906b000 ffffff80 bff13400 ffffffc0 0906b000 ffffff80 [ 1108.854522] 7bf0 b0e67c20 ffffffc0 08bc4b80 ffffff80 bff13400 ffffffc0 08bc47e4 ffffff80 [ 1108.862741] [ 1108.862741] X1: 0xfffffffffffedc62: [ 1108.867700] dc60 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.875924] dc80 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.884140] dca0 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.892358] dcc0 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.900576] dce0 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.908793] dd00 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.917009] dd20 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.925227] dd40 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.933446] dd60 ******** ******** ******** ******** ******** ******** ******** ******** [ 1108.941666] [ 1108.941666] X7: 0xffffffc0ae6cfdb0: [ 1108.946625] fdb0 0000c350 00000000 00000001 00000000 00000000 00000000 ae6cfeb0 ffffffc0 [ 1108.954840] fdd0 00000000 00000000 00000001 00000000 00000000 00000000 0000c350 00000001 [ 1108.963055] fdf0 ae6cfe90 ffffffc0 0813dd98 ffffff80 b55c0418 0000007f 00000000 00000000 [ 1108.971271] fe10 ffffffff ffffffff b76b299c 0000007f ae6cfe60 ffffffc0 080efae0 00000001 [ 1108.979489] fe30 b0e67ce0 ffffffc0 00000000 00000000 00000000 00000000 0c0b8de9 00000102 [ 1108.987707] fe50 0c0aca99 00000102 0813c598 ffffff80 bff0ed40 ffffffc0 00000001 00000825 [ 1108.995923] fe70 08bc80e8 ffffff80 696c616d 6d656d2d 7275702d 00006567 b11f3100 ffffffc0 [ 1109.004140] fe90 00000000 00000000 08085f30 ffffff80 00000000 00000000 b6ee1020 0000007f [ 1109.012359] [ 1109.012359] X8: 0xffffffc0b0e67c60: [ 1109.017318] 7c60 b1092800 ffffffc0 091a8000 ffffff80 00000000 00000000 00009f4c 00000000 [ 1109.025536] 7c80 b0e67ca0 ffffffc0 08bc82a8 ffffff80 b0e67d98 ffffffc0 00000100 00000000 [ 1109.033752] 7ca0 b0e67d40 ffffffc0 08bc8348 ffffff80 b0e67d98 ffffffc0 00000064 00000000 [ 1109.041970] 7cc0 b11bb580 ffffffc0 b0e67d30 ffffffc0 0808e7e4 ffffff80 b166c080 00000001 [ 1109.050186] 7ce0 bff0f2d1 ffffffc0 ae6cfe30 ffffffc0 bff0f170 ffffffc0 09828528 00000102 [ 1109.058405] 7d00 0980fe88 00000102 0813c598 ffffff80 bff0ed40 ffffffc0 00000001 000004e0 [ 1109.066621] 7d20 08bc829c ffffff80 6863736b 72666465 303a7165 00000000 b166c080 ffffffc0 [ 1109.074839] 7d40 b0e67d70 ffffffc0 08bc8068 ffffff80 026e40e0 00000000 001a13c8 00000000 [ 1109.083058] [ 1109.083058] X9: 0xffffffc0b0e63f80: [ 1109.088015] 3f80 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.096232] 3fa0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.104448] 3fc0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.112665] 3fe0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.120883] 4000 00000000 00000000 ffffffff ffffffff b166c080 ffffffc0 00000003 00000001 [ 1109.129100] 4020 57ac6e9d 00000000 32273028 0d0f1c33 10233816 201d111a 3b013e3c 532f2f26 [ 1109.137317] 4040 ae7a3648 ffffffc0 ae354db8 ffffffc0 ae7a36c0 ffffffc0 ae7a36c0 ffffffc0 [ 1109.145535] 4060 00000001 00000000 aca585e0 ffffffc0 aca584e0 ffffffc0 07fb7c71 00000000 [ 1109.153753] [ 1109.153753] X19: 0xffffffc0bff13380: [ 1109.158799] 3380 00000000[ 1109.160774] mali e82c0000.mali: Reset interrupt didn't reach CPU. Check interrupt assignments.
[ 1109.169934] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.177103] 33a0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.185319] 33c0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.193537] 33e0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.201755] 3400 fab6faaf 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.209971] 3420 00000000 00000000 00000000 00000000 00031548 00000001 00000000 00000000 [ 1109.218187] 3440 00000000 00000000 00000001 00000000 00000000 00000000 00000000 00000000 [ 1109.226402] 3460 00017d53 00000000 0002db81 00000000 00000000 00000000 00000000 00000000 [ 1109.234619] [ 1109.234619] X20: 0xffffffc0b166c000: [ 1109.239664] c000 0b030d00 08010211 100f0d05 07091214 040e0a0c ffffff00 00000001 00000000 [ 1109.247881] c020 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.256097] c040 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.264313] c060 00000000 00000010 00000000 00000000 ffffffff 0000003f ffffffff 0000003f [ 1109.272528] c080 00000002 00000000 b0e64000 ffffffc0 00000003 04208040 00000000 00000000 [ 1109.280745] c0a0 00000000 00000000 00000001 00000000 00031435 00000001 84b47180 ffffffc0 [ 1109.288960] c0c0 00000001 00000001 00000031 00000078 00000031 00000032 08be15c8 ffffff80 [ 1109.297178] c0e0 00000400 00000000 00400000 00000000 00000001 00000000 00000000 00000000 [ 1109.305395] [ 1109.305395] X21: 0xffffffc0bff13b68: [ 1109.310440] 3b68 bff13b60 ffffffc0 bff13b70 ffffffc0 bff13b70 ffffffc0 bff13b80 ffffffc0 [ 1109.318658] 3b88 bff13b80 ffffffc0 bff13b90 ffffffc0 bff13b90 ffffffc0 bff13ba0 ffffffc0 [ 1109.326875] 3ba8 bff13ba0 ffffffc0 bff13bb0 ffffffc0 bff13bb0 ffffffc0 00000000 00000064 [ 1109.335092] 3bc8 00000064 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.343309] 3be8 bff13be8 ffffffc0 bff13be8 ffffffc0 00000000 00000000 00000000 00000000 [ 1109.351526] 3c08 00000000 00000000 0810cb88 ffffff80 00020002 00000000 00000000 00000000 [ 1109.359742] 3c28 006303e4 00000000 389fd980 00000000 f80df80d 00000000 00000000 00000000 [ 1109.367959] 3c48 bff13400 ffffffc0 091c42a0 ffffff80 00000000 00000000 00000000 00000000 [ 1109.376176] [ 1109.376176] X22: 0xffffffc0b166c000: [ 1109.381221] c000 0b030d00 08010211 100f0d05 07091214 040e0a0c ffffff00 00000001 00000000 [ 1109.389439] c020 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.397656] c040 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.405873] c060 00000000 00000010 00000000 00000000 ffffffff 0000003f ffffffff 0000003f [ 1109.414090] c080 00000002 00000000 b0e64000 ffffffc0 00000003 04208040 00000000 00000000 [ 1109.422307] c0a0 00000000 00000000 00000001 00000000 00031435 00000001 84b47180 ffffffc0 [ 1109.430524] c0c0 00000001 00000001 00000031 00000078 00000031 00000032 08be15c8 ffffff80 [ 1109.438741] c0e0 00000400 00000000 00400000 00000000 00000001 00000000 00000000 00000000 [ 1109.446959] [ 1109.446959] X25: 0xffffffc0b166c648: [ 1109.452004] c648 b166c648 ffffffc0 b166c648 ffffffc0 b10c8910 ffffffc0 b10c8910 ffffffc0 [ 1109.460220] c668 b0e67ea0 ffffffc0 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.468436] c688 00000073 00000000 00000000 00000000 00000073 00000000 00000000 00000000 [ 1109.476653] c6a8 00000000 00000000 00000000 00000000 00000000 00000000 00009814 00000000 [ 1109.484869] c6c8 00000004 00000000 09ec2856 00000001 09ec2856 00000001 00000000 00000000 [ 1109.493087] c6e8 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.501305] c708 b166c708 ffffffc0 b166c708 ffffffc0 b166c718 ffffffc0 b166c718 ffffffc0 [ 1109.509521] c728 b166c728 ffffffc0 b166c728 ffffffc0 00000000 00000000 b0e0b880 ffffffc0 [ 1109.517740] [ 1109.517740] X26: 0xffffffc0bff13380: [ 1109.522785] 3380 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.531001] 33a0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.539217] 33c0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.547434] 33e0 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.555651] 3400 fab6faaf 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1109.563868] 3420 00000000 00000000 00000000 00000000 00031548 00000001 00000000 00000000 [ 1109.572085] 3440 00000000 00000000 00000001 00000000 00000000 00000000 00000000 00000000 [ 1109.580301] 3460 00017d53 00000000 0002db81 00000000 00000000 00000000 00000000 00000000 [ 1109.588521] [ 1109.588521] X29: 0xffffffc0b0e67b10: [ 1109.593567] 7b10 b166c080 ffffffc0 09095000 ffffff80 00000000 00000000 b166c6c8 ffffffc0 [ 1109.601784] 7b30 bff13400 ffffffc0 08bc4fc4 ffffff80 00000001 00000000 b0e67b90 ffffffc0 [ 1109.610001] 7b50 0810c82c ffffff80 b0e67b90 ffffffc0 08112428 ffffff80 60000185 00000000 [ 1109.618218] 7b70 b0e67ba0 ffffffc0 0810c538 ffffff80 ffffffff ffffffff 0810c574 ffffff80 [ 1109.626436] 7b90 b0e67bc0 ffffffc0 0810c82c ffffff80 bff13400 ffffffc0 0810c820 ffffff80 [ 1109.634653] 7bb0 bff13400 ffffffc0 b166c080 ffffffc0 b0e67bf0 ffffffc0 080eead8 ffffff80 [ 1109.642869] 7bd0 bff13400 ffffffc0 0906b000 ffffff80 bff13400 ffffffc0 0906b000 ffffff80 [ 1109.651086] 7bf0 b0e67c20 ffffffc0 08bc4b80 ffffff80 bff13400 ffffffc0 08bc47e4 ffffff80 [ 1109.659303] [ 1109.660788] Process kschedfreq:0 (pid: 1248, stack limit = 0xffffffc0b0e64020) [ 1109.668006] Stack: (0xffffffc0b0e67b90 to 0xffffffc0b0e68000) [ 1109.673748] 7b80: ffffffc0b0e67bc0 ffffff800810c82c [ 1109.681574] 7ba0: ffffffc0bff13400 ffffff800810c820 ffffffc0bff13400 ffffffc0b166c080 [ 1109.689399] 7bc0: ffffffc0b0e67bf0 ffffff80080eead8 ffffffc0bff13400 ffffff800906b000 [ 1109.697225] 7be0: ffffffc0bff13400 ffffff800906b000 ffffffc0b0e67c20 ffffff8008bc4b80 [ 1109.705052] 7c00: ffffffc0bff13400 ffffff8008bc47e4 ffffffc000000001 ffffffc0b166c080 [ 1109.712877] 7c20: ffffffc0b0e67c80 ffffff8008bc4fc4 ffffffc0b0e64000 0000000000000001 [ 1109.720703] 7c40: 00000000000186a0 ffffffc0b0e64000 ffffff8008be0000 00000000001a13c8 [ 1109.728529] 7c60: ffffffc0b1092800 ffffff80091a8000 0000000000000000 0000000000009f4c [ 1109.736355] 7c80: ffffffc0b0e67ca0 ffffff8008bc82a8 ffffffc0b0e67d98 0000000000000100 [ 1109.744181] 7ca0: ffffffc0b0e67d40 ffffff8008bc8348 ffffffc0b0e67d98 0000000000000064 [ 1109.752006] 7cc0: ffffffc0b11bb580 ffffffc0b0e67d30 ffffff800808e7e4 00000001b166c080 [ 1109.759832] 7ce0: ffffffc0bff0f2d1 ffffffc0ae6cfe30 ffffffc0bff0f170 0000010209828528 [ 1109.767658] 7d00: 000001020980fe88 ffffff800813c598 ffffffc0bff0ed40 000004e000000001 [ 1109.775483] 7d20: ffffff8008bc829c 726664656863736b 00000000303a7165 ffffffc0b166c080 [ 1109.783309] 7d40: ffffffc0b0e67d70 ffffff8008bc8068 00000000026e40e0 00000000001a13c8 [ 1109.791135] 7d60: 00000000000186a0 0000000108142a54 ffffffc0b0e67da0 ffffff800811a770 [ 1109.798961] 7d80: 000001020980f324 ffffffc0b0e64000 ffffffc0b11bb580 00000000026e40e0 [ 1109.806787] 7da0: ffffffc0b0e67e20 ffffff80080e1a14 ffffffc0b11bb400 ffffffc0b0e64000 [ 1109.814613] 7dc0: ffffff80091c41c8 ffffffc0b1092800 ffffff800811a6b0 0000000000000000 [ 1109.822438] 7de0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.830263] 7e00: ffffffc0b11bb400 ffffffc0b0e64000 ffffff80091c41c8 ffffffc000000032 [ 1109.838089] 7e20: 0000000000000000 ffffff8008085ed0 ffffff80080e192c ffffffc0b11bb400 [ 1109.845914] 7e40: 0000000000000000 0000000000000000 0000000000000000 ffffff80080efe18 [ 1109.853740] 7e60: 0000000000000000 0000000000000000 0000000000000000 ffffffc0b1092800 [ 1109.861565] 7e80: ffffffc000000000 ffffff8000000000 ffffffc0b0e67e90 ffffffc0b0e67e90 [ 1109.869392] 7ea0: 0000000000000000 ffffff8000000000 ffffffc0b0e67eb0 ffffffc0b0e67eb0 [ 1109.877217] 7ec0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.885042] 7ee0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.892867] 7f00: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.900692] 7f20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.908518] 7f40: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.916344] 7f60: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.924169] 7f80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.931994] 7fa0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.939820] 7fc0: 0000000000000000 0000000000000005 0000000000000000 0000000000000000 [ 1109.947646] 7fe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.955470] Call trace: [ 1109.957911] Exception stack(0xffffffc0b0e679c0 to 0xffffffc0b0e67af0) [ 1109.964348] 79c0: ffffffc0bff13400 0000008000000000 ffffffc0b0e67b90 ffffff8008112428 [ 1109.972173] 79e0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [ 1109.979999] 7a00: 0000000000000000 0000000000000000 0000000000000000 0000000000000009 [ 1109.987824] 7a20: 0000000000000010 0000000000000010 0000000000000000 000000000000068a [ 1109.995650] 7a40: ffffffc0b0e67a90 ffffff8008bc8bac 0000000000000180 ffffff800928c688 [ 1110.003476] 7a60: 00000000000bc75e fffffffffffedce2 0000000000000000 0000000000000002 [ 1110.011301] 7a80: 00000040b6ea8000 0000000000000001 ffffff8009095000 ffffffc0ae6cfe30 [ 1110.019127] 7aa0: ffffffc0b0e67ce0 ffffffc0b0e64000 00000000000009d0 ffffff8008be13cc [ 1110.026952] 7ac0: 0000000034d5d91d 0000000000000000 0000000000000000 0000000000000000 [ 1110.034777] 7ae0: 0000000000000000 0000000000000000 [ 1110.039651] [<ffffff8008112428>] walt_dec_cumulative_runnable_avg+0x40/0x44 [ 1110.046609] [<ffffff800810c82c>] dequeue_task_rt+0x40/0x8c [ 1110.052093] [<ffffff80080eead8>] deactivate_task+0x98/0xbc [ 1110.057580] [<ffffff8008bc4b80>] __schedule+0x44c/0x7c0 [ 1110.062800] [<ffffff8008bc4fc4>] schedule+0x40/0xa0 [ 1110.067674] [<ffffff8008bc82a8>] schedule_hrtimeout_range_clock+0x94/0x100 [ 1110.074544] [<ffffff8008bc8348>] schedule_hrtimeout_range+0x34/0x40 [ 1110.080806] [<ffffff8008bc8068>] usleep_range+0x4c/0x58 [ 1110.086028] [<ffffff800811a770>] cpufreq_sched_thread+0xc0/0x1e4 [ 1110.092032] [<ffffff80080e1a14>] kthread+0xe8/0xfc [ 1110.096821] [<ffffff8008085ed0>] ret_from_fork+0x10/0x40 [ 1110.102129] Code: b7f80081 f9400bf3 a8c37bfd d65f03c0 (d4210000)
eas-dev mailing list eas-dev@lists.linaro.org https://lists.linaro.org/mailman/listinfo/eas-dev
Hi Steven,
On Fri, Aug 25, 2017 at 10:47:07AM +0800, Steven Miao wrote:
Hi leo,
On Wed, Aug 23, 2017 at 4:00 PM, Leo Yan leo.yan@linaro.org wrote:
Hi Vikram, Joonwoo,
[ + EAS mailing list ]
On Hikey960 with EASv1.3, I encountered many times for WALT panic, it reports the bug from below two functions; you also could see log in the below. Before I dig into this, could you give some suggestion for this? Or if there have some existed fixing for this? Thanks in advance.
I've been running EAS r1.3 kernel for 3 weeks with the default hikey960 defconfig on several hikey960 board, haven't seen this panic yet.
kernel verison: 4a80fc6 ARM64: hikey960: Enable uart3 c5ad7df HACK: clk-hi3660: Fix clk_factor_uart3 to be 100000000, not 200000000 43c93f7 Merge remote-tracking branch 'common/android-4.4' into android-hikey-linaro-4.4-aosp 7eeebce6 ANDROID: binder: Don't BUG_ON(!spin_is_locked()). 78ba0ad Merge 4.4.79 into android-4.4 ...
Does this issue appear frequently?
As Joonwoo meantioned, I have applied patch 426442 [1] so the WALT panic can emerge more obvious; at my side usally I can reproduce this issue within 1 hours for workload testing [2].
[1] https://android-review.googlesource.com/#/c/kernel/common/+/426442/ [2] http://termbin.com/gh4l
Thanks, Leo Yan
-
Dietmar Eggemann -
Joonwoo Park -
joonwoop＠codeaurora.org
-
Leo Yan -
Steven Miao