Sorry to reply to myself, but after a while the panda doesn't reply any more over network and the syslog has lots of entries like this one:
Aug 27 13:00:38 panda kernel: [87136.303894] [<c044c1e9>] (__schedule_bug+0x39/0x50) from [<c045219f>] (__schedule+0x51f/0x5c4)
Aug 27 13:00:38 panda kernel: [87136.303924] [<c045219f>] (__schedule+0x51f/0x5c4) from [<c045250b>] (schedule_preempt_disabled+0xf/0x10)
Aug 27 13:00:38 panda kernel: [87136.303924] [<c045250b>] (schedule_preempt_disabled+0xf/0x10) from [<c0058bbb>] (cpu_startup_entry+0x12f/0x164)
Aug 27 13:00:38 panda kernel: [87136.303955] [<c0058bbb>] (cpu_startup_entry+0x12f/0x164) from [<80448ea9>] (0x80448ea9)
Aug 27 13:00:38 panda kernel: [87136.307403] BUG: scheduling while atomic: swapper/1/0/0xffff0000
Aug 27 13:00:38 panda kernel: [87136.313781] Modules linked in: snd_soc_omap_abe_twl6040 snd_soc_omap_mcpdm snd_soc_twl6040 snd_soc_omap snd_soc_omap_mcbsp snd_soc_core snd_compress regmap_spi snd_pcm snd_page_alloc snd_timer snd soundcore gator
Aug 27 13:00:38 panda kernel: [87136.313842] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G W 3.10.1.0-1-linaro-omap #1ubuntu1~ci+130723003933-Ubuntu
Aug 27 13:00:38 panda kernel: [87136.313842] [<c001175d>] (unwind_backtrace+0x1/0x9c) from [<c000f625>] (show_stack+0x11/0x14)
Aug 27 13:00:38 panda kernel: [87136.313873] [<c000f625>] (show_stack+0x11/0x14) from [<c044c1e9>] (__schedule_bug+0x39/0x50)
Aug 27 13:00:38 panda kernel: [87136.313903] [<c044c1e9>] (__schedule_bug+0x39/0x50) from [<c045219f>] (__schedule+0x51f/0x5c4)
Aug 27 13:00:38 panda kernel: [87136.313934] [<c045219f>] (__schedule+0x51f/0x5c4) from [<c045250b>] (schedule_preempt_disabled+0xf/0x10)
Aug 27 13:00:38 panda kernel: [87136.313934] [<c045250b>] (schedule_preempt_disabled+0xf/0x10) from [<c0058bbb>] (cpu_startup_entry+0x12f/0x164)
Aug 27 13:00:38 panda kernel: [87136.313964] [<c0058bbb>] (cpu_startup_entry+0x12f/0x164) from [<80448ea9>] (0x80448ea9)
Aug 27 13:00:38 panda kernel: [87136.317382] BUG: scheduling while atomic: swapper/1/0/0xffff0000
Aug 27 13:00:38 panda kernel: [87136.323760] Modules linked in: snd_soc_omap_abe_twl6040 snd_soc_omap_mcpdm snd_soc_twl6040 snd_soc_omap snd_soc_omap_mcbsp snd_soc_core snd_compress regmap_spi snd_pcm snd_page_alloc snd_timer snd soundcore gator
Aug 27 13:00:38 panda kernel: [87136.323822] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G W 3.10.1.0-1-linaro-omap #1ubuntu1~ci+130723003933-Ubuntu
Aug 27 13:00:38 panda kernel: [87136.323822] [<c001175d>] (unwind_backtrace+0x1/0x9c) from [<c000f625>] (show_stack+0x11/0x14)
Aug 27 13:00:38 panda kernel: [87136.323822] [<c000f625>] (show_stack+0x11/0x14) from [<c044c1e9>] (__schedule_bug+0x39/0x50)
Aug 27 13:00:38 panda kernel: [87136.323883] [<c044c1e9>] (__schedule_bug+0x39/0x50) from [<c045219f>] (__schedule+0x51f/0x5c4)
Aug 27 13:00:38 panda kernel: [87136.323913] [<c045219f>] (__schedule+0x51f/0x5c4) from [<c045250b>] (schedule_preempt_disabled+0xf/0x10)
Aug 27 13:00:38 panda kernel: [87136.323913] [<c045250b>] (schedule_preempt_disabled+0xf/0x10) from [<c0058bbb>] (cpu_startup_entry+0x12f/0x164)
Aug 27 13:00:38 panda kernel: [87136.323944] [<c0058bbb>] (cpu_startup_entry+0x12f/0x164) from [<80448ea9>] (0x80448ea9)
Aug 27 13:00:38 panda kernel: [87136.327392] BUG: scheduling while atomic: swapper/1/0/0xffff0000
Aug 27 13:00:38 panda kernel: [87136.333770] Modules linked in: snd_soc_omap_abe_twl6040 snd_soc_omap_mcpdm snd_soc_twl6040 snd_soc_omap snd_soc_omap_mcbsp snd_soc_core snd_compress regmap_spi snd_pcm snd_page_alloc snd_timer snd soundcore gator