[bug]sched clock overflows in 208 days (i386 and amd64)

 

[Symptom]

Ubuntu OS 장비의 uptime이 208일이 넘을 경우 장비가 재부팅되는 현상.

 

[LOG]

dmesg 혹은 message 등, 로그에 아래와 같은 로그가 출력

"May 28 00:07:26  [18446744056.812596] BUG: soft lockup - CPU#0 stuck for 17163091997s! [swapper:0]
May 28 00:07:26  [18446744056.812599] Modules linked in: bonding usb_storage qla2xxx r8169 r8168 ixgbe igb e1000e e100 e1000
May 28 00:07:26  [18446744056.812608] CPU 0
May 28 00:07:26  [18446744056.812609] Modules linked in: bonding usb_storage qla2xxx r8169 r8168 ixgbe igb e1000e e100 e1000
May 28 00:07:26  [18446744056.812616]
May 28 00:07:26  [18446744056.812619] Pid: 0, comm: swapper Not tainted 2.6.35.14 #16 X9SCL/X9SCM/X9SCL/X9SCM
May 28 00:07:26  [18446744056.812622] RIP: 0010:[<ffffffff81009dd7>]  [<ffffffff81009dd7>] mwait_idle+0x57/0x80
May 28 00:07:26  [18446744056.812630] RSP: 0018:ffffffff81801f28  EFLAGS: 00000246
May 28 00:07:26  [18446744056.812632] RAX: 0000000000000000 RBX: ffffffff81801f28 RCX: 0000000000000000
May 28 00:07:26  [18446744056.812635] RDX: 0000000000000000 RSI: ffffffff81801fd8 RDI: ffffffff818cd308
May 28 00:07:26  [18446744056.812637] RBP: ffffffff810029ce R08: 0000000000000000 R09: 0000000000000001
May 28 00:07:26  [18446744056.812639] R10: 0000000000000001 R11: 0000000000000000 R12: 0000000000000000
May 28 00:07:26  [18446744056.812642] R13: 0000000000000000 R14: ffff8804393957c0 R15: 0000000000011f00
May 28 00:07:26  [18446744056.812645] FS:  0000000000000000(0000) GS:ffff880001a00000(0000) knlGS:0000000000000000
May 28 00:07:26  [18446744056.812648] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
May 28 00:07:26  [18446744056.812650] CR2: 0000000006dcda58 CR3: 00000004393bf000 CR4: 00000000000406f0
May 28 00:07:26  [18446744056.812652] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
May 28 00:07:26  [18446744056.812655] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
May 28 00:07:26  [18446744056.812658] Process swapper (pid: 0, threadinfo ffffffff81800000, task ffffffff8180d020)
May 28 00:07:26  [18446744056.812660] Stack:
May 28 00:07:26  [18446744056.815374]  ffffffff81801f48 ffffffff810006cb 0000000000000000 6db6db6db6db6db7
May 28 00:07:26  [18446744056.815376] <0> ffffffff81801f58 ffffffff8140701c ffffffff81801f98 ffffffff81865bae
May 28 00:07:26  [18446744056.815379] <0> 0000000001ed83a8 ffffffff8189b340 000000000193f364 000000000000180a
May 28 00:07:26  [18446744056.815382] Call Trace:
May 28 00:07:26  [18446744056.818540]  [<ffffffff810006cb>] ? cpu_idle+0x4b/0x90
May 28 00:07:26  [18446744056.812596] BUG: soft lockup - CPU#0 stuck for 17163091997s! [swapper:0]
May 28 00:07:26  [18446744056.812599] Modules linked in: bonding usb_storage qla2xxx r8169 r8168 ixgbe igb e1000e e100 e1000
May 28 00:07:26  [18446744056.812608] CPU 0
May 28 00:07:26  [18446744056.812609] Modules linked in: bonding usb_storage qla2xxx r8169 r8168 ixgbe igb e1000e e100 e1000
May 28 00:07:26  [18446744056.812616]
May 28 00:07:26  [18446744056.812619] Pid: 0, comm: swapper Not tainted 2.6.35.14 #16 X9SCL/X9SCM/X9SCL/X9SCM
May 28 00:07:26  [18446744056.812622] RIP: 0010:[<ffffffff81009dd7>]  [<ffffffff81009dd7>] mwait_idle+0x57/0x80
May 28 00:07:26  [18446744056.812630] RSP: 0018:ffffffff81801f28  EFLAGS: 00000246
May 28 00:07:26  [18446744056.812632] RAX: 0000000000000000 RBX: ffffffff81801f28 RCX: 0000000000000000
May 28 00:07:26  [18446744056.812635] RDX: 0000000000000000 RSI: ffffffff81801fd8 RDI: ffffffff818cd308
May 28 00:07:26  [18446744056.812637] RBP: ffffffff810029ce R08: 0000000000000000 R09: 0000000000000001
May 28 00:07:26  [18446744056.812639] R10: 0000000000000001 R11: 0000000000000000 R12: 0000000000000000
May 28 00:07:26  [18446744056.812642] R13: 0000000000000000 R14: ffff8804393957c0 R15: 0000000000011f00
May 28 00:07:26  [18446744056.812645] FS:  0000000000000000(0000) GS:ffff880001a00000(0000) knlGS:0000000000000000
May 28 00:07:26  [18446744056.812648] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
May 28 00:07:26  [18446744056.812650] CR2: 0000000006dcda58 CR3: 00000004393bf000 CR4: 00000000000406f0
May 28 00:07:26  [18446744056.812652] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
May 28 00:07:26  [18446744056.812655] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
May 28 00:07:26  [18446744056.812658] Process swapper (pid: 0, threadinfo ffffffff81800000, task ffffffff8180d020)
May 28 00:07:26  [18446744056.812660] Stack:
May 28 00:07:26  [18446744056.815374]  ffffffff81801f48 ffffffff810006cb 0000000000000000 6db6db6db6db6db7
May 28 00:07:26  [18446744056.815376] <0> ffffffff81801f58 ffffffff8140701c ffffffff81801f98 ffffffff81865bae
May 28 00:07:26  [18446744056.815379] <0> 0000000001ed83a8 ffffffff8189b340 000000000193f364 000000000000180a
May 28 00:07:26  [18446744056.815382] Call Trace:
May 28 00:07:26  [18446744056.818540]  [<ffffffff810006cb>] ? cpu_idle+0x4b/0x90
May 28 00:07:26  [18446744056.818544]  [<ffffffff8140701c>] ? rest_init+0x80/0x84
May 28 00:07:26  [18446744056.818548]  [<ffffffff81865bae>] ? start_kernel+0x34d/0x359
May 28 00:07:26  [18446744056.818551]  [<ffffffff8186531d>] ? x86_64_start_reservations+0x12d/0x131
May 28 00:07:26  [18446744056.818553]  [<ffffffff81865408>] ? x86_64_start_kernel+0xe7/0xee
May 28 00:07:26  [18446744056.818554] Code: 33 31 d2 65 48 8b 34 25 88 b5 00 00 48 89 d1 48 8d 86 38 e0 ff ff 0f 01 c8 0f ae f0 f6 86 38 e0 ff ff 08 75 18 31 c0 fb 0f 01 c9 <5d> c3 0f 1f 80 00 00 00 00 0f ae ba 38 e0 ff ff eb c4 fb 5d 0f
May 28 00:07:26  [18446744056.840428] Call Trace:
May 28 00:07:26  [18446744056.840430]  [<ffffffff810006cb>] ? cpu_idle+0x4b/0x90
May 28 00:07:26  [18446744056.840432]  [<ffffffff8140701c>] ? rest_init+0x80/0x84
May 28 00:07:26  [18446744056.840435]  [<ffffffff81865bae>] ? start_kernel+0x34d/0x359
May 28 00:07:26  [18446744056.840437]  [<ffffffff8186531d>] ? x86_64_start_reservations+0x12d/0x131
May 28 00:07:26  [18446744056.840440]  [<ffffffff81865408>] ? x86_64_start_kernel+0xe7/0xee
May 28 00:07:26  [18446744056.840442] BUG: soft lockup - CPU#1 stuck for 17163091997s! [swapper:0]
May 28 00:07:26  [18446744056.840444] Modules linked in: bonding usb_storage qla2xxx r8169 r8168 ixgbe igb e1000e e100 e1000
May 28 00:07:26  [18446744056.840450] CPU 1 
May 28 00:07:26  [18446744056.840450] Modules linked in: bonding usb_storage qla2xxx r8169 r8168 ixgbe igb e1000e e100 e1000
May 28 00:07:26  [18446744056.840455]
May 28 00:07:26  [18446744056.840457] Pid: 0, comm: swapper Not tainted 2.6.35.14 #16 X9SCL/X9SCM/X9SCL/X9SCM
May 28 00:07:26  [18446744056.840459] RIP: 0010:[<ffffffff81009dd7>]  [<ffffffff81009dd7>] mwait_idle+0x57/0x80
May 28 00:07:26  [18446744056.840463] RSP: 0018:ffff88043fc97f08  EFLAGS: 00000246
May 28 00:07:26  [18446744056.840464] RAX: 0000000000000000 RBX: ffff88043fc97f08 RCX: 0000000000000000
May 28 00:07:26  [18446744056.840466] RDX: 0000000000000000 RSI: ffff88043fc97fd8 RDI: ffffffff818cd308
May 28 00:07:26  [18446744056.840468] RBP: ffffffff810029ce R08: 0000000000000000 R09: 0000000000000001
May 28 00:07:26  [18446744056.840469] R10: 0000000000000001 R11: 0000000000000000 R12: 0000000000000000
May 28 00:07:26  [18446744056.840471] R13: 0000000000000001 R14: ffff8804393957c0 R15: 0000000000011f00
May 28 00:07:26  [18446744056.818544]  [<ffffffff8140701c>] ? rest_init+0x80/0x84
May 28 00:07:26  [18446744056.818548]  [<ffffffff81865bae>] ? start_kernel+0x34d/0x359
May 28 00:07:26  [18446744056.818551]  [<ffffffff8186531d>] ? x86_64_start_reservations+0x12d/0x131
May 28 00:07:26  [18446744056.818553]  [<ffffffff81865408>] ? x86_64_start_kernel+0xe7/0xee
May 28 00:07:26  [18446744056.818554] Code: 33 31 d2 65 48 8b 34 25 88 b5 00 00 48 89 d1 48 8d 86 38 e0 ff ff 0f 01 c8 0f ae f0 f6 86 38 e0 ff ff 08 75 18 31 c0 fb 0f 01 c9 <5d> c3 0f 1f 80 00 00 00 00 0f ae ba 38 e0 ff ff eb c4 fb 5d 0f
May 28 00:07:26  [18446744056.840428] Call Trace:
May 28 00:07:26  [18446744056.840430]  [<ffffffff810006cb>] ? cpu_idle+0x4b/0x90"

 

[Solution]

Ubuntu Kernel 버그로 Kernel Upgrade 필요.

package linux - 2.6.35-32.65 버전 및 기타 버전들에서 패치 완료.

(자세한 건 아래 링크 참조)

 

https://bugs.launchpad.net/ubuntu/+source+linux/+bug/

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/805341