Login or Register to Ask a Question and Join Our Community


Red Hat

comm: mysqld Not tainted ... Kernel Panic , System totally unresponsive

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Operating Systems Linux Red Hat comm: mysqld Not tainted ... Kernel Panic , System totally unresponsive
# 1  
Old 08-26-2010
Question comm: mysqld Not tainted ... Kernel Panic , System totally unresponsive
Hi,

I am experiencing frequent system hangs, hard kernel panics, etc almost thrice a day. The system would be totally unresponsive and the only way is to reboot is hard power recycling (plug out the power cable and plug in back after 30 secs). I enabled kdump, but unfortunately the kdump files are as huge as 16GB and unable to analyze. The repeated errors I get in the /var/log/messages is
Quote:
Aug 24 18:05:35 blr-cos-mdb01 kernel: BUG: soft lockup - CPU#0 stuck for 10s! [mysqld:5365]
Aug 24 18:05:35 blr-cos-mdb01 kernel: CPU 0:
Aug 24 18:05:45 blr-cos-mdb01 kernel: Modules linked in: ipv6 xfrm_nalgo crypto_api hidp l2cap bluetooth lockd sunrpc cpufreq_ondemand acpi_cpufreq freq_table dm_multipath scsi_dh video backlight sbs power_meter hwmon i2c_ec i2c_core dell_wmi wmi button battery asus_acpi acpi_memhotplug ac parport_pc lp parport floppy snd_hda_intel sr_mod tpm_infineon snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq cdrom tpm snd_seq_device snd_pcm_oss snd_mixer_oss tpm_bios snd_pcm e1000e snd_timer shpchp serio_raw snd_page_alloc snd_hwdep pcspkr sg snd soundcore dm_raid45 dm_message dm_region_hash dm_mem_cache dm_snapshot dm_zero dm_mirror dm_log dm_mod ahci libata sd_mod scsi_mod ext3 jbd uhci_hcd ohci_hcd ehci_hcd
Aug 24 18:05:45 blr-cos-mdb01 kernel: Pid: 5365, comm: mysqld Not tainted 2.6.18-194.3.1.el5 #1
Aug 24 18:05:45 blr-cos-mdb01 kernel: RIP: 0010:[] [] __d_lookup+0xe2/0xff
Aug 24 18:05:45 blr-cos-mdb01 kernel: RSP: 0018:ffff8101a415fc88 EFLAGS: 00000282
Aug 24 18:05:45 blr-cos-mdb01 kernel: RAX: ffff8103c6c864c8 RBX: ffff8103c6c864c8 RCX: 0000000000000015
Aug 24 18:05:45 blr-cos-mdb01 kernel: RDX: 00000000000db1d6 RSI: ffff8101a415fd28 RDI: ffff8103c83274b0
Aug 24 18:05:45 blr-cos-mdb01 kernel: RBP: ffff810417bbf800 R08: 0000000000008001 R09: ffff81041747e5c0
Aug 24 18:05:45 blr-cos-mdb01 kernel: R10: ffff810188c8c580 R11: ffffffff8002c3e0 R12: ffff81040c0840c0
Aug 24 18:05:45 blr-cos-mdb01 kernel: R13: 0000000000000000 R14: ffff8101a394e348 R15: ffff8101a394e348
Aug 24 18:05:45 blr-cos-mdb01 kernel: FS: 00000000405c8940(0063) GS:ffffffff803ca000(0000) knlGS:0000000000000000
Aug 24 18:05:45 blr-cos-mdb01 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Aug 24 18:05:45 blr-cos-mdb01 kernel: CR2: 00002aace6224000 CR3: 0000000401db3000 CR4: 00000000000006e0
Aug 24 18:05:45 blr-cos-mdb01 kernel:
Aug 24 18:05:45 blr-cos-mdb01 kernel: Call Trace:
Aug 24 18:05:45 blr-cos-mdb01 kernel: [] __d_lookup+0xb0/0xff
Aug 24 18:05:45 blr-cos-mdb01 kernel: [] do_lookup+0x2c/0x1e6
Aug 24 18:05:45 blr-cos-mdb01 kernel: [] __link_path_walk+0xa01/0xf42
Aug 24 18:05:45 blr-cos-mdb01 kernel: [] link_path_walk+0x42/0xb2
Aug 24 18:05:45 blr-cos-mdb01 kernel: [] do_path_lookup+0x275/0x2f1
Aug 24 18:05:45 blr-cos-mdb01 kernel: [] __path_lookup_intent_open+0x56/0x97
Aug 24 18:05:45 blr-cos-mdb01 kernel: [] open_namei+0x73/0x6d5
Aug 24 18:05:45 blr-cos-mdb01 kernel: [] do_page_fault+0x4fe/0x874
Aug 24 18:05:45 blr-cos-mdb01 kernel: [] do_filp_open+0x1c/0x38
Aug 24 18:05:45 blr-cos-mdb01 kernel: [] _atomic_dec_and_lock+0x39/0x57
Aug 24 18:05:45 blr-cos-mdb01 kernel: [] do_sys_open+0x44/0xbe
Aug 24 18:05:45 blr-cos-mdb01 kernel: [] tracesys+0xd5/0xe0
Aug 24 18:05:45 blr-cos-mdb01 kernel:
Aug 24 18:22:23 blr-cos-mdb01 syslogd 1.4.1: restart.
I have CentOS release 5.5 (Final) with kernel-2.6.18-194.3.1.el5. The hardware is HP dc7900 with 16 GB RAM, Intel Core 2 Duo E8400/3Ghz/4GB RAM, 160GB HDD. I have installed MySQL builds from Percona viz
Percona-XtraDB-1.0.6-10.2-5.1.45-10.2.rhel5
Percona-Server-server-51-5.1.47-rel11.1.51.rhel5
Percona-XtraDB-1.0.3-5-5.1.34-5.rhel5
Percona-Server-shared-compat-5.1.43-3
Percona-Server-client-51-5.1.47-rel11.1.51.rhel5
Percona-Server-test-51-5.1.47-rel11.1.51.rhel5
Percona-Server-devel-51-5.1.47-rel11.1.51.rhel5
Percona-Server-shared-51-5.1.47-rel11.1.51.rhel5

The uname -a produces Linux blr-cos-mdb01.digi.com 2.6.18-194.3.1.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux

What could be the issue and how to resolve it ?

Regards
Prashant
# 2  
Old 08-26-2010
The kernel is not supposed to panic under any circumstances, there's no "normal" circumstances beyond a hardware failure or kernel bug that should cause one. As such, there's no magic technique for avoiding potential kernel panics. Try upgrading your kernel.
# 3  
Old 08-30-2010
MySQL
Issue resolved. BAD ... BAD MEMORY, after removing one if the DIMM's, the server no longer crashes.

I wonder how memtest passed the Memory !!!!
# 4  
Old 08-30-2010
did you actually run memtest86 or memtest86+ or did you rely on the BIOS address test? A BIOS address test will not resolve issues like you describe. Memtest86+, run for several hours has never failed in identifying bad RAM for me.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Red Hat

Kernel panic-not syncing

dear experts linux i am using RHEL 5.5 run on a physical server. i am front of a big trouble, the Rhel can't run correctly , it show me this error message in black screen: reading all physical volumes. this my take a while .. found volume group "VolGroup00" using metadata type lvm2 9... (1 Reply)
Discussion started by: wassimpb
1 Replies

2. SCO

Kernel Panic

Hello when i booting my SCO Server I become this Message PLS can anyone help me? Cheers Christoph (1 Reply)
Discussion started by: cga
1 Replies

3. Solaris

solaris 10 system unresponsive

Hi guys, I have solaris 10 installed in sun240 server. While I was practicing, the system became unresponsive. How can I fix this issue? Thanks for your help (3 Replies)
Discussion started by: cjashu
3 Replies

4. UNIX for Advanced & Expert Users

what does the kernel panic mean?

Hi, all: How should I obtain the reason of kernel panic? How should I debug the program which result in a kernel panic? What can I get from the panic information? li, kunlun (3 Replies)
Discussion started by: liklstar
3 Replies

5. Solaris

Solaris 10 - Unable to boot the system "panic: cannot open /kernel/amd64/unix"

Hi All, I have installed Solaris 10 on my AMD 64 3000+ system. I was playing with grub commands eeprom and bootadm commands. I screwed my boot-file and now am unable to boot the system. Gets error msg as "panic: cannot open /kernel/amd64/unix". I booted the system is filesafe and tried update the... (2 Replies)
Discussion started by: Manjunath K V
2 Replies

6. AIX

Disabling saving of system dump while kernel panic

Hi, Is there an option to disable saving of system dump when a user invokes a kernel panic using sysdumpstart command. Also, is there any other way of invoking a kernel panic other than using sysdumpstart? Thanks, Kanna (3 Replies)
Discussion started by: kanna_geekworkz
3 Replies

7. Linux

Kernel panic - not syncing: cannot execute a PAE-enabled kernel on PAE-less CPU

ok so I just installed fedora core 6 on my dell inspiron 700m and I go to boot into linux and I get this error. Has anyone seen this before? I also had XP Pro and Vista installed on this pc prior to putting fedora core 6 on the machine. I'm trying to setup a triple boot system. Please Help... (2 Replies)
Discussion started by: dave043
2 Replies

8. Linux

kernel panic

I have compiled kernel version 2.6.12 in my linux box mandrake 10.1. I have followed www.digitalhermit.com/linux/Kernel-Build-HOWTO.html instructions. I have copied .config file from /usr/src/linux and lightly modified it (only the processor type) by xconfig. I have compiled the source and modules.... (3 Replies)
Discussion started by: Davide71
3 Replies

9. UNIX for Dummies Questions & Answers

kernel panic

hello :-) I have one problem with my LiLO... I installed PLD Linux on my computer but when it boots, the last thing I can see is: request_module Root fs not mounted VFS: Cannot open root device 03:01 Kernel panic: VFS: unable to mount root fs on 03:01 and the system stops :-( what... (1 Reply)
Discussion started by: pgas
1 Replies

10. UNIX for Dummies Questions & Answers

RH 7.3 Kernel Panic.

I've installed rh 7.3 a fewday ago and I'm trying to boot it and I get this" Kernel panic: no init found. try passing init=options to kernel" ????? What do I do. Thanks. F,Ruiz. (2 Replies)
Discussion started by: ruizf
2 Replies
Login or Register to Ask a Question