Check hung process and restart

 
Thread Tools Search this Thread
Homework and Emergencies Emergency UNIX and Linux Support Check hung process and restart
# 1  
Old 05-03-2012
Check hung process and restart

Hi all

I have networker running on a RHEL 5.7 and over time it hangs. So the solution backup team proposed is to check if the process is hung, to stop and start it.

Unfortunately for me, the rc script only allows three commands, start, stop and status (no restart option) so I managed to set following script but when I executed it-even when networker has been stopped I get the OK message in my /var/log/messages. Why is that? Can someone please help me look into this? Where did I go wrong? Sorry I am rushing this, they need to get this setup on prod servers by today at COB...


Code:
#!/bin/bash
cmdstop='/etc/rc.d/init.d/networker stop'
cmdstart='/etc/rc.d/init.d/networker start'

if [ "${?}" != 0 ] ; then
 echo "`date` CRITICAL:Networker hung, will be restarted" >>/var/log/messages
$cmdstart
 else
echo "`date` OK:Networker running" >>/var/log/messages
fi
exit

# 2  
Old 05-03-2012
This variable assignment:
Code:
cmdstart='/etc/rc.d/init.d/networker start'

always succeeds, so $? is always 0
# 3  
Old 05-03-2012
@Scrutinizer

But what is the process hangs? Will it still show 0?
# 4  
Old 05-03-2012
It does not matter if the process hangs, the script does not check for that, it also does not restart the process and all it will do is write "`date` OK:Networker running" into the log.
# 5  
Old 05-03-2012
well, how do I incorporate stop and start into the script that looks for hung process? Smilie
Sorry I am so lost!!
# 6  
Old 05-03-2012
How would you characterize the state as "hung"?

Maybe a better approach is to stop and start it on a regular basis.
# 7  
Old 05-03-2012
@otheus

I've attached the screenshot of what was done when we last found that the proces was hung...
Check hung process and restart-db2vend-hungjpg

Is it a good idea to stop & start it a regular interval instead? But how do I check with command if the process is hung? I use
Code:
ps auxw | grep db2vend


Last edited by hedkandi; 05-03-2012 at 03:28 AM.. Reason: addition
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to check if the process is Hung?

I wish to monitor if my Tomcat process if Running, Hung, or Shutdown. I cannot use any third party monitoring tools so i decided to use one of these to test if the tomcat server is responding or not . 1. nslookup 2. telnet 3. ps The reason I do not use wget / curl is because it will... (2 Replies)
Discussion started by: mohtashims
2 Replies

2. UNIX for Dummies Questions & Answers

View User Mode Call Stack of Hung Process

I have a multithreaded usermode program(actually a daemon) which is in hanged state. To debug it I tried attaching the process to gdb, but the gdb hangs. gstack also gets hanged. I peeped into the proc file system and saw the process to be in sleeping state. /proc/sysrq-trigger I guess... (1 Reply)
Discussion started by: rupeshkp728
1 Replies

3. Red Hat

How to find the process which is caused system hung state?

when system is hung state due to swap, we will reboot it through ILO. i want to know which process caused system hung. (1 Reply)
Discussion started by: Naveen.6025
1 Replies

4. Shell Programming and Scripting

Script to restart process

HI, I am trying to write a scrip which would restart active process. This is what i have written till now. $ xms show pr PRESE.* NAME STATUS ROLE PID RSTRTS STARTED MACHINE... (8 Replies)
Discussion started by: Siddheshk
8 Replies

5. Shell Programming and Scripting

Script to restart a process

I have written a script which checks for a file if that is being updated or not. If the files is not being updated then it will restart the process. #!/bin/sh DATE=`date +%Y%m%d%H%M%S` LOG_FILE=/var/xmp/log/XMP_* INCEPT=`ls -l $LOG_FILE |awk '{print $5}'` PROC=`xms show pr |grep -i... (3 Replies)
Discussion started by: Siddheshk
3 Replies

6. Shell Programming and Scripting

zombie processes and hung process termination

Is there a way I can run a command that will run in the kernel or in the memory and automatically kill certain scripts if they get to <defunct> processes, without having to be monitoring the server manually? I have a Perl script which runs for 20k members and normally does not have any problems,... (2 Replies)
Discussion started by: ukndoit
2 Replies

7. Shell Programming and Scripting

Monitoring for a hung process

A coworker has a shell script that runs from a scheduler at the 3am. The shell script runs sqlplus passing in a sql statement, which generate a file. This is done 21 times for 21 different sql statements. Recently, one of the sqlplus processes got hung. Is there a way to monitor how long the... (2 Replies)
Discussion started by: prismtx
2 Replies

8. SuSE

Restart process

I have a process that gradually eats up memory, it's currently at 80.2% and slowing down the linux server > ps aux | grep SNMPME root 3129 0.0 80.2 3591752 2480700 ? Sl Feb13 5:04 /opt/nampe/lib/snmpme/SNMPME config/startup.xml Is there a command I can execute to restart this... (3 Replies)
Discussion started by: brendan76
3 Replies

9. Shell Programming and Scripting

need help to write script to check the process health and automatically restart it

I am working on a project, which need to constantly watch the process, and check its status, if it was dead, it should be restart automatically. Please kindly refer me to URL which teach how to write this kind of script, or service. Thanks. (1 Reply)
Discussion started by: dragondad
1 Replies

10. UNIX for Dummies Questions & Answers

SCO Openserver 5.0.7 Hung process problem

Hi guys I installed this new server with 5.0.7 openserver and i'm getting a lot of this process, if a stop and restart the printer spooler they go away but after a few minutes they appear again.This is how it looks like. root 372 615 0 - - 00:00:00 <defunct> root ... (0 Replies)
Discussion started by: josramon
0 Replies
Login or Register to Ask a Question