Defunct Processes


 
Thread Tools Search this Thread
Operating Systems AIX Defunct Processes
# 1  
Old 09-09-2008
Error Defunct Processes

Hi,

Can any one help me to get rid of defunct process on UNIX IBM AIX box. These processes started when the system was rebooted almost after 1 1/2 years. Once one defunct process is created then all the user ids get infected and in turn creates numerous defunct processes. We have tried rebooting the system many times but the defunct processes start coming almost 20 hrs after the reboot. When the system is back after reboot we dont see any defunct processes pilling at that time but it comes after 15-20 hrs and keep on increasing there after. Does any one have any idea what would be triggereing this and what measures can be taken to get rid of this ? This is affecting the whole sytem and application on it and we are not able to find any solution to this issues. Any help will be greatly appreciated.

Thanks.
# 2  
Old 09-09-2008
I have bad, worse and plain ugly news for you:

The bad news is that rebooting is the only way to get rid of zombies (this is what the "defunct" processes are usually called).

The worse news is that AIX is not to blame for these processes. The OS (read: the kernel) maintains a so-called "process table", where every running process gets an entry. All the processes are additionally organised in an hierarchy. At the root of this hierarchy is a process called "init" which is the process number 0. In fact when the machine is booted the first process to be started is this init, which in turn works through /etc/inittab (hence the name), and so on and so on until the system is up and alive.

When a user logs on there is a getty process (one of the childs of init), which starts a shell (now a child of getty) for the user, who now can start processes (childs of his shell), .... If a process is declared to run independently from the users shell (a background process or a process started with nohup) it is "passed over" to the init process as a child.

A Zombie is now a process which parent has died without cleaning up its leftovers. It still has an entry in the process table but this entry cannot be removed because the parent process which would normally control it is gone. From the whole tree some branch is cut off but a leaf from the branch is dangling - in the nowhere.

Now, after the bad and the worse, the very bad news: zombies are - excusively - a symptom of sluggish programming. A process is responsible for cleaning up its environment and if it doesn't do so on a regular basis its a case of "programmer has not found out how to program in the Unix environment yet". PEBKAC on behalf of your software vendor.

I hope this helps.

bakunin
# 3  
Old 09-09-2008
Like bakunin - I would suggest something that may not sit well either. If it started recently, the first and only place to look is at:

patches, upgrades, or new software

added to the system about just before the problem started.

Any new/updated cron entries, or new/updated scripts that run under "at" also need to be checked out. If you allow any of these on your system.
# 4  
Old 09-09-2008
I have seen this type of problem after a reboot in the following circumstances (granted, it was Solaris, but the principle is the same):
1. Server is booted
2. A new filesystem is mounted, but not updated in vfstab
3. A utility to do some task (like cleanup) is called from the new filesystem
4. Server is rebooted
5. The filesystem is not mounted at boot time
6. The utility is called to perform the cleanup task - but there's nothing there.
7. The application crashes quietly, you end up with defunct processes.
# 5  
Old 09-09-2008
Zombies have a living parent who can't be bothered to clean up. If that parent dies, it's surviving child processes, including zombies, will get a new ppid of 1 and now the parent will be init. init always reaps all child zombies.

So...want a zombie to go away? Look it up in ps and note the ppid. The is the pid of the misbehaving parent. Kill the parent. In a second or two the zombies should be gone.

At least this is the way unix is supposed to work. Is AIX really different than this?
# 6  
Old 09-09-2008
Quote:
Originally Posted by Perderabo
At least this is the way unix is supposed to work. Is AIX really different than this?
In (more than) one word: you don't want to know. ;-)) *)

Some (few) zombies can be cleaned up this way, most can't. I don't know why, but common knowledge and my experience is "zombies can only be cleaned by a reboot".

bakunin

___________
*) Years ago, when AIX 3.2.5 was current, there was a saying that this was the best MVS IBM had ever built. Go figure....
# 7  
Old 09-09-2008
Thanks all for your thoughts ...
Can any one share me a hint as to why these processes are starting after 15-20 hrs of the reboot . This is been happening from last few days .. Is there any script or anything i should check which could generate this at this time ?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. HP-UX

How to clear defunct processes??

How to find out and clear the 'defunct' processes at OS and DB level ..?? (6 Replies)
Discussion started by: SurajR
6 Replies

2. HP-UX

Defunct Processes; Maximum Process

I'd like to remove this defunct processes without rebooting. Or, I could remove this defuncts when the maximum allowable process is nearly reached, then send an email to the user/team. How can I determine the maximum allowable process that a server can handle? Any thoughts? Jin@PRODUCTION: >... (3 Replies)
Discussion started by: Jin_
3 Replies

3. UNIX for Advanced & Expert Users

defunct processes?

HiI had a tool fail recently, on analysis I found it was cleaning up orphaned directories that had been created by specific processes that had died for some reason, thus failing to clean up after themselves.The directories were of the form /dir.pid. The tool would look to see if any instances of... (2 Replies)
Discussion started by: steadyonabix
2 Replies

4. HP-UX

Defunct processes are getting generated

Dear Sir / Madam, We have two socket programs which are running in background. These two socket programs are creating defunct processes in HP-UX which were not happening in Compaq True64 UNIX. What is the root cause of these defunct processes and how can we correct it? Your suggestion will be... (3 Replies)
Discussion started by: kcsahoo
3 Replies

5. Shell Programming and Scripting

Abnormal behaviour of Defunct processes.

Hi All, Sorry to bother you all if my query is silly. Can you please clarify my doubts on defunct processes. Actually coming to the scenario, i have a server which is under cluster environment. Server B is having problems with the defunct process. There was a cron scheduled on Server B which... (0 Replies)
Discussion started by: reddybs
0 Replies

6. UNIX for Dummies Questions & Answers

whats the difference between zombie orpha and defunct processes

can some one please explain zombie orphan defunct and how they r related (3 Replies)
Discussion started by: pbsrinivas
3 Replies

7. UNIX for Dummies Questions & Answers

Question about defunct processes

Hey guys/gals. I am new to the site and fairly new to Unix/Linux. I have a master netbackup server (solaris) that I get pinged on occasionally regarding defunct processes. (Usually over 50 or so of them.) I know what a defunct process is, but what is it a product of? Just plain sloppy code? (From... (1 Reply)
Discussion started by: EMCSANMAN
1 Replies

8. Solaris

how do I kill defunct processes?

mqm 17700 16815 0 0:00 <defunct> kill -9 does not work, even as root (10 Replies)
Discussion started by: csaunders
10 Replies

9. Programming

defunct process!!

how can i assure that daemon process which is being run as init child,can be removed immediately from system when it goes defunct or to avoid daemon process becoming defunt? (5 Replies)
Discussion started by: rish2005
5 Replies

10. UNIX for Dummies Questions & Answers

<DEFUNCT> Processes

When I ps -ef I see about 3 or 4 <DEFUNCT> things - what are these, and what causes them? Are they a concern? How can I fix them? Thanks, Michael (4 Replies)
Discussion started by: cuppjr
4 Replies
Login or Register to Ask a Question