OK, let us go over your provided outputs.

First: all the paging-related columns (re, pi, po, fr, sr, cy, in, sy) being 0 means that the machine has very much memory compared o what it needs. Its kernel doesn't even bother to look for pages it could steal, so the machine really must have plenty. Not even the file-cache seems to reach its saturation. Post the output of svmon -G and we could perhaps show you how much the machine needs and hw much it really has in comparison.

Second: if you look at the columns with the run- and blocked-queue (leftmost, "r" and "b") you see occasional 1s in the blocked-column. This is not a problem in and of itself, but one starts to wonder where it comes from. Nonzero entries in "b" mean that there is a process ready to run, which can't because of some outside factor prohibiting it. Usually this is a side effect of paging (the process waits until its memory is paged in again), but this is not the case here.

Third: now we inspect the rightmost part of the output, which shows how the processor(s) is used. "us" (process spends time in user space) and "sy" (process spends time in system space) are near 0, so the system does next to nothing. But "wa" (wait) is non-zero and this corresponds to the blocked-entries. It means that a process, otherwise ready to run, is waiting for I/O. So it looks like the machine is slightly I/O-bound. This could come from:

- disks (or SAN, whatever) pose a bottleneck
- network over which data are transferred is slow
- another I/O-path - serial line, whatever - is the culprit

Now to the Zombie-problem: when a process ends, it sets an exit code. If you run a system command at the shell level and query the errorlevel you query in fact the exit code of the program. When a program now calls another program (a "fork") it usually does so in a way that it gets this exit code upon termination of the child process. As long as the exit code is not queried by the parent process the entry in the process table remains.

Now it happens sometimes that a parent process terminates (voluntarily or involuntarily) before it can reap its children. These children processes become zombies, because nobody will ever query their exit code. The programs themselves are long gone from memory but the entry in the process table still exists and will do so sometimes until next reboot. It is difficult to remove them.

If your program creates such zombies on a regular basis then this is a case of very sloppy programming. I suggest beating your programmer with the print version of the AIX Programmers Reference on the head until he understands basic UNIX programming concepts.

I hope this helps.

bakunin