Linux

Hi everybody,

I want to find out all the processes that ran before a server crashed. Is that possible?

I've looked in /var/log/messages and found out that the system was out of memory.

A user probably wrote a script (in Perl or Python) that used up all available memory and crashed the server.

I'm using Red Hat Enterprise Linux Server release 5.4 (Tikanga).

Thanks in advance!

Dave

Hm, if you have nothing set up that tracks stuff like this and there is no more details in /var/log/messages, I would set up some ps aux, maybe in combination with pmap and compare the numbers of memory usage from a fresh start to when it is about to have low memory (check with free or cat /proc/meminfo) triggered by cron every 10 minutes maybe.

Hi zaxxon!

Thanks for the prompt response.

That's a great idea. So there is no native log file or daemon that tracks this sort of information in RHEL? I wanted to find out the script/user that used up all the memory, so I can avoid situations like this in the future.

If there is no such service, I will create the cron job.

Cheers,

Dave

I am not sure if RHEL does provide something specially for this case, but usually on Linux'es you see messages about processes that will be killed in a dire attempt to free memory. I usually noticed that by a box that was still up and running but had no sshd or rsyncd running anymore.

Are messages about processes being killed stored anywhere?

I did notice this line in /var/log/messages

Feb 8 19:46:18 computer-name kernel: Out of memory: Killed process 19136 (emacs-x).

But I don't know what process 19136 is.

Thanks again!

Dave

My crystal ball tells me it was emacs.

But the process killed isn't necessarily the one that caused the out of memory condition. The kernel tries to identify it but when the whole system is memory starved, EVERYTHING is fighting for memory...

debug0:2> eps() <Enter>

The eps() command will give you a process listing.

This works with SCO Unix boxes that have had a kernel panic.