UNIX for Dummies Questions & Answers

Hello,

I hope someone can explain something to me just so I can understand why this took place:

We have Avaya telephony servers that are running RHEL 5 on them. A week ago, callers were dialing into the server and could not hear a ".wav" file that was supposed to be played in an AVP IVR. Not real important.

Come to find out, the / mount was out of storage. The strange thing was nothing really was adding up. I performed the df and du commands. I also performed the find command with the size flags to see if there were some large log files . . . nothing really showed up that was large. The / mount btw was 20 Gigs.

While doing a ps aux command I saw some tcpdumps taking place (over a year ago). It was piping the output to the tmp directory. I looked and these traces were NOT in the tmp directory.

Once I killed the processes, the storage came back!!!!

I just was hoping that someone could tell me where these files were at because I sure didn't see them. I can only assume that the trace was taking place, then an admin forgot to kill them. Then maybe some /tmp cleanup job removed the files but had I not performed the ps aux command I would have never known about this.

I understand that a reboot would have absolutely fixed this problem, but I hate to reboot unless absolutely necessary.

Thanks for sheding light on this. Linux is a learning experience for me every day and I always learn something valuable from these forums!

Tim.

On Linux, if you delete ("unlink") a file while a process has it open, all that happens is that the directory entry is removed but the file itself remains until the process closes the file-descriptor. At this point, if there are no directory entries referencing the i-node of the file, the file itself is deleted.

Before the tcpdump processes were killed, you could have seen this using the lsof command.

I would guess that someone deleted the files but didn't kill the processes. The processes went on happy writing to the "deleted" files and filling up the filesystem until you came and killed them. Since killing the process implicitly closes all FDs, the files were finally deleted from the filesystem.

Does this help?