UNIX for Advanced & Expert Users

I have some Solaris processes that run weeks at a time that create rather large log files that I would like to archive/compress daily. Instead of stopping the process, what can be done so that the log file is backed up and shrunk, but the process can still log to the open file handle without major interruption.

Depending on the process, it may be possible to send it a HUP to have it reopen it's filehandles.

If that's the case, the steps are relatively simple:

1) move the active file to a different name (process still logging to it)
2) HUP the process (closes current filehandle, opens new one to new file)
3) Then, compress the old file.

Some programs support this - others don't. Some require that the new file be present and have the correct permissions before opening it - others don't, and will create a new file automatically on a HUP.

Best of luck!

Hey correct me, if wrong.
But default action of SIGHUP is abrupt termination of process, unless "handled".
So , if the process, which are logging entries in log file, do not handle SIGHUP, it will not work out.

Yes, that's mostly correct. It's not abrupt termination, like a SIGABRT would be, but it will terminate if it's not handled.

If you've got some means of starting the application using different parameters, so you can set up a dummy instance to test with, you can see how it handles things.

The SIGHUP/SIGABRT solution isn't going to work for some of the items. Is there another solution, like perhaps something where I can lock the file to new updates from the active process, copy it, then somehow zero the file out, then remove the lock.

Just to be clear, SIGHUP and SIGABRT are two very different things. I was just trying to say that while the default action of SIGHUP is to exit the process, SIGABRT actually kills it without calling exit, so it doesn't hit any cleanup/shutdown code in the process, which SIGHUP would.

That being said...permit me to delve into details.

For the purposes of this discussion, there are two general ways that a process might be writing to a file.

1) The file is opened, written to, and closed each time a write is needed.
2) The file is opened at some point, and kept open for the life of the process, and writes happen during that period.

If your process does #1, then you should be able to move the file to a different name, and you won't lose any data. A write that's current taking place when you move the file to a different name won't care (since it's still got an open filehandle) and the next open/write/close will (hopefully) create a new file at the correct name.

If you have a process doing #2, then if you move the file, the filehandle will still point to it, so you will have to either HUP (if the process handles it) or terminate and restart the process to get the process to close the current filehandle, and open a new one to the right name.

So, if it's #2, you have a few options, besides restarting the process.

You could, instead of logging to a file, log to a named pipe (a "fifo") and then have a different program read from the pipe and write to a file, which could then have the needed signal handling to deal with closing and opening a new filehandle on the HUP.

You could write to a socket, and do much the same thing.

You could also create a device and log to that...this is pretty much how /dev/log works...on Linux at least.

An example of how this works, for you to play with:

Code:

$ mkfifo mypipe
$ while true; do echo `date` > mypipe && sleep 1; done

(from another window)

Code:

$ tail -f mypipe

I'm not actually sure if mkfifo is the right command on Solaris. Also, the syntax above is for bash, FYI.

Honestly, this is somewhat kludgy, and takes some significant systems programming chops to do robustly. Which is why well written unix programs that do logging to files should, IMHO, support HUP as a means to re-open log files. It's really the most straightforward method.

---------- Post updated at 12:20 PM ---------- Previous update was at 12:08 PM ----------

Oh, and as far as what you propose goes...

File locking is not all that easy to manage.

But, you could perhaps

1) send a SIGSTOP to the process (i.e. ^z, suspend it)
2) copy the data from the file to a new one.
3) cat /dev/null > file (zeroing it out)
4) send a SIGCONT to the process to have it resume

Some programs don't tolerate being STOPed very well. Some terminate when that happens. I have no idea what will happen with yours.

If the process isn't designed with having its log files rotated out from under it, even trying SIGSTOP/process file/SIGCONT may not work. For example, if the file isn't opened in append mode, you'll just wind up with at best a sparse file with nulls where the data used to be as the process picks up writing at the same point in the file it would have written anyway.