Shell Programming and Scripting

Hi all,

I have a shell script that I schedule to run from 0700 - 1900, every 15minutes. For each run it creates a lock file and remove it on exit.

While there is a bit of a flaw to it where the lock file don't get remove if someone kills the script while it is running, I am fairly satisfied with it. No one should be killing the script on purpose anyway unless the server itself rebooted and the script of course did not run its course where it is supposed to remove the lock file on exit.

Can someone advise how I can somehow check the timestamp of the lock file? For example, if the lock file is 30 or 45 minutes old, I assume the script may have been killed in which case instead of exiting because it exists, I want to remove it and continue the run of the script.

BTW, OS is Solaris 9.

Any advise much appreciated. Thanks in advance.

Code:

find /lock/file/location -type f -name "name.luck" -mmin -45 -exec rm -f {} \;

Eons ago I ended up writing C code to wrap execution of arbitrary commands with lock management just for cases like this. Meaning, I needed a way to coordinate the use of various scripts, tools, etc... that didn't already have a means of serializing their runs.

A couple of years ago I re-wrote that program and a few others and put them up on Github for anyone to grab. But I don't think you need to bother looking for packages to compile for what you want anymore. I just checked and found a command called "flock" that's already installed on my Ubuntu system that does the same sorts of things. So check "man flock" and see if it does what you want.

The main feature I think you need is for your "lockfile" to actually use file locking calls to coordinate access. In other words, the mere existence of the lockfile isn't enough to stop execution. What should block execution of another instance of the command is if you can't establish an exclusive lock on the file. And even if the previous command died without removing the lockfile, everything will just work. In fact you don't even need to remove the lockfile at all.

Instead of just removing the lock file, you might want to determine whether or not the script is still running.

A fairly common way to do this is to have the script that creates the lock file writes its PID into the lock file (echo $$ > lock). Then if the lock file is present when it isn't expected, you can cat the lock file and look for that PID in the output from ps to determine if the lock owner is still running or died before removing the lock file.

Even easier than using ps, you can use something like:

Code:

if kill -0 $(cat lock)
then    echo 'lock owner still running'
else    echo 'lock owner is gone'
fi

as long as the process that created the lock and the process running the above code are running with the same UID and permissions.

Use the script itself as the lock file:

Code:

#!/bin/bash

for pid in `fuser $0 2> /dev/null`; do
    if [ $pid -ne $$ ]; then
        echo "Script is already running, exiting..."
        exit 1
    fi
done

...

Here's a quick demonstration how "flock" can be used to coordinate serialized execution of a command. When you enter the following,

Code:

iter=0
while [ ${iter} -lt 10 ]
do
    flock -w 30 /var/tmp/test-flock.lock -c 'sleep 4; date' &
    iter="$(( ${iter} + 1 ))"
done

will start up 10 "flock" commands in parallel. But they will only run their respective payloads "sleep 4; date" when they can get an exclusive lock on the file "/var/tmp/test-flock.lock".

As a result, they will be serialized and run one at a time. You can tell that's the case because the time echo'd to your terminal will be 4 seconds apart.

Also, since each "flock" will only wait 30 second to get the exclusive lock, two of them will timeout and NOT run the "sleep 4; date" commands. And since it's using actual filesystem lock calls, you don't need to create or remove the lockfile. If it doesn't exist, it will be created for you. And there's no need to remove either.

Also, handling common signals might help you to do this automatically.
However, off-course its not possible when someone kills the script with kill -9 (SIGKILL) which is used quite often when its not necessary. Unfortunately it cant be trapped.

But, for other common signals, e.g. ctrl+c, you can do something like..

Code:

trap clean_jobs SIGHUP SIGINT SIGTERM

Code:

function clean_jobs {
  #rm locks
  #do other cleanup stuffs
}