Shell Programming and Scripting

I'm changing my mindset from a few big processes moving data from a few sources under an external, dependency-based scheduler to multiple processes moving data from many sources run by each client cron and possibly interfering with each other. It has the benefits of more granular code but I'm worried about tripping myself up. The incoming data will be delivered (atomically) into a known directory, but we cannot predict when so we have the script to process & remove them run every few minutes. The problem is that it could run a long time if we have a huge volume of data. I am therefore looking at locking mechanisms.

The flock manual page refers to wrapping up commands in parenthesis , redirecting a file descriptor to a file, something like this:-

Code:

(
flock -s 200
    # ... commands executed under lock ...
) 200>/var/lock/mylockfile

It has been suggested elsewhere that one could use braces {&} to acheive the same and avoid spawning another process. Sadly, both of these would make my code a little ugly because I want an exclusive lock in force for almost the whole script. It's not massive (only about 600 lines) but it adds an indentation I'd rather not have and I'm torn because I want to be good about indenting code too.

Additionally, whist debugging, I don't want to have to worry about where the end of the block of code is and generating very confusing syntax errors.

I have discovered that this seems to work just as well and doesn't leave me feeling I should be indenting the code wrapped in parenthesis/braces or worried when looking to grow/debug:-

Code:

#define any functions first

# Set/test lock
exec 200>/var/lock/mylockfile
flock -w 1 -x 200 || exit                 # Do not run concurrently

# My main script starts here
sleep 10                                  # Just for illustration, honest!
# My main script ends here

flock -u /var/lock/mylockfile             # Is this necessary?

The script is simply run by cron and I don't want more than one at a time, hence the exclusive lock. There would never need to be output written to the lock file.

So, my questions:-



Thanks, in advance,
Robin