Shell Programming and Scripting

Also, using find instead of ls can be better as ls will list directories, and find can be used for full paths.

so it can look like:


for F in `find /my/dir/ -maxdepth 1 -type f`
do
VAL=`lsof $F`
if [ -z $VAL ]
then
your_function_here $F
fi
done

Useful for "eye-balling" the status of a process or whatever. But: No way to trigger a command (automatically) upon a condition, and Do you really want to sit there and watch for files to come in? If you do, then :

Code:

watch ls -l $MAILBOX

is fine.

Never. You'd have to kill the process. It's equivalent to removing the while loop and putting the script as a cron job that runs ever 5 minutes.

It's running the pipeline of ls and grep. The grep is looking for any line. If ls finds no files, it outputs no line, and the grep search fails. If the grep search fails, it exits with 1 which (in BASH logic) means false, and so the if condition fails. If the grep finds at least one line, this means there are files and so the if condition succeeds and the THEN portion is executed.

Are there other files in the directory besides these?

There are at least four solutions to this. In the worst case, you can use what is suggested by ddreggors. Here are solutions to try:
ddreggors solution is a bit resource-intensive, and VERY linux-specific, but it will work if the uploading process does not close/reopen the file in between writes AND if it removes the file after failure (if the upload process is interrupted and the file is not completely transferred). It will fail if there are any other benign processes reading the incoming files. Here is a re-write of that solution which is more efficient (ie, doesn't use additional forks):

Code:

cd $MAILBOX
LSOF=/usr/sbin/lsof
for f in *.dat; do
  if  $LSOF $MAILBOX/$f ; then
     # trigger action goes here.
     sftp $MAILBOX/$f someone@somewhere:destination/$f
  fi
done

Nice rewrite and great points but I thought I might mention that in the quote above you make it seem like it will NOT work if "the upload process is interrupted and the file is not completely transferred". While that is not what one would want the process will still work, you will just end up with a blank or partially written file. What I mean to say is that the loop will still run and do what it is meant to do but it will preform the actions on a file that is not intact. This would happen with ANY process as far as I know. If the transfer fails then you have a bad file and no way to tell without looking inside manually (vi, nano, etc...) or diff against the original (which you might not have).

Now on the other hand, it would be a very nice feature if the upload/transfer process failed and the file was removed (as you already mentioned).

As to the Linux specific code... my bad, I am from a Linux world and should remember that nix* means *more* than Linux usually.

I also make the assumtion that this user wants a quick easy to create solution. I base this on very little knowledge granted but there seems to be an air of "quick dirty hack" (not to be taken as a bad thing, just a real world thing) written in the initial question.

You're right. I shouldn't have been so critical of your solution. The user clearly stated linux and flock (as a shell-script) is distribution-specific.

Thanks to both of you guys, you brought up some really valid points. After reading your points, I realized that a "quick dirty hack" might not be the best solution as it is prone to numerous errors, mainly the sftp portion kicking off before the files are finished transferring before being sent back out. So I am probably going to add as much logic as possible.

There will always be 4 files ending in *.dat, but the files names will change. They are the only files and after transferring the directory can be totally cleaned. Flock sounds like a good idea, but I 'm not sure if the host that initially sends the files supports flock. I think I can use the lsof and I was also looking into incron and fileschanged utilities. Have you guys ever used these within a script?

thanks

honestly I have never worked with either, but incron sounds like it is a good start:

(taken from Debian -- Details of package incron in sid)

- notifying programs (e.g. server daemons) about changes in configuration
- guarding changes in critical files (with their eventual recovery)
- file usage monitoring, statistics
- automatic on-crash cleanup
- automatic on-change backup or versioning
- new mail notification (for maildir)
- server upload notification
- installation management (outside packaging systems)
- ... and many others

but fileschanged may not help:

(taken from fileschanged)

- The file or directory that you want to monitor must exist at program start.

I am using rhel 4 so incron is not available. I have isntalled fileschanged and it will monitor the creation of files as well and I have tested it, but then it goes backthe same question before what if the next sftp step starts to process before the files are finished?

thanks