Sponsored Content
Full Discussion: script to monitor directory
Top Forums Shell Programming and Scripting script to monitor directory Post 302227320 by otheus on Thursday 21st of August 2008 04:00:45 AM
Old 08-21-2008
Quote:
Originally Posted by nulinux
what about using the watch command?
Useful for "eye-balling" the status of a process or whatever. But: No way to trigger a command (automatically) upon a condition, and Do you really want to sit there and watch for files to come in? If you do, then :

Code:
watch ls -l $MAILBOX

is fine.

Quote:
also in your while statement, when would the condition not be true?
Never. You'd have to kill the process. It's equivalent to removing the while loop and putting the script as a cron job that runs ever 5 minutes.

Quote:
What is your if statement actually doing?
It's running the pipeline of ls and grep. The grep is looking for any line. If ls finds no files, it outputs no line, and the grep search fails. If the grep search fails, it exits with 1 which (in BASH logic) means false, and so the if condition fails. If the grep finds at least one line, this means there are files and so the if condition succeeds and the THEN portion is executed.

Quote:
Originally Posted by nulinux
We do remove them after transfer. another note is that although the names will change as will the byte size, they will always end in *.dat.
Are there other files in the directory besides these?

Quote:
Originally Posted by nulinux
One of other problems is as you mention what if the files are detected before they are finished transferring to the host, before they are sent back out?
There are at least four solutions to this. In the worst case, you can use what is suggested by ddreggors. Here are solutions to try:
  1. Move-after-write. Modify (or configure) the process that places the incoming file. When creating the file, it names the file in a distinctive way (different extension, prefixed with ., different path, etc). After closing, it moves / renames the file in a way your script expects.
  2. Keep control-log file. Modify (or configure) the process that places the incoming file. After closing the file, it appends the name of the file to a control file, kept inside the mailbox folder (as "something.ctl"). Your script will rename this file, and then read it for a list of file names to sftp.
  3. Use flock. This will only work IF the process, which places the incoming files, uses flock(2) on files that it creates. Use the script to use flock (shell command) to each file before it is moved.

ddreggors solution is a bit resource-intensive, and VERY linux-specific, but it will work if the uploading process does not close/reopen the file in between writes AND if it removes the file after failure (if the upload process is interrupted and the file is not completely transferred). It will fail if there are any other benign processes reading the incoming files. Here is a re-write of that solution which is more efficient (ie, doesn't use additional forks):

Code:
cd $MAILBOX
LSOF=/usr/sbin/lsof
for f in *.dat; do
  if  $LSOF $MAILBOX/$f ; then
     # trigger action goes here.
     sftp $MAILBOX/$f someone@somewhere:destination/$f
  fi
done


Last edited by otheus; 08-21-2008 at 05:01 AM.. Reason: formatting fix
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Hep with script to monitor directory

Hello, I am a newbie who is attempting to write a script to monitor a directory for a set of 3 files that I am expecting to get ftp'd. Occasionally, we suspend operations for maintenance etc. but we still get the files so there can be more than 1 set. If there is more than 1 set, I would like... (2 Replies)
Discussion started by: cmf00186
2 Replies

2. Shell Programming and Scripting

script to monitor files in a directory and sending the alert

Hi All, We are having important config files in an directory which was accessable by all /auto/config/Testbed/>ls config1.intial config2.intial config3.inital often we find that some of the lines are missing in config files, we doubt if some one is removing. I would like to write... (0 Replies)
Discussion started by: shellscripter
0 Replies

3. Shell Programming and Scripting

Monitor capacity of directory

Good morning. I have been attempting to find a way to monitor the capacity of a directory so that when it reaches 80% or higher I can send an event. I was able to find a script that does this for the whole drive by I can not seem to figure out how to do this for just a single directory. ... (1 Reply)
Discussion started by: LRoberts
1 Replies

4. AIX

monitor directory events

I'm am looking for a cheap way to trigger a script when a new file is written in a specific directory. AIX 5.3. It is a production system, so no kernel patching (i.e. inotify). Filemon and audtiing are too expensive. Thanks in advance. (2 Replies)
Discussion started by: pbillast
2 Replies

5. UNIX for Dummies Questions & Answers

Monitor directory and email

Hello all, Can anyone please guide / help me in the following task.... I have a directory where some external users will upload pdf files. The filename of these pdf will be of a particular format (<id>-<first name>_<last name>_<some number>.pdf) I want to make a script such that it takes... (6 Replies)
Discussion started by: dhawalkv
6 Replies

6. Shell Programming and Scripting

Directory Monitor in KSh

Hi, I need to write a directory monitor, i.e. a korn shell script which would Report changes to the directory contents, like: added file1, deleted file2, updated file3 , created subdir (optional)... There is no specific file pattern. So far I have written a little script... (1 Reply)
Discussion started by: olegkon
1 Replies

7. Shell Programming and Scripting

Script to monitor directory size of specific users

Hi, i am new to shell scripts, i need to write a script that can monitor size of directory of specific users. Please help. Thanks, Nitin (2 Replies)
Discussion started by: nicksrulz
2 Replies

8. Shell Programming and Scripting

Shell script to monitor new file in a directory and mail the file content

Hi I am looking for a help in designing a bash script on linux which can do below:- 1) Look in a specific directory for any new files 2) Mail the content of the new file Appreciate any help Regards Neha (5 Replies)
Discussion started by: neha0785
5 Replies

9. Shell Programming and Scripting

awk to monitor size in directory

i'm trying to find the most efficient way to monitor specific files in different directories in one go. /var/log/ /var/app/ /var/db/ each one of these directories can have subdirectories which need to be looked into as well. I want to find any file in this directory that has the name... (7 Replies)
Discussion started by: SkySmart
7 Replies

10. Shell Programming and Scripting

Bash script monitor directory and subdirectories for new pdfs

I need bash script that monitor folders for new pdf files and create xml file for rss feed with newest files on the list. I have some script, but it reports errors. #!/bin/bash SYSDIR="/var/www/html/Intranet" HTTPLINK="http://TYPE.IP.ADDRESS.HERE/pdfs" FEEDTITLE="Najnoviji dokumenti na... (20 Replies)
Discussion started by: markus1981
20 Replies
All times are GMT -4. The time now is 06:48 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy