04-06-2006
82,
3
Join Date: Mar 2006
Last Activity: 17 June 2019, 11:06 AM EDT
Posts: 82
Thanks Given: 1
Thanked 3 Times in 2 Posts
Building a better mouse trap, or How many lines of code does it take to trap a mouse?
Hello all,
I'm hoping to get a little insight from some of the wily veterans amongst you.
I've written a script to check for new outgoing files to our vendors located on our ssl server. It seems to be working ok, but the final question here, will be one of logic, and/or a better way to do it.
First a little background; The program is run every 5 minutes from cron. The files are uploaded via NFS or CIFS. So file dates can't be fully trusted. So, I use find -cmin for the dates. Files remain on the server for 10 days.
Process;
1) Check for PID file. If PID file exist, exit. (program still running) If not generate PID file.
2) Check filesystem size for changes since the last run. If no changes, clean up PID file and exit. (No new files) If changed sleep 1 minute. (File(s) may still be transferring) Loop until changes stop. Add total sleep time to find time. Continue to step 3 (Transfer done)
3) Using the find command. Build a file containing the list of new files in ftp directory newer than specified cmin time.
4) Filter through the file built in step 3. Generate email for each vendor with file names and send to contact for vendor.
5) Clean up PID file. Copy stat files to backups for comparison on the next program run. exit.
Like I said, this is working, but a few files slip through the cracks.
What I would like to know is: If you have any thoghts on better ways to do this.
One Idea I've been looking into is:
Generate a full file list every 5 minutes and use diff to generate the outgoing file list?
Also, This started out as a small server. So, checking for filesystem changes was no problem. Now I have roughly 180 vendors accessing the site. With all the changes to the filesystem size the program will somtimes run for 15 - 20 minutes. Regardless of how the list is built. I would think that once it is generated I could just check file sizes on those files for changes. Once they finish transferring, generate the mail, and wait for the next go-round to pick up additional files.
So what's the general consensus? Thoughts, Ideas, Opinions?
Thanks in Advance,
MPH
I'd rather have a bottle in front of me, than a frontal labotomy.
