Shell Programming and Scripting

Hello Folks,

Looking for a script, where i can check for the existing of the files in a directory and copying to another.
How can i achieve it in loop for over period of time.

whatever files comes into the folder copied in another without duplicate and should be continuous loop.


Regards,
Sadique

Dear sadique.manzar,

I have a few to questions pose in response first:-Most importantly, What have you tried so far?

There are probably many ways to achieve most tasks, so giving us an idea of your style and thoughts will help us guide you to an answer most suitable to you so you can adjust it to suit your needs in future.


We're all here to learn and getting the relevant information will help us all.


Kind regards,
Robin

hereis what i am trying:

Code:

#!/bin/sh
DATE=`date +"%d-%m-%Y %H:%M:%S"`
path=/xyz/S1
while true
do
	for  files in `printf "%s\n" $path/*.RTM 
	do
		if [[ -f $files ]] && [[ -s $files ]]
		then
			cp -n $files  /xxx
		else
			echo "file doesn't exist" >/dev/null
		fi	
	done
done

The main things to consider are:

What about files that get modified? Do you want to copy them too? Or do files in this directory never get modified?

What about files that are still being written to/created? Do you want to copy these incomplete files regardless?

What lag in the copying can you tolerate? If a cron job ran every 5 minutes would that suffice? If you are trying to mirror in real-time then some mirroring suite would probably do the job better.

Dear Hicksd8,

First of all nice question.



Regards,
Sadique

This could prove quite difficult if the length of time between the file being a .temp and being moved out is just a few seconds.

Do you have any access to the script(s) which is either creating the files or moving the files? It might be easier to rename the files from .temp to .temp2 and you copy all .temp2 files, then rename them suitable for the move script. Can you get into the overall chain of the process?

I'm scratching my head as to how you can guarantee catching all files if they only exist for a second or two. It would only take your script not to be scheduled (due to CPU load or whatever) for a few seconds in order to miss files going through.

Would iWatch be a possible help here? I think you can defined directories to watch and can take action when a file is updated.

The question is also 'What removes the files?' You may still miss files if you don't get to them before the delete happens. How much slack are we thinking? If this is (and I'm just guessing) an FTP server that people deliver to and there is another server that polls it to collect and remove files then there will be a race to get to the file, which is never a good position to be in.

Perhaps your process could be adjusted so that there is an in-bound directory for clients to create files and an out-bound for something to poll and remove the files (presumably after reading them) If you can separate these and have your code do the move after it has done the action you want, this might give you a working solution.

How time critical is the movement of files? If you are trading shares or processing payments then maybe your process could copy the file to the out-bound directory and clean up the in-bound file when it has done what extra you need.


Can you elaborate on what is going on?



Thanks, in advance,
Robin