Shell Programming and Scripting

HI,

I am running a shell script on a folder where our sources files will be published for every 5 mins, I am copying all the source files(i.e .csv format) to a .txt as ls *.csv > camp.txt.

Now when the above command is executing some files which are not copied completely , their name is also getting captured in the camp.txt, is there any way we can exclude all the files which are copying state i.e files which are not copied completely from source to destination can be excluded from writing into the camp.txt


example

a.csv copied completely
b.csv copied completely
c. csv coping started but not finished 100%

my result should include.

camp.txt
a.csv
b.csv


please help me in how to exclude the files in the copying state.

Thanks,subhasri

Assumin that this "ls" is running on the destination directory independently of the copy process and sometimes executes while the copy process is still running or perhaps after the copy process has failed.
I guess that you will need to copy the files to a temporary filename, then rename the files to their correct name after to copy completes. The convention is to append ".tmp" to the real filename (e.g. a.csv.tmp).

so how about this way:

Prepare pair of cvs file, and its md5 file, every name.cvs should have a md5 file: name.cvs.md5

so before list the cvs name to camp.txt, check its md5 hash key in name.cvs.md5 is same as the real md5 hash key.

HI,

I didn't get the .tmp thing, when the source files are in copying state in the source directory they are not having any .tmp extension , this process will be required for a datamigration application so after the file is copied in the source directory with in 30 mins it has to be loaded in the table so copying the source files to temp location will delay that process.

And I haven't got the md5 hash key stuff, how can we check that can you please give me some more details.

Thanks,
Subhasri

in solaris system, there is a command md5 (or md5sum), maybe the name is different in other systesms.

Code:

$ md5 1.csv
9c02cfb4ae36f88fcf71c8047169da74        1.csv

you can build the md5 key list in pair of csv file

before list, check the md5 key if they are same.

The original proposal was to give the file a different name when it is being copied and then move to the final name once done. eg: mv /my/migration/dir/Monday_0124.txt.tmp /my/migration/dir/Monday_0124.txt

The mv command should operate pretty much instantaneously when the source and destination filenames are within the same filesystem, as this only involves an update of the File table, not a physical move (or copy) of the bytes within the file.

If you can't control the published file extension (i.e. the .tmp suggestion), you might check if the file is still open with the fuser command and discard any file in the o (open) state.