check if some file is in copy process, then transfer it
my user copy large files, and it's take 10min for file to be copied to the server (/tmp/user/ files/), if in the meantime start my scheduled script, then it will copy a part of some file to server1
my idea is to check the file size twice in a short period (1-2 seconds) of time, then compare, if the size is different then script transfer1.sh will copy it the next time wen is started from cron,
If anyone has an idea how to do this or any other suggestions?
PS another thing, files name will be random, but every file have "TEST" in name.
when my script run there will be about 5 files to transfer, and this control is just for case if one of the files is in copy process, but other files must be transfered to server1
30 * * * * /myscript/transfer1.sh
example, folder contains /tmp/user/ files/
TEST1.out
TEST2.out
TEST3.out
TEST4.out (example this file is still copying and will bee skiped with transfer to server1)
after transfer delete all files except TEST4.out, that file will bee copyied next time when script is started
Bad idea -- how long is long enough? Best idea: write under another name like same.temp and then rename when done. Second choice: use fuser to see if it is open, but even if not open, file may be incomplete due to a failed writer, unless you have a way to validate it, such as a trailer record.
Instead of checking the file size you should check if the file is open. You can use the fuser utility to do that:
Tested with bash on OpenBSD
---------- Post updated at 12:57 PM ---------- Previous update was at 12:54 PM ----------
Quote:
Originally Posted by DGPickett
Bad idea -- how long is long enough? Best idea: write under another name like same.temp and then rename when done. Second choice: use fuser to see if it is open, but even if not open, file may be incomplete due to a failed writer, unless you have a way to validate it, such as a trailer record.
PS: Comment s/b after the do line!
Agreed. Copy/rename is the fool proof way to avoid problems.
Well, some people send additional short ack files, perhaps with a line or byte count or checksum, but that is excess complication if you can rename.
Since fuser puts the pids on stdout and the rest (path name, colon, open type indicators) on stderr, you can just test for pids or you can ps -fp the pids to show who is on the file!
another thing, files name will be random, but every file have "TEST" in name.
when my script run there will be about 5 files to transfer, and this control is just for case if one of the files is in copy process, but other files must be transfered to server1
Hello Guys,
I am trying scp few file within if statement, but getting error, can someone please help to understand, what mistake I am making ?
if && ] ;
then
echo " Files transferred to Log servers successfully. "
else
echo " One or more file transfer failed over... (10 Replies)
Hi Unix community,
I got this code from you guys and I tried to modify it to use for my csv dir transfer
basically i want the .csv file to copy itself and populate it to the archive dir.
#!/bin/ksh
dir1="/home/pumela/unixtestprod"
cd "$dir1"
echo "code is running"
for srcd in... (13 Replies)
In shell script, I want to transfer files continuously and make sure transfer is successful.
Please advise... how to make sure ftp transfer is successful?
Also is there any option such as sftp -b where I can pass multiple put <file name> commands to ftp
Thanks! (1 Reply)
All,
I have to write a script to do the following requirement.
There is a file called BUSINESS_DATE.TXT.
This file get updated once the oracle partition created. In Oracle, Partition will be created every day. There is a seperate script scheduled to take care ORACLE partition creation.
The... (3 Replies)
Using RCP command we can transfer file from one server to another server.
While transferring we can rename the file also e.g.
File name = FILE123.txt (lying on Source server = oldserver)
Target Server Name = newyour
Renamed File = FILE456.txt
rcp FILE123.txt newyour:./FILE456.txt... (1 Reply)
Hello,
Can I copy a binary file while the file is being written to by another process?
Another process (program) “P1” creates and opens (for writing) binary file “ABC” on local disk. Process P1 continuously write into ABC file every couple of seconds, adding 512-byte blocks of data. ABC file... (1 Reply)
Hi all,
I have a script that is monitoring a hot folder. This script works fine with one exception when the script is executed while a file is being copied to the hot folder.
What is the easiest method to check if the copy file is completed? I'd like to get the solution in bash :) (8 Replies)
Hi everyone. I am trying to write a bash script that will copy files from one directory to another but I need to be able to check the directory that I'm copying the files to and see if the file already exists. If it does I need to add a number at the end of the copied file. Thanks for your help. (3 Replies)
Hi,
I would like to perform bash which would check the file A.txt to be size 0 or not. If the size is 0, I would copy file B.txt to replace A.txt.
Please help.
Thanks.
-Jason (6 Replies)