rsync mirror of download directory


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting rsync mirror of download directory
# 1  
Old 04-29-2009
rsync mirror of download directory

I have a download directory for NDM Secure+ to receive files from other system. I need to mirror this directory to another box, but need to skip files that are still coming in.
I have no control over file names or destination directories as they are configured on the sending side.

How can I keep rsync from copying files that are still in the download process.

I run a cron every 5 minutes to keep files synced, is there something I can add as an exclude for these files?

Thanks
# 2  
Old 04-29-2009
This actually can be a hard problem. How are the incoming files stored? FTP ? SSH? shell script?

Can you run the command "lsof" ? If not, can you install it ? If you can run it, try:
Code:
 DIR=<dir-to-mirror>
 /usr/sbin/lsof -F nc  +D $DIR  |sed -n 's/^n//p;' |sort -u

You can output that to a file and had it to rsync via --exclude-from=FILE option.
Don't forget to put the output outside of $DIR and remove it after each rsync.
# 3  
Old 04-29-2009
NDM is just SCP on steroids I think, so the files would be coming in via ssh.

The files actually have a header and trailer stamp and I was going to write a script to look for both before syncing. I was hoping I had just overlooked a basic configuration option.
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Rsync Error: rsync: link_stat failed: No such file or directory (2)

I wish to copy all the files & folder under /web/Transfer_Files/data/ on mymac1 (Linux) to remote server mybank.intra.com (Solaris 10) /tmp/ location I am using Ansible tool synchronize module which triggers the unix rsync command as below:rsync --delay-updates -F --compress --archive --rsh=ssh... (2 Replies)
Discussion started by: mohtashims
2 Replies

2. UNIX for Beginners Questions & Answers

Move directory with rsync and delete from source

I need a rsync command which will exclude certain files and directories from source and copy the rest. I got this command working, sudo rsync -avzh --exclude 'bin' --exclude 'braintree' --exclude 'colinmollenhour' --exclude 'composer' --exclude 'doctrine' --exclude 'fabpot' --exclude... (2 Replies)
Discussion started by: Siddheshk
2 Replies

3. Shell Programming and Scripting

Rsync defined directory

i want to copy with rsync a specified (sub)-directory and the according folders and files below. my test failed : cd /dir /usr/bin/rsync -ravz -e ssh --include='*/' --include='*/00/*/*' --exclude='*' . remote_host:/tmp i want to copy in these example the directory "00" and the according... (2 Replies)
Discussion started by: bora99
2 Replies

4. HP-UX

What is the difference between DRD and Root Mirror Disk using LVM mirror ?

what is the difference between DRD and Root Mirror Disk using LVM mirror ? (3 Replies)
Discussion started by: maxim42
3 Replies

5. Shell Programming and Scripting

rsync: skip mail directory

i want to skip the /home/user/mail directory in an rsync but only in this level not like /home/user/public_html/anotherscript/mail if i have a command like this rsync -av --exclude mail /home/$USER root@$SERVERIP:/home2 does it exclude only the mail directory directly under... (1 Reply)
Discussion started by: vanessafan99
1 Replies

6. Shell Programming and Scripting

wget command help to download packages from mirror

Index of /distribution/openSUSE-stable/repo/oss/suse/i586 How can i download packages with wget from that mirror with a string "alsa"? Such as: alsa-1.0.23-2.12.i586.rpm alsa-devel-1.0.23-2.12.i586.rpm alsa-oss-1.0.17-29.2.i586.rpm Not manually downloading packages. But i'm trying to... (9 Replies)
Discussion started by: cola
9 Replies

7. UNIX for Advanced & Expert Users

How to rsync or tar directory trees, with hidden directory, but without files?

I want to backup all the directory tress, including hidden directories, without copying any files. find . -type d gives the perfect list. When I tried tar, it won't work for me because it tars all the files. find . -type d | xargs tar -cvf a.tar So i tried rsync. On my own test box, the... (4 Replies)
Discussion started by: fld2007
4 Replies

8. UNIX for Advanced & Expert Users

rsync deletes original directory

Hi I have a strange problem. Sometimes when I execute the below command something wierd happens. rsync -avz -e ssh travegre@travegre.net: ../travegre.net/ the folder named "hm" that is held in travegre.net and is coppied along with all the other folders and data at travegre.net, gets deleted... (4 Replies)
Discussion started by: travegre
4 Replies

9. UNIX for Advanced & Expert Users

Why rsync deletes source directory?

Hi I have a strange problem. Sometimes when I execute the below command something wierd happens. rsync -avz -e ssh travegre@travegre.net: ../travegre.net/ the folder named "hm" that is held in travegre.net and is coppied along with all the other folders and data at travegre.net, gets... (1 Reply)
Discussion started by: travegre
1 Replies
Login or Register to Ask a Question