Does rsync check and ignore files that already exist?


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Does rsync check and ignore files that already exist?
# 1  
Old 09-04-2015
Does rsync check and ignore files that already exist?

Hi,

We have two (2) servers named primary and standby. There is a directory named /db01/archive that we need to keep in-sync.

Files get transferred from primary and standby. Sometimes when we do a failover or when there is a network issue, some files fail to get transferred.

I want to use rsync to sync the files between the primary and the standby. Does rsync checks and ignores files that already exist on each server or does it check if a file has been modified, is therefore newer and then copy it over?

For example, if file01 to file10 are on the primary and file01 to file05 is already on the standby and a network error happens where file06 to file10 were not transferred and I run rsync, can I use rsync to tell it to copy file06 to file10 because they are missing and copy file01 to file05 only if they had been modified and are now newer than the files that are already in the standby?

Similarly, if we failed over from primary to standby and standby has the file11 to file15 and file13 is missing on primary, can I this time run rsync to copy files from the standby to primary where it only copy over file13 and the rest of the other files BUT only if they had been modified and are now newer than the files that are in the primary.

Basically, what I am ultimately hoping to use rsync with is being able to tell it which of the servers is the source and which one is the target whenever I run rsync. I need to be able to do so since because the role between the servers changes when a failover occurs. Is this possible?

Any advice much appreciated. Thanks in advance.
# 2  
Old 09-04-2015
rsync is pretty smart. If all that changes is the meta data (like times), a full copy isn't done if the content is the same. For speed rsync by default looks at time and size to "skip" things it believes don't need to be examined further, but you can disable that with an option.
This User Gave Thanks to cjcox For This Post:
# 3  
Old 09-08-2015
Quote:
Originally Posted by cjcox
rsync is pretty smart. If all that changes is the meta data (like times), a full copy isn't done if the content is the same. For speed rsync by default looks at time and size to "skip" things it believes don't need to be examined further, but you can disable that with an option.

Thanks, had re-read the man page and I am able to test also using the -in dry run option.

I assume rsync will do a full-on sync between the source and destination. Quite embarrassing, I should have known it will consider the source as the 'master' and the destination as the 'slave'

Anyway, I am using rsync with the -niazv option and it starts as below:

Code:
sending incremental file list
.d..t...... ./
<f+++++++++ testdb_1_1.dbf

Do you know what the .d..t...... ./ mean? Just curious, whether it means this is the root/parent directory and it is skipping it?
# 4  
Old 09-08-2015
If something underneath a directory gets changed in certain ways a directory attribute might be changed. The first char is "." in your example which means nothing was transferred, so there might have been a modification time "t" attribute change made due to some change to something underneath the directory. That "<" on the file means a transfer was made, and so one of the likely culprits of the modification time change to the directory.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Check if 10 files exist

Hi All, Whenever i get 10 files(file names like sales*) then another file need to create. May i know how to implement this in KSH. (4 Replies)
Discussion started by: siddireddy
4 Replies

2. Shell Programming and Scripting

Rsync files check

Hello everybody, I sent some files a remote server using simple ssh command line: rsync -e 'ssh -p SSH-PORT' -vr --log-file=/var/log/rsync.log /home/USER/www/* USER@IP:/home/USER/www Then I would like to check if files in server1 are the same, file size check or any other way to make... (4 Replies)
Discussion started by: Abu Rayane
4 Replies

3. UNIX for Dummies Questions & Answers

Rsync copy files if dont exist

I have a setup where I have two drives. TV TVbackup For what ever reason, I have a lot of content on my TVbackup drive which isn't on my TV drive. I want to copy all the files across which are on TVbackup but are not currently on TV. If there is a file with the same name but a... (2 Replies)
Discussion started by: Spadez
2 Replies

4. Shell Programming and Scripting

Shell script to check files if exist else touch the file

Hi All, Thanks in Advance I wrote the following code if then echo "version is 1.1" for i in "subscriber promplan mapping dedicatedaccount faflistSub faflistAcc accumulator pam_account" do FILE="SDP_DUMP_$i.csv" echo "$FILE" ... (5 Replies)
Discussion started by: aealexanderraj
5 Replies

5. Shell Programming and Scripting

Check if file exist

Hi, I am trying to create a bash script which will check if file exist then remove that file else do nothing. I have to do same process for three files in same script. I have written code for one file and trying to run it. if then rm -r /user1/abc/File1 fi When I run this code it... (1 Reply)
Discussion started by: palak08
1 Replies

6. Shell Programming and Scripting

ignore fields to check in grep

Hi, I have a pipe delimited file. I am checking for junk characters ( non printable characters and unicode values). I am using the following code grep '' file.txt But i want to ignore the name fields. For example field2 is firstname so i want to ignore if the junk characters occur... (4 Replies)
Discussion started by: ashwin3086
4 Replies

7. Shell Programming and Scripting

check/wait for files to exist before continuing

I'm attempting to write a pretty simple script. It opens a Filemaker file successfully. That Filemaker file takes around 30-90 seconds to finish. When it's done, it writes a few .xml files into the same directory where my shell script and the Filemaker script reside. In my script, how can I... (2 Replies)
Discussion started by: alternapop
2 Replies

8. Shell Programming and Scripting

check the directory exist

I have the below script to check whether directory is exist or not , now I sure the directory /abc NOT exist , but when run the script , it still pop the result is "the directory exist" , could suggest what is wrong ? thx ll -d /abc > /dev/null 2>&1 if then echo "the directory exist !!" ... (7 Replies)
Discussion started by: ust
7 Replies

9. Shell Programming and Scripting

Check if certain files exist in a directory, if not add name to a textfile

We recieve some logs on our windows box via FTP on a daily basis, in the same directory. I would like to check for missing logs files and add their name to a text file. Something like... Check if C:\logs\file1_currentdate exists (if not, add file1_currentdate to... (1 Reply)
Discussion started by: SunnyK
1 Replies

10. Shell Programming and Scripting

How to check a file exist and do a copy of other files

Hi, I would like to perform bash which would check the file A.txt to be size 0 or not. If the size is 0, I would copy file B.txt to replace A.txt. Please help. Thanks. -Jason (6 Replies)
Discussion started by: ahjiefreak
6 Replies
Login or Register to Ask a Question