Rsync files check


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Rsync files check
# 1  
Old 11-29-2016
Rsync files check

Hello everybody,

I sent some files a remote server using simple ssh command line:

Code:
rsync -e 'ssh -p SSH-PORT' -vr --log-file=/var/log/rsync.log /home/USER/www/* USER@IP:/home/USER/www

Then I would like to check if files in server1 are the same, file size check or any other way to make sure everything is fine.

How can I do this kind of comparison?

Thanks in advance
# 2  
Old 11-30-2016
rsync does that, called a hash code. If the hash code fails rsync barfs all over the place. However to answer your question:

There are several programs for this, but cksum is more than adequate.
Example
In the oldfile directory
Code:
cd /path/to/oldfile
old=$(cksum oldfile)
cd /path/to/new/file (or use ssh for a remote file): new=$(ssh USER@IP cd /path/to/newfile && cksum newfile)
new=$(cksum oldfile)
# you get output that looks like this :
# echo "$old" 
# 2201537306 122235 oldfile
# first number == checksum, second number == line count, name of file
# test example code 
if [ "$old" = "$new"  ] ; then 
   echo 'OK'
else
   echo 'NOT OK'
fi

Beware that ssh can add extra text all by itself - nothing to do with cksum
This User Gave Thanks to jim mcnamara For This Post:
# 3  
Old 11-30-2016
I think I should loop through the specified directory and get all files, this is my last step:

Code:
dirName='/root/Scripts/Misc/'

cd $dirName


listFiles=$(find $dirName -iname '*' -exec ls -l {} \; | awk '{print $9}')

md5=$(cksum $listFiles)

But it does not look to work properly
# 4  
Old 11-30-2016
If you want the md5 operation output, use md5sum instead of ckcum

If you have lots of files and you should be ensuring that they are identical (i.e. no files missing or extra on the target) you can use sum, cksum or md5sum on a wildcard and catch the output to a file, then compare the files. As an alternate with md5sum you can run it on the source system into a file, transfer the file and run it again on the target system using the -c flag, a bit like this:-
Code:
md5sum -c /tmp/md5sums.file 2>/dev/null |grep ": FAILED"

Any extra files on the target side will be ignored though. Is that acceptable?



Robin
This User Gave Thanks to rbatte1 For This Post:
# 5  
Old 12-01-2016
Well, this is the first part of the script, hope is good:

Code:
#!/bin/bash

dirName='/root/Scripts/Misc'

# Loop through directories
for elems in $(find $dirName -mindepth 1)
do

# if is file
if [[ -f $elems ]]; then

# Get file md5sum
md5=$(md5sum $elems | awk '{print $1}')

# if is directory
else

# get how many files inside this directory
md5=$(find $elems -type f | wc -l)

fi

# Get files size
fileSize=$(stat -c %s $elems)

# Print everything in one line
echo -e "$elems : $md5 : $fileSize" >> log

done

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Ubuntu

Rsync - not copy certain files

I use this rsync --progress -r -u /media/andy/MAXTOR_SDB1/Ubuntu_Mate_18.04/* /home/andy/Ubuntu_18.04_Programs/Is there a way to have it not copy files such as these? 2019-02-25_11:04 I found this but can not figure out what it's doing. 3. Exclude a specific file To exclude a... (5 Replies)
Discussion started by: drew77
5 Replies

2. Shell Programming and Scripting

Rsync and Move files

I Have a requirement where i have to sync two directories one on source location server A and other on destination location server B as i do not have ssh access from server A----------->B I am doing rsync from server B, The Requirement is as follows Two directories on the source and... (4 Replies)
Discussion started by: James0806
4 Replies

3. Shell Programming and Scripting

Rsync Error: rsync: link_stat failed: No such file or directory (2)

I wish to copy all the files & folder under /web/Transfer_Files/data/ on mymac1 (Linux) to remote server mybank.intra.com (Solaris 10) /tmp/ location I am using Ansible tool synchronize module which triggers the unix rsync command as below:rsync --delay-updates -F --compress --archive --rsh=ssh... (2 Replies)
Discussion started by: mohtashims
2 Replies

4. UNIX for Dummies Questions & Answers

Does rsync check and ignore files that already exist?

Hi, We have two (2) servers named primary and standby. There is a directory named /db01/archive that we need to keep in-sync. Files get transferred from primary and standby. Sometimes when we do a failover or when there is a network issue, some files fail to get transferred. I want to use... (3 Replies)
Discussion started by: newbie_01
3 Replies

5. Shell Programming and Scripting

Perl code to check date and check files in particular dir

Hi Experts, I am checking how to get day in Perl. If it is “Monday” I need to process…below is the pseudo code. Can you please prove the code for below condition. if (today=="Monday" ) { while (current_time LESS THAN 9:01 AM) ... (1 Reply)
Discussion started by: ajaypatil_am
1 Replies

6. Linux

Modifying Rsync script to check source

Hi I am running rsyncs between a raid attached server and a nas device, which works fine. I am using the --delete to ensure that its kept in sync. And running a log at the end of the sync for me to check. this i am running from a cron Problem is if the part of the raid fails on the server the... (1 Reply)
Discussion started by: treds
1 Replies

7. Shell Programming and Scripting

rsync copy files once

Hi This is my situation I have files on the left which I want to copy to the right. Once the files are copied to the right, they are processed and then deleted. The next time rsync runs I dont want it to copy the same files again, it should only copy any new files to the right. I have been... (4 Replies)
Discussion started by: duonut
4 Replies

8. Shell Programming and Scripting

Rsync temporary files

Hi, I am looking to use rsync in a very specific way, and even though I have trawled the rsync man pages I have not succeeded in seeing a way of doing the following: The temporary files created by rsync should not be created in the destination directory. (I have used --temp-dir option to... (0 Replies)
Discussion started by: LostInTheWoods
0 Replies

9. Solaris

Using RSYNC to copy files locally

Has anyone ever used rsync to copy files locally on one server? (in this case from one SAN volume to another). I am familiar with using rsync to copy files between servers, but not locally, I would usually use cp or or tar or something. Is rsync slower? Does it use additional overhead of the... (4 Replies)
Discussion started by: BG_JrAdmin
4 Replies

10. UNIX for Dummies Questions & Answers

rsync, which files where moved?

Hello, I am using rsync to make sure that my folder "local" mirrors the remote directory "remote". When a file is copied from "remote" to "local", I need to apply a bash script to it. What would be a neat way to do that? Thanks ps: is there a way to edit the title of the thread (I am a bit... (5 Replies)
Discussion started by: JCR
5 Replies
Login or Register to Ask a Question