Compare the checksum of files in 2 different folders | Unix Linux Forums | Shell Programming and Scripting

  Go Back    


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

Compare the checksum of files in 2 different folders

Shell Programming and Scripting


Closed Thread    
 
Thread Tools Search this Thread Display Modes
    #1  
Old 12-18-2009
reldb reldb is offline
Registered User
 
Join Date: Jul 2006
Last Activity: 1 August 2014, 5:45 AM EDT
Posts: 35
Thanks: 0
Thanked 0 Times in 0 Posts
Compare the checksum of files in 2 different folders

Hi

I have 2 different folders on different machines. they are supposed to be same but some time for unknown reason they are not. then we have to generate a report for files which are not matching.

I was doing as below -


Code:
cd folder1
find . -type f | sort | cksum >1.txt

cd folder2
find . -type f | sort | cksum >2.txt

1)
Code:
diff -w 1.txt 2.txt

still it shows lot of lines in diff even when file name/path/size/checksum is same. is there any nice and clean way to report the expected result.

2) i was able to generate good results (but not good format) by using -

Code:
cat 1.txt 2.txt | sort -k 3,3 | uniq -u

it does not tell me which file is from which folder (there is no < or > sign as we have in diff)

is there any other best way to do it ?

Thanks
Rel

Last edited by Franklin52; 12-19-2009 at 07:40 AM.. Reason: Please use code tags!!
Sponsored Links
    #2  
Old 12-18-2009
durden_tyler's Avatar
durden_tyler durden_tyler is offline Forum Advisor  
Registered User
 
Join Date: Apr 2009
Last Activity: 27 September 2014, 12:05 PM EDT
Posts: 1,841
Thanks: 7
Thanked 266 Times in 241 Posts
Quote:
Originally Posted by reldb View Post
...
I was doing as below -

cd folder1
find . -type f | sort | cksum >1.txt

cd folder2
find . -type f | sort | cksum >2.txt

1) diff -w 1.txt 2.txt

still it shows lot of lines in diff even when file name/path/size/checksum is same. ...
You may want to list, sort and find cksum of each file as shown in the following example:


Code:
$ # go to directory dir1
$ cd dir1
$
$ ls | sort | xargs cksum
1958180199 14 file1
1994293486 14 file2
1996893289 14 file3
$
$ # now do the same for directory dir2
$ cd ../dir2
$
$ ls | sort | xargs cksum
1958180199 14 file1
1994293486 14 file2
1996893289 14 file3
$
$

And as you are doing currently, redirect the output to two files and find the diff.

tyler_durden
Sponsored Links
    #3  
Old 12-18-2009
reldb reldb is offline
Registered User
 
Join Date: Jul 2006
Last Activity: 1 August 2014, 5:45 AM EDT
Posts: 35
Thanks: 0
Thanked 0 Times in 0 Posts
tyler_durden
thanks for suggestion
there are 2 issues

1) first ls wont show the files in sub directories. so we have to use find command. thats fine

2) main issue is still the same. if you do the diff of output from folder 1 and output from folder 2 it wont show properly as both the folder will have different no files so sort will be differnet and so the diff
    #4  
Old 12-19-2009
ichigo ichigo is offline
Ghostdog (Read Only)
 
Join Date: Dec 2009
Last Activity: 19 December 2009, 6:21 AM EST
Posts: 65
Thanks: 0
Thanked 0 Times in 0 Posts

Code:
diff directory1 directory2

Sponsored Links
    #5  
Old 12-19-2009
gh0std0g74 gh0std0g74 is offline
Ghostdog (Read Only)
 
Join Date: Dec 2009
Last Activity: 19 December 2009, 9:21 AM EST
Posts: 6
Thanks: 0
Thanked 0 Times in 0 Posts
if you can use Python, you can use its filecmp module

Code:
#!/usr/bin/env python
import filecmp
import os
directory1 = os.path.join("./","dir1")
directory2 = os.path.join("./","dir2")
cmp = filecmp.dircmp(directory1,directory2)
print cmp.report()
# print cmp.common
# print cmp.same_files # diff_files etc....

sample output

Code:
$ ls -1 dir1 dir2
dir1:
1.txt
2.txt
3.txt

dir2:
1.txt
3.txt
4.txt

$ ./python.py
diff ./dir1 ./dir2
Only in ./dir1 : ['2.txt']
Only in ./dir2 : ['4.txt']
Identical files : ['1.txt', '3.txt']
None

Sponsored Links
    #6  
Old 12-19-2009
pludi's Avatar
pludi pludi is offline Forum Advisor  
Cat herder
 
Join Date: Dec 2008
Last Activity: 28 March 2014, 8:35 AM EDT
Location: Vienna, Austria, Earth
Posts: 5,522
Thanks: 38
Thanked 335 Times in 308 Posts
How about using chksums -c option? On the first machine run
Code:
cd /dir1
find . -type f -exec chksum {} \; > chksums

Copy it over to the other machine and run
Code:
cd /dir2
chksum -c chksums

That should give you a listing of those files that differ.
Sponsored Links
    #7  
Old 12-19-2009
durden_tyler's Avatar
durden_tyler durden_tyler is offline Forum Advisor  
Registered User
 
Join Date: Apr 2009
Last Activity: 27 September 2014, 12:05 PM EDT
Posts: 1,841
Thanks: 7
Thanked 266 Times in 241 Posts
Or if you would like to program in Perl, then you may want to consider using the File:: Dircmp module -


Code:
$
$ cat -n dc.pl
     1  #!/usr/bin/perl -w
     2  use File::Dircmp;
     3  @r = dircmp("dir1","dir2");
     4  foreach $i (@r) {
     5    print $i,"\n";
     6  }
$
$ perl dc.pl
Only in dir1: dir11
Only in dir2: dir21
Files dir1/file2 and dir2/file2 are identical
Files dir1/file1 and dir2/file1 are identical
Files dir1/file3 and dir2/file3 differ
$
$

tyler_durden
Sponsored Links
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Compare 2 folders... protocomm Shell Programming and Scripting 8 03-19-2009 07:14 AM
To compare selective file in different folders gmahesh2k UNIX for Dummies Questions & Answers 0 05-15-2008 02:03 AM
removing old files except configuration files and folders jamcalicut Shell Programming and Scripting 1 11-09-2007 03:34 PM
Remote compare of folders sunilav Shell Programming and Scripting 1 04-04-2007 05:04 AM
Checksum Key Files adamevans AIX 1 01-23-2007 01:19 AM



All times are GMT -4. The time now is 10:16 AM.