Checksum Comparison and listing out the differences


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Checksum Comparison and listing out the differences
# 1  
Old 02-03-2014
Checksum Comparison and listing out the differences

Hi All,

We have a requiremetn where i have created a shell script that will compute the checksum of remote directoryand the local directory files
and get those details in two different files in the below format

File1:

Code:
0bee89b07a248e27c83fc3d5951213c1 /home/test1/test1/test1.sh
d41d8cd98f00b204e9800998ecf8427e /home/test1/test1/test2.sh

File2(Remote server)
Code:
 e48bc2344e4f2e2778309fa3abf8fb3f /home/test1/test1/test1.sh
d41d8cd98f00b204e9800998ecf8427e /home/test1/test1/test2.sh
d41d8cd98f00b204e9800998ecf8427e /home/test1/test1/test4

Here first column is checksum and the second column is the filename with path
Based on their cheksum Now i need to compare both the files in such a manner that it should display the output like this
Code:
 /home/test1/test1/test1.sh differs 
/home/test1/test1/test2.sh same
/home/test1/test1/test4 only present in file2_name.

This all is required to do the overall code comparison for the client

Last edited by bartus11; 02-03-2014 at 10:37 AM.. Reason: Please use [code][/code] tags.
# 2  
Old 02-03-2014
You could try something like:
Code:
awk '
NR == 1 {
        f1 = FILENAME
}
FNR == NR {
        cs[$2] = $1
        next
}
$2 in cs {
        if($1 == cs[$2])
                print $2, "same"
        else    print $2, "differs"
        delete cs[$2]
        next
}
{       print $2, "only present in", FILENAME
}
END {   for(i in cs)
                print i, "only present in", f1
}' File1 File2

If you want to try this on a Solaris/SunOS system, use /usr/xpg4/bin/awk, /usr/xpg6/bin/awk, or nawk instead of the default /usr/bin/awk.
# 3  
Old 02-04-2014
Hi Don,

Thanks that worked like a charm.

I have developed my script. Now i need to format it to give a profeessional look as this needs to be presented to the client. Could you please hlep me in this.


Below is the script
Code:
#!/bin/bash
function ssh_server_local()
{
if [ -r dir1 ];
then
for i in `cat dir1`
do
find $i -type f -exec md5sum {} +|sort -k2 > $name_local
done
else
echo "Please enter the folder path in dir1 file"
fi
}
function ssh_server_remote()
{
if [ -r dir2 ];
then
for i in `cat dir2`
do
ssh $username@$ip_remote "find $i -type f -exec md5sum {} +|sort -k2; exit" > $name_remote
done
else
echo "Please enter the folder path in dir2 file"
fi
}
function comparison()
{
awk '
NR == 1 {
f1 = FILENAME
}
FNR == NR {
cs[$2] = $1
next
}
$2 in cs {
if($1 == cs[$2])
print $2, "same"
else print $2, "differs"
delete cs[$2]
next
}
{ print $2, "only present in", FILENAME
}
END { for(i in cs)
print i, "only present in", f1
}' $name_remote $name_local >result.txt
}
 
echo "Please confirm if the direcotry path has been added in the dir1 y/n"
read d1
echo "Please confirm if the direcotry path has been added in the dir2 y/n"
read d2
echo "Please enter the user to which file belongs in the remote server"
read username
echo "Please enter the ip address of remote server"
read ip_remote
echo "Please enter the name of the remote server"
read name_remote
echo "Please enter the ip address of local server"
read ip_local
echo "Please enter the name of local server"
read name_local
ssh_server_local
ssh_server_remote
comparison


Last edited by Franklin52; 02-04-2014 at 09:25 AM.. Reason: Please use code tags
# 4  
Old 02-04-2014
For starters, you can change the useless use of cat and dangerous backticks for i in `cat dir1` ; do ... done into

Code:
while read -r i
do
...
done < dir

And add more error checking, so the script quits instead of spitting errors when given nonsense values or when missing important files. I often use this kind of construct:

Code:
die() { # Print error message and quit, use 'die "string"' or 'die "string" exitcode'
        echo "$1" >&2
        exit ${2-1}
}

# Get input from user
...

[[ -z "$string" ]] && die "String is blank"
[[ -e "filename" ]] || die "filename does not exist"

...


Last edited by Corona688; 02-04-2014 at 12:46 PM..
# 5  
Old 02-04-2014
Quote:
Originally Posted by Ekamjot
Hi Don,

Thanks that worked like a charm.

I have developed my script. Now i need to format it to give a profeessional look as this needs to be presented to the client. Could you please hlep me in this.


Below is the script
... ... ...
No. This forum is intended to help you learn how to use the tools available on Linux and UNIX Systems so you can effectively use these wonderful platforms. We are not here to do your job for you.

On the other hand, if you like the code I write and would like to hire me to turn sample code into something that could be used in a production environment, send me private mail giving details of what job you want me to do for you and how much you will pay me to do the job.
These 4 Users Gave Thanks to Don Cragun For This Post:
# 6  
Old 02-04-2014
Quote:
Originally Posted by Ekamjot
Now i need to format it to give a profeessional look as this needs to be presented to the client.
I would say the most important thing to give it a professional look is by indenting your code.

Writing a nicely indented code clearly shows that you've a clear sense of what you're writing and you certainly know how to "respect programming".
# 7  
Old 02-04-2014
Can I also suggest:

1. Check command line parameters and print quick usage info, consider -h help parameter and the ability to pass prompted value in as command options.

2 .Proper validation of entered data/parameters, e.g. ensure response to y/n prompts is valid (what are you using d1 and d2 for anyway), ip addresses are pingable

3. Good naming of function and variables - try to describe what they do/store e.g. "comparison" is less than informative.

4. Try to write a manual page that fully describes the script and any pre-requisites.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to get checksum of itself

is there a way to get a script to do a checksum on itself? something like this: #!/bin/sh myexpectedsig=$(cksum $0 | awk '{print $1}') if ; then exit else who uptime date fi im looking for something that would always represent the running script, which is why im... (6 Replies)
Discussion started by: SkySmart
6 Replies

2. UNIX for Dummies Questions & Answers

[Solved] How to remove listing of current user cmd from ps -ef listing?

Hi All, Could you please help to resolve my following issues: Problem Description: Suppose my user name is "MI90". i.e. $USER = MI90 when i run below command, i get all the processes running on the system containing name MQ. ps -ef | grep MQ But sometimes it lists... (8 Replies)
Discussion started by: KDMishra
8 Replies

3. IP Networking

Wireshark UDP checksum bad checksum

Hello I am communicating with two devices using my computer over UDP protocol. The application is running fine. When I monitored the UDP traffic using Wireshark software, I found that there were too many Checksum errors. Please find attached the png file showing this error. I am about to... (0 Replies)
Discussion started by: AustinCann
0 Replies

4. Solaris

md5 checksum what does it do

Hello good people, I came across md5 checksum. Can anyone please explain to me what it does and if possible an example of how to use it? Thank you very much (1 Reply)
Discussion started by: cjashu
1 Replies

5. Shell Programming and Scripting

Checksum+SFTP

Hi ALL, I use solaris OS and SFTP to get/put files from remote server.I use the below command , sftp user@host<<EOF cd "dir" get --checksum "filename" EOF I am getting a strange error as " get --checksum INVALID paramter". It has been working succesfully since last 3 years but all of a... (1 Reply)
Discussion started by: mohanpadamata
1 Replies

6. Shell Programming and Scripting

Differences between 2 Flat Files and process the differences

Hi Hope you are having a great weeknd !! I had a question and need your expertise for this : I have 2 files File1 & File2(of same structure) which I need to compare on some columns. I need to find the values which are there in File2 but not in File 1 and put the Differences in another file... (5 Replies)
Discussion started by: newbie_8398
5 Replies

7. Solaris

checksum

Anyone can tell me the different between "cksum" and "sum" command on Solaris? I read the man pages but still not get it. And how to display the md5 checksum for a file. Thanks, (1 Reply)
Discussion started by: redstone
1 Replies

8. SCO

checksum

Does anyone know the answer to this? When I run "sum -r" on a file that I've down loaded from the sco website, the 1st set of numbers differs from the checksum on the download page but the 2nd set matches. If I try to install the patch, I get errors. Anyone has an answer? (3 Replies)
Discussion started by: jn5519
3 Replies

9. Shell Programming and Scripting

Checksum question

in HPUX: I am copying oracle datafiles from one mountpoint to another the total size is about 250Gb. I wanted to perform a checksum on the target and make sure the files came overy properly. Mountpoints: /s01 to /u01 /s02 to /u02 I tried using "SUM" on these mountpoints but its taking... (1 Reply)
Discussion started by: jigarlakhani
1 Replies

10. UNIX for Dummies Questions & Answers

Recursive directory listing without listing files

Does any one know how to get a recursive directory listing in long format (showing owner, group, permission etc) without listing the files contained in the directories. The following command also shows the files but I only want to see the directories. ls -lrtR * (4 Replies)
Discussion started by: psingh
4 Replies
Login or Register to Ask a Question