Shell Programming and Scripting

Hi all, I am brand new to these forums and I am a brand new UNIX Administartor. Don't know any scripting yet , and would like to learn as my boss is slowly letting me understand everything about being a Sys/*Nix Admin. He created this script which I am trying to replicate because I lost it to the dreaded /tmp folder. It was like 20 lines long. It seemed crazy easy to him . Oh well!

On to the problem...

I have a generated Directory Sync Report list of files that need to be synced across all servers/hosts (HP-UX ones). They are all supposed to be properly updated from the server which we push all the changes from, to all the other mirrors!

I have to use MD5 and SSH and SCP to do this. I have a script that generates hostnames/server-names (hostname.sh) which I am to use. My script should read the output of the file list (I think it was using SCP to get the files over from the other servers, and check the MD5 hash of each file, use SSH to check the same files across the other servers!

I REALIZE THIS MAY SEEM LIKE A HOMEWORK QUESTION BUT BELIEVE ME IT IS WORK RELATED.

AGAIN, I NEVER MADE A SCRIPT BEFORE!

I just need to be pointed in the right direction so that I can learn how to script myself these little things which come so easily to more experienced people!

Thanks a lot for any input!

Ivan

md5 creates a distinct hash string based on the actual data in a file, ssh lets you run md5 on remote nodes, and scp copies files to remote nodes.

Assume the output of md5 looks like:

Code:

MD5 (t.lis) = ac0395498ed01862baf46065b2b92b16

The hash is the string at the end so it is the fourth field, fields defined by spaces is the default.

Assume that /path/to/master is on the server that you want as the master. And has the files to be "synced".

node1 node2 node3 are the names of remote servers/mirrors.

Code:

for fname in /path/to/master/*
do
    hash=$(md5 $fname | awk '{print $4}' )  # get hash for each master file
    bname=$(basename $fname)                 # just the file name, no path
    for box in node1 node2 node3       # check each remote mirror
    do
        t=$(ssh myusername@$box "md5 /path/to/file/$bname" ) 
        test_hash=$(echo "$t" | awk '{print $4}')
        if [ "$hash"  != "$test_hash ] ;  then         # files are not identical
             scp $fname myusername@$box:/path/to/file/$bname
             ssh /path/to/file/$bname "chmod 775 /path/to/file/$bname"
        fi
    done
done

Ask your boss about the chmod statement. You may not need it.

Thank you very much for your quick reply. This seems to be something I can shoot off of. Thank you for an introduction.

I think though that the script should not really base itself on a "master" server. I want to check all the servers at once, and find out the latest, most important files, whether they are on the main server or not. Do you understand what I mean.

I think I can work off of your example and see if I can work something out. But if you have any other suggestions I would love to hear them!

My boss had his script starting with declaring something and then he used the Vi file with the /pathname/files which differed for that declaration in a string:
Something to the effect of how you proposed:

Code:

for box in node1 node2 node3       # check each remote mirror
    do
        t=$(ssh myusername@$box "md5 /path/to/file/$bname" ) 
        test_hash=$(echo "$t" | awk '{print $4}')
        if [ "$hash"  != "$test_hash ] ;  then         # files are not identical
             scp $fname myusername@$box:/path/to/file/$bname
             ssh /path/to/file/$bname "chmod 775 /path/to/file/$bname"
        fi

I think he declares the "t" beforehand, and then assigns the values from one of the hostname sripts!

Code:

Ask your boss about the chmod statement.  You may not need it.

I don't think I need to change permissions on the files, he didn't use that in his script!

Thank you so much for all your help!
Ivan

P.S. Hey while I have you, how can I read his scripts that he made. They are executable and just say hostname, and host. If I open them with Vi they just read "executable" on the bottom! Weird! I'm trying to understand his approach to scripting, he's amazing!

You should check out rsync, it is very versatile, works over ssh connections and can sync up files based on checksum, last-modified time and more.

Right I did see everyone talking about that program. I will have to look into that. I just wanted to also learn scripting to aid me in my job. But yeah that program seems like a very good solution!

Thanks

Hey Jim,

I was trying to use your example but getting confused with the following:

Code:

bname=$(basename $fname)                 # just the file name, no path

What is basename? WHat do you mean "just the file name"?

Thanks,

basename is a standard Unix and GNU/Linux utility.
For example:

Code:

$ pwd
/dev/fs/C/SUA/home/mpf
$ basename $(pwd)
mpf
$

use the man utility to read up more about basename, i.e.

Code:

$ man basename