Visit Our UNIX and Linux User Community


Need to create a simple script using MD5, SSH...


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need to create a simple script using MD5, SSH...
# 1  
Old 03-01-2011
Error Need to create a simple script using MD5, SSH...

Hi all, I am brand new to these forums and I am a brand new UNIX Administartor. Don't know any scripting yet Smilie, and would like to learn as my boss is slowly letting me understand everything about being a Sys/*Nix Admin. He created this script which I am trying to replicate because I lost it to the dreaded /tmp folder. It was like 20 lines long. It seemed crazy easy to him Smilie. Oh well!

On to the problem...

I have a generated Directory Sync Report list of files that need to be synced across all servers/hosts (HP-UX ones). They are all supposed to be properly updated from the server which we push all the changes from, to all the other mirrors!

I have to use MD5 and SSH and SCP to do this. I have a script that generates hostnames/server-names (hostname.sh) which I am to use. My script should read the output of the file list (I think it was using SCP to get the files over from the other servers, and check the MD5 hash of each file, use SSH to check the same files across the other servers!

I REALIZE THIS MAY SEEM LIKE A HOMEWORK QUESTION BUT BELIEVE ME IT IS WORK RELATED.

AGAIN, I NEVER MADE A SCRIPT BEFORE!Smilie

I just need to be pointed in the right direction so that I can learn how to script myself these little things which come so easily to more experienced people!

Thanks a lot for any input!

Ivan

Last edited by zixzix01; 03-02-2011 at 10:58 AM..
# 2  
Old 03-02-2011
md5 creates a distinct hash string based on the actual data in a file, ssh lets you run md5 on remote nodes, and scp copies files to remote nodes.

Assume the output of md5 looks like:
Code:
MD5 (t.lis) = ac0395498ed01862baf46065b2b92b16

The hash is the string at the end so it is the fourth field, fields defined by spaces is the default.

Assume that /path/to/master is on the server that you want as the master. And has the files to be "synced".

node1 node2 node3 are the names of remote servers/mirrors.

Code:
for fname in /path/to/master/*
do
    hash=$(md5 $fname | awk '{print $4}' )  # get hash for each master file
    bname=$(basename $fname)                 # just the file name, no path
    for box in node1 node2 node3       # check each remote mirror
    do
        t=$(ssh myusername@$box "md5 /path/to/file/$bname" ) 
        test_hash=$(echo "$t" | awk '{print $4}')
        if [ "$hash"  != "$test_hash ] ;  then         # files are not identical
             scp $fname myusername@$box:/path/to/file/$bname
             ssh /path/to/file/$bname "chmod 775 /path/to/file/$bname"
        fi
    done
done

Ask your boss about the chmod statement. You may not need it.
This User Gave Thanks to jim mcnamara For This Post:
# 3  
Old 03-02-2011
Thank you very much for your quick reply. This seems to be something I can shoot off of. Thank you for an introduction.

I think though that the script should not really base itself on a "master" server. I want to check all the servers at once, and find out the latest, most important files, whether they are on the main server or not. Do you understand what I mean.

I think I can work off of your example and see if I can work something out. But if you have any other suggestions I would love to hear them!

My boss had his script starting with declaring something and then he used the Vi file with the /pathname/files which differed for that declaration in a string:
Something to the effect of how you proposed:

Code:
for box in node1 node2 node3       # check each remote mirror
    do
        t=$(ssh myusername@$box "md5 /path/to/file/$bname" ) 
        test_hash=$(echo "$t" | awk '{print $4}')
        if [ "$hash"  != "$test_hash ] ;  then         # files are not identical
             scp $fname myusername@$box:/path/to/file/$bname
             ssh /path/to/file/$bname "chmod 775 /path/to/file/$bname"
        fi

I think he declares the "t" beforehand, and then assigns the values from one of the hostname sripts!

Code:
Ask your boss about the chmod statement.  You may not need it.

I don't think I need to change permissions on the files, he didn't use that in his script!

Thank you so much for all your help!
Ivan

P.S. Hey while I have you, how can I read his scripts that he made. They are executable and just say hostname, and host. If I open them with Vi they just read "executable" on the bottom! Weird! I'm trying to understand his approach to scripting, he's amazing!
# 4  
Old 03-02-2011
You should check out rsync, it is very versatile, works over ssh connections and can sync up files based on checksum, last-modified time and more.
This User Gave Thanks to Chubler_XL For This Post:
# 5  
Old 03-03-2011
Right I did see everyone talking about that program. I will have to look into that. I just wanted to also learn scripting to aid me in my job. But yeah that program seems like a very good solution!

Thanks
# 6  
Old 03-04-2011
Hey Jim,

I was trying to use your example but getting confused with the following:

Code:
bname=$(basename $fname)                 # just the file name, no path

What is basename? WHat do you mean "just the file name"?

Thanks,
# 7  
Old 03-04-2011
basename is a standard Unix and GNU/Linux utility.
Quote:
The basename utility deletes any prefix ending with the last slash `/'
character present in a string (after first stripping trailing slashes), and
a suffix, if given. The suffix is not stripped if it is identical to the
remaining characters in string. The resulting filename is written to the
standard output. A non-existent suffix is ignored.
For example:
Code:
$ pwd
/dev/fs/C/SUA/home/mpf
$ basename $(pwd)
mpf
$

use the man utility to read up more about basename, i.e.
Code:
$ man basename


Previous Thread | Next Thread
Test Your Knowledge in Computers #829
Difficulty: Medium
The JSON standard supports object references.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help making simple perl or bash script to create a simple matrix

Hello all! This is my first post and I'm very new to programming. I would like help creating a simple perl or bash script that I will be using in my work as a junior bioinformatician. Essentially, I would like to take a tab-delimted or .csv text with 3 columns and write them to a "3D" matrix: ... (16 Replies)
Discussion started by: torchij
16 Replies

2. Shell Programming and Scripting

Create simple script

Dear all, I have a directory named A and some subdirectories named B, C, D with .xml files. I want to use the following command to strip the file. sed -re ':start s/<*>//g; /</ {N; b start}' file.xml > file.xml At the same time, I want to remove the blank lines using sed '/^$/d' How can... (6 Replies)
Discussion started by: corfuitl
6 Replies

3. Shell Programming and Scripting

How to create a simple copy script?

Guys I want to do this: copy: /var/router/system1/config/backup/install.put /var/router/system2/config/backup/install.put /var/router/system3/config/backup/install.put /var/router/system4/config/backup/install.put into: /var/router/system1/config/install.dat... (22 Replies)
Discussion started by: DallasT
22 Replies

4. Shell Programming and Scripting

Create md5 sums and archive the resulting md5 files

Hello everyone, I am looking to basically creating md5sum files for all iso files in a directory and archive the resulting md5 files into a single archive in that very same directory. I worked out a clumsy solution such as: #find files for which md5sum are to be created and store the... (1 Reply)
Discussion started by: SurfTranquille
1 Replies

5. Shell Programming and Scripting

How to create a simple shell script to backup

Hello - I am in process of deleting many files which are older than 4 weeks. For example I am inside: /subsystem/prod/ Files are with various extentions, but anything older than 4 weeks should be deleted. What would be the most simplest script to acheive this? (4 Replies)
Discussion started by: DallasT
4 Replies

6. Solaris

How to create a simple background script on Solaris

I have a local account for a unix server. The idle timeout for the account is around 10 mins. I have to login to the server multiple times during the day. Is there a way to increase the idle timeout or may be a script that I can run on background so it is not idle. Something like echo date every 9... (3 Replies)
Discussion started by: vinaysa
3 Replies

7. Shell Programming and Scripting

Simple SSH script

I have several unix servers, I need to logon to each server and find out if an id exists on that server. I need a simple script for this, i have come up with the following script, but I cannot bring the output of a child process on the remote server. for i in `cat SERVER_LIST` do ssh $i... (5 Replies)
Discussion started by: ramky79
5 Replies

8. Shell Programming and Scripting

Simple Script to create folders

Hi I want to write a small script that will create folders named from `AAAA' all the way to `ZZZZ'. That is: `AAAA' `AAAB' `AAAC' ... `AABA' `AABB' `AABC' ... `ABAA' `ABAB' `ABAC' ... `ABBA' ... `ZZZZ' (4 Replies)
Discussion started by: ksk
4 Replies

9. Shell Programming and Scripting

Modifying simple commands to create a script

Can anyone direct me to a resource that explains scripting in simple terms? I have visited many sites and browsed this forum and have yet to find simple explanations. (8 Replies)
Discussion started by: rocinante
8 Replies

10. Shell Programming and Scripting

How to create md5 Hash variable?

I have a script that runs the grub-md5-crypt command based on whether the pass_value variable is a non-zero string. The md5 hash is being created in the /opt/hostconfigs/$HOST file, but I can't echo $md5_value. It is blank. Is there a way to create and echo a md5 hash variable? if then... (1 Reply)
Discussion started by: cstovall
1 Replies

Featured Tech Videos