Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Need to create a simple script using MD5, SSH...

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need to create a simple script using MD5, SSH...
# 36  
Old 03-09-2011
Lol, I guess I am trained now to spot errors and remember them. You keep putting the " in
echo "trying "${fname} (${bname})" >&2
But now I notice it! Anyway, yeah here it is:

Quote:
box 1(root):/home/izivanov# sh dirsync6.sh
trying /opt/dba/scp/xxxxxxx (xxxxxxxxxxx)
local hash is xxxxxxxxxxxxxxxxxxxxx
MD5 (/opt/dba/scp/xxxxxxxxxx) = xxxxxxxxxxxxxxxxxxxxx
~
~
~
~
A couple of cannot open things on the bottom. Not really sure what that is!

---------- Post updated at 12:54 PM ---------- Previous update was at 12:50 PM ----------

Also what should I re-name the variable t with. My boss said that its not best practice to do that cause if anyone else looks at the script they won't know what the t stands for! I'm not sure what it stands for either! I know that it compares the hash-es of the files!

---------- Post updated at 12:55 PM ---------- Previous update was at 12:54 PM ----------

right, I was about to ask you about what to do if the files are different. My sync report got changed!
Last edited by zixzix01; 03-09-2011 at 06:02 PM..
# 37  
Old 03-09-2011
It's because not all your files are under /opt/dba/scp/. Check your list file and tell me, which ones should be converted from (whatever)/filename into /opt/dba/scp/filename, and which ones should be left as-is?

Or can they all be left as-is? In that case:

Code:
while read fname; do 
    #bname=$(basename $fname) 
    echo "trying ${fname}" >&2

    hash=$(/usr/local/bin/md5 "$bname" | awk '{print $4}')

    echo "local hash is ${hash}" >&2

    # should just print the md5's
    /usr/local/bin/hosts | while read box
    do
        T=$(ssh ${box} "/usr/local/bin/md5" "$fname" | awk '{ print $4;}')
        [ "$T" = "$hash" ] || echo "${box} $fname differs!"
    done

done < /home/izivanov/iz3

As for the syntax errors, ordinarily i'd try shell code I post on my own system before offering it to you but that's impossible for this script...
This User Gave Thanks to Corona688 For This Post:
zixzix01
# 38  
Old 03-09-2011
No I'm just kidding. I'm incredibly thankful and learning a ton just from these little things you're showing me! THank you so much!

Here is the output of my directory sync report. Thats the original notification about what needs to be updated! Maybe that will help you!

Quote:
root@: updating host

root@: updating host

root@: updating host

root@: updating host

root@: updating host

root@: updating host

root@box_: box_bkpa: /opt/sysadm/bin/xxxxxxxxx.sh: need to remove
root@box_: box_bkpa: /opt/sysadm/bin/xxxxxxxxxxx.sh: need to remove
root@box_: /opt/dba/scp/xxxxxxxxxxx: need to update
root@box_: /opt/dba/scpxxxxxxxxxxxx: need to update
root@box_: /opt/dba/scp/xxxxxxxxxxxxx: need to update
root@box_: /opt/dba/scp/xxxxxxxxxxxx: need to update
root@box_: /etc/sudoers: need to update
root@box_: /opt/dba/bin/xxxxxxxxxxxxx: need to update
So thats my original file I'm going off of!

Here is the ouput of the last change you suggested!

Quote:
(root):/home/izivanov# sh dirsync6.sh
trying /opt/dba/scp/xxxxxxxxxxxx (xxxxxxxxxxxx)
local hash is xxxxxxxxxxxxxxxxxxxxx
~
~
~
~
And sorry I had to remove the company server names. It might be confusing but I can't post that out here!
Last edited by zixzix01; 03-09-2011 at 06:03 PM..
# 39  
Old 03-09-2011
I'd be more interested in the contents of /home/izivanov/iz3 actually...

I spotted an error in my script. I left a bname in when inappropriate.

And one really should check the return value of md5 here. It bizzarely spews its errors to stdout, so whenever something goes wrong, the loop checks garbage vs garbage and decides everything's OK.

Code:
# If you don't have mktemp, TMP=/tmp/$$ will do.  mktemp is more secure.
TMP=`mktemp`
while read fname; do 
    #bname=$(basename $fname) 
    echo "trying ${fname}" >&2

    if ! /usr/local/bin/md5 "$fname" > "$TMP"
    then
        echo "md5 on ${bname} failed: "
        # print the error
        cat "$TMP"
        continue
    fi

    hash=$(awk '{print $4}' "$TMP")

    echo "local hash is ${hash}" >&2

    # should just print the md5's
    /usr/local/bin/hosts | while read box
    do
        T=$(ssh ${box} "/usr/local/bin/md5" "$fname" | awk '{ print $4;}')
        [ "$T" = "$hash" ] || echo "${box} $fname differs!"
    done

done < /home/izivanov/iz3

# clean up temp file
rm -f "$TMP"

This User Gave Thanks to Corona688 For This Post:
zixzix01
# 40  
Old 03-09-2011
Here is the iz3 file!

Quote:
/opt/dba/scp/xxxxxxxx
/opt/dba/scp/xxxxxxxxxxxxxx
/opt/dba/scp/xxxxxxxxxxxxxxx
/opt/dba/scp/xxxxxxxxxxxxx
/etc/sudoers
/opt/dba/bin/xxxxxxxxxxxxxxxxx
And here is the output of me just ssh-ing and checking a hash on a file across box 2

Quote:
(root):/home/izivanov# ssh root@box_dsvb /usr/local/bin/md5 /opt/dba/scp/start.progdsvb
MD5 (/opt/dba/scp/start.progdsvb) = 3351d46dfeae6011d2e074e4cdeb7dff
---------- Post updated at 01:24 PM ---------- Previous update was at 01:20 PM ----------

And the last code suggestion you had! I think it gives the same output as before!

Quote:
(root):/home/izivanov# sh dirsync6.sh
trying /opt/dba/scp/xxxxxxxxxxxxx
local hash is xxxxxxxxxxxxxxxxxxxxxxxxx
trying /opt/dba/scp/xxxxxxxxxxxxxxxxxxxx
local hash is xxxxxxxxxxxxxxxxxxxxx

~
~
~
~
Last edited by zixzix01; 03-09-2011 at 06:04 PM..
# 41  
Old 03-09-2011
certainly not the same output; for one thing it properly takes the md5 of sudoers now and never did before.

But argh, there's NO REASON for it to immediately break the loop like that given the data you've shown me. There's not even any break statement in it, it can't break until read actually fails! And because you're always stripping out the box names I'll never know what it really, really says and what your your hosts command actually, really prints. I could easily be missing something obvious. Smilie

You never told me what one bit of code did. I'll update it:
Code:
while read fname; do 
    bname=$(basename $fname) 
    echo "trying ${fname} (${bname})" >&2

    hash=$(/usr/local/bin/md5 "$fname" | awk '{print $4}')

    echo "local hash is ${hash}"

    # should just print the md5's
    /usr/local/bin/hosts | while read box
    do
        echo "${box}"
        # should print md5's repeatedly
        ssh ${box} "/usr/local/bin/md5" "$fname"
    done

done < /home/izivanov/iz3

---------- Post updated at 01:24 PM ---------- Previous update was at 01:07 PM ----------

[edit] The only possible thing that could be eating all the filenames before they're read is ssh. You don't have the usual md5sum command; maybe it's still trying to read from standard input, and in doing so, eating all the output from hosts. Try this:
Code:
# If you don't have mktemp, TMP=/tmp/$$ will do.  mktemp is more secure.
TMP=`mktemp`
while read fname; do 
    #bname=$(basename $fname) 
    echo "trying ${fname}" >&2

    if ! /usr/local/bin/md5 "$fname" > "$TMP" </dev/null
    then
        echo "md5 on ${bname} failed: "
        # print the error
        cat "$TMP"
        continue
    fi

    hash=$(awk '{print $4}' "$TMP")

    echo "local hash is ${hash}" >&2

    # should just print the md5's
    /usr/local/bin/hosts | while read box
    do
        # redirect its stdin to prevent it reading any filenames
        ssh "${box}" "/usr/local/bin/md5" "${fname}" > "$TMP" </dev/null
        # Skip errors
        [ "$?" -eq 0 ] || continue

        remotehash=$(awk '{print $4;}' < "$TMP")
        [ "$remotehash" = "$hash" ] || echo "${box} $fname differs!"
    done

done < /home/izivanov/iz3

# clean up temp file
rm -f "$TMP"

This User Gave Thanks to Corona688 For This Post:
zixzix01
# 42  
Old 03-09-2011
Sorry about thatthe box names are just first three initals of company with standard developers names

like:

xxxbox1
xxxbox2
xxxbox3

THe xxxbox1 one is our push server. From there we check all the rest!

Here is the output of the last code:
Code:
while read fname; do 
    bname=$(basename $fname) 
    echo "trying ${fname} (${bname})" >&2

    hash=$(/usr/local/bin/md5 "$fname" | awk '{print $4}')

    echo "local hash is ${hash}"

    # should just print the md5's
    /usr/local/bin/hosts | while read box
    do
        echo "${box}"
        # should print md5's repeatedly
        ssh ${box} "/usr/local/bin/md5" "$fname"
    done
done < /home/izivanov/iz3

And what the output is:

Quote:
xxxbox1(root):/home/izivanov# sh dirsync6.sh
trying /opt/dba/scp/xxxxxxxxxxxxx
local hash is xxxxxxxxxxxxxxxxxxxxxxx

~
~
~
local hash is xxxxxxxxxxxxxxxxxx
xxxbox3 /opt/dba/bin/xxxxxxxxxxxxxx differs!
We got a differ on one of the servers! Yeah something is definitely off! What do you need to see to have a better understanding!?

BTW we don't have md5sum ( we have md5), its an HP-UX environment. Isn't that only on the Linux type systems!?

---------- Post updated at 02:47 PM ---------- Previous update was at 02:44 PM ----------

did you get my message about what the host and hostname files look like and their output?

---------- Post updated at 02:56 PM ---------- Previous update was at 02:47 PM ----------

sorry the last code i actually used was:
Code:
#!/usr/bin/sh

# If you don't have mktemp, TMP=/tmp/$$ will do.  mktemp is more secure.
TMP=`mktemp`
while read fname; do
    #bname=$(basename $fname)
    echo "trying ${fname}" >&2

    if ! /usr/local/bin/md5 "$fname" > "$TMP" </dev/null
    then
        echo "md5 on ${bname} failed: "
        # print the error
        cat "$TMP"
        continue
    fi

    hash=$(awk '{print $4}' "$TMP")

    echo "local hash is ${hash}" >&2

    # should just print the md5's
    /usr/local/bin/hosts | while read box
    do
        # redirect its stdin to prevent it reading any filenames
        ssh "${box}" "/usr/local/bin/md5" "${fname}" > "$TMP" </dev/null

The only thing I added was #!/usr/bin/sh, cause my boss told me I would need it!
Last edited by zixzix01; 03-09-2011 at 06:06 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help making simple perl or bash script to create a simple matrix

Hello all! This is my first post and I'm very new to programming. I would like help creating a simple perl or bash script that I will be using in my work as a junior bioinformatician. Essentially, I would like to take a tab-delimted or .csv text with 3 columns and write them to a "3D" matrix: ... (16 Replies)
Discussion started by: torchij
16 Replies

2. Shell Programming and Scripting

Create simple script

Dear all, I have a directory named A and some subdirectories named B, C, D with .xml files. I want to use the following command to strip the file. sed -re ':start s/<*>//g; /</ {N; b start}' file.xml > file.xml At the same time, I want to remove the blank lines using sed '/^$/d' How can... (6 Replies)
Discussion started by: corfuitl
6 Replies

3. Shell Programming and Scripting

How to create a simple copy script?

Guys I want to do this: copy: /var/router/system1/config/backup/install.put /var/router/system2/config/backup/install.put /var/router/system3/config/backup/install.put /var/router/system4/config/backup/install.put into: /var/router/system1/config/install.dat... (22 Replies)
Discussion started by: DallasT
22 Replies

4. Shell Programming and Scripting

Create md5 sums and archive the resulting md5 files

Hello everyone, I am looking to basically creating md5sum files for all iso files in a directory and archive the resulting md5 files into a single archive in that very same directory. I worked out a clumsy solution such as: #find files for which md5sum are to be created and store the... (1 Reply)
Discussion started by: SurfTranquille
1 Replies

5. Shell Programming and Scripting

How to create a simple shell script to backup

Hello - I am in process of deleting many files which are older than 4 weeks. For example I am inside: /subsystem/prod/ Files are with various extentions, but anything older than 4 weeks should be deleted. What would be the most simplest script to acheive this? (4 Replies)
Discussion started by: DallasT
4 Replies

6. Solaris

How to create a simple background script on Solaris

I have a local account for a unix server. The idle timeout for the account is around 10 mins. I have to login to the server multiple times during the day. Is there a way to increase the idle timeout or may be a script that I can run on background so it is not idle. Something like echo date every 9... (3 Replies)
Discussion started by: vinaysa
3 Replies

7. Shell Programming and Scripting

Simple SSH script

I have several unix servers, I need to logon to each server and find out if an id exists on that server. I need a simple script for this, i have come up with the following script, but I cannot bring the output of a child process on the remote server. for i in `cat SERVER_LIST` do ssh $i... (5 Replies)
Discussion started by: ramky79
5 Replies

8. Shell Programming and Scripting

Simple Script to create folders

Hi I want to write a small script that will create folders named from `AAAA' all the way to `ZZZZ'. That is: `AAAA' `AAAB' `AAAC' ... `AABA' `AABB' `AABC' ... `ABAA' `ABAB' `ABAC' ... `ABBA' ... `ZZZZ' (4 Replies)
Discussion started by: ksk
4 Replies

9. Shell Programming and Scripting

Modifying simple commands to create a script

Can anyone direct me to a resource that explains scripting in simple terms? I have visited many sites and browsed this forum and have yet to find simple explanations. (8 Replies)
Discussion started by: rocinante
8 Replies

10. Shell Programming and Scripting

How to create md5 Hash variable?

I have a script that runs the grub-md5-crypt command based on whether the pass_value variable is a non-zero string. The md5 hash is being created in the /opt/hostconfigs/$HOST file, but I can't echo $md5_value. It is blank. Is there a way to create and echo a md5 hash variable? if then... (1 Reply)
Discussion started by: cstovall
1 Replies
Login or Register to Ask a Question