Expect in Bash - and then compare md5sum

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Expect in Bash - and then compare md5sum
# 1  
Old 07-20-2016
Expect in Bash - and then compare md5sum

I'm running on a staging server. I will need to use expect and I think ssh or scp to the other boxes.

I need to see something like this....
Code:
Enter:Host 1
Enter:Host 2

Enter full directory path to compare: example /apps/acd/jboss-customer1/

Enter User Id: 
Enter Password:

( Assumes they are the same for both boxes )

Then I need to basically create two files on the servers and scp them back to the main box say in /tmp and do a
Code:
 find $host1 -type f -name "*" | xargs -n 1 md5sum | sort -n >/usr/local/scripts/apps-ard-jboss-jpannprod-pd1.txt

I need to do that on both boxes and then get them back to the main server to do the compare?

It's a bit jumbled in my mind sorry if I'm not being clear.

Code:
#!/bin/bash
clear
echo "================================== "
echo "Enter your Host 1 Name to compare"
echo "================================== "
echo -n "Host1 Name:"
read host1
clear
echo "================================== "
echo "Enter your Host 2 Name to compare"
echo "================================== "
echo -n "Host2 Name:"
read host2
clear
echo "================================== "
echo "Enter your Usr/Pw Authentication"
echo "================================== "
echo -n "UserName:"
read user
echo -n "Password:"
read password
/usr/bin/expect   <<EOF
set force_conservative 0
set timeout 4
spawn ssh $user@$name
expect "password:"
send  "$password\r"
expect "$"
send  "echo find $host1 -type f -name "*" | xargs -n 1 md5sum >/usr/local/scripts/apps-ard-jboss-jpannprod-pd.txt\r"

send "exit\r"


Last edited by rbatte1; 07-21-2016 at 08:19 AM.. Reason: Added CODE tags for sample input & required output section
# 2  
Old 07-21-2016
I am not sure that I understand the difference between this thread and your earlier thread: Compare md5sum two servers' setup. But, whether or not they are the same topic, I don't understand why you are trying to rewrite code to perform what rsync already does??? What makes you think that rsync only compares file sizes to determine if a file has changed?
# 3  
Old 07-22-2016
Quote:
Originally Posted by Don Cragun
But, whether or not they are the same topic, I don't understand why you are trying to rewrite code to perform what rsync already does??? What makes you think that rsync only compares file sizes to determine if a file has changed?
Amen to that!

In addition: whenever you start to use expect this is the confession that something went horribly wrong in your setup.

If you plan your authorisation correctly you should never have to rely on expect (which is just a way of having passwords written in cleartext into scripts typed automatically into forms) but use (preexchanged) keys or something similar. Instead of fiddling around with expect-scripts you should simply test for a possible ssh-connection and raise an error if that doesn't work (because keys are not exchanged, are invalid or some other reason).

Here is how i do that. It first tests for IP-connectivity and then - if that is possible - for ssh-connectivity:

Code:
# ------------------------------------------------------------------------------
# f_CheckConnectivity                          checking connectivity for a host
# ------------------------------------------------------------------------------
# Author.....: Wolf Machowitsch
# last update: 2016 01 25    by: Wolf Machowitsch
# ------------------------------------------------------------------------------
# Revision Log:
# - 0.99   2007 01 18   Original Creation
#                       -
#
# - 1.00   2015 04 28   Production Version
#                       removed unecessary subshells
#
# - 1.01   2016 01 25   bugfix
#                       - changed ssh-call to cover for new hosts:
#                         added "-o 'CheckHostIP=no'"
#                         added "-o 'StrictHostKeyChecking=no'"
#
# ------------------------------------------------------------------------------
# Usage:
#     f_CheckConnectivity  char Hostname [ char user ]
#     checks the connectivity for the host given in $1 optionally using a
#     username given in $2. If no user name is given the current user is
#     assumed.
#
#     Example:  f_CheckConnectivity $host    # checks if $host can be worked on
#                                            # using the current user (usually
#                                            # this will be root)
#
# Prerequisites:
# -   to use this function, the FPATH variable must be set
#
# ------------------------------------------------------------------------------
# Documentation:
#     f_CheckConnectivity() checks in a successive manner. First an (IP)-ping
#     is issued. If this is successful a connection test is done by issuing
#     a command via ssh. f_CheckConnectivity() does NOT try to correct any
#     errors, merely stating them.
#
#     Parameters: char Host
#
#     returns:    0: connectivity test passed
#                 1: no ssh connection
#                 2: no IP connection
#                 3: parameter/other/internal error
#
# ------------------------------------------------------------------------------
# known bugs:
#
#     none
# ------------------------------------------------------------------------------
# .....................(C) 2007 Wolf Machowitsch ...............................
# ------------------------------------------------------------------------------

f_CheckConnectivity ()
{

$chFullDebug
                                                 # internal variables
typeset -i iRetVal=0                             # return value (see docu)
typeset    chHost="$1"                           # hostname
typeset    chUser="$2"                           # optional username

if [ -z "$1" ] ; then                            # check for prereqs
     iRetVal=3
elif [ "$chUser" == "" ] ; then
     chUser="$(who am i | cut -d' ' -f1)"
fi

if [ $iRetVal -eq 0 ] ; then
     if ! ping -c1 $chHost 1>/dev/null 2>&1 ; then
          iRetVal=2
     fi
fi
if [ $iRetVal -eq 0 ] ; then
     if ! ssh -nq                  \
              -o 'BatchMode=yes'   \
              -o 'CheckHostIP=no'  \
              -o 'StrictHostKeyChecking=no' \
                 ${chUser}@${chHost} date 1>/dev/null 2>&1; then
          iRetVal=1
     fi
fi

return $iRetVal

}
# --- EOF f_CheckConnectivity

Notice that you will have to adjust this if you deal with hosts in DMZs or behind firewalls, because they often block attempts to ping the host.

I hope this helps.

bakunin
This User Gave Thanks to bakunin For This Post:
# 4  
Old 07-22-2016
Hi.

I'll take the blame for this.

I had hoped that my quote from the rsync man page would encourage the OP to look farther into rsync. A more complete explanation of the workings of rsync can be found at rsync - Wikipedia, the free encyclopedia

If that is too difficult or does not meet the OP's requirements, then the MD5, sort, and compare steps seem reasonable to me.

Best wishes ... cheers, drl
# 5  
Old 07-22-2016
I don't need to check connectivity but if config files are the same in certain directories. I am also not trying to sync them exactly the config files will have different IP addresses in them and different database names, this is why I can't just use rsync but was trying to hash something out that would tell me if they are different and list out the differences through making a couple of files and then doing a compare. Perhaps rsync will do this but I didn't think so.
# 6  
Old 07-22-2016
Quote:
Originally Posted by xgringo
I don't need to check connectivity but if config files are the same in certain directories. I am also not trying to sync them exactly the config files will have different IP addresses in them and different database names, this is why I can't just use rsync but was trying to hash something out that would tell me if they are different and list out the differences through making a couple of files and then doing a compare. Perhaps rsync will do this but I didn't think so.
So the script that you are hoping to create just gives you a list of files that need to be investigated further; it doesn't sync files. Isn't that exactly what:
Code:
rsync -n ... list_of_config_files_in_certain_directories ...

does?

The first time through it is likely to give you lots of false positives due to timestamp differences. But if you use rsync to synchronize those false positives (to sync the timestamps) and files that really need to be synced, subsequent runs should give you a greatly reduced list of files on which you actually need to perform your detailed md5sum checks.

And, obviously, if there are intentional IP and database name differences in some of your config files, md5sum isn't going to help you determine if there are also other differences in those files that shouldn't be present or that should cause a more complicated partial sync.
# 7  
Old 07-22-2016
Don Cragun is right, rsync would be the solution of first choice.

But still, if this is ruled out somehow for some reason we have yet to find out:

Quote:
Originally Posted by xgringo
I don't need to check connectivity but if config files are the same in certain directories.
If you do not need to check for connectivity then all the better. But in your threads title and the first post you mentioned you have to use "expect" and this can conceivably only be to transmit passwords. If you can guarantee somehow your ssh-keys can always be readily exchanged (though i'd still check that before i'd base my procedures on that) you can simply use ssh to remote-execute any command(s) you want and that would probably be a cleaner solution than using expect to initiate a pseudo-interactive ssh-session.

If you need to remotely execute arbitrary commands then you might want to do it this way (sketch):

Code:
while read HOST USER ; do
     if ! ssh -nqo 'BatchMode=yes' "$USER"@"$HOST" some_command
          echo "Error: some_command returned $?"
     fi
done <<_EOF_
host1 user1
host2 user2
....
_EOF_

Again, I'd check if indeed the keys are exchanged by including the function i showed you and call it somewhere inside the loop as part of my errror checking, but that is up to you. You should be able to build your custom routine around the shown mechanism pretty easily.

I hope this helps.

bakunin
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Bash script + expect

im very happy to back for this forum I have servers with alias of double dns extentions: sample: servera.test.com servera.test1.com serverb.test.com serverb.test1.com I need to login to that severs and executing the set of commands if test.com failed then try to login via... (0 Replies)
Discussion started by: prakash0106
0 Replies

2. Shell Programming and Scripting

Compare two md5sum

Hello, First of all I want to apologize because i'm not a admin or coder and maybe all my efforts to write only this small script in my life would need one week full time reading man pages and forums but... I don't have the money to offer me to get this time and the script I want to do seems... (5 Replies)
Discussion started by: toscan
5 Replies

3. Shell Programming and Scripting

Bash expect problem

Hey there :) I have a Bash Script and I'm trying to update Roundcube, but theres a user interactive line like: bin/installto.sh /var/www/mail/rc Upgrading from 1.1.3. Do you want to continue? (y/N) I'm trying to avoid this user interaction like this: cd roundcubemail-1.2.1 >/dev/null... (5 Replies)
Discussion started by: Aeris
5 Replies

4. Shell Programming and Scripting

Bash script with expect

Dear all Hi I want use expect in bash so that we can not use these with each other /bin/bash. With. /usr/bin/expect How can use these with on script or how can call a script from other script #!/bin/bash clear echo "================================== " echo "Enter your Esxi IP"... (3 Replies)
Discussion started by: Baber
3 Replies

5. Shell Programming and Scripting

Compare md5sum two servers' setup

I'm trying to think of a way to compare two boxes and make sure their files will be the same. There may be extra files on one side and some on the other. I also need to make sure the file content is identical. So far I've gotten this to create a file find /directorypath/ -type f -name... (3 Replies)
Discussion started by: xgringo
3 Replies

6. Shell Programming and Scripting

Problems with expect and sftp in bash

I'm having trouble with some automated sftp pulls. I'm using expect inside bash scripts and spawning SFTP. Some times the expect seems bog down. I have tried to put sleeps in my code to give everything time to work before I move on to next step but I till continue to get issues. For example when... (2 Replies)
Discussion started by: gosteen
2 Replies

7. Shell Programming and Scripting

Expect - bash and variables

I was wondering if anyone could provide some assistance. I trying to run an expect script within bash and get the results of a variable called RESULT. I Have tried a few things but none of them have worked. I know that the child process (the expect script) in this instance cannot set a variable... (6 Replies)
Discussion started by: ylafont
6 Replies

8. Shell Programming and Scripting

Compare files in directories with md5sum

And not to start. I can compare files, that's easy. The problem is that I compare files in a directory, and check if these files exist in another directory. The problem is that the file names are not the same. So I have to compare with "md5sum" or something similar. How I can do? All this in... (7 Replies)
Discussion started by: Jomeaide
7 Replies

9. Shell Programming and Scripting

Expect in bash to get the return value

cat test.sh #!/bin/sh expect <<- EOF set timeout 5 spawn ssh -o StrictHostKeyChecking=no lyang0@128.224.178.245 -C mkdir -p /tmp expect { "Password:" {send "root\r"} } spawn scp -o StrictHostKeyChecking=no /tmp/1 lyang0@128.224.178.245:/tmp/ ... (1 Reply)
Discussion started by: yanglei_fage
1 Replies

10. Shell Programming and Scripting

expect in bash script

Hi, I'm writing a shell script that calls a few commands that prompt the user for two simple yes/no questions. if the answers are consistent (the first is a yes, the second is a no), what would my expect script look like? Google is only giving me answers for scripts where I telnet or ssh. right now... (3 Replies)
Discussion started by: js741
3 Replies
Login or Register to Ask a Question