Assistance to connect to servers via ssh once and collect various commands into separate variables


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Assistance to connect to servers via ssh once and collect various commands into separate variables
# 1  
Old 09-16-2019
Assistance to connect to servers via ssh once and collect various commands into separate variables

Hello,

We use as bash script to connect to servers listed in an .csv file to issue commands against these servers and retrieve data to be saved in a .csv file. The data we want to collect is saved in variables. We issue an ssh command for each variable we want to capture. I'm thinking this is inefficient and slow and perhaps there is a better way to do this by connecting to the server through ssh once and issuing all the commands we need and still being able to save the commands in individual variables for our output file so I can decide in the output file to place the new commands we've captured as well as list some data from the input file into our final report.

Would someone be able to point me in the right direction on how I can connect once to a server via ssh, issue multiple commands to that server yet save the results from each command to their individual variable?

Here is what our script looks like:

Code:
#!/bin/bash

SOURCE_FILE=/home/user/scripts/input.csv
if test -f "$SOURCE_FILE"; then
    echo "$SOURCE_FILE exists"
fi

REPORT_FILE="/home/user/scripts/Report_$(date +"%F_%T").csv"

echo $REPORT_FILE

UNREACHABLE="255"
INVALID_PASSWORD="5"
NO_HOME_DIRECTORY="2"
PASSWORD_EXPIRED="1"
FLAVOUR=""
HOSTNAME=""

touch $REPORT_FILE

       echo "Input Hostname,\
Input IP,\
SSH Access,\
Tag,\
OS Type,\
Hostname,\
IP Adress,\
Uptime,\
ACTIVE KERNEL,\
INACTIVE KERNELS"  >> $REPORT_FILE

exec < $SOURCE_FILE || exit 1
read header # read (and ignore) the first line
while IFS=, read -r InputHost\
InputIP\
Tag\
UserAgentTag\
ResourceProvisioner\
OSType\
OSVersion\
OSDetail\
Location\
PowerState\
; do
        sshpass -p '********' ssh -qno StrictHostKeyChecking=no -o ConnectTimeout=1 userlogin@$InputIP 'ls -l /home/userlogin >/dev/null 2>&1' > /dev/null 2>&1
       
        status="$(echo $?)"
       
        echo "processing $InputHost"
       
        FLAVOUR=$(sshpass -p '********' ssh -qno StrictHostKeyChecking=no -o ConnectTimeout=1 userlogin@$InputIP 'cat /etc/redhat-release 2>/dev/null || lsb_release -a 2>/dev/null | grep Description | cut -f2')
       
        HOSTNAME=$(sshpass -p '********' ssh -qno StrictHostKeyChecking=no -o ConnectTimeout=1 userlogin@$InputIP 'hostname')

        IPADDRESS=$(sshpass -p '********' ssh -qno StrictHostKeyChecking=no -o ConnectTimeout=1 userlogin@$InputIP 'hostname -I')
       
        ACTIVE_KERNEL=$(sshpass -p '********' ssh -qno StrictHostKeyChecking=no -o ConnectTimeout=1 userlogin@$InputIP 'uname -r')

        INACTIVE_KERNEL=$(sshpass -p '********' ssh -qno StrictHostKeyChecking=no -o ConnectTimeout=1 userlogin@$InputIP 'rpm -qa | grep '^kernel-[0-9]' |grep -vE `uname -r`')
       
        UPTIME=$(sshpass -p '********' ssh -qno StrictHostKeyChecking=no -o ConnectTimeout=1 userlogin@$InputIP 'uptime | cut -d "," -f 1')


        if [ "$status" == "$UNREACHABLE" ]
        then
                echo "$InputHost,\
                $InputIP,\
                cannot connect,\
                $Tag,\
                $FLAVOUR,\
                $HOSTNAME,\
                $IPADDRESS,\
                $UPTIME,\
                "$ACTIVE_KERNEL,\
                $INACTIVE_KERNEL >> $REPORT_FILE
        elif [ "$status" == "$INVALID_PASSWORD" ]
        then
                echo "$InputHost,\
                $InputIP,\
                invalid account access,\
                $Tag,\
                $FLAVOUR,\
                $HOSTNAME,\
                $IPADDRESS,\
                $UPTIME,\
                "$ACTIVE_KERNEL,\
                $INACTIVE_KERNEL >> $REPORT_FILE
        elif [ "$status" == "$NO_HOME_DIRECTORY" ]
        then
                echo "$InputHost,\
                $Tag,\
                $FLAVOUR,\
                $HOSTNAME,\
                $IPADDRESS,\
                $UPTIME,\
                "$ACTIVE_KERNEL,\
                $INACTIVE_KERNEL >> $REPORT_FILE

        elif [ "$status" == "$PASSWORD_EXPIRED" ]
        then
                echo "$InputHost,\
                $InputIP,\
                password expired,\
                $Tag,\
                $FLAVOUR,\
                $HOSTNAME,\
                $IPADDRESS,\
                $UPTIME,\
                "$ACTIVE_KERNEL,\
                $INACTIVE_KERNEL >> $REPORT_FILE
        else
             echo "$InputHost,\
                $InputIP,\
                successful ssh access,\
                $Tag,\
                $FLAVOUR,\
                $HOSTNAME,\
                $IPADDRESS,\
                $UPTIME,\
                "$ACTIVE_KERNEL,\
                $INACTIVE_KERNEL >> $REPORT_FILE

        fi
done < $SOURCE_FILE

Thank you.
# 2  
Old 09-16-2019
If it was me....

I would write a script on the server side to read all the vars you are interested in and store the vars in a JSON file where the filename contains the timestamp, or put the timestamp in a JSON var as well..

Then I would have a remote call to get the JSON file and process the JSON file locally.
This User Gave Thanks to Neo For This Post:
# 3  
Old 09-16-2019
Thank you @Neo for your reply and suggestion.

I do see the benefit in doing this on each server. From my usecase we won't have this server script on every newly built server and I only scan a certain list of servers based on another report which I then use as my input file. For my usecase I will need to continue with using the script as we've currently written it to reach out to each server, collect data into variables and concatenate this collected data along with other variables from my input file.

I'm hoping there is a way to issue one ssh connection to each server and issue various command line instructions like I do in the script I've provided and collect each result in their own variable.

Thanks in advance for any help anyone can provide me.
# 4  
Old 09-16-2019
You could write a script (local or remote) and execute that, or use "here documents" like

Code:
IFS=$'\t' read FLAVOUR HOSTNAME IPADDRESS ACTIVE_KERNEL INACTIVE_KERNEL UPTIME <<- EOFREAD
        $(ssh -q user@targethost <<- EOFSSH | tr $'\n' $'\t'
        cat /etc/redhat-release 2>/dev/null || lsb_release -a 2>/dev/null | grep Description | cut -f2
        hostname
        hostname -I
        uname -r
        rpm -qa | grep '^kernel-[0-9]' |grep -vE `uname -r`
        uptime | cut -d "," -f1
        EOFSSH
        )
 EOFREAD

Try this to improve readability:
Code:
case "$status" in
        $UNREACHABLE)           OUT="cannot connect";;
        $INVALID_PASSWORD)      OUT="invalid account access";;
        $NO_HOME_DIRECTORY)     OUT="";;
        $PASSWORD_EXPIRED)      OUT="password expired";;
        $SUCCESS)               OUT="successful ssh access";;
esac
echo "$InputHost,\
      $InputIP,\
      $OUT,\
      $Tag,\
      $FLAVOUR,\
      $HOSTNAME,\
      $IPADDRESS,\
      $UPTIME,\
      $ACTIVE_KERNEL,\
      $INACTIVE_KERNEL"

(or use a shell array)


You duplicate the stdin redirection by exec < $SOURCE_FILE before the while loop and then redirecting the loop's stdin again.

Last edited by RudiC; 09-17-2019 at 10:22 AM..
This User Gave Thanks to RudiC For This Post:
# 5  
Old 09-17-2019
Thank you @RudiC for the reply! Very much appreciate you taking the time to review my post. I will also look into using the case statement to clean up the code readability!

For this code suggestion, I would like to clarify a few things.

Could I still use sshpass when connecting and could I use the $InputIP from the input file? So would your code suggestion look like the following:

Code:
IFS=$'\t' read FLAVOUR HOSTNAME IPADDRESS ACTIVE_KERNEL INACTIVE_KERNEL UPTIME <<- EOFREAD
        $(sshpass -p '*******' ssh -qno StrictHostKeyChecking=no -o ConnectTimeout=1 root@$InputIP <<- EOFSSH | tr $'\n' $'\t'

Secondly...each command instruction, I would like to assign a variable to each. Would I be able to do that as follows based on your code suggestion:

Code:
        FLAVOUR=$(cat /etc/redhat-release 2>/dev/null || lsb_release -a 2>/dev/null | grep Description | cut -f2)
        HOSTNAME=$(hostname)
        IPADDRESS=$(hostname -I)
        ACTIVE_KERNEL=$(uname -r)
        INACTIVE_KERNEL=$(rpm -qa | grep '^kernel-[0-9]' |grep -vE `uname -r`)
        UPTIME=$(uptime | cut -d "," -f1)

The reason I would like to assign variables to each result is so that I may place these into the final report where necessary.

Thanks!

Last edited by greavette; 09-17-2019 at 01:25 AM..
# 6  
Old 09-17-2019
Not sure I understand your questions / problems.
Of course, you can adapt the "command substitution" $(...) to fit your needs - in fact, you are required to do so as I only gave a proof of concept.
And, the read VAR1 VAR2 ... assigns each individual command line's output to its respective individual variable - did you try that? The way you propose can't be used as those variables used it the one single ssh session would be defined on the remote system and not expandable (not even existent) on your local system.
# 7  
Old 10-09-2019
Hello RudiC,

I'm finally getting around to testing the code snippets you've provided. Here is the code I've added to my script as per the suggestion. The code is allowing me to connect to the list of servers from the $SOURCE_FILE, but I'm not capturing the vars from the first line of the 'IFS=$'\t' read' line. There is also no progress on which Input ip I'm connecting too when I'm in this read loop so I can't see if it's working.

My script ended after running through about 50 servers with the following error - id: cannot find name for group ID 1000.

Here is how I added the code to my script. I don't think I've added it correctly. Any pointers on what I did wrong?

Code:
exec < $SOURCE_FILE || exit 1
read header # read (and ignore) the first line
while IFS=, read -r InputHost\
 InputIP\
 MaintenanceWindowTag\
 UserAgentTag\
 ResourceProvisioner\
 OSType\
 OSVersion\
 OSDetail\
 Location\
 PowerState\
 ; do

IFS=$'\t' read FLAVOUR HOSTNAME IPADDRESS ACTIVE_KERNEL INACTIVE_KERNEL UPTIME <<- EOFREAD
        $(sshpass -p 'xxxxxxx' ssh -qno StrictHostKeyChecking=no -o ConnectTimeout=1 user@$InputIP <<- EOFSSH | tr $'\n' $'\t'
        cat /etc/redhat-release 2>/dev/null || lsb_release -a 2>/dev/null | grep Description | cut -f2
        hostname
        hostname -I
        uname -r
        rpm -qa | grep '^kernel-[0-9]' |grep -vE `uname -r`
        uptime | cut -d "," -f1
EOFSSH
        )
EOFREAD

       if [ "$status" == "$UNREACHABLE" ]
        then
                echo "$InputHost,\
                $InputIP,\
                cannot connect,\
                $Tag,\
                $FLAVOUR,\
                $HOSTNAME,\
                $IPADDRESS,\
                $UPTIME,\
                "$ACTIVE_KERNEL,\
                $INACTIVE_KERNEL >> $REPORT_FILE
        elif [ "$status" == "$INVALID_PASSWORD" ]
        then
                echo "$InputHost,\
                $InputIP,\
                invalid account access,\
                $Tag,\
                $FLAVOUR,\
                $HOSTNAME,\
                $IPADDRESS,\
                $UPTIME,\
                "$ACTIVE_KERNEL,\
                $INACTIVE_KERNEL >> $REPORT_FILE
        elif [ "$status" == "$NO_HOME_DIRECTORY" ]
        then
                echo "$InputHost,\
                $Tag,\
                $FLAVOUR,\
                $HOSTNAME,\
                $IPADDRESS,\
                $UPTIME,\
                "$ACTIVE_KERNEL,\
                $INACTIVE_KERNEL >> $REPORT_FILE

        elif [ "$status" == "$PASSWORD_EXPIRED" ]
        then
                echo "$InputHost,\
                $InputIP,\
                password expired,\
                $Tag,\
                $FLAVOUR,\
                $HOSTNAME,\
                $IPADDRESS,\
                $UPTIME,\
                "$ACTIVE_KERNEL,\
                $INACTIVE_KERNEL >> $REPORT_FILE
        else
             echo "$InputHost,\
                $InputIP,\
                successful ssh access,\
                $Tag,\
                $FLAVOUR,\
                $HOSTNAME,\
                $IPADDRESS,\
                $UPTIME,\
                "$ACTIVE_KERNEL,\
                $INACTIVE_KERNEL >> $REPORT_FILE

        fi
done < $SOURCE_FILE

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Connect direct - SFTP - List of servers that I can connect

Greetings Experts, I am working for a bank client and have a question on connect-direct and SFTP. We are using Linux RedHat servers. We use connect-direct to transfer (NDM) files from one server to another server. At times, we manually transfer the files using SFTP from one server to another... (2 Replies)
Discussion started by: chill3chee
2 Replies

2. Shell Programming and Scripting

Find active SSH servers w/ ssh keys on LAN

Hi, I am trying to complete my bash script in order to find which SSH servers on LAN are still active with the ssh keys, but i am frozen at this step: #!/bin/bash # LAN SSH KEYS DISCOVERY SCRIPT </etc/passwd \ grep /bin/bash | cut -d: -f6 | sudo xargs -i -- sh -c ' && cat... (11 Replies)
Discussion started by: syrius
11 Replies

3. Shell Programming and Scripting

Shell script to connect to multiple ssh servers

Hello, I have access to several linux servers (mostly centos based) located in a DC in another country. from day to day I need to login to each of them to do some work (they dont have gui/window manager installed, I work only from console), or even to just do a check like df -h for disc usage.... (3 Replies)
Discussion started by: MaRiOsGR
3 Replies

4. UNIX for Advanced & Expert Users

Collect files from different servers to a single server and append them

Hi, I have script1.sh on 3 servers. I want to collect output report generated by them to a single server and append all the reports. Please tell me how can i do this? (2 Replies)
Discussion started by: pratikm23
2 Replies

5. Shell Programming and Scripting

connect to multiple servers using SSH and execute commands

Requirement: Run a shell script with below inputs file name checksum path the script should go to multiple servers (around 35) and verify the input cksum and if there is a mismatch display a simple message to the user that cksum verification failed. host details, user id /... (1 Reply)
Discussion started by: amicableperson
1 Replies

6. UNIX for Dummies Questions & Answers

need assistance on Calling DB user from separate file in Shell script

Hi All, I need to execute a SQL via shell script and i am connecting to Oracle DB by this way $USERNAME1/$PASSWORD1@$STRING1 and i need to get username, password and string from someother file stored in the Unix Directory. $Username, $Password and $String is stored in File A in Path A and i want... (1 Reply)
Discussion started by: sathish.tn
1 Replies

7. Shell Programming and Scripting

Send Remote Commands via SSH with variables

Hi there I found the Command to send commands to other servers like: sv01> ssh user@sv02 'ps -ef' But I cant use Variables from a script i want to execute on another server like: sv01> ssh user@sv02 'cd $SCRIPTHOME' although the variable is set on sv01. How can I run commands on sv02 with... (2 Replies)
Discussion started by: DarkSwiss
2 Replies

8. Shell Programming and Scripting

Cant separate variables

Hey guys, new problem....im not being able to seperate variables. the code runs like this... OPTIONS=$(awk '{print $2}' /etc/fstab} a=$(zenity --list --text "Mount points selection" --radiolist --column "choice" --column "mountpt" FALSE $OPTIONS); echo $a Note: the result i get is that... (2 Replies)
Discussion started by: dplate07
2 Replies

9. Shell Programming and Scripting

Need assistance with appending strings using sed and variables

HI, Can't seem to find anything on the forums to fix this. I have a file, one line within this will not have a specific string at the end. I have the string, but need to append it to the specific line which has it missing. I need to use a variable for this, $string - I am using double... (13 Replies)
Discussion started by: mandriver
13 Replies

10. Shell Programming and Scripting

need script to connect sftp servers

Dear friends, i need to connect sftp server from my home directory using script . Please can anyone help me on this. (1 Reply)
Discussion started by: kittusri9
1 Replies
Login or Register to Ask a Question