Sponsored Content
Top Forums Shell Programming and Scripting Script works, but I think it could be better and faster Post 303025744 by cornelvis on Sunday 11th of November 2018 07:22:20 AM
Old 11-11-2018
Script works, but I think it could be better and faster

Hi All,

I'm new to the forum and to bash scripting. I did some stuff with VB.net, Batch, and VBScripting in the past, but because I shifted over to Linux, I am learning to script in Bash at this moment. So bear with me if I seem to script like a newbie, that's just because I am ;-)

OK, I wrote this script. It does the job, but it's rather slow. I get the server names, files, and password from separate text files. I place the servers and files into a list and loop them, but I configured only 2 local VM's at the moment and it runs rather slow already. I think I know why... It is logging into the server every time and then gets the file(s) I need. How do I overcome this? So, in my opinion, it just needs to login once and then gets the files off and after that move on to the other server.
Let me know what your thoughts are

Here it is:

Code:
#!/usr/bin/env bash

############################################ Global Declarations #################################################

home_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
dest_dir="/home/cornelis/config_files_properties"
ssh_user="root"
ssh_pass=$home_dir"/password_file" # Not needed on the script server
logfile="/tmp/config_files_properties.log"


############################################### Do the job #####################################################

## first thing to check if we are root, if not close
#
#if [ $(whoami) != 'root' ]; then
#        echo "Must be root to run $0"
#        exit 1;
#fi
# redirect all output to log file

exec &> >(while read -r line; do printf '%s %s\n' "$(date)" "$line"; done >> $logfile)

## check if destination folder exists and if not make it

if [[ ! -d "$dest_dir" ]]; then
    mkdir -p $dest_dir
fi

# Deleting existing files in the destination folder (if empty send error to /dev/null)

# extra error handling just in case, before removing anything

cd $dest_dir
if [[ "$?" = "0" ]]; then


# empty folder

shopt -s dotglob && rm -r $dest_dir/* > /dev/null 2>&1

# get the files off the servers and place them in the destination folder in its own folder

index=0
while read line; do
  server_array[index]="$line"
    for server in ${server_array[*]}
    do
    # create the folders named per server
        mkdir -p $dest_dir/$server
          #find the correct files
          index2=0
         while read line2; do
           file_array[index2]="$line2"
           for file in ${file_array[*]}
            do
            #copy the files to the folders
           sshpass -f $ssh_pass scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -rpqv $ssh_user@$server:$file $dest_dir/$server
           done
          done < $home_dir/files
    done

done < $home_dir/test_server_names
# change the directory and run SED on the files and folders
cd $dest_dir || exit 1

find ./ -type f -print0 | xargs -0 sed -Ei 's/([^&]password=)([^\t "]*)/\1xxxxxx/gI; s/(password.*>).*(<)/\1xxxxxx\2/gI; s/(password" value=").[^"]*/\1xxxxxx/gI; s/=admin:.[^\t ]*/=admin:xxxxxx/g; /"Password">/ {n;N;s/([\t ]*).*(\n.*<\/)/\1xxxxxx\2/gI}'

#zip the folders into one zip file

zip -rq9 properties.zip *


else
	echo "Cannot change directory to delete old files! Aborting! " 1>&2 >> $logfile
	exit 1
fi



# open the folder with contents

xdg-open $dest_dir

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script works fine until I | more

Hello all, This beats me. I have a script that executes some commands and redirects their output to some text files that I will parse. The commands are along the lines of: dsmadmc -id=admin -pa=admin -outfile=/home/tools/qlog.txt q log f=d If I just run the script it works. If I execute... (2 Replies)
Discussion started by: Skovian
2 Replies

2. Shell Programming and Scripting

perl - how come this script works?

#!/usr/bin/perl open (DATA, file.txt); @array = <DATA>; close (DATA); open (DATA, ">$file.txt"); for (@array) { s/text/replace text/; push(@contents,$_); } seek(DATA, 0, 0); print DATA (@contents); close(DATA); could someone please explain how this works. i've been... (3 Replies)
Discussion started by: mjays
3 Replies

3. UNIX for Advanced & Expert Users

Country Codes script faster response ;please help

Dear all I have group of input lines which look like this These input lines is placed in a file named phonelines.txt and there is a script which match $4 and $5 with country codes placed in another file named country-codes.txt and its contents is : Italy 39 Libyana 21892 Thuraya... (12 Replies)
Discussion started by: zanetti321
12 Replies

4. Shell Programming and Scripting

Can anyone make this script run faster?

One of our servers runs Solaris 8 and does not have "ls -lh" as a valid command. I wrote the following script to make the ls output easier to read and emulate "ls -lh" functionality. The script works, but it is slow when executed on a directory that contains a large number of files. Can anyone make... (10 Replies)
Discussion started by: shew01
10 Replies

5. Shell Programming and Scripting

Script works with bash 3.0 but not 3.2.

Hello, So my knowledge of bash scripting is not that great and I have been trying to solve this problem on my own for awhile to no avail. Here's the error I get when running it with an OS that uses bash 3.2.x: testagain.sh: line 10: *-1: syntax error: operand expected (error token is... (2 Replies)
Discussion started by: forkandspoon
2 Replies

6. Shell Programming and Scripting

Script to parse a file faster

My example file is as given below: conn=1 uid=oracle conn=2 uid=db2 conn=3 uid=oracle conn=4 uid=hash conn=5 uid=skher conn=6 uid=oracle conn=7 uid=mpalkar conn=8 uid=anarke conn=1 op=-1 msgId=-1 - fd=104 slot=104 LDAPS connection from 10.10.5.6 to 10.18.6.5 conn=2 op=-1 msgId=-1 -... (7 Replies)
Discussion started by: sags007_99
7 Replies

7. Shell Programming and Scripting

Make script faster

Hi all, In bash scripting, I use to read files: cat $file | while read line; do ... doneHowever, it's a very slow way to read file line by line. E.g. In a file that has 3 columns, and less than 400 rows, like this: I run next script: cat $line | while read line; do ## Reads each... (10 Replies)
Discussion started by: AlbertGM
10 Replies

8. Shell Programming and Scripting

Making script run faster

Can someone help me edit the below script to make it run faster? Shell: bash OS: Linux Red Hat The point of the script is to grab entire chunks of information that concerns the service "MEMORY_CHECK". For each chunk, the beginning starts with "service {", and ends with "}". I should... (15 Replies)
Discussion started by: SkySmart
15 Replies

9. Shell Programming and Scripting

Optimize shell script to run faster

data.file: contact { contact_name=royce-rolls modified_attributes=0 modified_host_attributes=0 modified_service_attributes=0 host_notification_period=24x7 service_notification_period=24x7 last_host_notification=0 last_service_notification=0 host_notifications_enabled=1... (8 Replies)
Discussion started by: SkySmart
8 Replies

10. Shell Programming and Scripting

Shell script works fine as a standalone script but not as part of a bigger script

Hello all, I am facing a weird issue while executing a code below - #!/bin/bash cd /wload/baot/home/baotasa0/sandboxes_finance/ext_ukba_bde/pset sh UKBA_publish.sh UKBA 28082015 3 if then echo "Param file conversion for all the areas are completed, please check in your home directory"... (2 Replies)
Discussion started by: ektubbe
2 Replies
All times are GMT -4. The time now is 12:00 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy