Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

ksh scripting SSH to Compare File Sizes

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting ksh scripting SSH to Compare File Sizes
# 1  
Old 10-31-2018
ksh scripting SSH to Compare File Sizes
Hello,
I currently have very little experience with Shell scripting and trying to create a script for the purpose of collecting the size of a couple sizes on 4 different Hosts. The Idea is to collected the information from the files in which the script is kicked off on, store the values into Variables, SSH to the next Host and repeat for the remaining 2. With Variables Collected I would like to compare the sizes and output that they matched.

This is what I have come up with so far but not the most familiar with constructs and especially the SSH Process.

Code:
#!/bin/ksh

FILENAME1=/root/directory1/files/file1.fil
FILE1SIZEa=$(stat -c%s "$FILENAME1")

FILENAME2=/root/directory2/files/file2.fil
FILE2SIZEa=$(stat -c%s "$FILENAME2")

ssh user@server2 |

FILENAME3=/root/directory1/files/file1.fil
FILE1SIZEb=$(stat -c%s "$FILENAME3")

FILENAME4=/root/directory2/files/file2.fil
FILE2SIZEb=$(stat -c%s "$FILENAME4")

ssh user@server3 |

FILENAME5=/root/directory1/files/file1.fil
FILE1SIZEc=$(stat -c%s "$FILENAME5")

FILENAME6=/root/directory2/files/file2.fil
FILE2SIZEc=$(stat -c%s "$FILENAME6")

ssh user@server4 |

FILENAME7=/root/directory1/files/file1.fil
FILE1SIZEd=$(stat -c%s "$FILENAME7")

FILENAME8=/root/directory2/files/file2.fil
FILE2SIZEd=$(stat -c%s "$FILENAME8")


if && [[ "$FILE1SIZEa" =~ ^($FILE1SIZEb|$FILE1SIZEc|$FILE1SIZEd)$ ]]; then
	print "FILE1 Sizes Match"
else
	print "Something is Wrong with FILE1"
fi

if && [[ "$FILE2SIZEa" =~ ^($FILE2SIZEb|$FILE2SIZEc|$FILE2SIZEd)$ ]]; then
	print "FILE2 Sizes Match"
else
	print "Something is Wrong with FILE2"
fi

If someone could help get this functioning, or possibly has suggestions for more efficient scripting with less repetition It would be greatly appreciated.


Thanks for your consideration.
# 2  
Old 10-31-2018
You should build a function that do what you want.



Then loop over the servers with a "while" or a "for" instruction and call that function.


You should also choose whether you want to hold this server list into a config file or a variable.
# 3  
Old 10-31-2018
I appreciate the advice, I was kind of thinking the same thing myself while looking at it that I needed some sort of for loop to iterate through the servers executing the same command while storing file locations in variables and simply comparing each server to the 1st. I don't really have any idea of how i could apply a for loop to ssh multiple servers that were in an Array for instance. Though I did rework the code a bit and came up with the following solution, though you couldn't be as specific in your output you could easily spot an error.

Code:
#!/bin/ksh


FILE1=/root/directory1/files/file1.fil
FILE2=/root/directory2/files/file2.fil
FILESZ1==$(stat -c%s "$FILE1")
FILESZ2==$(stat -c%s "$FILE2")

filecheck() {
CHKFILE1=$(stat -c%s "$FILE1")
CHKFILE2=$(stat -c%s "$FILE2")
if [[ $FILESZ1 = $CHKFILE1 ]]; then
	print "file sizes match"
else
	print "file sizes do not match"
fi
if [[ $FILESZ2 = $CHKFILE3 ]]; then
	print "file sizes match"
else
	print "file sizes do not match"
fi
}

ssh user@server2 "$(typeset -f filecheck); filecheck;
ssh user@server3 "$(typeset -f filecheck); filecheck;
ssh user@server4 "$(typeset -f filecheck); filecheck;

Would the following code do the trick or am I missing something?
Thanks for the suggestions.
# 4  
Old 10-31-2018
You will have to define the function on each node separately, and the local variables like FILESZ1 won't be available there.


How about creating a server file like
Code:
$ cat svctl
user@server2
user@server3
user@server4

to which you could even add your local server entry (user@server1) as the first line provided it has an sshd service running, and then, using shell arrays for the file sizes
Code:
$ FILE1=/root/directory1/files/file1.fil
$ FILE2=/root/directory2/files/file2.fil 
$ while read SV; do { read FILESZ1[++CNT]; read FILESZ2[CNT]; } <<< $(ssh -n $SV "stat -c%s $FILE1; stat -c%s $FILE2"); done <  svctl

leaving you with two file size arrays of four elements each to work upon locally. Admittedly, this has been tested on bash only, as I don't have a ksh machine here, but corrections - if necessary at all - should be minor.
This User Gave Thanks to RudiC For This Post:
Abstract3000
# 5  
Old 10-31-2018
Thanks for the feedback and option, Though being very new this that code is a bit over my head so I'm currently Dissecting it right now to try and get a better understanding as to what exactly is going on piece by piece.

So if i have this correct for the 1st part you could create a file like SV.fil in the same directory with code such as:

Code:
$ cat svctl
user@server1
user@server2
user@server3
user@server4

I get that the cat command is simply stating "Read the following Contents" what is the "svctl" all about?

Next it appears you are starting a while loop that's reading that SV.fil that would have to be in the same directory in the loop it appears you have created 2 Arrays "read FILESZ1[++CNT] read FILESZ2[CNT]", which i'm guessing the "++" increments it, though why is that not in "FILESZ2[CNT]" ? Did I not understand that portion correctly?

The rest of it look self explanatory iterating through the servers collecting the file size of each file adding them to each array (yet then there's that svctl again?)

Once this line has been executed I then have 2 Arrays in which I can simply use functions to compare all values and give outputs Smilie

Sorry I'm new and this scripting, and appreciate you taking the time Smilie
Last edited by RudiC; 10-31-2018 at 11:59 AM..
# 6  
Old 10-31-2018
We need a text file containing the login information of the servers to visit. Choose a file name to taste, call it SV.fil, or, as I did, svctl (~ "server control"). No "extension" (remainder from MSDOS times) needed. As I can see, you already added your local host for ease of operation.



Yes,cat copies its stdin (or the files given as arguments) to stdout - that was just for showing readers the contents of that file.


In the while loop, we're reading the server info line by line into the SV variable, then execute a "compound command" consisting of two reads to populate two arrays' elements indexed by the CNT variable which in turn is pre-incremented once per loop, so both arrays are indexed by the same value. stdin of this compound command is redirected from a "here string", consisting of a "command substitution" $(...) that provides the stdout of the entire ssh command including the two stat commands executed on the remore node for the two target files.



With four servers in the control file, we should end up with eight file sizes, four for file1 and four for file2, in arrays FILESZ1 and FILESZ2, indexed 1 ... 4 (given CNT was unset or reset to 0 upfront). You now can work on those array elements, as you did before, like
Code:
if [ ${FILESZ1[1]} -eq ${FILESZ1[2]} ]; then

. You could do it e.g. in a for loop 2 ... 4 to improve efficiency and readability.
This User Gave Thanks to RudiC For This Post:
Abstract3000
# 7  
Old 10-31-2018
Another approach using ksh. I setup two arrays of the FILES to check and the HOSTS.

Then as and example, I print each file name and it's size on host numbers 0,1 and 2.

Code:
#!/bin/ksh

set -A FILES /root/directory1/files/file1.fil /root/directory2/files/file2.fil
set -A HOSTS user@server2 user@server3

i=0
while [[ $i -lt ${#HOSTS[@]} ]]
do
   set -A FS$i $(ssh ${HOSTS[i]} "stat -c%s ${FILES[@]}")
   ((i++))
done

i=0
while [[ $i -lt ${#FILES[@]} ]]
do
  echo ${FILES[i]} ${HOSTS[0]}=${FS0[i]} ${HOSTS[1]}=${FS1[i]} ${HOSTS[2]}=${FS2[i]}
  ((i++))
done

These 2 Users Gave Thanks to Chubler_XL For This Post:
Abstract3000 RudiC
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Ksh: how compare content of a file with an other array

Hi, I created a skript in ksh which generate a file with semicolon as separator, this is an example of the file a created: example content file: hello;AAAA;2014-08-17 hello;BBBB;2014-08-17 hello;CCCC;2014-08-17 I would need to compare the content in of the second column of this file... (3 Replies)
Discussion started by: jmartin
3 Replies

2. UNIX for Dummies Questions & Answers

ksh scripting, skip server if asks for password with SSH

Hi, I am running a script that connets to a list of servers with SSH and runs a command but I have some servers that are asking for password (authorized keys is not configured properly). Is there any way to do so that if I get a prompt for password just skip that entry? my script: ... (1 Reply)
Discussion started by: galuzan
1 Replies

3. Shell Programming and Scripting

Using csh / awk / sed to compare database sizes in a txt file

Hello, I have an output file showing database sizes across the 3 environments that I use (LIVE, TEST & DEVELOPMENT). I am trying to write a script that lets me know if the size of a db on one environment is different to its corresponding db on the other environments. Here is an example... (4 Replies)
Discussion started by: stevie_g
4 Replies

4. Shell Programming and Scripting

Script to compare file sizes

I need to write a bash script larger X Y that compares the sizes of two specified files X and Y, and reports which file is larger. For example, if X is larger, the output should be "File X is larger", while if Y is larger, the output should be "File Y is larger". If the files are exactly the... (3 Replies)
Discussion started by: julia_21436
3 Replies

5. UNIX for Dummies Questions & Answers

Compare two file sizes.

Hi everyone! I need to compare two file sizes. One of them (size) will be stored in a flat file and the other coming from a listed file. I can now get the first file size using: SIZE=`ls -l $DOCTYPE | awk '{print $5}'` 1. How can I store this value in a flat file? 2. How... (2 Replies)
Discussion started by: mrreds
2 Replies

6. HP-UX

compare file percent sizes

I need to get a file size and compare it to a previous day file size. If it's larger or smaller by 50 percent I'll replace the new with the old. I know how to get the file sizes but do not know how to calculate if it's 50 percent difference. Thanks for your help. (2 Replies)
Discussion started by: jkuchar747
2 Replies

7. Shell Programming and Scripting

KSH: Compare variable to $1 in an input file

Hello, I am working with KSH on AIX and I have 2 files generated from different sources... as seen below: FILE1 FILE2 AAA AAA@ABS0001C BBB BBB@ABS0003D CCC CCC@ABS0023A DDD DDD@ABC0145D EEE EEE@ABS0090A FFF FFF@ABS0002A GGG GGG@ABC0150D HHH FILE1 is main main data source,... (4 Replies)
Discussion started by: right_coaster
4 Replies

8. Shell Programming and Scripting

how to compare file sizes

hi ls -l * | sed 's/\+/ /g' | cut -f5 -d " " >out1 ls -l * | sed 's/\+/ /g' | cut -f5 -d " " >out2 diff out1 out2 i tried this it will work fine and i can see difference but i need a script which should neglect, if the difference b/w files is small and it should display... (5 Replies)
Discussion started by: revenna
5 Replies

9. Shell Programming and Scripting

to compare total directory structure and get sizes of all f on two different servers

Hello every one, Iam newbie to this forum and shell programming &scripting. i needed to compare each and every folder of two separate servers. Actually I have copied some directory structure from one server to second server, to build on second server the files all should be copied... (3 Replies)
Discussion started by: mannam srinivas
3 Replies

10. Shell Programming and Scripting

compare file sizes

Is there a command that will return the name of the largest file within a directory? If so, can I set the returned filename into a variable? (4 Replies)
Discussion started by: joli
4 Replies
Login or Register to Ask a Question