Bash script to diff two arrays


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Bash script to diff two arrays
# 8  
Old 02-08-2016
In addition to what has already been said, you might also want to consider:
  1. There will never be any matches in the arrays created by the statements:
    Code:
                filea=($(find /var/tmp/dir1 -type f -follow -print))
                fileb=($(find /var/tmp/dir2 -type f -follow -print))

    because every element in ${filea[@]} will start with /var/tmp/dir1 and every element in ${fileb[@]} will start with /var/tmp/dir2. With your nested for loops, you will be comparing every file in ${filea[@]} to every file in ${fileb[@]}. One would assume that you really just want to compare files with the same pathnames relative to /var/tmp/dir[12].
  2. And, if you're just trying to determine if two files have the same contents, checking the exit status of:
    Code:
    cmp -s "$afile" "$bfile"

    would be much more efficient than using:
    Code:
    diff -a --suppress-common-lines -y "$afile" "$bfile"

    and checking whether the output is an empty string.
# 9  
Old 02-08-2016
Hi Don,

I am looking to compare/diff the internal contents of two files, one will be in a SVN repo and the other will be in a server.

Essentially if someone edits the server config file I want to just test for any differences and if one does exist no matter what it is then copy that file to the repository.

I am just struggling to get the loop to accept the values from both arrays as arguments for cmp or diff

If i echo the values of $afile and $bfile they output on screen but cmp/diff returns no output even when i know there is differences in the file

Appreciate your response it makes sense but i know i still have a lot of learning to do!

Cheers
Justin
# 10  
Old 02-08-2016
I see no reason for two arrays (or even one for that matter). Presumably you have one master source file hierarchy and you want to update copies of the regular files in that master source file hierarchy in a slave file hierarchy for cases where the corresponding master and slave files are different (or the slave file is missing). That could be done with something like:
Code:
#!/bin/bash
MASTER='/var/tmp/dir1'
SLAVE='/var/tmp/dir2'

cd "$MASTER"
find . -type f -follow -print | while read file
do	if [ ! -x "$SLAVE/$file" ] || ! cmp -s "$file" "$SLAVE/$file"
	then	cp -f "$file" "$SLAVE/$file"
	fi
done

and on many systems could be replaced by a single invocation of rsync with appropriate arguments.
This User Gave Thanks to Don Cragun For This Post:
# 11  
Old 02-08-2016
I had to muck around with sed to get the "./" stripped off the front and added $MASTER to front of other strings just for my own sanity when troubleshooting/testing it out, but it all works as i want.

i just need to add some more script to automate the checkout and commit but that i have worked out

Really appreciate your help and keep up the good work! Smilie

Code:
#!/bin/bash
MASTER='/var/tmp/dir1'
SLAVE='/var/tmp/dir2'

cd "$MASTER"
find . -type f -follow -print | sed 's/..//' | while read filea
do

if [ ! -x "$SLAVE/$file" ] || ! cmp -s "$MASTER/$filea" "$SLAVE/$filea"

        then
        cp -f $MASTER/$filea $SLAVE/$filea
        fi

        done

# 12  
Old 02-08-2016
Quote:
Originally Posted by jlykke
I had to muck around with sed to get the "./" stripped off the front and added $MASTER to front of other strings just for my own sanity when troubleshooting/testing it out, but it all works as i want.

i just need to add some more script to automate the checkout and commit but that i have worked out

Really appreciate your help and keep up the good work! Smilie

Code:
#!/bin/bash
MASTER='/var/tmp/dir1'
SLAVE='/var/tmp/dir2'

cd "$MASTER"
find . -type f -follow -print | sed 's/..//' | while read filea
do

if [ ! -x "$SLAVE/$file" ] || ! cmp -s "$MASTER/$filea" "$SLAVE/$filea"

        then
        cp -f $MASTER/$filea $SLAVE/$filea
        fi

        done

I assume that you are aware that all of the following pathnames name exactly the same file when the current working directory is /var/tmp/dir1:
Code:
/var/tmp/dir1/./file
/var/tmp/dir1//file
/var/tmp/dir1/file
./file
file
/var/tmp/dir1/././././././file
/var/tmp/dir1/.///////////file

So, you can include sed in that pipeline if you want to. It will give you exactly the same results, except it will run slower with the sed than without it.

And, you can change "$file" to "$MASTER/$file" in that loop if you want to. It will give you exactly the same results, except it will run slightly slower.

But, PLEASE, do not change:
Code:
        cp -f "$filea" "$SLAVE/$filea"

to:
Code:
        cp -f $MASTER/$filea $SLAVE/$filea

With your current filenames it happens to work, but those double quotes are there to protect you against the possibility of filenames containing <space>s and <tab>s. Get into the habit of quoting the expansion of any variables that contain user supplied strings and the expansion of any variables containing pathnames not explicitly created by your script. Adding the quotes will never hurt you. Smilie Leaving out the quotes will eventually lead to you getting an irate call from a customer somewhere around midnight on a three day weekend. Smilie
# 13  
Old 02-09-2016
Appreciate the help Don!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Using Diff to compare 2 arrays

I have two arrays and they look like this: array=(`cat /local/mnt/*sys/*includes|grep -v NEW`) array2=(`cat /tmp/*sys.z |grep -v NEW`) I am trying to compare them but I need to use the diff -u command. I am not sure how to do this. I cannot just do diff -u ${array} ${array2} I cannot... (4 Replies)
Discussion started by: newbie2010
4 Replies

2. Shell Programming and Scripting

Using arrays in bash using strings to bash built-in true

I have the following code and for some reason when I call the program using /home/tcdata/tatsh/trunk/hstmy/bin/bash/raytrac.bash --cmod=jcdint.cmod I get hasArgument = hasArgument = true Somehow the array element is returning even though I have not chosen the option. ... (41 Replies)
Discussion started by: kristinu
41 Replies

3. Shell Programming and Scripting

bc,getopt and arrays in bash

trying to sum elements in an array using bc and getopt,i have a file with names and thier vaules if the names appears 3 times i should multiply its value with 3 then find the sum of all the elements together cat foo.txt max 2.3 henry 3 fransis 4.5 max 2.3 henry 3 max 2.3 it should... (1 Reply)
Discussion started by: elginmulizwa
1 Replies

4. Shell Programming and Scripting

Yet another bash arrays question

Hi all, I have a file that contains many lines, but only a few are of my interest, so I'm cutting it with grep + awk, and the result I get is for example line 0 line 1 line 2 line 3 line n Now I want to store each line in an array "cell" so I can use it later calling to ${array},... (2 Replies)
Discussion started by: TuxSax
2 Replies

5. Shell Programming and Scripting

subtraction in bash arrays

hi i am using bash shell to perform some subraction. here is what i have: i have a while loop and am using i as a counter. diff= `expr ${ARRAY1} - ${ARRAY2}` for example array1 has -0.7145 and array2 has -0.7041. when i try the above command, i get expr: non-numeric argument. any... (6 Replies)
Discussion started by: npatwardhan
6 Replies

6. Shell Programming and Scripting

arrays and while loops in bash

hi guys, i have an array called ARRAY which has elements in it... i am trying to assign elements of ARRAY to master_array.. i get a =: command not found error.. i=0 while do ${master_array}=${ARRAY} ((i++)) done is there something i am missing? (4 Replies)
Discussion started by: npatwardhan
4 Replies

7. Shell Programming and Scripting

arrays in bash

hi guys, i have the following script and when i run it i get blank lines on the screen.. i am trying to display the contents of array var.. #!/usr/bin/bash var=`awk 'NR>20&&NR<31' try.sum | awk '{print $4}'` echo "${var}" (1 Reply)
Discussion started by: npatwardhan
1 Replies

8. Shell Programming and Scripting

Searching Bash Arrays

Hi, I am writing a bash shell script. I would like to execute a statement only if an array contains a specific value. For example: array=(1 3 5 7) I would like to execute the statement only if the value 3 is present in ${array}. Thanks for any help, Mike (1 Reply)
Discussion started by: msb65
1 Replies

9. Shell Programming and Scripting

Arrays in bash.need help

:confused: Is it possible to delete array elements dynamically.For instance,consider an array( a b c d ) ,now can i delete array (the third element 'c').So that the array becomes array(a b d).. Thanks in advance!! (1 Reply)
Discussion started by: tj23
1 Replies

10. UNIX for Dummies Questions & Answers

bash: setting arrays

Ok, I searched the threads a couple of times but couldn't find anything really relevant. Here's my problem, maybe you can help: I am running version 1.14.7 of the bash shell, on Red Hat Linux. I am trying to set an array like so: bash$> letters=(x y z) spaces are between the letters but... (3 Replies)
Discussion started by: Kriton
3 Replies
Login or Register to Ask a Question