Shell Programming and Scripting

This is probably rehash but I did look.

I want a bash script that will take Item 1 in File1, traverse all lines in File2 and output if a match exists. Continuing the pattern recursively, Item2, File1, traverse all lines in File2 for a match, continue this pattern until all lines in File one have been processed.

Now, check this out, sample data.

File1 - single column of hostnames, using the short name
vsie1p990
vsie1p991
vsie1p992
...

File2 - multi-column, comma separated, the first column is the hostname(shortname)

format: shortname, IP Address, fqdn
> vsie1p992,191.167.44.212,vsie1p992.srv.us.company.com

I tried the following, but something is just not quite right:

Code:

     #!/bin/bash
    echo "Report Generated"
    date
     count=0
     while read list ; do
    {
      IT=`grep -i "$list" $2`
      If [ -n "$IT" ] ; then
         echo "Match Found: $list"
         count=`expr "$count" + 1`
      fi
     }
     done <$1
     echo "Total Matches = $count"

Example run: > ./checkit.sh list1 list2


Any help, advice, guidance would be greatly appreciated.

-Richard

In what way is it just not quite right?

Will a given hostname ever appear on more than one line in list2?

Can a hostname in list1 ever be a substring of another hostname in list1 or list2?

Does the output have to be in the order of hostnames in list1? Is it ok if the output is in the order of appearance of where those hostnames appear in list2?

Hey Don, so if I have a match, I want to echo out the matching item in File2, example

Code:

File1/Item1 = donsServer

File2/Item 5444 = donsServer,192.168.1.44,donesServer.svr.us.domain.com

how would you echo out the match from File2?

I.E.,

Code:

Match Found = donsServer,192.168.1.44,donsServer.svr.us.domain.com

* for each match found

One other thing that is confusing is, IT seems blank, but I know it is not
if I add:

Code:

      while read list ; do
      { 
         IT=`grep -i "$list" $2`
         echo "IT = $IT"

it's blank, ugh...

Thanks for your time

Moderator's Comments:

Please use CODE tags for all sample input, output, and code.

Assuming that hostnames in file1 are distinct and none of them are a subset of any other hostname, the command:

Code:

grep -Fi -f file1 file2

will give you all lines in file2 that contain any of the hostnames listed in file1 using a case insensitive match.

So, in your code:

Code:

      while read list ; do
      { 
         IT=`grep -i "$list" $2`
         echo "IT = $IT"

where is the closing } and where is the closing done? If you're creating a subshell, variables defined in the subshell won't be available in the parent.

Ok, so here is the deal, I have 179 entries in file1 and I have 118875 entries in file2.

If I do this:

Code:

cat file2 | grep -i <entry from file1>

, it returns a match

Code:

server,ip address, fqdn

The point is, all 179 entries "do" exist in file2, so I simply (ugh :-)) want
to export the hostname,ipaddress,fqdn for each match from file2

Here is the complete code (not working yet):

Code:

#!/bin/bash

clear

echo "Report Generated:" >> outfile
date >> outfile
echo >> outfile

count=0

while read list ; do
{ 
IT=`grep -i "$list" $2`
echo "IT = $IT >> outfile #this value is blank, don't understand why
count=`expr "$count" + 1`
echo "Processing Transaction Number: $count"; >> outfile
echo "$list" >> outfile
if [ -n "$IT" ] ; then
echo "Match Found: $list and $2" >> outfile
fi
}
done <$1
echo "Total Matches: $count" >> outfile

The irony is, code similar to this has worked fine for me in the past with (or using) other data sets. Obviously my "lack of understanding" is now exhibited because the code is not behaving as I would expect it would. Thus my confusion.

I'm surprised that bash ran this script without complaining about the missing " in the line:

Code:

echo "IT = $IT >> outfile #this value is blank, don't understand why

Your code would be easier to read if you would use indentation to make the structure obvious.

The braces ({ and }) in your code are unnecessary, but shouldn't affect the behavior of this script.

You could simplify some of your code by redirecting the output of the while loop instead of redirecting the output of each command in the loop, but that shouldn't affect the output produced by this script either.

What is the name of your script? What are the permissions on your script? How did you invoke your script?

If every line in file1 matches a line in file2 when using the command:

Code:

grep -i <entry_from_file1> file2

and you add the missing quote in:

Code:

echo "IT = $IT" >> outfile #this value is blank, don't understand why

and your script is executable, and you invoke it with something like the command:

Code:

./YourScriptName file1 file2

it seems like you should get something in outfile as long as file1 and file2 do not contain any whitespace characters. (If they do contain whitespace characters and you quoted them appropriately when you invoked your script, your script is still missing quotes around uses of "$1" and "$2" (but, again, if that is the problem, bash should have given you syntax errors).

What does "this value is blank" mean? Is there no output in outfile from the echo? Is there output from each echo, but it is just IT = ? What is in outfile when your script completes?

Please explain how this script is behaving differently than you expect.

Please explain what you expect this script to do.

Please show us what the script is doing?

Please answer the questions I asked in post #2 in this thread.

Does the grep command I suggested in post #4 in this thread produce any output? If so, is it suitable for the main body of your report?

How did you create your files? Using a windows text editor? Then no surprise there's no matches due to the trailing CR char (excerpt from a script run with -vx set):

Code:

grep -i $'vsie1p990\r' file2

You can find out with e.g.

Code:

cat -vet file1
vsie1p990^M$
vsie1p991^M$
vsie1p992^M$

, or hexdump / od.