Shell Programming and Scripting

hello

I need to compare 2 text files. File 1 has 2 columns and file 2 has 1 to many.
Sample:

File 1:

111 555
222 666
333 777
444 755

File 2:

000 110 113 114
844 111 555 999
202 777 865 098 023
222 313 499
065 655 333 011 890 777
433

Results should be from file1:
222 666
444 755

I need to find all the lines from file 1 that the data in column 1 and 2 do not also exist in a row in anywhere in file 2. The files are a few thousand lines each.

I've looked at awk and nawk but to be honest I'm not sure where to begin with this.

Thanks very much

try using associative arrays -

Code:

#!/bin/ksh
# this looks at all the columns in searchfile 
# prints a line from outputfile when no column in 
#    the outputfile line matches any column in 
#     searchfile
awk '
	FILENAME=="searchfile" {
	 for(i=1;i<=NF;i++ ) {Keys[$i]++}
	}
	FILENAME=="outputfile" {
		found=0
		for(i=1;i<=NF;i++) { if(Keys[$i]>0) {found=1; break;}}
		if (found == 0) {
			print $0
		}
	}
' searchfile outputfile

In the aux file you have the solution

Code:

#!/bin/bash

echo > aux
for i in `cat file1`
do
  bool=1
  for j in `cat file2`
  do
    if [ `echo $i` = `echo $j` ]
      then
        bool=0
    fi
  done
  if [ $bool -eq 1 ]
    then
      echo $i >> aux
  fi
done

Ask if you have questions, see you

Awesome.

A couple of little changes I needed to make for it to work exactly as I needed:

awk '
FILENAME=="searchfile" {
for(i=1;i<=NF;i++ ) {Keys[$i]++}
}
FILENAME=="outputfile" {
found=0
for(i=1;i<=NF;i++) { if(Keys[$i]>0) {found++; }}
if (found != 2 ) {
print $0
}
}
' searchfile outputfile

Many thanks for your help on this!!!

can you guys tell me what's the search file and output file here. My guess is that output file is empty before the process and the search file is file2. Am i wrong in my assumption? Let me know.

The search file is the example "file 2" from my original post, the output file would be "file 1." File 1 has 2 columns to find in any row of file 2. The output of the script goes to standard out and is all rows from "file 1" whose columns do not have matching values in either column of of any row of file 2. Take a good look at the example in my original post to understand what the script does.
Cheers

Greetings folks

I've found a problem with the code, it does everything described in the original thread except it doesn't do it row by row.
For example: if i have the following files

File1
111 222
333 444
555 666

File2
000 999 211 333
111 020 222
444 990 433

The output should be:
333 444
555 666

Actual result is:
555 666

Because it finds the columns 333 and 444 from file1 in 2 different rows of file2. I need the rows from file1 to be compared to match 2 columns in of any 1 row in file2. I've been playing with this for 2 days and am once again stumped.

Thanks in advance!

#!/bin/ksh
# this looks at all the columns in hcs.txt
# prints a line from wmh.txt when no column in
# the wmh.txt line matches any column in
# hcs.txt
awk '
FILENAME=="file2" {
for(i=1;i<=NF;i++ ) {Keys[$i]++}
}
FILENAME=="file1" {
found=0
for(i=1;i<=NF;i++)
{ if(Keys[$i]>0) {found++; }}
if (found != 2 ) {
print $0; found = 0; break;
}
}
' file2 file1