I'm trying to compare multiple columns between two files. I would like to use columns 1,2 from file1 and search file2 in columns 2,3 for any matches. If they match, then print columns 2,3 from file2.
Your basic problem is that you are testing for A[$1$2] when file 2, and you should be testing for A[$2$3].
This is a bit simpler as it doesn't keep redundant information. You only need to know that you saw the fields in file one, so setting the value to 1, rather than keeping the data in those fields is all that is necessary.
Also, a personal preference of mine to separate the items used as the index with a comma when it makes sense. This prevents "abc|def" in file one from matching "ab|cdef" in file 2.
Agama, thank you very much for your help and useful tips. As you can see, I'm fairly new to using awk but I'm beginning to realize how powerful a tool it really is. Please let me know if you know of any good 'awk' resources- this is a tool I would definitely like to know better
---------- Post updated at 06:30 PM ---------- Previous update was at 05:42 PM ----------
Actually, I have one followup question. How would the commands look if I wanted to print $1,$2 from file1 that were not in $2,$3 of file2?
You are most welcome! I remember all too well some of my early frustrations with awk and am more than happy to pass my experience along to others.
If you like real, paper, books, the O'Reilly - Sed & Awk 2Nd Edition book O'Reilly - Sed & Awk 2Nd Edition is decent. If you like reading html/pdf the doc linked from this page seems good: The GNU Awk User's Guide Personally, I've gotten by with just the manual pages.
Quote:
Originally Posted by pinseeker
Actually, I have one followup question. How would the commands look if I wanted to print $1,$2 from file1 that were not in $2,$3 of file2?
With a few small changes, your programme can print the fields from the first file that weren't found in the second.
Code:
# print data from file 1 that is not in file 2
awk -F\| '
FILENAME=="file1" {
a[$1"|"$2] = 1;
next;
}
FILENAME=="file2" {
a[$2"|"$3] = 0;
}
END {
for( x in a )
if( a[x] ) # still set, not seein in file 2
print x;
}
' file1 file2
A couple of things to note...
1) I use lowercase variable names -- uppercase imples an awk maintained/generated variable like NF, NR or FILENAME.
2) I've changed the index to be a single string of both fields with a vertical bar. This still prevents abc|def in one file from being matched with ab|cdef in another, and lets me print the index out nicely without any extra logic to mash it up.
Thank you once again - the updated program works perfectly! Just as important, thank you for the references and the explanations in the code. I have been struggling with this for the past several days and other methods I have tried have taken too long to complete (file1/file2 in my case are a few hundred thousand to several million lines in length). When I first tried AWK to perform this task, it seemed fast enough, but I just couldn't get the code right to produce the right results.....best regards.
Hi,
I'm trying to copy and paste the sixth column from a bunch of files into a single file having each column pasted in separate columns (and not one after each other in just one column.)
I tried this code but works only partially because it copied and pasted 50 rows of each column... (6 Replies)
I have two csv files : say a.csv, b.csv
a.csv looks like this :
property1,property2,100
property3,property4,200
In a.csv, the combination of column1 and column2 will be unique
b.csv looks like this
property1,property2, 300, t1
property1,property2, 400,t2
property3, property4,800,t1... (2 Replies)
Hi,
I want to compare the columns of two files excluding column 2 from both the files. I tried this awk command.
awk -F":" 'NR==FNR{++a;next} !(a)' file1.txt file2.txt
.
Example: File1.txt
123:09-15-2011:abc:123456
123:09-15-2011:abc:234567
123:09-15-2011:abc:345678
... (5 Replies)
How can I use awk to create a new file that has 2 columns, each colums comes form a different file.
example:
I need column 3 from file1 and column 5 from file2 to make file3. (3 Replies)
Hi All,
as you can see I'm pretty new to this board. :D
I'm struggling around with small script to search a few fields in another file.
Basically I have file1 looking like this:
15:38:28 sz:10001 pr:14.16
15:38:28 sz:10002 pr:18.41
15:38:29 sz:10003 pr:19.28
15:38:30 sz:10004... (1 Reply)
08-04-2010
