Comparing fields in 1 file to another file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Comparing fields in 1 file to another file
# 1  
Old 07-20-2010
Comparing fields in 1 file to another file

Need help with doing field comparisons.

File1
204.11.23.1 fastmovie.mp4
209.13.1.1 slowmovie.mp4

file 2
NY USA 201.1.1.1 200 freemovie.mp4
CA USA 204.11.23.1 404 notfastmovie.mp4
CA USA 204.11.23.1 200 fastmovie.mp4

basically need to take the first file and find exact matches in field 3 and 5 of the second file and I would print the entire line from file 2. So the output here would only be line 3 of file 2
# 2  
Old 07-20-2010
Hi,

try:

Code:
awk 'NR==FNR{a[$1$2]}{if ($3$5 in a) print $0}' file1 file2

gives me:
Code:
CA USA 204.11.23.1 200 fastmovie.mp4

HTH Chris
# 3  
Old 07-20-2010
Code:
awk 'NR==FNR{a[$1]=$2;next}($5==a[$3])' File1 File2

# 4  
Old 07-20-2010
Ok a couple things. If file 2 is huge on the magnitude of over 6 gig is awk the best way to handle it? finally in file 2 the filename field wouldn't be an exact match in file 2 it would actually be listed like /home/httpd/www/fastmovie.mp4
# 5  
Old 07-20-2010
Quote:
Originally Posted by rdcwayx
Code:
awk 'NR==FNR{a[$1]=$2;next}($5==a[$3])' File1 File2

I have a similar probelm to the OP but need to compare jsut the 1st fields of the 2 files and print out the whole line as he needs. However, I thought i'd try his problem and work from it, but your solution only seems to print out the whole of file 1...could you re-look into this please?

---------- Post updated at 11:01 AM ---------- Previous update was at 10:59 AM ----------

Quote:
Originally Posted by Christoph Spohr
Hi,

try:

Code:
awk 'NR==FNR{a[$1$2]}{if ($3$5 in a) print $0}' file1 file2

gives me:
Code:
CA USA 204.11.23.1 200 fastmovie.mp4

HTH Chris
As I stated I have a similar problem, but tried to work through this example to solve mine (I want to compare jsut the 1st field of each file), but your code seems to give a syntax error, although I have it down exactly the same. Any idea's?
# 6  
Old 07-20-2010
Quote:
Originally Posted by linuxkid
I have a similar probelm to the OP but need to compare jsut the 1st fields of the 2 files and print out the whole line as he needs. However, I thought i'd try his problem and work from it, but your solution only seems to print out the whole of file 1...could you re-look into this please?[COLOR="#738fbf"]
Code:
$ cat File1
204.11.23.1 fastmovie.mp4
209.13.1.1 slowmovie.mp4

$ Cat File2
NY USA 201.1.1.1 200 freemovie.mp4
CA USA 204.11.23.1 404 notfastmovie.mp4
CA USA 204.11.23.1 200 fastmovie.mp4

$ awk 'NR==FNR{a[$1]=$2;next}($5==a[$3])' File1 File2
CA USA 204.11.23.1 200 fastmovie.mp4

If on Solaris, use /usr/bin/nawk or /usr/xpg4/bin/awk
# 7  
Old 07-20-2010
Quote:
Originally Posted by rdcwayx
Code:
$ cat File1
204.11.23.1 fastmovie.mp4
209.13.1.1 slowmovie.mp4
 
$ Cat File2
NY USA 201.1.1.1 200 freemovie.mp4
CA USA 204.11.23.1 404 notfastmovie.mp4
CA USA 204.11.23.1 200 fastmovie.mp4
 
$ awk 'NR==FNR{a[$1]=$2;next}($5==a[$3])' File1 File2
CA USA 204.11.23.1 200 fastmovie.mp4

What would I need to do to be able to compare just the 1st fields in the 2 files? The whole equals signs confuse me slightly. Many thanks
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Speed : awk command to count the occurrences of fields from one file present in the other file

Hi, file1.txt AAA BBB CCC DDD file2.txt abc|AAA|AAAabcbcs|fnwufnq bca|nwruqf|AAA|fwfwwefwef fmimwe|BBB|fnqwufw|wufbqw wcdbi|CCC|wefnwin|wfwwf DDD|wabvfav|wqef|fwbwqfwfe i need the count of rows of file1.txt present in the file2.txt required output: AAA 2 (10 Replies)
Discussion started by: mdkm
10 Replies

2. Shell Programming and Scripting

File comparing and appending based on fields

I want to compare 2 files, locus_file.txt is a very large file and attr.txt is a small file. I want to match the first 2 columns of the first file to the second column of attr.txt and print the attributes together. locus_file.txt:large file LOC_Os02g47020, LOC_Os03g57840,0.88725114... (3 Replies)
Discussion started by: Sanchari
3 Replies

3. Shell Programming and Scripting

Join fields comparing 4 fields using awk

Hi All, I am looking for an awk script to do the following Join the fields together only if the first 4 fields are same. Can it be done with join function in awk?? a,b,c,d,8,,, a,b,c,d,,7,, a,b,c,d,,,9, a,b,p,e,8,,, a.b,p,e,,9,, a,b,p,z,,,,9 a,b,p,z,,8,, desired output: ... (1 Reply)
Discussion started by: aksijain
1 Replies

4. Shell Programming and Scripting

Script for Comparing directories and file from a text file

Hello all, I need to write a script which has following requirement: Need to read the filenames from text file and then search for the above read files in the required directory and if match found backup them in a backup folder. And also need to compare and verify whether the files in the... (7 Replies)
Discussion started by: saurau
7 Replies

5. Shell Programming and Scripting

Comparing two csv file fields using awk script

Hi All, I want to remove the rows from File1.csv by comparing the columns/fields in the File2.csv. I only need the records whose first column is same and the second column is different for the same record in both files.Here is an example on what I need. File1.csv: RAJAK|ACTIVE|1... (2 Replies)
Discussion started by: rajak.net
2 Replies

6. UNIX for Dummies Questions & Answers

Match the amount fields in the source file vs trigger file

Hello, I have to write a script to compare the sum of the amount fields in a source file and the amount field in another file. details are: based on the name of the source file (say SALES as an example), a file already available in a path will be grabbed (say SALES_ParmFile) and this file... (4 Replies)
Discussion started by: vijaylak
4 Replies

7. Shell Programming and Scripting

Remove duplicate lines from first file comparing second file

Hi, I have two files with below data:: file1:- 123|aaa|ppp 445|fff|yyy 999|ttt|jjj 555|hhh|hhh file2:- 445|fff|yyy 555|hhh|hhh The records present in file1, not present in file 2 should be writtent to the out put file. output:- 123|aaa|ppp 999|ttt|jjj Is there any one line... (3 Replies)
Discussion started by: gani_85
3 Replies

8. Shell Programming and Scripting

help with comparing fields

I have a file having records like below Field1;Filed2;Filed3 < '393200103052';'H3G';'20081204' < '393200103059';'TIM';'20110111' < '393200103061';'TIM';'20060206' < '393200103064';'OPI';'20110623' > '393200103052';'HKG';'20081204' > '393200103056';'TIM';'20110111' >... (5 Replies)
Discussion started by: shruthi123
5 Replies

9. Shell Programming and Scripting

Comparing rows in same file and writing the result in new file

Help needed... Can you tell me how to compare the last two couple entries in a file and print their result in new file..:confused: I have one file Check1.txt \abc1 12345 \abc2 12327 \abc1 12345 \abc2 12330 I want to compare the entries in Check1 and write to... (1 Reply)
Discussion started by: kichu
1 Replies

10. Shell Programming and Scripting

Need help in comparing a file with log file: perl code

Dear Members, I need a perl code: 1. Which will open first file with two columns separated by tab: 37 Distribution and seasonal variation of trace metals 85 Seasonal variability of the mixed layer in the central Bay 99 Dynamics of transparent exopolymeric particles (TEP) 103 Bacterial... (0 Replies)
Discussion started by: srsahu75
0 Replies
Login or Register to Ask a Question