Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Comparing fields in 1 file to another file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Comparing fields in 1 file to another file
# 8  
Old 07-20-2010
If you only compare 1st field, then your output should be two lines.

Code:
CA USA 204.11.23.1 404 notfastmovie.mp4
CA USA 204.11.23.1 200 fastmovie.mp4

But your expect is only one line, then we will think 5th field also need be compared.

Here is the code which only compare 1st field in File1

Code:
$ awk 'NR==FNR{a[$1]=1;next} a[$3] {print}' File1 File2
CA USA 204.11.23.1 404 notfastmovie.mp4
CA USA 204.11.23.1 200 fastmovie.mp4

# 9  
Old 07-20-2010
Quote:
Originally Posted by rdcwayx
If you only compare 1st field, then your output should be two lines.

Code:
CA USA 204.11.23.1 404 notfastmovie.mp4
CA USA 204.11.23.1 200 fastmovie.mp4

But your expect is only one line, then we will think 5th field also need be compared.

Here is the code which only compare 1st field in File1

Code:
$ awk 'NR==FNR{a[$1]=1;next} a[$3] {print}' File1 File2
CA USA 204.11.23.1 404 notfastmovie.mp4
CA USA 204.11.23.1 200 fastmovie.mp4


I'm sorry but it jsut doesn't work, even the last code just printed out the whole file again for some reason. I have a slightly different file to the OP, however the layout is exactly the same. I'm doing bash scripting on unix if that makes any difference at all. I don't know why it doesn't work and its really frustrating me!!!!!!!!!!!!

---------- Post updated at 04:14 PM ---------- Previous update was at 11:34 AM ----------
Quote:
Originally Posted by rdcwayx
Code:
$ cat File1
204.11.23.1 fastmovie.mp4
209.13.1.1 slowmovie.mp4
 
$ Cat File2
NY USA 201.1.1.1 200 freemovie.mp4
CA USA 204.11.23.1 404 notfastmovie.mp4
CA USA 204.11.23.1 200 fastmovie.mp4
 
$ awk 'NR==FNR{a[$1]=$2;next}($5==a[$3])' File1 File2
CA USA 204.11.23.1 200 fastmovie.mp4

If on Solaris, use /usr/bin/nawk or /usr/xpg4/bin/awk

I am on Solaris! SORTED! thanks mate! only took me a day!
# 10  
Old 07-20-2010
Quote:
Originally Posted by rdcwayx
If you only compare 1st field, then your output should be two lines.

Code:
CA USA 204.11.23.1 404 notfastmovie.mp4
CA USA 204.11.23.1 200 fastmovie.mp4

But your expect is only one line, then we will think 5th field also need be compared.

Here is the code which only compare 1st field in File1

Code:
$ awk 'NR==FNR{a[$1]=1;next} a[$3] {print}' File1 File2
CA USA 204.11.23.1 404 notfastmovie.mp4
CA USA 204.11.23.1 200 fastmovie.mp4


So my issue with the file being too large was easily solved by just splitting it into 6 files. The 1 field match example works but I need the 1st field to be an exact match and I need the 2nd field to be contained within the 5th field of file 2. So here are the original examples revised to correctly reflect full path being in 5th field of file 2.

File1
204.11.23.1 fastmovie.mp4
209.13.1.1 slowmovie.mp4

file 2
NY USA 201.1.1.1 200 /home/httpd/www/freemovie.mp4
CA USA 204.11.23.1 404 /home/httpd/www/notfastmovie.mp4
CA USA 204.11.23.1 200 /home/httpd/www/fastmovie.mp4
# 11  
Old 07-20-2010
Try:
Code:
awk -F"[ /]" 'NR==FNR{a[$2];next} $NF in a' File1 File2
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Speed : awk command to count the occurrences of fields from one file present in the other file

Hi, file1.txt AAA BBB CCC DDD file2.txt abc|AAA|AAAabcbcs|fnwufnq bca|nwruqf|AAA|fwfwwefwef fmimwe|BBB|fnqwufw|wufbqw wcdbi|CCC|wefnwin|wfwwf DDD|wabvfav|wqef|fwbwqfwfe i need the count of rows of file1.txt present in the file2.txt required output: AAA 2 (10 Replies)
Discussion started by: mdkm
10 Replies

2. Shell Programming and Scripting

File comparing and appending based on fields

I want to compare 2 files, locus_file.txt is a very large file and attr.txt is a small file. I want to match the first 2 columns of the first file to the second column of attr.txt and print the attributes together. locus_file.txt:large file LOC_Os02g47020, LOC_Os03g57840,0.88725114... (3 Replies)
Discussion started by: Sanchari
3 Replies

3. Shell Programming and Scripting

Join fields comparing 4 fields using awk

Hi All, I am looking for an awk script to do the following Join the fields together only if the first 4 fields are same. Can it be done with join function in awk?? a,b,c,d,8,,, a,b,c,d,,7,, a,b,c,d,,,9, a,b,p,e,8,,, a.b,p,e,,9,, a,b,p,z,,,,9 a,b,p,z,,8,, desired output: ... (1 Reply)
Discussion started by: aksijain
1 Replies

4. Shell Programming and Scripting

Script for Comparing directories and file from a text file

Hello all, I need to write a script which has following requirement: Need to read the filenames from text file and then search for the above read files in the required directory and if match found backup them in a backup folder. And also need to compare and verify whether the files in the... (7 Replies)
Discussion started by: saurau
7 Replies

5. Shell Programming and Scripting

Comparing two csv file fields using awk script

Hi All, I want to remove the rows from File1.csv by comparing the columns/fields in the File2.csv. I only need the records whose first column is same and the second column is different for the same record in both files.Here is an example on what I need. File1.csv: RAJAK|ACTIVE|1... (2 Replies)
Discussion started by: rajak.net
2 Replies

6. UNIX for Dummies Questions & Answers

Match the amount fields in the source file vs trigger file

Hello, I have to write a script to compare the sum of the amount fields in a source file and the amount field in another file. details are: based on the name of the source file (say SALES as an example), a file already available in a path will be grabbed (say SALES_ParmFile) and this file... (4 Replies)
Discussion started by: vijaylak
4 Replies

7. Shell Programming and Scripting

Remove duplicate lines from first file comparing second file

Hi, I have two files with below data:: file1:- 123|aaa|ppp 445|fff|yyy 999|ttt|jjj 555|hhh|hhh file2:- 445|fff|yyy 555|hhh|hhh The records present in file1, not present in file 2 should be writtent to the out put file. output:- 123|aaa|ppp 999|ttt|jjj Is there any one line... (3 Replies)
Discussion started by: gani_85
3 Replies

8. Shell Programming and Scripting

help with comparing fields

I have a file having records like below Field1;Filed2;Filed3 < '393200103052';'H3G';'20081204' < '393200103059';'TIM';'20110111' < '393200103061';'TIM';'20060206' < '393200103064';'OPI';'20110623' > '393200103052';'HKG';'20081204' > '393200103056';'TIM';'20110111' >... (5 Replies)
Discussion started by: shruthi123
5 Replies

9. Shell Programming and Scripting

Comparing rows in same file and writing the result in new file

Help needed... Can you tell me how to compare the last two couple entries in a file and print their result in new file..:confused: I have one file Check1.txt \abc1 12345 \abc2 12327 \abc1 12345 \abc2 12330 I want to compare the entries in Check1 and write to... (1 Reply)
Discussion started by: kichu
1 Replies

10. Shell Programming and Scripting

Need help in comparing a file with log file: perl code

Dear Members, I need a perl code: 1. Which will open first file with two columns separated by tab: 37 Distribution and seasonal variation of trace metals 85 Seasonal variability of the mixed layer in the central Bay 99 Dynamics of transparent exopolymeric particles (TEP) 103 Bacterial... (0 Replies)
Discussion started by: srsahu75
0 Replies
Login or Register to Ask a Question