Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Urgent Help Required for File Comparison using Awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Urgent Help Required for File Comparison using Awk
# 1  
Old 05-17-2011
MySQL Urgent Help Required for File Comparison using Awk
Hello All,
I am having a below requirement.

File1 contains
Code:
KEY|VIN|SEQUENCE|COST
 
101 | XXX111 | 1 | 234.22
234 | XXX111 | 2 | 134.32
444 | ABC234 | 1 | 100.22
555 | DFF611 | 1 | 734.82

FILE 2 Contains only VINs
Code:
XXX111
DFF611

Now if the VIN from file 1 is present in File 2 I want to write the whole row for the VIN in a seperate file. Please help.

Desired Output
===========
Code:
101 | XXX111 | 1 | 234.22
234 | XXX111 | 2 | 134.32
555 | DFF611 | 1 | 734.82


Please help me with an awk script which can do this task faster. Its really urgent for me
# 2  
Old 05-18-2011
Code:
awk 'NR==FNR{++a[$0]}a[$3]' file2 file1

This User Gave Thanks to danmero For This Post:
dinesh1985
# 3  
Old 05-18-2011
Try this

Code:
awk -F"|" 'NR==FNR{a[$0]}{if($2 in a){print}} file2 file1

regards,
Ahamed
This User Gave Thanks to ahamed101 For This Post:
dinesh1985
# 4  
Old 05-18-2011
MySQL Thanks a lot
Hi Ahmed,
It works ...Thanks a lot...
I want to do this processing for millions of records. Hope it completes quickly...
Thanks for your timely help..

---------- Post updated at 11:35 PM ---------- Previous update was at 11:34 PM ----------

Quote:
Originally Posted by danmero
Code:
awk 'NR==FNR{++a[$0]}a[$3]' file2 file1

Hello Danmero..
Thanks a lot for your reply...I tried it..But it is not working..Below code is working for me

---------- Post updated 05-18-11 at 12:16 AM ---------- Previous update was 05-17-11 at 11:35 PM ----------

Quote:
Originally Posted by ahamed101
Try this

Code:
awk -F"|" 'NR==FNR{a[$0]}{if($2 in a){print}} file2 file1

regards,
Ahamed
Thanks a lot Ahmed..u made my day....!!! it processed millions of records in minutes SmilieSmilie Smilie
# 5  
Old 05-18-2011
Quote:
Originally Posted by dinesh1985
Hello Danmero..
Thanks a lot for your reply...I tried it..But it is not working..Below code is working for me
Datasample posted on your OP is not what you have.
My solution works on your original datasample and ahamed101's solution don't, because you have a space char before and after the field on file1

Code:
# cat file1
KEY|VIN|SEQUENCE|COST

101 | XXX111 | 1 | 234.22
234 | XXX111 | 2 | 134.32
444 | ABC234 | 1 | 100.22
555 | DFF611 | 1 | 734.82
# cat file2
XXX111
DFF611
# awk 'NR==FNR{++a[$0]}a[$3]' file2 file1
101 | XXX111 | 1 | 234.22
234 | XXX111 | 2 | 134.32
555 | DFF611 | 1 | 734.82
# awk -F"|" 'NR==FNR{a[$0]}{if($2 in a){print}} file2 file1
> ^C
# awk -F"|" 'NR==FNR{a[$0]}{if($2 in a){print}}' file2 file1




.... no output.....

Record " XXX111 " will never match "XXX111"
# 6  
Old 05-18-2011
danmero,
Yes, you are right, if there is a space it will not work.

This should do it
Code:
awk 'NR==FNR{a[$0]}{if($3 in a){print}}' file2 file1

regards,
Ahamed
# 7  
Old 05-18-2011
Quote:
Originally Posted by ahamed101
danmero,
Yes, you are right, if there is a space it will not work.

This should do it
Code:
awk 'NR==FNR{a[$0]}{if($3 in a){print}}' file2 file1

You can replace {if($3 in a){print}} by $3 in a Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

File comparison and proccessing using awk

Hi Guys, I am having two requirement in one of my scripts. please help out to find a fast solution using AWK (since there is lot of data to be processed) 1) First snippet - File1 has two columns and file2 has three columns If any value of column 1 of file1 matches with column 1... (4 Replies)
Discussion started by: stormfield
4 Replies

2. Shell Programming and Scripting

File comparison using awk

Hi All, i have two files file1 ,file 2 file 1 col1|col2|col3|col4|col5|col6|col7|col8 11346925|0|2009-09-20|9999-12-31|100|0 11346925|0|2009-09-20|9999-12-31|120|0 12954311|0|2009-09-11|9999-12-31|100|0 12954311|0|2009-07-23|2999-12-31|120|0 12954312|0|2009-09-11|9999-12-31|100|0... (9 Replies)
Discussion started by: mohanalakshmi
9 Replies

3. Shell Programming and Scripting

File comparison using awk

my files are as follows fileA sepearated by tab /t 00 lieferungen 00 attractiop 01 done 02 forness 03 rasp 04 alwaysisng 04 funny 05 done1 fileB funnymou120112 funnymou234470 mou3raspnhdhv rddfgmoudone1438748 so all those record which are greater than 3 and which are not... (6 Replies)
Discussion started by: rajniman
6 Replies

4. UNIX for Dummies Questions & Answers

Urgent Help Required ! Please Help

HI All , Pardon me for asking some very basic questions, I would be grateful if someone can help. I am trying to execute a shell script which runs multiple processes in background. It includes various operations like copying , DB operations etc etc. Now problem is that the complete... (1 Reply)
Discussion started by: gpta_varun
1 Replies

5. Shell Programming and Scripting

awk program for file comparison

Hello there, I'm trying to write an awk program in bash shell with the following three input files: File 1 1001 1 2 3 1002 4 5 6 1003 7 8 9 1004 10 11 12 File 2 1001 11 22 33 1002 44 55 66 1004 100 111 122 ... (4 Replies)
Discussion started by: kbirde
4 Replies

6. UNIX for Advanced & Expert Users

Urgent Help required : awk/sed help to find pattern and delete till end of line

Hi, I need help with using an awk or sed filter on the below line ALTER TABLE "ACCOUNT" ADD CONSTRAINT "ACCOUNT_PK" PRIMARY KEY ("ACCT_ID") USING INDEX PCTFREE 10 INITRANS 2 MAXTRANS 255 STORAGE(INITIAL 65536 FREELISTS 1 FREELIST GROUPS 1) TABLESPACE "WMC_DATA" LOGGING ENABLE Look for... (1 Reply)
Discussion started by: rajan_san
1 Replies

7. AIX

Urgent : Help required

Hi all, Could you please give me the command to know which is the default block size for a file on AIX ? Thank a lot ! :b: (2 Replies)
Discussion started by: V3l0
2 Replies

8. Shell Programming and Scripting

Urgent help required

I have a text file, with a fixed format of certain data. I have to extract certain fields of data. Then store it in a structure. Is it possible to use shell commands by making a system call from a C PROGRAM and then store the extracted data in the structure (which has been declared in the C... (1 Reply)
Discussion started by: umanglalani
1 Replies

9. UNIX for Dummies Questions & Answers

Urgent Help required

Hi UNIX Gurus, I have got the following requirement and totally :confused: how to proceed. A file is dummy.lst is there in the following format:-- ID NAME TYPE_ID -------- --------- ----------- 1947 Asia ... (1 Reply)
Discussion started by: rahul26
1 Replies

10. Shell Programming and Scripting

Urgent help required in deleting a line without opening a file usinga shell script

Hi, I need a help in deleting a line matching a particular pattern in a file using shell script without opening the file. The file is a .c/.cpp file. Is it possible? Thanks (6 Replies)
Discussion started by: naan
6 Replies
Login or Register to Ask a Question