Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

merge two files with awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting merge two files with awk
# 1  
Old 09-30-2012
merge two files with awk
I have two file like follows. I want to merge them according the first field of file1. The separator of file1 is tab, while the second one is ",". I tried some code, but can't get the results.

File1:
HTML Code:
AABB	6072	28	5922
BBCC	316	147	162
CCDD	907	71	231
File2:
HTML Code:
 CCDD,hTRBV12-4,hTRBJ2-3,319895
DDEE,hTRBV20-1,hTRBJ1-5,252661
Output:
HTML Code:
CCDD	907	71	231      hTRBV12-4        hTRBJ2-3       319895

Code:
awk 'NR==FNR{a[$1];next}$1 in a{print $0}' file1 FS, file2

# 2  
Old 09-30-2012
Try
Code:
join -t'     ' -j 1 file1 <(sed 's/,/        /g' file2)

Input the tab as ctrl+v ctrl+i.

Your awk line can also be made to work:
Code:
awk -F'      ' 'NR==FNR{a[$1];next}$1 in a' file1 <(sed 's/,/        /g' file2)

Last edited by mirni; 09-30-2012 at 04:07 AM..
This User Gave Thanks to mirni For This Post:
xshang
# 3  
Old 09-30-2012
Slight addition to your own awk cmd will make it work:
Code:
awk 'NR==FNR{a[$1]=$0;next}$1 in a{print a[$1], $2, $3, $4}' file1 FS="," OFS="\t" file2

This User Gave Thanks to RudiC For This Post:
xshang
# 4  
Old 09-30-2012
another awk version:
Code:
awk 'NR==FNR{A[$1]=$0; next} $1 in A && $1=A[$1]' file1 FS=, OFS='\t' file2

This User Gave Thanks to Scrutinizer For This Post:
xshang
# 5  
Old 09-30-2012
Quote:
Originally Posted by Scrutinizer
Code:
...  && $1=A[$1] ...

NICE!
This User Gave Thanks to RudiC For This Post:
xshang
# 6  
Old 09-30-2012
Quote:
Originally Posted by RudiC
NICE!
Thanks Smilie It later crossed my mind that it could be reduced a bit further still:
Code:
awk 'NR==FNR{A[$1]=$0; next} $1=A[$1]' file1 FS=, OFS='\t' file2

These 2 Users Gave Thanks to Scrutinizer For This Post:
mirni xshang
# 7  
Old 09-30-2012
Quote:
Originally Posted by Scrutinizer
Thanks Smilie It later crossed my mind that it could be reduced a bit further still:
Code:
awk 'NR==FNR{A[$1]=$0; next} $1=A[$1]' file1 FS=, OFS='\t' file2

Nice code. It works, but another problem appears.

The fields in file2 cover some fields in file1's in output, however, the value of "NF" show the right number. Confused.

Besides, what should I do if I want to change the position of file1 and file2?
I try the following code but it don't work.
Code:
awk 'NR==FNR{A[$1]=$0; next} $1=A[$1]' file2 FS='\t' OFS=, file1

Last edited by xshang; 09-30-2012 at 11:36 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Merge columns from two files using awk

I have two csv files : say a.csv, b.csv a.csv looks like this : property1,property2,100 property3,property4,200 In a.csv, the combination of column1 and column2 will be unique b.csv looks like this property1,property2, 300, t1 property1,property2, 400,t2 property3, property4,800,t1... (2 Replies)
Discussion started by: Lakshmikumari
2 Replies

2. Shell Programming and Scripting

Merge two files on awk/shell

Hi, i have two files like these: FILE 1 00:0f:35:1b:0c:00 1402691094.750049000 00:0f:35:1b:0c:00 1402691087.474893000 44:d3:ca:fd:a2:08 1402691091.865127000 30:e4:db:c1:df:de 1402691090.192464000 FILE 2_ 00:0F:35 Cisco Systems, Inc 30:E4:DB Cisco Systems, Inc I need a file 3, that... (5 Replies)
Discussion started by: bertiko
5 Replies

3. Shell Programming and Scripting

awk merge two files

file1 AAA3:WWW1:DDD1:XXX8:DDD2:XXX9 AAA6:WWW2:FFF1:XXX130:FFF1:XXX104:FFF1:XXX16 AAA7:WWW3:ZZZ1:XXX4:ZZZ2:XXX5:ZZZ3:XXX6:ZZZ4:XXX7file2 XXX8:EEE1:EEE2 XXX9:KKK1:KKK2 XXX130:OOO1:OOO2 XXX104:PPP1:PPP2 XXX16:RRR1:RRR1 XXX4:UUU1:UUU2 XXX5:III1:III2 XXX7:JJJ1:JJJ2Result... (2 Replies)
Discussion started by: vikus
2 Replies

4. Shell Programming and Scripting

awk match and merge with 2 files

Dear Awk experts! I have been trying to get a one liner for a match and merge operation, but having difficulties as I'm an awk newb. I always get stuck on the match and merge with 2 files. I have 2 files as follows: File 1: field 1,field 2,field 3,field 4,field 5,field 6,field 7,field... (6 Replies)
Discussion started by: landossa
6 Replies

5. Shell Programming and Scripting

Merge files using AWK

I want to merge data from 2nd file to 1st file based on 1st column File1 ==== data1,12,comp1 data1,13,comp2 data3,14,, File2 ==== data1,11,host1,lit data2,11,host2,lit3 data3,11,host3,lit4 Required Ouput (5 Replies)
Discussion started by: greycells
5 Replies

6. Shell Programming and Scripting

Merge 2 csv files with awk

I have 2 files pipe delimted and want to merge them based on a key e.g file 1 123$aaa$yyy$zzz 345$xab$yzy$zyz 456$sss$ttt$foo 799$aaa$ggg$dee file 2 123$hhh 345$ddd 456$xxx 888$zzz so if the key is the first field, and the result should be the common key between file 1 and 2 (6 Replies)
Discussion started by: loloAix
6 Replies

7. Shell Programming and Scripting

how to merge selective content between files using awk?

Hello, I'm new to awk. I've 2 files where H stands for header and T for trailer. The number following T gives the record count in a file. file 1 looks like this: H|A|B|C 1|2|3 1|2|4 2|3|5 T|3 file 2 looks like this: H|A|B|C 4|5|6 7|8|9 T|2 Need to merge the above 2 files such... (2 Replies)
Discussion started by: skumre
2 Replies

8. Shell Programming and Scripting

merge two files into one file use awk

Hi, guys. I have one question: I have two files: passwd and shadow (the number of records in these files are not equal)the contents of them are below: passwd: ************** ftp:x:24:24: sshd:x:71:65: uucp:x:10:14: brownj:x:5005:1000: sherrys: x :5006:1000: ... ************* ... (2 Replies)
Discussion started by: daikeyang
2 Replies

9. Shell Programming and Scripting

Merge files of differrent size with one field common in both files using awk

hi, i am facing a problem in merging two files using awk, the problem is as stated below, file1: A|B|C|D|E|F|G|H|I|1 M|N|O|P|Q|R|S|T|U|2 AA|BB|CC|DD|EE|FF|GG|HH|II|1 .... .... .... file2 : 1|Mn|op|qr (2 Replies)
Discussion started by: shashi1982
2 Replies

10. Shell Programming and Scripting

merge two files using awk

Hi Guys, I wonder whether is possible to merge two files using awk. I have two files one with 7 columns and another one with 9 columns and the first column on both files is identical so will be my key to merge the files. Any ideas.Thanks in advance. Harby. (2 Replies)
Discussion started by: hariza
2 Replies
Login or Register to Ask a Question