Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Merge Two files on the basis of 2 columns at diffrent position

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Merge Two files on the basis of 2 columns at diffrent position
# 1  
Old 02-26-2015
Merge Two files on the basis of 2 columns at diffrent position
Hello,

I am trying to merge two files with multiple records having a common columns but on first file its on 7th column and on 2nd file it on 6th column.

First file(file1.txt) - 7th Column is unique
Code:
H|123|Alex|Ren|W|9856746|2345789|20152202|
H|97654|Ray|John||9855678|2345790|20152201|
H|91|Roj|John||9851234|2345791|20152200|

Second File(file2.txt) - 3rd column is unique and matching first file 7th column
Code:
L|23|2345789|92311|T|76|
L|24|2345789|92312|T|76|
L|25|2345789|92313|T|76|
L|91|2345790|91111|T|90|
L|101|2345791|93311|W|86|

So the output should be like this -
Code:
H|123|Alex|Ren|W|9856746|2345789|20152202|
L|23|2345789|92311|T|76|
L|24|2345789|92312|T|76|
L|25|2345789|92313|T|76|
H|97654|Ray|John||9855678|2345790|20152201|
L|25|2345789|92313|T|76|
H|91|Roj|John||9851234|2345791|20152200|
L|101|2345791|93311|W|86|

I tried using awk to swap the file 1, 7th Coumn to 3rd Column using
Code:
awk -F'|' '{t=$7 ; $7=$3 ; $3=t; print;}' OFS='|' file1.txt > formatted_file1.txt

Now I used sort using key on both the 3rd column to merge but do not have any clue to move back column 3rd to column 7th in merge file for all the records starting with "H".

Please advice or let me know something easy way to get the output.

Thank You.
# 2  
Old 02-26-2015
Hi,
Can you explain how to obtain sort in red ?
Code:
H|123|Alex|Ren|W|9856746|2345789|20152202|
L|23|2345789|92311|T|76|
L|24|2345789|92312|T|76|
L|25|2345789|92313|T|76|
H|97654|Ray|John||9855678|2345790|20152201|
L|25|2345789|92313|T|76|
H|91|Roj|John||9851234|2345791|20152200|
L|101|2345791|93311|W|86|

Regards.
# 3  
Old 02-26-2015
Do you mean the third column not the 6th column?

In the second file the 3rd column is not unique.
They seem to be sorted in order?
If so, and if the first file the 7th column is truly unique you could try this

Code:
awk -F\| 'NR==FNR{A[$7]=$0; next} $3 in A{print A[$3]; delete A[$3]}1' file1.txt file2.txt

Code:
H|123|Alex|Ren|W|9856746|2345789|20152202|
L|23|2345789|92311|T|76|
L|24|2345789|92312|T|76|
L|25|2345789|92313|T|76|
H|97654|Ray|John||9855678|2345790|20152201|
L|91|2345790|91111|T|90|
H|91|Roj|John||9851234|2345791|20152200|
L|101|2345791|93311|W|86|

# 4  
Old 02-26-2015
Hi disedorgue,

I swaped the column 7 in first file from column 3 and now its easy to sort the file.

Code:
sort -t\| -k3,3 file1.txt file2.txt > merge_file.txt

But here problem is that when I creates a merge_file.txt. The file 1 column is already swaped so I need to change it position back to normal in merge_file.txt. Means the records starts with "H|" need to be changed - Again swapping required to change its position back to original.

Hi Scrutinizer,

Your code gives only the file2.txt results. I am not getting the desired output for multiple records.
# 5  
Old 02-26-2015
Quote:
Originally Posted by Mannu2525
[..]
Hi Scrutinizer,

Your code gives only the file2.txt results. I am not getting the desired output for multiple records.
I used you input samples and got the result in post #3

What did you get? What is your OS and version?
# 6  
Old 02-26-2015
Sorry Scrutinizer, You code is working. Thank you so much!!
# 7  
Old 02-26-2015
Try also
Code:
awk 'FNR==NR {$1=$7 "|" $1;print;next} {$1=$3 "|" $1} 1' FS="|" OFS="|"  file[34] | sort -t\| -k1,1 | cut -d\| -f2-
H|123|Alex|Ren|W|9856746|2345789|20152202|
L|23|2345789|92311|T|76|
L|24|2345789|92312|T|76|
L|25|2345789|92313|T|76|
H|97654|Ray|John||9855678|2345790|20152201|
L|91|2345790|91111|T|90|
H|91|Roj|John||9851234|2345791|20152200|
L|101|2345791|93311|W|86|

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Merge columns from two files using awk

I have two csv files : say a.csv, b.csv a.csv looks like this : property1,property2,100 property3,property4,200 In a.csv, the combination of column1 and column2 will be unique b.csv looks like this property1,property2, 300, t1 property1,property2, 400,t2 property3, property4,800,t1... (2 Replies)
Discussion started by: Lakshmikumari
2 Replies

2. Shell Programming and Scripting

Merge specific columns of two files

Hello, I have two tab delimited text files. Both files have the same number of rows but not necessarily the same number of columns. The column headers look like, File 1: f0order CVorder Name f0 RI_9 E99 E199 E299 E399 E499 E599 E699 E799 E899 E999 File 2:... (9 Replies)
Discussion started by: LMHmedchem
9 Replies

3. Shell Programming and Scripting

Merge columns from multiple files

Hello and Good day I have a lot of files with same number of rows and columns.$2 and $3 are the same in all files . I need to merge $2,$3,$6 from first file and $6 from another files. File1: $1 $2 $3 $4 $5 $6... (8 Replies)
Discussion started by: ali.seifaddini
8 Replies

4. Shell Programming and Scripting

Merge 2 files with one reference columns

Hi All Source1 servername1,patchid1 servername1,patchid2 servername1,patchid3 servername2,patchid1 servername2,patchid2 servername3,patchid4 servername3,patchid5 Source2 servername1,appname1 servername1,appname2 servername1,appname3 servername2,appname1 servername2,appname2... (13 Replies)
Discussion started by: mv_mv
13 Replies

5. Shell Programming and Scripting

Merge columns on different files

Hello, I have two files that have this format: file 1 86.82 0.00 86.82 43.61 86.84 0.00 86.84 43.61 86.86 0.00 86.86 43.61 86.88 0.00 86.88 43.61 file 2 86.82 0.22 86.84 0.22 86.86 0.22 86.88 0.22 I would like to merge these two files such that the final file looks like... (5 Replies)
Discussion started by: kayak
5 Replies

6. Shell Programming and Scripting

How to merge some files with diffrent sizes into one excel file using shell?

Hii I have these files , and I want to merge them in an excel file each file have two columns file1 title1 1 1 2 2 3 3 file2 title2 5 5 6 6 7 7 8 8 9 9 (10 Replies)
Discussion started by: maryre89
10 Replies

7. Shell Programming and Scripting

Merge two files matching columns

Hi! I need to merge two files when col1 (x:x:x) matching and adds second column from file1.txt. # cat 1.txt aaa;a12 bbb;b13 ccc;c33 ddd;d55 eee;e11 # cat 2.txt bbb;b55;34444;d55 aaa;a15;35666;a44 I try with this awk and I get succesfully first column from 1.txt: # awk -F";"... (2 Replies)
Discussion started by: fhluque
2 Replies

8. Shell Programming and Scripting

Merge columns of different files

Hi, I have tab limited file 1 and tab limited file 2 The output should contain common first column vales and corresponding 2nd column values; AND also unique first column value with corresponding 2nd column value of the file that contains it and 0 for the second file. the output should... (10 Replies)
Discussion started by: polsum
10 Replies

9. Shell Programming and Scripting

merge the two files which has contain columns

Hi may i ask how to accomplish this task: I have 2 files which has multiple columns first file 1 a 2 b 3 c 4 d second file 14 a 9 .... 13 b 10.... 12 c 11... 11 d 12... I want to merge the second file to first file that will looks like this ... (2 Replies)
Discussion started by: jao_madn
2 Replies

10. Shell Programming and Scripting

Compare two files and merge columns in a third

Hi, I'm working with snmp, with a little script I'm able to obtain from a switch a list with a couple of values with this format Port Mac 1 00:0A:0B:0C:0D:0E .... (hundred of entries) Now with a simple arp on a router I am able to obtain another list 00:0A:0B:0C:0D:0E... (20 Replies)
Discussion started by: CM64
20 Replies
Login or Register to Ask a Question