Column Merge?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Column Merge?
# 1  
Old 10-22-2005
Tools Column Merge?

Hi all,

I need to join two columns from two different files file1.txt and file2.txt and append in a new file. Example :
$cat file1.txt
ABCD.ksh:010141
ABCD.ksh:010511
ABCD.ksh:010815
ABCD.ksh:011114
ABCD.ksh:011415
ABCD.ksh:011720
ABCD.ksh:012022
ABCD.ksh:012830
ABCD.ksh:014432
ABCD.ksh:020532

$cat file2.txt
ABCD.ksh:010148
ABCD.ksh:010522
ABCD.ksh:010849
ABCD.ksh:011124
ABCD.ksh:011421
ABCD.ksh:011731
ABCD.ksh:012511
ABCD.ksh:014012
ABCD.ksh:015512
ABCD.ksh:021010

I need to get the output of something like this (say with : separator)

ABCD.ksh 10141 10148
ABCD.ksh 10511 10522
ABCD.ksh 10815 10849
ABCD.ksh 11114 11124
ABCD.ksh 11415 11421
ABCD.ksh 11720 11731
ABCD.ksh 12022 12511
ABCD.ksh 12830 14012
ABCD.ksh 14432 15512
ABCD.ksh 20532 21010

A simple output of :

10141 10148
10511 10522
10815 10849
11114 11124
11415 11421
11720 11731
12022 12511
12830 14012
14432 15512
20532 21010

May also help.

I am trying $ join -t: file1.txt file2.txt
But it results the Cartesian product (10x10=100) rows.

Any help ??

~
# 2  
Old 10-25-2005
These both work for me...but maybe I am not understanding you correctly.

$join -t: file1.txt file2.txt | more
ABCD.ksh:010141:010148
ABCD.ksh:010141:010522
ABCD.ksh:010141:010849
ABCD.ksh:010141:011124
ABCD.ksh:010141:011421
ABCD.ksh:010141:011731
ABCD.ksh:010141:012511
ABCD.ksh:010141:014012
ABCD.ksh:010141:015512
ABCD.ksh:010141:021010
ABCD.ksh:010511:010148
ABCD.ksh:010511:010522
ABCD.ksh:010511:010849
ABCD.ksh:010511:011124
ABCD.ksh:010511:011421
ABCD.ksh:010511:011731
ABCD.ksh:010511:012511
ABCD.ksh:010511:014012
ABCD.ksh:010511:015512
ABCD.ksh:010511:021010
ABCD.ksh:010815:010148
ABCD.ksh:010815:010522

$join -t: file1.txt file2.txt | cut -d: -f2,3 | more
010141:010148
010141:010522
010141:010849
010141:011124
010141:011421
010141:011731
010141:012511
010141:014012
010141:015512
010141:021010
010511:010148
010511:010522
010511:010849
010511:011124
010511:011421
010511:011731
010511:012511
010511:014012
010511:015512
010511:021010
010815:010148
010815:010522
# 3  
Old 10-25-2005
Thanks. But..

while I am usine
$join -t: file1.txt file2.txt

I am getting the cartesian product - that is the all possible join combination 10x10 = 100 rows.

/home/mukher2> join -t: file1 file2|wc -l
100
/home/mukher2> wc -l file*
10 file1
10 file2
20 total
/home/mukher2>

I am using HP-UX B.11.11 and /usr/bin/join.


But the problem resolved when I used :

/home/mukher2> pr -tm file1 file2
ABCD.ksh:010141 ABCD.ksh:010148
ABCD.ksh:010511 ABCD.ksh:010522
ABCD.ksh:010815 ABCD.ksh:010849
ABCD.ksh:011114 ABCD.ksh:011124
ABCD.ksh:011415 ABCD.ksh:011421
ABCD.ksh:011720 ABCD.ksh:011731
ABCD.ksh:012022 ABCD.ksh:012511
ABCD.ksh:012830 ABCD.ksh:014012
ABCD.ksh:014432 ABCD.ksh:015512
ABCD.ksh:020532 ABCD.ksh:021010
/home/mukher2>

The desired output can be obtained by :

pr -tm file1 file2 |awk '{print $1":"$2}'
# 4  
Old 10-25-2005
sabyasm,

thanks for the reply, I understand what you are saying and glad that you have found the solution...this is another variation which makes it more pretty for you (there is probably a better way of doing it using awk BUT this is just faster for me):

$pr -tm file1.txt file2.txt | awk '{print $1":"$2}' | cut -d: -f2,4
010141:010148
010511:010522
010815:010849
011114:011124
011415:011421
011720:011731
012022:012511
012830:014012
014432:015512
020532:021010

OR

$pr -tm file1.txt file2.txt | awk '{print $1":"$2}' | cut -d: -f2,4 | sed s/\:/\ /g
010141 010148
010511 010522
010815 010849
011114 011124
011415 011421
011720 011731
012022 012511
012830 014012
014432 015512
020532 021010

Smilie
# 5  
Old 10-25-2005
thanks a lot! Smilie
# 6  
Old 10-25-2005
nawk -f syb.awk file1.txt file2.txt

syb.awk:
Code:
BEGIN {
  FS=":"
  OFS=" "
}

FNR==NR {
  arr[FNR] = $1 OFS $2
  next
}
{
   print arr[FNR], $2
}

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Merge csvs with column headers

hello gurus, Somebody must have done this before, I couldn't find anything. Please redirect me if this was solved before, and if not please help. To the problem now, I have multiple csv files (about 1000) which I need to concatenate by column header. The final file should have a superset... (4 Replies)
Discussion started by: abh.kumar
4 Replies

2. Shell Programming and Scripting

Merge column file

Hi All, I have on file1 and file2 some, $cat file1 aaa bbb ccc ddd eee fff ggg hhh iii jjj with line blank, and (12 Replies)
Discussion started by: aav1307
12 Replies

3. Shell Programming and Scripting

Multiple file merge by column

Hello all, I am quite new in linux shell scripting and I have this issue. I ve got some files including measurements taken every 10minutes for a whole day. File name format is: 00.00, 00.10, 00.20,....23.50 File structure is: x | y | temperature x and y is the same in all files (same... (12 Replies)
Discussion started by: atzounis
12 Replies

4. Shell Programming and Scripting

Merge column headers and transpose

Hello Everyone! I am new on this forum and this is my first post. I wish to apologize for my, not canonical, English. I would like to solve this problem but I have no clue of how do it!I will be grateful if someone could help me! I have a table like this: gene TF1 TF2 TF3 TF4 gene1 1 2 3 4... (5 Replies)
Discussion started by: giuliangiuseppe
5 Replies

5. Shell Programming and Scripting

Merge with common column

hi i have two files and i wanted to join them using common column. try to do this using "join" command but that did not help. File 1: 123 9a.vcf hy92.vcf hy90.vcf Index Ref Alt Ref Alt Ref Alt 315 14 0 7 4 ... (6 Replies)
Discussion started by: empyrean
6 Replies

6. Shell Programming and Scripting

Count and merge using common column

I have the following records from multiple files. 415 A G 415 A G 415 A T 415 A . 415 A . 421 G A 421 G A,C 421 G A 421 G A 421 G A,C 421 G . 427 A C 427 A ... (3 Replies)
Discussion started by: empyrean
3 Replies

7. Shell Programming and Scripting

column merge same value

Dear All, I have the following file: file1 "1" 189218400 189221399 3000 "0041 ENSMUSG00000000031" "0041" "+" "ENSMUSG00000000031" 189039418 189175306 "downstream" 178982 43094 "NearestStart" "1" 197067200 197068353 1154 "0057... (3 Replies)
Discussion started by: paolo.kunder
3 Replies

8. Shell Programming and Scripting

merge same pattern of same column in one line

Hello, I have some problem in the modified shell script. I would like to merge the same word in column 1 in one line. Example : A1 B1 2 A2 B1 4 A3 B1 7 A1 B2 1 A2 B2 10 A3 B2 8 Expected output : A1 B1 B2 2 1 A2 B1 B2 4 10 A3 ... (6 Replies)
Discussion started by: awil
6 Replies

9. UNIX for Advanced & Expert Users

merge two column multiple files into one

Hi I have multiple files each with two columns and I need to combine all those file into a tab delimited file. (multiple entry with same name separated by a comma) The content of the files are as follows: --- file1.txt: name var1 aaa xx aaa gg bbb yy ddd zz --- file2.txt ... (8 Replies)
Discussion started by: mary271
8 Replies

10. Shell Programming and Scripting

Merge Two Files based on First column

Hi, I need to join two files based on first column of both files.If first column of first file matches with the first column of second file, then the lines should be merged together and go for next line to check. It is something like: File one: 110001 abc efd 110002 fgh dfg 110003 ... (10 Replies)
Discussion started by: apjneeraj
10 Replies
Login or Register to Ask a Question