Merge columns from multiple files


Login or Register for Dates, Times and to Reply

 
Thread Tools Search this Thread
# 1  
Merge columns from multiple files

Hello and Good day

I have a lot of files with same number of rows and columns.$2 and $3 are the same in all files .
I need to merge $2,$3,$6 from first file and $6 from another files.

File1:
Code:
$1      $2            $3           $4            $5         $6
75	3.66192	101.54361	2.000		5.5	0.0804649
79	3.63195	101.54361	2.000		7.5	0.107302
79	3.63195	101.55112	2.000		7.5	0.107302
78	3.62445	101.54361	2.000		7	0.0998519
78	3.62445	101.55112	2.000		7	0.0998519
0	3.61696	101.54361	2.000		-32	0.000364633
0	3.61696	101.55112	2.000		-32	0.000364633

File2:
Code:
$1      $2            $3           $4            $5         $6
0	3.66192	101.54361	2.000		-32	0.000364633
0	3.63195	101.54361	2.000		-32	0.000364633
0	3.63195	101.55112	2.000		-32	0.000364633
0	3.62445	101.54361	2.000		-32	0.000364633
0	3.62445	101.55112	2.000		-32	0.000364633
0	3.61696	101.54361	2.000		-32	0.000364633
0	3.61696	101.55112	2.000		-32	0.000364633

File3:
Code:
$1      $2            $3           $4            $5         $6
80	3.66192	101.54361	2.000		8	0.115307
73	3.63195	101.54361	2.000		4.5	0.0696797
73	3.63195	101.55112	2.000		4.5	0.0696797
0	3.62445	101.54361	2.000		-32	0.000364633
0	3.62445	101.55112	2.000		-32	0.000364633
0	3.61696	101.54361	2.000		-32	0.000364633
0	3.61696	101.55112	2.000		-32	0.000364633

and so on....

Expected Result:
Code:
$2         $3           $6 File1       $6 File2     $6File3 .....
3.66192	101.54361   0.0804649       0.000364633   0.115307
3.63195	101.54361   0.107302	    0.000364633   0.0696797
3.63195	101.55112   0.107302	    0.000364633   0.000364633
3.62445	101.54361   0.0998519       0.000364633   0.000364633
3.62445	101.55112   0.0998519       0.000364633   0.000364633
3.61696	101.54361   0.000364633    0.000364633   0.000364633
3.61696	101.55112   0.000364633    0.000364633   0.000364633

Thanks in advance

Last edited by ali.seifaddini; 05-26-2014 at 06:32 AM..
# 2  
Code:
awk 'BEGIN{OFS = "\t"}
  NR == FNR {a[$2 " " $3] = $6; next}
  {a[$2 " " $3] = (a[$2 " " $3] OFS $6)}
  END {for(x in a) print (x OFS a[x])}' file1 file2 file3...

This User Gave Thanks to SriniShoo For This Post:
# 3  
Hi
Thanks for your response.
I have 144 files. Is it needed to put all files one by one in the script? (File1 file2...). How can be used loop for this script?
And also it is important assign name of each file to related column. Thanks

---------- Post updated at 12:48 PM ---------- Previous update was at 12:46 PM ----------

Quote:
Originally Posted by SriniShoo
Code:
awk 'BEGIN{OFS = "\t"}
  NR == FNR {a[$2 " " $3] = $6; next}
  {a[$2 " " $3] = (a[$2 " " $3] OFS $6)}
  END {for(x in a) print (x OFS a[x])}' file1 file2 file3...

Hi
Thanks for your response.
I have 144 files. Is it needed to put all files one by one in the script? (File1 file2...). How can be used loop for this script?
And also it is important assign name of each file to related column. Thanks
# 4  
Hello,

you can use following, just adding a * to SriniShoo's code.

Code:
 awk 'NR==FNR{a[$2 " " $3]=$6;next} {a[$2 " " $3]=a[$2 " "  $3] OFS $6} END{for(i in a){print i OFS a[i]}}' File*


Output will be as follows.

Code:
3.61696 101.55112 0.000364633 0.000364633 0.000364633
3.62445 101.55112 0.0998519 0.000364633 0.000364633
3.63195 101.54361 0.107302 0.000364633 0.0696797
3.61696 101.54361 0.000364633 0.000364633 0.000364633
3.66192 101.54361 0.0804649 0.000364633 0.115307
3.62445 101.54361 0.0998519 0.000364633 0.000364633
3.63195 101.55112 0.107302 0.000364633 0.0696797



Thanks,
R. Singh
# 5  
Quote:
Originally Posted by RavinderSingh13
Hello,

you can use following, just adding a * to SriniShoo's code.

Code:
 awk 'NR==FNR{a[$2 " " $3]=$6;next} {a[$2 " " $3]=a[$2 " "  $3] OFS $6} END{for(i in a){print i OFS a[i]}}' File*


Output will be as follows.

Code:
3.61696 101.55112 0.000364633 0.000364633 0.000364633
3.62445 101.55112 0.0998519 0.000364633 0.000364633
3.63195 101.54361 0.107302 0.000364633 0.0696797
3.61696 101.54361 0.000364633 0.000364633 0.000364633
3.66192 101.54361 0.0804649 0.000364633 0.115307
3.62445 101.54361 0.0998519 0.000364633 0.000364633
3.63195 101.55112 0.107302 0.000364633 0.0696797



Thanks,
R. Singh
Hello sir

Something is wrong with the script because there are some extra columns and also rows are miss ordered.
I attached sample data for your try.

Thank you so much.
Sample.rar
# 6  
Quote:
Originally Posted by SriniShoo
Code:
awk 'BEGIN{OFS = "\t"}
  NR == FNR {a[$2 " " $3] = $6; next}
  {a[$2 " " $3] = (a[$2 " " $3] OFS $6)}
  END {for(x in a) print (x OFS a[x])}' file1 file2 file3...

Hello sir
Is it possible to run the script with sample files which I attached on the above message.
There are some extra columns and the rows disarranged.
Regards.
# 7  
The awk for-in loop has no defined ordering.

Regards,
Alister
Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #379
Difficulty: Medium
Solaris is UNIX V7, UNIX 03 and UNIX 98 compliant.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Join and merge multiple files with duplicate key and fill void columns

Join and merge multiple files with duplicate key and fill void columns Hi guys, I have many files that I want to merge: file1.csv: 1|abc 1|def 2|ghi 2|jkl 3|mno 3|pqr file2.csv: (5 Replies)
Discussion started by: yjacknewton
5 Replies

2. UNIX for Beginners Questions & Answers

Merge multiple columns into one using cat

I will like to merge several files using 'cat', but I observe the output is not consistent. the merge begins at the last line of the first file. file1.txt: 1234 1234 1234 file2.txt: aaaa bbbb cccc dddd cat file1.txt file2.txt > file3.txt file3.txt: 1234 1234 1234aaaa bbbb cccc... (13 Replies)
Discussion started by: geomarine
13 Replies

3. Shell Programming and Scripting

Merge specific columns of two files

Hello, I have two tab delimited text files. Both files have the same number of rows but not necessarily the same number of columns. The column headers look like, File 1: f0order CVorder Name f0 RI_9 E99 E199 E299 E399 E499 E599 E699 E799 E899 E999 File 2:... (9 Replies)
Discussion started by: LMHmedchem
9 Replies

4. Shell Programming and Scripting

Merge files based on columns

011111123444 1234 1 20000 011111123444 1235 1 30000 011111123446 1234 3 40000 011111123447 1234 4 50000 011111123448 1234 3 50000 File2: 011111123444,Rsttponrfgtrgtrkrfrgtrgrer 011111123446,Rsttponrfgtrgtr 011111123447,Rsttponrfgtrguii 011111123448,Rsttponrfgtrgtjiiu I have 2 files... (4 Replies)
Discussion started by: vinus
4 Replies

5. Shell Programming and Scripting

Merge records based on multiple columns

Hi, I have a file with 16 columns and out of these 16 columns 14 are key columns, 15 th is order column and 16th column is having information. I need to concate the 16th column based on value of 1-14th column as key in order of 15th column. Here are the example file Input File (multiple... (3 Replies)
Discussion started by: Ravi Agrawal
3 Replies

6. Shell Programming and Scripting

Merge columns on different files

Hello, I have two files that have this format: file 1 86.82 0.00 86.82 43.61 86.84 0.00 86.84 43.61 86.86 0.00 86.86 43.61 86.88 0.00 86.88 43.61 file 2 86.82 0.22 86.84 0.22 86.86 0.22 86.88 0.22 I would like to merge these two files such that the final file looks like... (5 Replies)
Discussion started by: kayak
5 Replies

7. UNIX for Dummies Questions & Answers

Merge columns from multiple files

Hi all, I've searched the web for a long time trying to figure out how to merge columns from multiple files. I know paste will append columns like so: paste file1 file2 file3 file4 file5 ... But this becomes inconvenient when you want to append a large number of files into a single file. ... (2 Replies)
Discussion started by: torchij
2 Replies

8. UNIX for Dummies Questions & Answers

How do I merge multiple columns into one column?

Hi all, I'm looking for a way to merge multiple columns (from one file) into a single column in an output file. The file I have looks somewhat like this: @HWI-ST212 1:N:0 AGTCCTACCGGGAGT + @@@DDDDDHHHHHII @HWI-ST212 1:N:0 CGTTTAAAAATTTCT + @;@B;DDDDH?:F;F... (4 Replies)
Discussion started by: Vnguyen
4 Replies

9. Shell Programming and Scripting

Merge columns of different files

Hi, I have tab limited file 1 and tab limited file 2 The output should contain common first column vales and corresponding 2nd column values; AND also unique first column value with corresponding 2nd column value of the file that contains it and 0 for the second file. the output should... (10 Replies)
Discussion started by: polsum
10 Replies

10. Shell Programming and Scripting

merge the two files which has contain columns

Hi may i ask how to accomplish this task: I have 2 files which has multiple columns first file 1 a 2 b 3 c 4 d second file 14 a 9 .... 13 b 10.... 12 c 11... 11 d 12... I want to merge the second file to first file that will looks like this ... (2 Replies)
Discussion started by: jao_madn
2 Replies

Featured Tech Videos