Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Merging multiple files from multiple columns

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Merging multiple files from multiple columns
# 8  
Old 10-17-2012
Thanks pamu and Scrutinizer. I will try your suggestions and read up a bit on awk and will let you guys know.
Thanks a lot!

---------- Post updated at 02:40 PM ---------- Previous update was at 11:39 AM ----------

pamu, your code actually works very well! I would just need to adapt it because my files don't really have 10 columns, more like 80 but the principle is the same. It would just be quite a long command.

Scrutinizer, your code works as well but what if file1 has more columns? How could I remove some of them if I don't want them?
Also, do I have to run the command with the spacing as is? When I tried to put into one long line I got some syntax errors.
Code:
awk: f==2{F2[$2]=$8} f==3{F3a[$2]=$9 F3b[$2]=$10} f==1{$8=F2[$2] $9=F3a[$2] $10=F3b[$2] print}
awk:                                        ^ syntax error
awk: f==2{F2[$2]=$8} f==3{F3a[$2]=$9 F3b[$2]=$10} f==1{$8=F2[$2] $9=F3a[$2] $10=F3b[$2] print}
awk:                                                               ^ syntax error
awk: f==2{F2[$2]=$8} f==3{F3a[$2]=$9 F3b[$2]=$10} f==1{$8=F2[$2] $9=F3a[$2] $10=F3b[$2] print}
awk:                                                                           ^ syntax error
awk: f==2{F2[$2]=$8} f==3{F3a[$2]=$9 F3b[$2]=$10} f==1{$8=F2[$2] $9=F3a[$2] $10=F3b[$2] print}
awk:                                                                                    ^ syntax error

Thanks for all your help.
# 9  
Old 10-17-2012
If joining the lines in previous awk post then include a ; between joined lines.
This User Gave Thanks to rdrtx1 For This Post:
bartman2099
# 10  
Old 10-17-2012
Quote:
Originally Posted by bartman2099
[..]
Scrutinizer, your code works as well but what if file1 has more columns? How could I remove some of them if I don't want them?
Also, do I have to run the command with the spacing as is? When I tried to put into one long line I got some syntax errors.
[..]
Like rdrtx1 says, use semicolons, so it would look like this:
Code:
awk 'f==2{F2[$2]=$8} f==3{F3a[$2]=$9; F3b[$2]=$10} f==1{ $8=F2[$2]; $9=F3a[$2]; $10=F3b[$2]; print}' f=2 file2 f=3 file3 f=1 file1

But that also makes it more difficult to read IMO.

--
To remove columns, there are several methods. What does file1 look like?
This User Gave Thanks to Scrutinizer For This Post:
bartman2099
# 11  
Old 10-17-2012
Ok, so this is a full row from file1 which has 78 columns:

Code:
--- rs144097162 NA 58246214 C A 546966 0.99949 0.73373 1419.4 3.594 0 0 1189.9 0.129 0 0 843.87 2.127 0 0 1307.7 2.272 0 0 1246.7 0.291 0 0 2235.2 0.837 0 0 624.04 6.957 0 0 2696.9 2.146 0 0 11564 18.353 0 0 4134 12.969 0 7429.6 5.384 0 0.00079231 0.0015637 0.00036207 0 1 1 1 1 1 1 1 1 1 1 4.3291 1.5837 11.833 -1 -1 -1 4.3239 1.5824 11.815 9.5513e-07 0.5773 3.389 0.69154

Your command as far as I understand takes the values i want from files 2 and 3 and puts them in the same positions in file1 and prints the rest of file 1 as is. Let's say I want to do all that and on top keep a subset of columns, say column 65 and 75 from the remaining columns. Is that clear? Could I just replace "print" with "print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$65,$75" since now $8,$9 and $10 have the values from the other files?

I Think it does work!

---------- Post updated at 03:21 PM ---------- Previous update was at 03:08 PM ----------

Thanks again for all your help guys!
Last edited by Scrutinizer; 10-17-2012 at 11:29 AM.. Reason: tried what I suggested; code tags (mod)
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing carriage returns from multiple lines in multiple files of different number of columns

Hello Gurus, I have a multiple pipe separated files which have records going over multiple Lines. End of line separator is \n and records going over multiple lines have <CR> as separator. below is example from one file. 1|ABC DEF|100|10 2|PQ RS T|200|20 3| UVWXYZ|300|30 4| GHIJKL|400|40... (7 Replies)
Discussion started by: dJHa
7 Replies

2. Shell Programming and Scripting

Merging multiple files into one

Hi guys, could you please help me with this? I have multiple files with this structure: file1 xxx1 1.0 xxx2 3.5 xxx3 2.4 xxx4 3.0 … xxx1890 5.7 file2 xxx1 8.0 xxx3 7.5 xxx4 5.5 …. (4 Replies)
Discussion started by: coppuca
4 Replies

3. Shell Programming and Scripting

Merging Multiple Columns between two files

Hello guys, I have 2 CSV files which goes like this: CSV1: Breaking.csv: UTF-8 "Name","Description","Occupation","Email" "Walter White","","Chemistry Teacher","w.w@bb.com" "Jessie Pinkman","","Junkie","j.p@bb.com" "Hank Schrader","","DEA Agent","h.s@bb.com" CSV2: Bad.csv... (7 Replies)
Discussion started by: jeffreybsu
7 Replies

4. Shell Programming and Scripting

Compare multiple files with multiple number of columns

Hi, input file1 abcd 123 198 xyz1:0909090-0909091 ghij 234 999 xyz2:987654:987655 kilo 7890 7990 xyz3:12345-12357 prem 9 112 xyz5:97-1134 input file2 abcd 123 198 xyz1:0909090-0909091 -9.122 0 abed 88 98 xyz1:98989-090808 -1.234 1.345 ghij 234 999 xyz2:987654:987655 -10.87090909 5... (5 Replies)
Discussion started by: jacobs.smith
5 Replies

5. UNIX for Dummies Questions & Answers

cutting multiple columns into multiple files

Hypothetically, suppose that file1 id v1 v2 v3 v4 v5 v6 v7..........v100 1 1 1 1 1 1 2 2 .....50 2 1 1 1 1 1 2 2 .....50 3 1 1 1 1 1 2 2 .....50 4 1 1 1 1 1 2 2 .....50 5 1 1 1 1 1 2 2 .....50 I want to write a loop such that I take the id# and the first 5 columns (v1-v5) into the... (3 Replies)
Discussion started by: johnkim0806
3 Replies

6. Shell Programming and Scripting

Merging columns from multiple files

Hello, I have a number of tab delimited data files consists of two columns. Like that: File1 800.000000 0.002744 799.000000 0.002517 798.000000 0.002836 797.000000 0.002553 FIle2 800.000000 0.000261 799.000000 0.000001 798.000000 0.000551 797.000000 0.000275 File3... (19 Replies)
Discussion started by: erden
19 Replies

7. Shell Programming and Scripting

need help with post:extract multiple columns from multiple files

hello, I will would be grateful if anyone can help me reply to my post extract multiple cloumns from multiple files; skip rows and include filenames; awk Please see this thread. Thanks manishabh (0 Replies)
Discussion started by: manishabh
0 Replies

8. Shell Programming and Scripting

Combine multiple columns from multiple files

Hi there, I was wondering if someone can help me with this. I am trying the combine multiple columns from multiple files into one file. Example file 1: c0t0d0 c0t2d0 # hostname vgname c0t0d1 c0t2d1 # hostname vgname c0t0d2 c0t2d2 # hostname vgname c0t1d0 c0t3d0 # hostname vgname1... (5 Replies)
Discussion started by: martva
5 Replies

9. UNIX for Advanced & Expert Users

Merging multiple .so files

Hi All, How to merge independent .so files into an executable. Thanks in Advance, Regards, Kusu (2 Replies)
Discussion started by: Kusu
2 Replies

10. Shell Programming and Scripting

Merging columns from multiple files in one file

Hi, I want to select columns from multiple files and combine them in one file. The files are simulation-data-files with 23 columns each and about 50 rows. I now use: cut -f 11 Sweep?wing-30?scale=0.?0?fan2?.txt | pr -3 | awk '{printf("\n%s\t%s\t%s",$1,$2,$3)}' > ../Data_Processed/output.txtI... (1 Reply)
Discussion started by: isgoed
1 Replies
Login or Register to Ask a Question