Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Merging multiple files into one

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Merging multiple files into one
# 1  
Old 11-21-2013
Merging multiple files into one
Hi guys,

could you please help me with this?

I have multiple files with this structure:

file1
Code:
xxx1  1.0
xxx2  3.5
xxx3  2.4
xxx4  3.0
…

xxx1890  5.7

file2
Code:
xxx1  8.0
xxx3  7.5
xxx4  5.5
….
xxx789  6.3

I need to put them in one file based on the first column names:

Code:
xxx1  1.0 8.0
xxx2  3.5 --
xxx3  2.4 7.5
xxx4  3.0 5.5
…
xxx789  -- 6.3
….
xxx1890  5.7 --

Thank you!
Last edited by coppuca; 11-21-2013 at 06:46 AM.. Reason: Add CODE tags.
# 2  
Old 11-21-2013
Hi,

Please use code tag for your input file, which help to make thread more readable.

Use below code for the desire o/p.

Code:
 awk 'NR==FNR{a[$1]=$2;next}
a[$1]{print $0,a[$1];next}{print $0,"- -"}' file2 file1

Thanks
Pravin
This User Gave Thanks to pravin27 For This Post:
coppuca
# 3  
Old 11-21-2013
this would be simple for join if you didn't have missing columns..
Are they to be sorted by the first column? The tough part is adding '--'. Unless the inputs always sorted?
# 4  
Old 11-21-2013
it doesn't have to be '--', it can be just tab but it has to be something...
# 5  
Old 11-21-2013
Quote:
Originally Posted by coppuca
Hi guys,

could you please help me with this?

I have multiple files with this structure:

file1
Code:
xxx1  1.0
xxx2  3.5
xxx3  2.4
xxx4  3.0
…

xxx1890  5.7

file2
Code:
xxx1  8.0
xxx3  7.5
xxx4  5.5
….
xxx789  6.3

I need to put them in one file based on the first column names:

Code:
xxx1  1.0 8.0
xxx2  3.5 --
xxx3  2.4 7.5
xxx4  3.0 5.5
…
xxx789  -- 6.3
….
xxx1890  5.7 --

Thank you!
Try

Code:
awk '
  NR==FNR{
         A[$1]=$1 FS $2
         next
        }
        {
         A[$1] = A[$1] ? A[$1] FS $2 : $1 FS "---" FS $2
        }
     END{
           for(i in A)print split(A[i],T,FS) == 2 ? A[i] FS "---" : A[i]
        }' file1 file2 | sort -nk1.4

Resulting
Code:
xxx1 1.0 8.0
xxx2 3.5 ---
xxx3 2.4 7.5
xxx4 3.0 5.5
xxx789 --- 6.3
xxx1890 5.7 ---

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Merging Multiple Columns between two files

Hello guys, I have 2 CSV files which goes like this: CSV1: Breaking.csv: UTF-8 "Name","Description","Occupation","Email" "Walter White","","Chemistry Teacher","w.w@bb.com" "Jessie Pinkman","","Junkie","j.p@bb.com" "Hank Schrader","","DEA Agent","h.s@bb.com" CSV2: Bad.csv... (7 Replies)
Discussion started by: jeffreybsu
7 Replies

2. Shell Programming and Scripting

Perl - multiple keys and merging two files

Hi, I'm not a regular coder but some times I write some basic perl script, hence Perl is bit difficult for me :). I'm merging two files a.txt and b.txt into c.txt: a.txt ------ x001;frtb70;xyz;109 x001;frvt65;sec;239 x003;wqax34;jul;659 x004;yhud43;yhn;760 b.txt ------... (8 Replies)
Discussion started by: Lokesha
8 Replies

3. Shell Programming and Scripting

awk Merging multiple files with symbol representing new file

I just tried following ls *.dat|sort -t"_" -k2n,2|while read f1 && read f2; do awk '{print}' $f1 awk FNR==1'{print $1,$2,$3,$4,$5,"*","*","*" }' OFS="\t" $f2 awk '{print}' $f2 donegot following result 18-Dec-1983 11:45:00 AM 18.692 84.672 0 25.4 24 18-Dec-1983 ... (3 Replies)
Discussion started by: Akshay Hegde
3 Replies

4. Shell Programming and Scripting

Merging multiple files from multiple columns

Hi guys, I have very basic linux experience so I need some help with a problem. I have 3 files from which I want to extract columns based on common fields between them. File1: --- rs74078040 NA 51288690 T G 461652 0.99223 0.53611 3 --- rs77209296 NA 51303525 T G 461843 0.98973 0.60837 3... (10 Replies)
Discussion started by: bartman2099
10 Replies

5. Shell Programming and Scripting

Merging multiple files using lines from one file

I have been working of this script for a very long time and I have searched the internet for direction but I am stuck here. I have about 3000 files with two columns each. The length of each file is 50000. Each of these files is named this way b.4, b.5, b.6, b.7, b.8, b.9, b.10, b.11, b.12... (10 Replies)
Discussion started by: iconig
10 Replies

6. Shell Programming and Scripting

Merging information from multiple files to a single file

Hello, I am new to unix and need help with a problem. I have 2 files each containing multiple columns of information ie; File 1 : A B C D E 1 2 3 4 5 File 2 : F G 6 7 I would like to merge the information from File 2 to File 1 so that the data reads as follows; File 1: A... (4 Replies)
Discussion started by: crunchie
4 Replies

7. Shell Programming and Scripting

Merging columns from multiple files

Hello, I have a number of tab delimited data files consists of two columns. Like that: File1 800.000000 0.002744 799.000000 0.002517 798.000000 0.002836 797.000000 0.002553 FIle2 800.000000 0.000261 799.000000 0.000001 798.000000 0.000551 797.000000 0.000275 File3... (19 Replies)
Discussion started by: erden
19 Replies

8. UNIX for Advanced & Expert Users

Merging multiple .so files

Hi All, How to merge independent .so files into an executable. Thanks in Advance, Regards, Kusu (2 Replies)
Discussion started by: Kusu
2 Replies

9. Shell Programming and Scripting

Merging columns from multiple files in one file

Hi, I want to select columns from multiple files and combine them in one file. The files are simulation-data-files with 23 columns each and about 50 rows. I now use: cut -f 11 Sweep?wing-30?scale=0.?0?fan2?.txt | pr -3 | awk '{printf("\n%s\t%s\t%s",$1,$2,$3)}' > ../Data_Processed/output.txtI... (1 Reply)
Discussion started by: isgoed
1 Replies

10. Shell Programming and Scripting

merging multiple log files

Hello, I have 8 sql loader scripts which produce ".bad" file if there is any errors, how can I join the contents of these files together in one column? file 1 CA-94061-TSS Tkb Sport Shop CA-95133-V Vollyrite ... (3 Replies)
Discussion started by: jack1981
3 Replies
Login or Register to Ask a Question