Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Merge two text files (oh no, not again!)

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Merge two text files (oh no, not again!)
# 1  
Old 11-24-2015
Merge two text files (oh no, not again!)
Hello,

I'm new to this forum. I have always made good use of all the wise hints shown here. But this time I'm struggling with an issue that is driving me crazy.

I have two text files, I have to merge them based on the first column, resulting file must contain all record from the first file AND all records from the second one.

FILE1
Code:
RB-4.762/G10Q M1         04763-010M01R3-01       ;         ;         ;         ;  2744000;
RB-4.762/G10Q M2         04763-010M02R3-01       ;         ;         ;         ;  4020800;
RB-4.762/G10Q M3         04763-010M03R3-01       ;         ;         ;         ;  2587200;
RB-4.762/G10Q M3-R3      04763-010M03R3-01       ;         ;         ;         ;   470400;
RB-4.762/G10Q M4         04763-010M04R3-01       ;         ;         ;         ;  3124800;
RB-4.762/G10Q M5         04763-010M05R3-01       ;         ;         ;         ;  1142400;
RB-4.762/G10Q N          04763-010PM0R3-01       ;         ;         ;         ;  5062400;
RB-4.762/G10Q N-R3       04763-010PM0R3-01       ;         ;         ;         ;   940800;
RB-4.762/G10Q P1         04763-010P01R3-01       ;         ;         ;         ;  2900800;
RB-4.762/G10Q P1-R3      04763-010P01R3-01       ;         ;         ;         ;  1176000;

FILE2
Code:
RB-4.762/G10Q M1         04763-010M01R3-01       ;   940800;        0;  3460800;
RB-4.762/G10Q M2         04763-010M02R3-01       ;   940800;        0;  1769600;
RB-4.762/G10Q M3         04763-010M03R3-01       ;  1881600;        0;  1041600;
RB-4.762/G10Q M4         04763-010M04R3-01       ;   470400;        0;  1164800;
RB-4.762/G10Q M5         04763-010M05R3-01       ;   470400;        0;  1422400;
RB-4.762/G10Q N          04763-010PM0R3-01       ;  2822400;        0;  2721600;
RB-4.762/G10Q P1         04763-010P01R3-01       ;  2822400;        0;  2676800;
RB-4.762/G10Q P2         04763-010P02R3-01       ;   470400;        0;  1948800;
RB-4.762/G10Q P3         04763-010P03R3-01       ;        0;        0;   604800;
RB-4.762/G10Q P4         04763-010P04R3-01       ;   940800;        0;  1097600;

Expected result
Code:
RB-4.762/G10Q M1         04763-010M01R3-01       ;   940800;        0;  3460800;  2744000;
RB-4.762/G10Q M2         04763-010M02R3-01       ;   940800;        0;  1769600;  4020800;
RB-4.762/G10Q M3         04763-010M03R3-01       ;  1881600;        0;  1041600;  2587200;
RB-4.762/G10Q M3-R3      04763-010M03R3-01       ;        0;        0;        0;   470400;
RB-4.762/G10Q M4         04763-010M04R3-01       ;   470400;        0;  1164800;  3124800;
RB-4.762/G10Q M5         04763-010M05R3-01       ;   470400;        0;  1422400;  1142400;
RB-4.762/G10Q N          04763-010PM0R3-01       ;  2822400;        0;  2721600;  5062400;
RB-4.762/G10Q N-R3       04763-010PM0R3-01       ;        0;        0;        0;   940800;
RB-4.762/G10Q P1         04763-010P01R3-01       ;  2822400;        0;  2676800;  2900800;
RB-4.762/G10Q P1-R3      04763-010P01R3-01       ;        0;        0;        0;  1176000;
RB-4.762/G10Q P2         04763-010P02R3-01       ;   470400;        0;  1948800;        0;
RB-4.762/G10Q P3         04763-010P03R3-01       ;        0;        0;   604800;        0;
RB-4.762/G10Q P4         04763-010P04R3-01       ;   940800;        0;  1097600;        0;

Thanks a lot for any help.

Ema
# 2  
Old 11-24-2015
Are there clashing values? Is ";" the field separator ?
# 3  
Old 11-24-2015
No clashing values, and yes ";" is the Field Separator. I'm running AIX 7.1.0.0
The files look like in my above post. No hidden rows Smilie

I know how to get a merged file with just common records using AWK
Code:
awk -F\, '{FS=";"} NR==FNR {a[$1]; next} ($1) in a{ print $1";"$2".........

but here I need to have all records.
Last edited by emare; 11-24-2015 at 12:28 PM..
# 4  
Old 11-24-2015
Would this be close?
Code:
awk 'NR==FNR {T[$1]=$5; next} {print $0 T[$1],""}' FS=";" OFS=";" file1 file2
RB-4.762/G10Q M1         04763-010M01R3-01       ;   940800;        0;  3460800;  2744000;
RB-4.762/G10Q M2         04763-010M02R3-01       ;   940800;        0;  1769600;  4020800;
RB-4.762/G10Q M3         04763-010M03R3-01       ;  1881600;        0;  1041600;  2587200;
RB-4.762/G10Q M4         04763-010M04R3-01       ;   470400;        0;  1164800;  3124800;
RB-4.762/G10Q M5         04763-010M05R3-01       ;   470400;        0;  1422400;  1142400;
RB-4.762/G10Q N          04763-010PM0R3-01       ;  2822400;        0;  2721600;  5062400;
RB-4.762/G10Q P1         04763-010P01R3-01       ;  2822400;        0;  2676800;  2900800;
RB-4.762/G10Q P2         04763-010P02R3-01       ;   470400;        0;  1948800;;
RB-4.762/G10Q P3         04763-010P03R3-01       ;        0;        0;   604800;;
RB-4.762/G10Q P4         04763-010P04R3-01       ;   940800;        0;  1097600;;

---------- Post updated at 17:33 ---------- Previous update was at 17:28 ----------

Sorry, overlooked part of the requirement. Better:
Code:
awk 'NR==FNR {P[$1]=$5; T[$1]=$0; next} {print $0 P[$1],""; delete T[$1]} END {for (t in T) print T[t]}' FS=";" OFS=";" file1 file2
RB-4.762/G10Q M1         04763-010M01R3-01       ;   940800;        0;  3460800;  2744000;
RB-4.762/G10Q M2         04763-010M02R3-01       ;   940800;        0;  1769600;  4020800;
RB-4.762/G10Q M3         04763-010M03R3-01       ;  1881600;        0;  1041600;  2587200;
RB-4.762/G10Q M4         04763-010M04R3-01       ;   470400;        0;  1164800;  3124800;
RB-4.762/G10Q M5         04763-010M05R3-01       ;   470400;        0;  1422400;  1142400;
RB-4.762/G10Q N          04763-010PM0R3-01       ;  2822400;        0;  2721600;  5062400;
RB-4.762/G10Q P1         04763-010P01R3-01       ;  2822400;        0;  2676800;  2900800;
RB-4.762/G10Q P2         04763-010P02R3-01       ;   470400;        0;  1948800;;
RB-4.762/G10Q P3         04763-010P03R3-01       ;        0;        0;   604800;;
RB-4.762/G10Q P4         04763-010P04R3-01       ;   940800;        0;  1097600;;
RB-4.762/G10Q P1-R3      04763-010P01R3-01       ;         ;         ;         ;  1176000;
RB-4.762/G10Q N-R3       04763-010PM0R3-01       ;         ;         ;         ;   940800;
RB-4.762/G10Q M3-R3      04763-010M03R3-01       ;         ;         ;         ;   470400;

?
This User Gave Thanks to RudiC For This Post:
emare
# 5  
Old 11-24-2015
Thanks. It looks just what I need.
I'm checking the files (huge) and the output looks correct.

Ema
 
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Merge two text files by two fields and mixed output

Hello, I'm back again looking for your precious help- This time I need to merge two text files with matching two fields, output only common records with mixed output. Let's look at the example: FILE1 56153;AAA0708;3;TEST1TEST1; 89014;BBB0708;3;TEST2TEST2; 89014;BBB0708;4;TEST3TEST3; ... (7 Replies)
Discussion started by: emare
7 Replies

2. Shell Programming and Scripting

Merge and Sort tabular data from different text files

I have 42 text files; each containing up to 34 lines with following structure; file1 H-01 23 H-03 5 H-05 9 H-02 14 . . file2 H-01 17 H-02 43 H-04 7 H-05 8 H-03 7 . . file3 (6 Replies)
Discussion started by: Syeda Sumayya
6 Replies

3. Shell Programming and Scripting

Need to merge multiple text files vertically and place comma between fields

Hello expert friends, I'm writing a script to capture stats using sar and stuck up at report generation. I have around 10 files in a directory and need to merge them all vertically based on the time value of first column (output file should have only one time value) and insert comma after... (6 Replies)
Discussion started by: prvnrk
6 Replies

4. Shell Programming and Scripting

Merge the multiple text files into one file

Hi All, I am trying to merge all the text files into one file using below snippet cat /home/Temp/Test/Log/*.txt >> all.txt But it seems it is not working. I have multiple files like Output_ServerName1.txt, Output_ServreName2.txt I want to merge each file into one single file and... (6 Replies)
Discussion started by: sharsour
6 Replies

5. Shell Programming and Scripting

merge two text files of different size on common index

I have two text files. text file 1: ID filePath col1 col2 col3 1 10584588.mol 269.126 190.958 23.237 2 10584549.mol 281.001 200.889 27.7414 3 10584511.mol 408.824 158.316 29.8561 4 10584499.mol 245.632 153.241 25.2815 5 10584459.mol ... (8 Replies)
Discussion started by: LMHmedchem
8 Replies

6. Shell Programming and Scripting

Have several text files and want to merge into a single

Hello, I have several files that begin with db. in my directory and I would like to first take it from a specific word starting from $TTL until the end of the contents then do the same all the way down the directory then merge them into one txt file. Is this possible? I am using cygwin with... (4 Replies)
Discussion started by: richsark
4 Replies

7. AIX

merge text files

Hello. Could you please help to know the command to merge multiple text files into one? I am thinking to use: cat f1.txt f2.txt f3.txt > f4.txt Is it okay to use cat command for same purpose - Or could there be any disadvantage in using it? Thank you (4 Replies)
Discussion started by: panchpan
4 Replies

8. Shell Programming and Scripting

Merge text files while combining the multiple header/trailer records into one each.

Situation: Our system currently executes a job (COBOL Program) that generates an interface file to be sent to one of our vendors. Because this system processes information for over 100,000 employees/retirees (and growing), we'd like to multi-thread the job into processing-groups in order to... (4 Replies)
Discussion started by: oordonez
4 Replies

9. UNIX for Dummies Questions & Answers

Is there any non graphical tool that make selective merge between text files?

whitout using awk / sad and so on? (3 Replies)
Discussion started by: umen
3 Replies
Login or Register to Ask a Question