Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Merge files based on columns

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Merge files based on columns
# 1  
Old 07-11-2014
Merge files based on columns
Code:
011111123444 1234 1 20000 
011111123444 1235 1 30000
011111123446 1234 3 40000
011111123447 1234 4 50000
011111123448 1234 3 50000

File2:
Code:
011111123444,Rsttponrfgtrgtrkrfrgtrgrer
011111123446,Rsttponrfgtrgtr 
011111123447,Rsttponrfgtrguii
011111123448,Rsttponrfgtrgtjiiu

I have 2 files : File1 and File2. File1 is a fixed length file. File 2 is comma delimiter file
I want to join both files based on column 1 and append it in postion 17- 41(Max 25) on File1.
Note: If the length of column 2 of file 2 is more than 25 then trim it. Also in the output file and file 1 has one space in the begining
of each line.

Desired Output:
Code:
011111123444 Rsttponrfgtrgtrkrfrgtrgre 1234 1 20000 
011111123444 Rsttponrfgtrgtrkrfrgtrgre 1235 1 30000
011111123446 Rsttponrfgtrgtr 1234 3 20000
011111123447 Rsttponrfgtrguii 1234 4 50000
011111123448 Rsttponrfgtrgtjiiu 1234 3 50000


Can anyone please help? The input and output file postion got changed while posting , please see the attached file.
Last edited by Don Cragun; 07-11-2014 at 07:16 PM.. Reason: Add CODE tags.
# 2  
Old 07-11-2014
I have attached the sample file
File1.txt (1.5 KB) 
# 3  
Old 07-11-2014
Code:
awk '
        NR == FNR {
                n = split ( $0, R, "," )
                A[R[1]] = R[n]
                next
        }
        $1 in A {
                printf ( " %-14s %-29s %s\n", $1, A[$1], substr($0,47) )
        }
' file2 file1

# 4  
Old 07-11-2014
Join command is useful and the standard answer you will get but depending on the type of join one or both files must be sorted.

Often when I want to do this one file is changing constantly and huge, the other is relatively small and static or rarely changed. In that case, I usually read the smaller file into an associative array and match that way.

I've done this in both Bash and AWK (AWK is faster): For example in an AWK BEGIN block:
Code:
BEGIN { while (getline < "'"$indexFile"'" ) { split($0,myIndex,","); indexArr[imyIndex[1]]=myIndex[2] }
                     close("'"$indexFile"'") }

Read up on associative arrays as they are very powerful features of high level languages.

Mike

PS. I am just a Padawan compared to maser Yoda. I passed a Bash variable, Yoda did not. NR=FNR with two files indeed . . . Smilie
Last edited by Michael Stora; 07-11-2014 at 04:49 PM.. Reason: index is a reserved word in AWK so I changed to myIndex
# 5  
Old 07-11-2014
Thanks , i made minor change and Its working fine.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Merge selective columns from files based on common key

Hi, I am trying to selectively merge two files based on keys reported in the 1st column. File1: #file1-header1 file1-header2 111 qwe rtz uio 198 asd fgh jkl 165 yxc 789 poi uzt rew 89 lkj File2: #file2-header2 file2-header2 165 ghz nko2 ... (2 Replies)
Discussion started by: dovah
2 Replies

2. Shell Programming and Scripting

Merge specific columns of two files

Hello, I have two tab delimited text files. Both files have the same number of rows but not necessarily the same number of columns. The column headers look like, File 1: f0order CVorder Name f0 RI_9 E99 E199 E299 E399 E499 E599 E699 E799 E899 E999 File 2:... (9 Replies)
Discussion started by: LMHmedchem
9 Replies

3. Shell Programming and Scripting

Merge records based on multiple columns

Hi, I have a file with 16 columns and out of these 16 columns 14 are key columns, 15 th is order column and 16th column is having information. I need to concate the 16th column based on value of 1-14th column as key in order of 15th column. Here are the example file Input File (multiple... (3 Replies)
Discussion started by: Ravi Agrawal
3 Replies

4. Shell Programming and Scripting

Merge 2 files with one reference columns

Hi All Source1 servername1,patchid1 servername1,patchid2 servername1,patchid3 servername2,patchid1 servername2,patchid2 servername3,patchid4 servername3,patchid5 Source2 servername1,appname1 servername1,appname2 servername1,appname3 servername2,appname1 servername2,appname2... (13 Replies)
Discussion started by: mv_mv
13 Replies

5. Shell Programming and Scripting

Merge columns on different files

Hello, I have two files that have this format: file 1 86.82 0.00 86.82 43.61 86.84 0.00 86.84 43.61 86.86 0.00 86.86 43.61 86.88 0.00 86.88 43.61 file 2 86.82 0.22 86.84 0.22 86.86 0.22 86.88 0.22 I would like to merge these two files such that the final file looks like... (5 Replies)
Discussion started by: kayak
5 Replies

6. Shell Programming and Scripting

Merge two files matching columns

Hi! I need to merge two files when col1 (x:x:x) matching and adds second column from file1.txt. # cat 1.txt aaa;a12 bbb;b13 ccc;c33 ddd;d55 eee;e11 # cat 2.txt bbb;b55;34444;d55 aaa;a15;35666;a44 I try with this awk and I get succesfully first column from 1.txt: # awk -F";"... (2 Replies)
Discussion started by: fhluque
2 Replies

7. Shell Programming and Scripting

file merge based on common columns

I have two files 1.txt 34, ABC, 7, 8, 0.9 35, CDE, 6.5, -2, 0.01 2.txt 34, ABC, 9, 6, -1.9 35, CDE, 8.5, -2.3, 5.01 So in both files common columns are 1 and 2 so final o/p should look like 34, ABC, 7, 8, 0.9, 9, 6, -1.9 35, CDE, 6.5, -2, 0.01, 8.5, -2.3, 5.01 I tried using... (3 Replies)
Discussion started by: manas_ranjan
3 Replies

8. Shell Programming and Scripting

Merge columns of different files

Hi, I have tab limited file 1 and tab limited file 2 The output should contain common first column vales and corresponding 2nd column values; AND also unique first column value with corresponding 2nd column value of the file that contains it and 0 for the second file. the output should... (10 Replies)
Discussion started by: polsum
10 Replies

9. Shell Programming and Scripting

merge the two files which has contain columns

Hi may i ask how to accomplish this task: I have 2 files which has multiple columns first file 1 a 2 b 3 c 4 d second file 14 a 9 .... 13 b 10.... 12 c 11... 11 d 12... I want to merge the second file to first file that will looks like this ... (2 Replies)
Discussion started by: jao_madn
2 Replies

10. Shell Programming and Scripting

Compare two files and merge columns in a third

Hi, I'm working with snmp, with a little script I'm able to obtain from a switch a list with a couple of values with this format Port Mac 1 00:0A:0B:0C:0D:0E .... (hundred of entries) Now with a simple arp on a router I am able to obtain another list 00:0A:0B:0C:0D:0E... (20 Replies)
Discussion started by: CM64
20 Replies
Login or Register to Ask a Question