04-15-2014
Re:Merge records based on multiple columns
Hi,
I am not an expert shell scripting programmer ,but I have tried few options
1) For loop by reading each row and doing concatenated grep to find the matching column
2) I have also gone through the link (on this website itself )given below and tried to change given solution as per requirement
shell-programming-and-scripting
208027-merge-multiple-lines-same-file-common-key-using-awk.html
but ordering on the basis of 15th column is still an issue and also I am getting double quotes in concatenation of last field
I am ok with any solution being in KSH,BASH or AWK. Perl and any other language could be the last option but not preferable for now.
I am not sure about OS version too as I am out of my office and can't check
let me know if these informations are helpful
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Situation:
Our system currently executes a job (COBOL Program) that generates an interface file to be sent to one of our vendors. Because this system processes information for over 100,000 employees/retirees (and growing), we'd like to multi-thread the job into processing-groups in order to... (4 Replies)
Discussion started by: oordonez
4 Replies
2. Shell Programming and Scripting
Hi ,
I have the below source
source data
1|2|3|:123:abc|4
1|2|a| | 5
1|2|3|4|:a:s:D.....:n|t
Target data should be
1|2|3|:123:abc|4
1|2|3|:123:abc|4
1|2|a| | 5
1|2|3|4|:a:s:D.....:n|t
1|2|3|4|:a:s:D.....:n|t
1|2|3|4|:a:s:D.....:n|t
1|2|3|4|:a:s:D.....:n|t (3 Replies)
Discussion started by: mora
3 Replies
3. Shell Programming and Scripting
I have two files
1.txt
34, ABC, 7, 8, 0.9
35, CDE, 6.5, -2, 0.01
2.txt
34, ABC, 9, 6, -1.9
35, CDE, 8.5, -2.3, 5.01
So in both files common columns are 1 and 2
so final o/p should look like
34, ABC, 7, 8, 0.9, 9, 6, -1.9
35, CDE, 6.5, -2, 0.01, 8.5, -2.3, 5.01
I tried using... (3 Replies)
Discussion started by: manas_ranjan
3 Replies
4. Shell Programming and Scripting
Hi everyone,
I have a file result.txt with records as following and another file mirna.txt with a list of miRNAs e.g. miR22, miR123, miR13 etc.
Gene Transcript miRNA
Gar Nm_111233 miR22
Gar Nm_123440 miR22
Gar Nm_129939 miR22
Hel Nm_233900 miR13
Hel ... (6 Replies)
Discussion started by: miclow
6 Replies
5. UNIX for Dummies Questions & Answers
Hi all,
I'm looking for a way to merge multiple columns (from one file) into a single column in an output file.
The file I have looks somewhat like this:
@HWI-ST212 1:N:0 AGTCCTACCGGGAGT + @@@DDDDDHHHHHII
@HWI-ST212 1:N:0 CGTTTAAAAATTTCT + @;@B;DDDDH?:F;F... (4 Replies)
Discussion started by: Vnguyen
4 Replies
6. UNIX for Dummies Questions & Answers
Hi all, I've searched the web for a long time trying to figure out how to merge columns from multiple files.
I know paste will append columns like so:
paste file1 file2 file3 file4 file5 ...
But this becomes inconvenient when you want to append a large number of files into a single file.
... (2 Replies)
Discussion started by: torchij
2 Replies
7. Shell Programming and Scripting
Hello and Good day
I have a lot of files with same number of rows and columns.$2 and $3 are the same in all files .
I need to merge $2,$3,$6 from first file and $6 from another files.
File1:
$1 $2 $3 $4 $5 $6... (8 Replies)
Discussion started by: ali.seifaddini
8 Replies
8. Shell Programming and Scripting
011111123444 1234 1 20000
011111123444 1235 1 30000
011111123446 1234 3 40000
011111123447 1234 4 50000
011111123448 1234 3 50000
File2:
011111123444,Rsttponrfgtrgtrkrfrgtrgrer
011111123446,Rsttponrfgtrgtr
011111123447,Rsttponrfgtrguii
011111123448,Rsttponrfgtrgtjiiu
I have 2 files... (4 Replies)
Discussion started by: vinus
4 Replies
9. UNIX for Dummies Questions & Answers
Hi, I am trying to selectively merge two files based on keys reported in the 1st column.
File1:
#file1-header1
file1-header2
111 qwe rtz uio
198 asd fgh jkl
165 yxc
789 poi uzt rew
89 lkj
File2:
#file2-header2
file2-header2
165 ghz nko2 ... (2 Replies)
Discussion started by: dovah
2 Replies
10. UNIX for Beginners Questions & Answers
I will like to merge several files using 'cat', but I observe the output is not consistent. the merge begins at the last line of the first file.
file1.txt:
1234
1234
1234
file2.txt:
aaaa
bbbb
cccc
dddd
cat file1.txt file2.txt > file3.txt
file3.txt:
1234
1234
1234aaaa
bbbb
cccc... (13 Replies)
Discussion started by: geomarine
13 Replies
IGAWK(1) Utility Commands IGAWK(1)
NAME
igawk - gawk with include files
SYNOPSIS
igawk [ all gawk options ] -f program-file [ -- ] file ...
igawk [ all gawk options ] [ -- ] program-text file ...
DESCRIPTION
Igawk is a simple shell script that adds the ability to have ``include files'' to gawk(1).
AWK programs for igawk are the same as for gawk, except that, in addition, you may have lines like
@include getopt.awk
in your program to include the file getopt.awk from either the current directory or one of the other directories in the search path.
OPTIONS
See gawk(1) for a full description of the AWK language and the options that gawk supports.
EXAMPLES
cat << EOF > test.awk
@include getopt.awk
BEGIN {
while (getopt(ARGC, ARGV, "am:q") != -1)
...
}
EOF
igawk -f test.awk
SEE ALSO
gawk(1)
Effective AWK Programming, Edition 1.0, published by the Free Software Foundation, 1995.
AUTHOR
Arnold Robbins (arnold@skeeve.com).
Free Software Foundation Nov 3 1999 IGAWK(1)