04-15-2014
Re:Merge records based on multiple columns
Hi,
I am not an expert shell scripting programmer ,but I have tried few options
1) For loop by reading each row and doing concatenated grep to find the matching column
2) I have also gone through the link (on this website itself )given below and tried to change given solution as per requirement
shell-programming-and-scripting
208027-merge-multiple-lines-same-file-common-key-using-awk.html
but ordering on the basis of 15th column is still an issue and also I am getting double quotes in concatenation of last field
I am ok with any solution being in KSH,BASH or AWK. Perl and any other language could be the last option but not preferable for now.
I am not sure about OS version too as I am out of my office and can't check
let me know if these informations are helpful
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Situation:
Our system currently executes a job (COBOL Program) that generates an interface file to be sent to one of our vendors. Because this system processes information for over 100,000 employees/retirees (and growing), we'd like to multi-thread the job into processing-groups in order to... (4 Replies)
Discussion started by: oordonez
4 Replies
2. Shell Programming and Scripting
Hi ,
I have the below source
source data
1|2|3|:123:abc|4
1|2|a| | 5
1|2|3|4|:a:s:D.....:n|t
Target data should be
1|2|3|:123:abc|4
1|2|3|:123:abc|4
1|2|a| | 5
1|2|3|4|:a:s:D.....:n|t
1|2|3|4|:a:s:D.....:n|t
1|2|3|4|:a:s:D.....:n|t
1|2|3|4|:a:s:D.....:n|t (3 Replies)
Discussion started by: mora
3 Replies
3. Shell Programming and Scripting
I have two files
1.txt
34, ABC, 7, 8, 0.9
35, CDE, 6.5, -2, 0.01
2.txt
34, ABC, 9, 6, -1.9
35, CDE, 8.5, -2.3, 5.01
So in both files common columns are 1 and 2
so final o/p should look like
34, ABC, 7, 8, 0.9, 9, 6, -1.9
35, CDE, 6.5, -2, 0.01, 8.5, -2.3, 5.01
I tried using... (3 Replies)
Discussion started by: manas_ranjan
3 Replies
4. Shell Programming and Scripting
Hi everyone,
I have a file result.txt with records as following and another file mirna.txt with a list of miRNAs e.g. miR22, miR123, miR13 etc.
Gene Transcript miRNA
Gar Nm_111233 miR22
Gar Nm_123440 miR22
Gar Nm_129939 miR22
Hel Nm_233900 miR13
Hel ... (6 Replies)
Discussion started by: miclow
6 Replies
5. UNIX for Dummies Questions & Answers
Hi all,
I'm looking for a way to merge multiple columns (from one file) into a single column in an output file.
The file I have looks somewhat like this:
@HWI-ST212 1:N:0 AGTCCTACCGGGAGT + @@@DDDDDHHHHHII
@HWI-ST212 1:N:0 CGTTTAAAAATTTCT + @;@B;DDDDH?:F;F... (4 Replies)
Discussion started by: Vnguyen
4 Replies
6. UNIX for Dummies Questions & Answers
Hi all, I've searched the web for a long time trying to figure out how to merge columns from multiple files.
I know paste will append columns like so:
paste file1 file2 file3 file4 file5 ...
But this becomes inconvenient when you want to append a large number of files into a single file.
... (2 Replies)
Discussion started by: torchij
2 Replies
7. Shell Programming and Scripting
Hello and Good day
I have a lot of files with same number of rows and columns.$2 and $3 are the same in all files .
I need to merge $2,$3,$6 from first file and $6 from another files.
File1:
$1 $2 $3 $4 $5 $6... (8 Replies)
Discussion started by: ali.seifaddini
8 Replies
8. Shell Programming and Scripting
011111123444 1234 1 20000
011111123444 1235 1 30000
011111123446 1234 3 40000
011111123447 1234 4 50000
011111123448 1234 3 50000
File2:
011111123444,Rsttponrfgtrgtrkrfrgtrgrer
011111123446,Rsttponrfgtrgtr
011111123447,Rsttponrfgtrguii
011111123448,Rsttponrfgtrgtjiiu
I have 2 files... (4 Replies)
Discussion started by: vinus
4 Replies
9. UNIX for Dummies Questions & Answers
Hi, I am trying to selectively merge two files based on keys reported in the 1st column.
File1:
#file1-header1
file1-header2
111 qwe rtz uio
198 asd fgh jkl
165 yxc
789 poi uzt rew
89 lkj
File2:
#file2-header2
file2-header2
165 ghz nko2 ... (2 Replies)
Discussion started by: dovah
2 Replies
10. UNIX for Beginners Questions & Answers
I will like to merge several files using 'cat', but I observe the output is not consistent. the merge begins at the last line of the first file.
file1.txt:
1234
1234
1234
file2.txt:
aaaa
bbbb
cccc
dddd
cat file1.txt file2.txt > file3.txt
file3.txt:
1234
1234
1234aaaa
bbbb
cccc... (13 Replies)
Discussion started by: geomarine
13 Replies
GREP(1) General Commands Manual GREP(1)
NAME
grep - search a file for a pattern
SYNOPSIS
grep [ option ... ] pattern [ file ... ]
DESCRIPTION
Grep searches the input files (standard input default) for lines (with newlines excluded) that match the pattern, a regular expression as
defined in regexp(6). Normally, each line matching the pattern is `selected', and each selected line is copied to the standard output.
The options are
-c Print only a count of matching lines.
-h Do not print file name tags (headers) with output lines.
-i Ignore alphabetic case distinctions. The implementation folds into lower case all letters in the pattern and input before interpre-
tation. Matched lines are printed in their original form.
-l (ell) Print the names of files with selected lines; don't print the lines.
-L Print the names of files with no selected lines; the converse of -l.
-n Mark each printed line with its line number counted in its file.
-s Produce no output, but return status.
-v Reverse: print lines that do not match the pattern.
Output lines are tagged by file name when there is more than one input file. (To force this tagging, include /dev/null as a file name
argument.)
Care should be taken when using the shell metacharacters $*[^|()= and newline in pattern; it is safest to enclose the entire expression in
single quotes '...'.
SOURCE
/sys/src/cmd/grep.c
SEE ALSO
ed(1), awk(1), sed(1), sam(1), regexp(6)
DIAGNOSTICS
Exit status is null if any lines are selected, or non-null when no lines are selected or an error occurs.
GREP(1)