04-15-2014
Re:Merge records based on multiple columns
Hi,
I am not an expert shell scripting programmer ,but I have tried few options
1) For loop by reading each row and doing concatenated grep to find the matching column
2) I have also gone through the link (on this website itself )given below and tried to change given solution as per requirement
shell-programming-and-scripting
208027-merge-multiple-lines-same-file-common-key-using-awk.html
but ordering on the basis of 15th column is still an issue and also I am getting double quotes in concatenation of last field
I am ok with any solution being in KSH,BASH or AWK. Perl and any other language could be the last option but not preferable for now.
I am not sure about OS version too as I am out of my office and can't check
let me know if these informations are helpful
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Situation:
Our system currently executes a job (COBOL Program) that generates an interface file to be sent to one of our vendors. Because this system processes information for over 100,000 employees/retirees (and growing), we'd like to multi-thread the job into processing-groups in order to... (4 Replies)
Discussion started by: oordonez
4 Replies
2. Shell Programming and Scripting
Hi ,
I have the below source
source data
1|2|3|:123:abc|4
1|2|a| | 5
1|2|3|4|:a:s:D.....:n|t
Target data should be
1|2|3|:123:abc|4
1|2|3|:123:abc|4
1|2|a| | 5
1|2|3|4|:a:s:D.....:n|t
1|2|3|4|:a:s:D.....:n|t
1|2|3|4|:a:s:D.....:n|t
1|2|3|4|:a:s:D.....:n|t (3 Replies)
Discussion started by: mora
3 Replies
3. Shell Programming and Scripting
I have two files
1.txt
34, ABC, 7, 8, 0.9
35, CDE, 6.5, -2, 0.01
2.txt
34, ABC, 9, 6, -1.9
35, CDE, 8.5, -2.3, 5.01
So in both files common columns are 1 and 2
so final o/p should look like
34, ABC, 7, 8, 0.9, 9, 6, -1.9
35, CDE, 6.5, -2, 0.01, 8.5, -2.3, 5.01
I tried using... (3 Replies)
Discussion started by: manas_ranjan
3 Replies
4. Shell Programming and Scripting
Hi everyone,
I have a file result.txt with records as following and another file mirna.txt with a list of miRNAs e.g. miR22, miR123, miR13 etc.
Gene Transcript miRNA
Gar Nm_111233 miR22
Gar Nm_123440 miR22
Gar Nm_129939 miR22
Hel Nm_233900 miR13
Hel ... (6 Replies)
Discussion started by: miclow
6 Replies
5. UNIX for Dummies Questions & Answers
Hi all,
I'm looking for a way to merge multiple columns (from one file) into a single column in an output file.
The file I have looks somewhat like this:
@HWI-ST212 1:N:0 AGTCCTACCGGGAGT + @@@DDDDDHHHHHII
@HWI-ST212 1:N:0 CGTTTAAAAATTTCT + @;@B;DDDDH?:F;F... (4 Replies)
Discussion started by: Vnguyen
4 Replies
6. UNIX for Dummies Questions & Answers
Hi all, I've searched the web for a long time trying to figure out how to merge columns from multiple files.
I know paste will append columns like so:
paste file1 file2 file3 file4 file5 ...
But this becomes inconvenient when you want to append a large number of files into a single file.
... (2 Replies)
Discussion started by: torchij
2 Replies
7. Shell Programming and Scripting
Hello and Good day
I have a lot of files with same number of rows and columns.$2 and $3 are the same in all files .
I need to merge $2,$3,$6 from first file and $6 from another files.
File1:
$1 $2 $3 $4 $5 $6... (8 Replies)
Discussion started by: ali.seifaddini
8 Replies
8. Shell Programming and Scripting
011111123444 1234 1 20000
011111123444 1235 1 30000
011111123446 1234 3 40000
011111123447 1234 4 50000
011111123448 1234 3 50000
File2:
011111123444,Rsttponrfgtrgtrkrfrgtrgrer
011111123446,Rsttponrfgtrgtr
011111123447,Rsttponrfgtrguii
011111123448,Rsttponrfgtrgtjiiu
I have 2 files... (4 Replies)
Discussion started by: vinus
4 Replies
9. UNIX for Dummies Questions & Answers
Hi, I am trying to selectively merge two files based on keys reported in the 1st column.
File1:
#file1-header1
file1-header2
111 qwe rtz uio
198 asd fgh jkl
165 yxc
789 poi uzt rew
89 lkj
File2:
#file2-header2
file2-header2
165 ghz nko2 ... (2 Replies)
Discussion started by: dovah
2 Replies
10. UNIX for Beginners Questions & Answers
I will like to merge several files using 'cat', but I observe the output is not consistent. the merge begins at the last line of the first file.
file1.txt:
1234
1234
1234
file2.txt:
aaaa
bbbb
cccc
dddd
cat file1.txt file2.txt > file3.txt
file3.txt:
1234
1234
1234aaaa
bbbb
cccc... (13 Replies)
Discussion started by: geomarine
13 Replies
PSC(1) General Commands Manual PSC(1)
NAME
psc - prepare sc files
SYNOPSIS
psc [-fLkrSPv] [-s cell] [-R n] [-C n] [-n n] [-d c]
DESCRIPTION
Psc is used to prepare data for input to the spreadsheet calculator sc(1). It accepts normal ascii data on standard input. Standard out-
put is a sc file. With no options, psc starts the spreadsheet in cell A0. Strings are right justified. All data on a line is entered on
the same row; new input lines cause the output row number to increment by one. The default delimiters are tab and space. The column for-
mats are set to one larger than the number of columns required to hold the largest value in the column.
OPTIONS
-f Omit column width calculations. This option is for preparing data to be merged with an existing spreadsheet. If the option is not
specified, the column widths calculated for the data read by psc will override those already set in the existing spreadsheet.
-L Left justify strings.
-k Keep all delimiters. This option causes the output cell to change on each new delimiter encountered in the input stream. The
default action is to condense multiple delimiters to one, so that the cell only changes once per input data item.
-r Output the data by row first then column. For input consisting of a single column, this option will result in output of one row
with multiple columns instead of a single column spreadsheet.
-s cell
Start the top left corner of the spreadsheet in cell. For example, -s B33 will arrange the output data so that the spreadsheet
starts in column B, row 33.
-R n Increment by n on each new output row.
-C n Increment by n on each new output column.
-n n Output n rows before advancing to the next column. This option is used when the input is arranged in a single column and the
spreadsheet is to have multiple columns, each of which is to be length n.
-d c Use the single character c as the delimiter between input fields.
-P Plain numbers only. A field is a number only when there is no imbedded [-+eE].
-S All numbers are strings.
-v Print the version of psc
SEE ALSO
sc(1)
AUTHOR
Robert Bond
PSC 7.16 19 September 2002 PSC(1)