Sponsored Content
Full Discussion: Merge with common column
Top Forums Shell Programming and Scripting Merge with common column Post 302728473 by empyrean on Thursday 8th of November 2012 12:44:55 AM
Old 11-08-2012
Merge with common column

hi i have two files and i wanted to join them using common column. try to do this using "join" command but that did not help.

File 1:

Code:
123             9a.vcf               hy92.vcf          hy90.vcf
Index    Ref     Alt     Ref     Alt     Ref     Alt
315       14       0       7       4      17       2
321        5       9       3       8       7      11
327       13       0       9       3      16       1
336       13       1       8       4      18       1
445       14       0       9       3      18       1

file 2:
Code:
Gene     Position         Reference Base         Alternate Base          Alternate count         Alternate count         Snp format      Sequence
lcl|utg7180000001423     315      A      G      3                [A/G]   CTAGAATCAAAGCAAGAATACACTCTTTTTTTTGGAAAAGAATATCTCATGTTTGCTCTTAAACTCCAAAACATATATCTACTTCAAAGTAGTGTTTTCCTGAGAGAAAGCACTATCGATAAGTTATGTCCCAACTCCAAATTATTACATCAAAGTAGAAACAAGATGACATATATATATCAACCGTATATTCTGCACATGATGTG
lcl|utg7180000001423     321      G      A/A,C  41       39,2    [-/-]   CTAGAATCAAAGCAAGAATACACTCTTTTTTTTGGAAAAGAATATCTCATGTTTGCTCTTAAACTCCAAAACATATATCTACTTCAAAGTAGTGTTTTCCTGAGAGAAAGCACTATCGATAAGTTATGTCCCAACTCCAAATTATTACATCAAAGTAGAAACAAGATGACATATATATATCAACCGTATATTCTGCACATGATGTG
lcl|utg7180000001423     327      A      C      4                [A/C]   CTAGAATCACTAGAATCAAAGCAAGAATACACTCTTTTTTTTGGAAAAGAATATCTCATGTTTGCTCTTAAACTCCAAAACATATATCTACTTCAAAGTAGTGTTTTCCTGAGAGAAAGCACTATCGATAAGTTATGTCCCAACTCCAAATTATTACATCAAAGTAGAAACAAGATGACATATATATATCAACCGTATATTCTGCACATGATGTG

i wanted to match first column in 1st file with 2nd column in second file and get union of both files i.e i need all the fields from file 1 as well as file 2.

---------- Post updated at 12:44 AM ---------- Previous update was at 12:21 AM ----------

an update is that the following code is able to merge the columns but its taking off the headings. i need to retain them in the final file..

Code:
#!/bin/awk -f
  # store the first file, indexed by col2
  NR==FNR {f1[$1] = $0; next}
  # output only if file1 contains file2's col2
  ($2 in f1) {print $0,"\t",f1[$2]}


Last edited by Scrutinizer; 11-08-2012 at 01:44 AM.. Reason: Changed icode to code
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

merge based on common, awk help

All, $ cat x.txt z 11 az x 12 ax y 13 ay $ cat y.txt ay TT ax NN Output required: y 13 ay TT x 12 ax NN (3 Replies)
Discussion started by: jkl_jkl
3 Replies

2. Shell Programming and Scripting

merge rows based on a common column

Hi guys, Please guide me if you have a solution to this problem. I have tried paste -s but it's not giving the desired output. I have a file with the following content- A123 box1 B345 bat2 C431 my_id A123 service C431 box1 A123 my_id I need two different outputs- OUTPUT1 A123... (6 Replies)
Discussion started by: smriti_shridhar
6 Replies

3. Shell Programming and Scripting

"Join" or "Merge" more than 2 files into single output based on common key (column)

Hi All, I have working (Perl) code to combine 2 input files into a single output file using the join function that works to a point, but has the following limitations: 1. I am restrained to 2 input files only. 2. Only the "matched" fields are written out to the "matched" output file and... (1 Reply)
Discussion started by: Katabatic
1 Replies

4. UNIX for Dummies Questions & Answers

Writing a loop to merge multiple files by common column

I have 100 data files labelled 250.1.txt through 250.100.txt. The second column of the data files partially match (there is about %90 overlap). Each data file has 4 columns. I want the merge all these text files by the matching values in the second column. In the output, the first column should... (1 Reply)
Discussion started by: evelibertine
1 Replies

5. UNIX for Dummies Questions & Answers

Merge rows with common column

Dear all I have big file with two columns A_AA960715 GO:0006952 A_AA960715 GO:0008152 A_AA960715 GO:0016491 A_AA960715 GO:0007165 A_AA960715 GO:0005618 A_AA960716 GO:0006952 A_AA960716 GO:0005618 A_AA960716... (15 Replies)
Discussion started by: AAWT
15 Replies

6. Shell Programming and Scripting

file merge based on common columns

I have two files 1.txt 34, ABC, 7, 8, 0.9 35, CDE, 6.5, -2, 0.01 2.txt 34, ABC, 9, 6, -1.9 35, CDE, 8.5, -2.3, 5.01 So in both files common columns are 1 and 2 so final o/p should look like 34, ABC, 7, 8, 0.9, 9, 6, -1.9 35, CDE, 6.5, -2, 0.01, 8.5, -2.3, 5.01 I tried using... (3 Replies)
Discussion started by: manas_ranjan
3 Replies

7. Shell Programming and Scripting

Count and merge using common column

I have the following records from multiple files. 415 A G 415 A G 415 A T 415 A . 415 A . 421 G A 421 G A,C 421 G A 421 G A 421 G A,C 421 G . 427 A C 427 A ... (3 Replies)
Discussion started by: empyrean
3 Replies

8. Shell Programming and Scripting

Merge files based on both common and uncommon rows

Hi, I have two files A (2190 rows) and file B (1100 rows). I want to merge the contents of two files based on common field, also I need the unmatched rows from file A file A: ABC XYZ PQR file B: >LMN|chr1:11000-12456: >ABC|chr15:176578-187678: >PQR|chr3:14567-15866: output... (3 Replies)
Discussion started by: Diya123
3 Replies

9. Shell Programming and Scripting

Seperated by columns, merge in a file, sort them on common column

Hi All, I have 4 files in below format. I took them as an example. File 1: Cut from position 1-4 then 6-7 then 8-14 then rest left and make them as columns in one new file. Inserting character H to the initial of all line like HCTOT. CTOT 456787897 Low fever CTOR 556712345 High fever... (2 Replies)
Discussion started by: Mannu2525
2 Replies

10. Shell Programming and Scripting

Merge multiple files with common header

Hi all, Say i have multiple files x1 x2 x3 x4, all with common header (date, time, year, age),, How can I merge them to one singe file "X" in shell scripting Thanks for your suggestions. (2 Replies)
Discussion started by: msarguru
2 Replies
JOIN(1) 						      General Commands Manual							   JOIN(1)

NAME
join - relational database operator SYNOPSIS
join [ options ] file1 file2 DESCRIPTION
Join forms, on the standard output, a join of the two relations specified by the lines of file1 and file2. If file1 is `-', the standard input is used. File1 and file2 must be sorted in increasing ASCII collating sequence on the fields on which they are to be joined, normally the first in each line. There is one line in the output for each pair of lines in file1 and file2 that have identical join fields. The output line normally con- sists of the common field, then the rest of the line from file1, then the rest of the line from file2. Fields are normally separated by blank, tab or newline. In this case, multiple separators count as one, and leading separators are dis- carded. These options are recognized: -an In addition to the normal output, produce a line for each unpairable line in file n, where n is 1 or 2. -e s Replace empty output fields by string s. -jn m Join on the mth field of file n. If n is missing, use the mth field in each file. -o list Each output line comprises the fields specifed in list, each element of which has the form n.m, where n is a file number and m is a field number. -tc Use character c as a separator (tab character). Every appearance of c in a line is significant. SEE ALSO
sort(1), comm(1), awk(1) BUGS
With default field separation, the collating sequence is that of sort -b; with -t, the sequence is that of a plain sort. The conventions of join, sort, comm, uniq, look and awk(1) are wildly incongruous. JOIN(1)
All times are GMT -4. The time now is 01:46 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy