Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk to combine all matching fields in input but only print line with largest value in specific field

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk to combine all matching fields in input but only print line with largest value in specific field
# 1  
Old 12-29-2016
awk to combine all matching fields in input but only print line with largest value in specific field
In the below I am trying to use awk to match all the $13 values in input, which is tab-delimited,
that are in $1 of gene which is just a single column of text.

However only the line with the greatest $9 value in input needs to be printed.
So in the example below all the MECP2 and LTBP1 lines in input are found, but since line 2 MECP2 and line 5 LTBP1 have the greatest $9 value they are printed.
The awk does run but the output is not desired and I'm not sure if this is the best way. Thank you Smilie.

gene
Code:
MECP2
LTPB1
GJB2

input
Code:
219    NM_001110792    chrX    -    153295685    153363188    153295817    153363122    3    153295685,153297657,153363060,    153296901,153298008,153363188,    0    MECP2    cmpl    cmpl    2,2,0,
219    NM_004992    chrX    -    153287263    153363188    153295817    153357667    4    153287263,153297657,153357641,153363060,    153296901,153298008,153357765,153363188,    0    MECP2    cmpl    cmpl    2,2,0,-1,
1    NM_000627    chr2    +    33359663    33624575    33359804    33623612    30    33359663,33411922,33413643,33442618,33447146,33468728,33477743,33482350,33484654,33487788,33488360,33498722,33500031,33500867,33505105,33518226,33525517,33526588,33534500,33540210,33567904,33572433,33585663,33586495,33588456,33589282,33590399,33614250,33622199,33623430,    33360027,33412147,33413918,33442721,33447218,33468851,33477911,33482578,33484677,33487888,33488459,33498848,33500157,33500990,33505225,33518349,33525640,33526711,33534623,33540336,33568030,33572577,33585846,33586582,33588585,33589423,33590570,33614373,33622349,33624575,    0    LTBP1    cmpl    cmpl    0,1,1,0,1,1,1,1,1,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
1    NM_001166264    chr2    +    33359663    33624575    33359804    33623612    29    33359663,33411922,33413643,33442618,33447146,33468728,33477743,33482350,33484654,33487788,33488360,33498722,33500031,33500867,33505105,33518226,33525517,33526588,33534500,33540210,33572433,33585663,33586495,33588456,33589282,33590399,33614250,33622199,33623430,    33360027,33412147,33413918,33442721,33447218,33468851,33477911,33482578,33484677,33487888,33488459,33498848,33500157,33500990,33505225,33518349,33525640,33526711,33534623,33540336,33572577,33585846,33586582,33588585,33589423,33590570,33614373,33622349,33624575,    0    LTBP1    cmpl    cmpl    0,1,1,0,1,1,1,1,1,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
1    NM_206943    chr2    +    33172368    33624575    33172391    33623612    34    33172368,33173941,33245975,33335648,33359859,33411922,33413643,33442618,33447146,33468728,33477743,33482350,33484654,33487788,33488360,33498722,33500031,33500867,33505105,33518226,33525517,33526588,33534500,33540210,33567904,33572433,33585663,33586495,33588456,33589282,33590399,33614250,33622199,33623430,    33172885,33174012,33246273,33335818,33360027,33412147,33413918,33442721,33447218,33468851,33477911,33482578,33484677,33487888,33488459,33498848,33500157,33500990,33505225,33518349,33525640,33526711,33534623,33540336,33568030,33572577,33585846,33586582,33588585,33589423,33590570,33614373,33622349,33624575,    0    LTBP1    cmpl    cmpl    0,2,1,2,1,1,1,0,1,1,1,1,1,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
1    NM_001166265    chr2    +    33359663    33624575    33359804    33623612    30    33359663,33411922,33413643,33442618,33447146,33468728,33477743,33482509,33484654,33487788,33488360,33498722,33500031,33500867,33505105,33518226,33525517,33526588,33534500,33540210,33567904,33572433,33585663,33586495,33588456,33589282,33590399,33614250,33622199,33623430,    33360027,33412147,33413918,33442721,33447218,33468851,33477911,33482578,33484677,33487888,33488459,33498848,33500157,33500990,33505225,33518349,33525640,33526711,33534623,33540336,33568030,33572577,33585846,33586582,33588585,33589423,33590570,33614373,33622349,33624575,    0    LTBP1    cmpl    cmpl    0,1,1,0,1,1,1,1,1,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,
1    NM_001166265    chr2    +    33359663    33624575    33359804    33623612    30    33359663,33411922,33413643,33442618,33447146,33468728,33477743,33482509,33484654,33487788,33488360,33498722,33500031,33500867,33505105,33518226,33525517,33526588,33534500,33540210,33567904,33572433,33585663,33586495,33588456,33589282,33590399,33614250,33622199,33623430,    33360027,33412147,33413918,33442721,33447218,33468851,33477911,33482578,33484677,33487888,33488459,33498848,33500157,33500990,33505225,33518349,33525640,33526711,33534623,33540336,33568030,33572577,33585846,33586582,33588585,33589423,33590570,33614373,33622349,33624575,    0    LTBP1    cmpl    cmpl    0,1,1,0,1,1,1,1,1,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,

desired output line 2 and line 5
Code:
219    NM_004992    chrX    -    153287263    153363188    153295817    153357667    4    153287263,153297657,153357641,153363060,    153296901,153298008,153357765,153363188,    0    MECP2    cmpl    cmpl    2,2,0,-1,
1    NM_206943    chr2    +    33172368    33624575    33172391    33623612    34    33172368,33173941,33245975,33335648,33359859,33411922,33413643,33442618,33447146,33468728,33477743,33482350,33484654,33487788,33488360,33498722,33500031,33500867,33505105,33518226,33525517,33526588,33534500,33540210,33567904,33572433,33585663,33586495,33588456,33589282,33590399,33614250,33622199,33623430,    33172885,33174012,33246273,33335818,33360027,33412147,33413918,33442721,33447218,33468851,33477911,33482578,33484677,33487888,33488459,33498848,33500157,33500990,33505225,33518349,33525640,33526711,33534623,33540336,33568030,33572577,33585846,33586582,33588585,33589423,33590570,33614373,33622349,33624575,    0    LTBP1    cmpl    cmpl    0,2,1,2,1,1,1,0,1,1,1,1,1,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,

awk
Code:
awk -F'\t' 'NR==FNR{A[$1];next}$13 in A && $9>a {a=$9; b=$0} END {print b}' gene input > out

current output skipping MECP2 line
Code:
1    NM_206943    chr2    +    33172368    33624575    33172391    33623612    34    33172368,33173941,33245975,33335648,33359859,33411922,33413643,33442618,33447146,33468728,33477743,33482350,33484654,33487788,33488360,33498722,33500031,33500867,33505105,33518226,33525517,33526588,33534500,33540210,33567904,33572433,33585663,33586495,33588456,33589282,33590399,33614250,33622199,33623430,    33172885,33174012,33246273,33335818,33360027,33412147,33413918,33442721,33447218,33468851,33477911,33482578,33484677,33487888,33488459,33498848,33500157,33500990,33505225,33518349,33525640,33526711,33534623,33540336,33568030,33572577,33585846,33586582,33588585,33589423,33590570,33614373,33622349,33624575,    0    LTBP1    cmpl    cmpl    0,2,1,2,1,1,1,0,1,1,1,1,1,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,

---------- Post updated at 01:40 PM ---------- Previous update was at 08:34 AM ----------

This seems to work:

Code:
awk '{if ($9>max[$13])   # combine all matching $13 and grad the max value of $9
         {max[$13]=$9; val[$13]=$0}# read them into variable
     } 
     END {for (i in val) print val[i]}' input > out   # print variable with line

Thank you :).

Last edited by cmccabe; 12-29-2016 at 11:06 AM.. Reason: fixed format, added current output
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Continued trouble matching fields in different files and selective field printing ([g]awk)

I apologize in advance, but I continue to have trouble searching for matches between two files and then printing portions of each to output in awk and would very much appreciate some help. I have data as follows: File1 PS012,002 PRQ 0 1 1 17 1 0 -1 3 2 1 2 -1 ... (7 Replies)
Discussion started by: jvoot
7 Replies

2. Shell Programming and Scripting

awk to print fields that match using conditions and a default value for non-matching in two files

Trying to use awk to match the contents of each line in file1 with $5 in file2. Both files are tab-delimited and there may be a space or special character in the name being matched in file2, for example in file1 the name is BRCA1 but in file2 the name is BRCA 1 or in file1 name is BCR but in file2... (6 Replies)
Discussion started by: cmccabe
6 Replies

3. Shell Programming and Scripting

awk to combine matches and use a field to adjust coordinates in other fields

Trying to output a result that uses the data from file to combine and subtract specific lines. If $4 matches in each line then the last $6 value is added to $2 and that becomes the new$3. Each matching line in combined into one with $1 then the original $2 then the new$3 then $5. For the cases... (4 Replies)
Discussion started by: cmccabe
4 Replies

4. Shell Programming and Scripting

How to print with awk specific field different from specific character?

Hello, i need help with awk. I have this file: cat number DirB port 67 er_enc_out 0 er_bad_os 0 DirB port 71 er_enc_out 56 er_bad_os 0 DirB port 74 er_enc_out 0 er_bad_os 0 DirB port 75 ... (4 Replies)
Discussion started by: elilmal
4 Replies

5. Shell Programming and Scripting

How to print 1st field and last 2 fields together and the rest of the fields after it using awk?

Hi experts, I need to print the first field first then last two fields should come next and then i need to print rest of the fields. Input : a1,abc,jsd,fhf,fkk,b1,b2 a2,acb,dfg,ghj,b3,c4 a3,djf,wdjg,fkg,dff,ggk,d4,d5 Expected output: a1,b1,b2,abc,jsd,fhf,fkk... (6 Replies)
Discussion started by: 100bees
6 Replies

6. UNIX for Advanced & Expert Users

awk print all fields except matching regex

grep -v will exclude matching lines, but I want something that will print all lines but exclude a matching field. The pattern that I want excluded is '/mnt/svn' If there is a better solution than awk I am happy to hear about it, but I would like to see this done in awk as well. I know I can... (11 Replies)
Discussion started by: glev2005
11 Replies

7. Shell Programming and Scripting

How to Print from nth field to mth fields using awk

Hi, Is there any short method to print from a particular field till another filed using awk? Example File: File1 ==== 1|2|acv|vbc|......|100|342 2|3|afg|nhj|.......|100|346 Expected output: File2 ==== acv|vbc|.....|100 afg|nhj|.....|100 (8 Replies)
Discussion started by: machomaddy
8 Replies

8. Shell Programming and Scripting

awk - print all fields except for last field

How do I print all the fields of a record except for the $(NF) field? (4 Replies)
Discussion started by: locoroco
4 Replies

9. Shell Programming and Scripting

AWK : Add Fields of lines with matching field

Dear All, I would like to add values of a field, if the lines match in a certain field. Then I would like to divide the sum though the number of lines that have a matched field. This is the Input: Input: Test1 5 Test1 10 Test2 2 Test2 5 Test2 13 Test3 4 Output: Test1 7.5 Test1 7.5... (6 Replies)
Discussion started by: DerSeb
6 Replies

10. Shell Programming and Scripting

Print matching field using awk

Hi All, I have a string like below: str="Hold=True Map=False 'This will map the data' Run=Yes Modify=False" I want to print the field Run=Yes and retrive the value "Yes". I cannot use simple awk command because the position of the "Run" will be different at different times. Is there a way... (6 Replies)
Discussion started by: deepakgang
6 Replies
Login or Register to Ask a Question