Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

awk colon separated items

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers awk colon separated items
# 1  
Old 03-15-2016
awk colon separated items
Hi,

I need to filter my data based on items in column 23. Column 1 until column 23 are tab separated. This is how column 23 looks like:

PRIMARY=<0/1:504:499,5:.:.:.:0.01:1:15:.>

I want to extract lines if items 7 (separated by : ) in column 23 are more than 0.25 . In example above , item $7 is 0.01 .

Previously, I just need to count items that pass my criteria . I used this code and it worked perfectly .

Code:
awk -F"\t" 'NR>1 {split($23,a,":"); count[a[7]]++} END {for (i in count) if(i>0.25) sum +=count[i]; print sum}' file.tsv


Now , I need to extract information in all columns too. So i tried to modify the code but I did not work.

Code:
awk -F"\t" 'NR>1 {split($23,a,":"); count[a[7]]++} END {for (i in count) if(i>0.25) print $0}' file.tsv

The output that I want would look the same as my input file, but only with lines that pass the filter.

I have attached the sample input file
sample.txt (1.9 KB) 
# 2  
Old 03-15-2016
Hello janshamsani,

Let's say following is your Input_file(I have taken data from your Input_file only as there was no line having value more than .25 so add one more line to it as follows.).
Code:
1       11291088        T       TA      1       TCGA-AP-A0LH_W_IlluminaGA-DNASeq_exome  15      PASS    1       INS     1011    ZERO    frameshift_variant      MTOR   HIGH     tta/ttTa        CODING  protein_coding  ENST00000361445 GT:DP:AD:BQ:MQ:SB:FA:SS:SSC:MQA NORMAL=<0/1:507:500,7:.:.:.:0.014:0:15:.>       PRIMARY=<0/1:504:499,5:.:.:.:0.01:1:15:.>
1       16255141        GGA     G       91      TCGA-AX-A0IW_W_IlluminaGA-DNASeq_exome  15      PASS    1       DEL     638     ZERO    frameshift_variant      SPEN   HIGH     gag/    CODING  protein_coding  ENST00000375759 GT:DP:AD:BQ:MQ:SB:FA:SS:SSC:MQA NORMAL=<0/1:266:261,5:.:.:.:0.019:0:15:.>       PRIMARY=<0/1:372:367,5:.:.:.:0.013:1:15:.>
1       16255169        G       GA      1       TCGA-AP-A0LH_W_IlluminaGA-DNASeq_exome  15      PASS    1       INS     1020    ZERO    frameshift_variant      SPEN   HIGH     gaa/gAaa        CODING  protein_coding  ENST00000375759 GT:DP:AD:BQ:MQ:SB:FA:SS:SSC:MQA NORMAL=<0/1:510:499,11:.:.:.:0.022:0:15:.>      PRIMARY=<0/1:510:500,10:.:.:.:0.02:1:15:.>
1       16255186        C       CA      1       TCGA-AP-A0LH_W_IlluminaGA-DNASeq_exome  15      PASS    1       INS     1018    ZERO    frameshift_variant      SPEN   HIGH     aaa/Aaaa        CODING  protein_coding  ENST00000375759 GT:DP:AD:BQ:MQ:SB:FA:SS:SSC:MQA NORMAL=<0/1:509:498,11:.:.:.:0.022:0:15:.>      PRIMARY=<0/1:509:500,9:.:.:.:0.018:1:15:.>
1       16255424        G       GA      1       TCGA-AP-A0LH_W_IlluminaGA-DNASeq_exome  15      PASS    1       INS     1012    ZERO    frameshift_variant      SPEN   HIGH     gaa/gAaa        CODING  protein_coding  ENST00000375759 GT:DP:AD:BQ:MQ:SB:FA:SS:SSC:MQA NORMAL=<0/1:507:500,7:.:.:.:0.014:0:15:.>       PRIMARY=<0/1:505:499,6:.:.:.:0.012:1:15:.>
1       16255559        G       GA      1       TCGA-AP-A0LH_W_IlluminaGA-DNASeq_exome  15      PASS    1       INS     1010    ZERO    frameshift_variant      SPEN   HIGH     gaa/gAaa        CODING  protein_coding  ENST00000375759 GT:DP:AD:BQ:MQ:SB:FA:SS:SSC:MQA NORMAL=<0/1:504:499,5:.:.:.:0.01:0:15:.>        PRIMARY=<0/1:506:500,6:.:.:.:0.012:1:15:.>
1       16255727        C       CA      1       TCGA-AP-A0LH_W_IlluminaGA-DNASeq_exome  15      PASS    1       INS     684     ZERO    frameshift_variant      SPEN   HIGH     caa/cAaa        CODING  protein_coding  ENST00000375759 GT:DP:AD:BQ:MQ:SB:FA:SS:SSC:MQA NORMAL=<0/1:387:381,6:.:.:.:0.016:0:15:.>       PRIMARY=<0/1:297:292,5:.:.:.:0.017:1:15:.>
1       16255727        C       CA      1       TCGA-AP-A0LH_W_IlluminaGA-DNASeq_exome  15      PASS    1       INS     684     ZERO    frameshift_variant      SPEN    HIGH    caa/cAaa        CODING  protein_coding  ENST00000375759 GT:DP:AD:BQ:MQ:SB:FA:SS:SSC:MQA NORMAL=<0/1:387:381,6:.:.:.:0.016:0:15:.>       PRIMARY=<0/1:297:292,5:.:.:.:0.27:1:15:.>

Now following code may help you in same.
Code:
awk '{split($NF, array,":");if(array[7]>0.25){print $0}}'  Input_file

Output will be as follows then.
Code:
1       16255727        C       CA      1       TCGA-AP-A0LH_W_IlluminaGA-DNASeq_exome  15      PASS    1       INS     684     ZERO    frameshift_variant      SPEN    HIGH    caa/cAaa        CODING  protein_coding  ENST00000375759 GT:DP:AD:BQ:MQ:SB:FA:SS:SSC:MQA NORMAL=<0/1:387:381,6:.:.:.:0.016:0:15:.>       PRIMARY=<0/1:297:292,5:.:.:.:0.27:1:15:.>

Hope this helps you. Let me know if you have any queries on same.


Thanks,
R. Singh
This User Gave Thanks to RavinderSingh13 For This Post:
janshamsani
# 3  
Old 03-15-2016
Hi,

The code works perfectly. Thank you
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk unique count of partial match with semi-colon

Trying to get the unique count of the below input, but if the text in beginning of $5 is a partial match to another line in the file then it is not unique. awk awk '!seen++ {n++} END {print n}' input 7 input chr1 159174749 159174770 chr1:159174749-159174770 ACKR1 chr1 ... (2 Replies)
Discussion started by: cmccabe
2 Replies

2. Shell Programming and Scripting

awk does not find ids with semi-colon in the name

I am using awk to search $5 of the "input" file using the "list" file as the search criteria. So if the id in line 1 of "list" is found in "search" then it is counted in the ids found. However, if the line in "list" is not found in "search", then it is outputted as is missing. The awk below runs... (3 Replies)
Discussion started by: cmccabe
3 Replies

3. Homework & Coursework Questions

C++ Attempting to modify this function to read from a (;) semi-colon-separated file

After some thought. I am uncomfortable issuing my professors name where, there may be unintended side effects from any negative responses/feedback. Willing to re post if I can omit school / professor publicly, but can message moderator for validation? I am here for knowledge and understanding,... (1 Reply)
Discussion started by: briandanielz
1 Replies

4. Shell Programming and Scripting

awk :help to parse a file to change to separated by colon ":"

Hi experts , I am trying to get the below output: file : 0/6/4/1 0x0019503C2E26 5 UP lan5 snap5 1 ETHER Yes 224 0/6/4/0 0x0019503C2E25 6 UP lan6 snap6 2 ETHER Yes 224 0/2/1/0 0x0019503E6900 0 UP lan0 snap0 3 ETHER Yes 224... (8 Replies)
Discussion started by: rveri
8 Replies

5. Shell Programming and Scripting

How to sort a column in UNIX that is colon separated ":" ?

Experts, how to sort this fields with numerical order : -How to use the sort command in this case, I was thinking with -k but it is not working, lan5000 lan5000:1 lan5000:10 lan5000:11 lan5000:12 lan5000:13 lan5000:14 lan5000:15 lan5000:16 lan5000:17 ... (6 Replies)
Discussion started by: rveri
6 Replies

6. Shell Programming and Scripting

Colon in awk script output

I'm using AIX 5.3 and running a awk replace to modify data as follows: echo 1234: 1234 123 123 444 555 666 7777 | awk '/^:/{split($2,N);n=N} {n=$1} {sub(n,n+10000000)}1' 10001234 1234 123 123 444 555 666 7777 dumb question.. how do I get the colon back in, so it outputs 10001234: 1234... (4 Replies)
Discussion started by: say170
4 Replies

7. Shell Programming and Scripting

Running multiple commands stored as a semi-colon separated string

Hi, Is there a way in Korn Shell that I can run multiple commands stored as a semi-colon separated string, e.g., # vs="echo a; echo b;" # $vs a; echo b; I want to be able to store commands in a variable, then run all of it once and pipe the whole output to another program without using... (2 Replies)
Discussion started by: svhyd
2 Replies

8. Shell Programming and Scripting

awk between items including items

OS=HP-UX ksh The following works, except I want to include the <start> and <end> in the output. awk -F '<start>' 'BEGIN{RS="<end>"; OFS="\n"; ORS=""} {print $2} somefile.log' The following work in bash but not in ksh sed -n '/^<start>/,/^<end>/{/LABEL$/!p}' somefile.log (4 Replies)
Discussion started by: Ikon
4 Replies

9. UNIX for Dummies Questions & Answers

Multiple field separators in awk? (First a space, then a colon)

How do I deal with extracting a portion of a record when multiple field separators are involved. Let's say I have: Mike Harrington;(555) 555-5555:250:100:175 Christian Dobbins;(555) 555-2358:155:90:201 Susan Dalsass;(555) 555-6279:250:60:50 Archie McNichol;(555) 555-1348:250:100:175 Jody... (3 Replies)
Discussion started by: doubleminus
3 Replies

10. Shell Programming and Scripting

two lines into one colon separated line...

Does anyone know how to get these two output lines into one colon ':' separated line with some unix command? Maybe nawk. I've tried to read the nawk and awk man pages but I don't get it right. Are these commands the one to use? Output from find command: # /sw/tools/matlab/7.0.1/man... (2 Replies)
Discussion started by: tonlu
2 Replies
Login or Register to Ask a Question