Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk to output the percentage of a field compared to length

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk to output the percentage of a field compared to length
# 1  
Old 09-30-2015
awk to output the percentage of a field compared to length
The awk below using the sample input would output the following: Basically, it averages the text in $5 that matches if $7 < 30 .
Code:
awk '{if(len==0){last=$5;total=$7;len=1;getline}if($5!=last){printf("%s\t%f\n", last, total/len);last=$5;total=$7;len=1}else{total+=$7;len+=1}}END{printf("%s\t%f\n", last, total/len)}' Input.txt > output.txt

Sample Input
Code:
chr 1   955542  955763  +   AGRN:exon.1 1   0 
chr1   955542  955763  +   AGRN:exon.1 2   0 
chr 1   955542  955763  +   AGRN:exon.1 3   0 
chr 1   955542  955763  +   AGRN:exon.1 4   1 
chr 1   955542  955763  +   AGRN:exon.1 5   1 
chr 1   955542  955763  +   AGRN:exon.1 6   1 
.... 
.... 
chr 1   955542  955763  +   AGRN:exon.1 218 32 
chr 1   955542  955763  +   AGRN:exon.1 219 32 
chr 1   955542  955763  +   AGRN:exon.1 220 32 
chr 1   955542  955763  +   AGRN:exon.1 221 29



Output
Code:
 AGRN:exon.1 4.5714285

My question is I can not seem to add the correct syntax that will also output the total # of lines in $6 that represent $5 and the % of 7 < 30 I know my words may not be all that helpful so hopefully the desired output will help. Thank you Smilie.


Desired output
Code:
  
ID             Average Reads      % of Baits 
AGRN:exon.1    4.5714285          3.16742     (221 (# of lines in $6 /   the # 0f lines < 30 in $7)


the boild is only to show the math and does not need rto be included.
Last edited by cmccabe; 09-30-2015 at 05:57 PM.. Reason: fixed formatting; [mod] extra code tags, added details
# 2  
Old 09-30-2015
All-in-one-line confuses me.
Multi-line and the repeated stuff put in functions:
Code:
awk '
function prt(){
  printf("%s\t%f\t%f\n", last, total/len, t6/t7)
}
function resetvars(){
  last=$5; total=$7; len=1; t6=0; t7=0
}
{
  if (len==0) { resetvars(); getline }
  if ($5!=last) { prt(); resetvars() } else { total+=$7; len+=1; t6+=1; if ($7<30) { t7+=1 } }
}
END { prt() }
' input.txt

Now it is easy (at least for me) to add new code.
You might also want to initialize the new variables.
# 3  
Old 09-30-2015
The input data is very large (~720MB)

Using that code I get the below:

Code:
+    1.000000    -nan
AGRN:exon.1    0.000000    -nan
+    112.000000    2.333333

I thought I followed your code and it is much easier to read, but the calculations don't look right. Thank you very much Smilie.
# 4  
Old 10-01-2015
Looking at the fields in your file,
Code:
awk '{for (i=1; i<=NF; i++) print i, $i}' file
1 chr
2 1
3 955542
4 955763
5 +
6 AGRN:exon.1
7 1
8 0

1 chr1
2 955542
3 955763
4 +
5 AGRN:exon.1
6 2
7 0

, I'm a bit lost about what you want to average, as $5 is either a + sign, or the text "AGRN:exon.1". Same is valid for $7. And, the condition $7 < 30 is never tested in your code.
Where does the line AGRN:exon.1 4.5714285 come from? I can't seem to see the arithmetics...

You may want to revise your spec to enable others the jump in helping.
# 5  
Old 10-01-2015
@RudiC, certainly it's always chr1; some Excel/Outlook/IE has added the occasional spaces.
# 6  
Old 10-01-2015
Still $5 would be a string, no?
# 7  
Old 10-01-2015
Yes, $5 is a string and the format of the data will always be:
Code:
1 chr1
2 955542
3 955763
4 +
5 AGRN:exon.1
6 2
7 0

The awk(maybe not the best) calculates the average for all the $5 that are same and uses the value in $7 only if it is < 30. In the desired output that is the 4.5 #. What I would also like to include is % of $6 that makes up that number. I am not sure the best way and included the math in post 1 to try and help. Did this help any? Thank you Smilie.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Using awk to add length of matching characters between field in file

The awk below produces the current output, which will add +1 to $3. However, I am trying to add the length of the matching characters between $5 and $6 to $3. I have tried using sub as a variable to store the length but am not able to do so correctly. I added comments to each line and the... (4 Replies)
Discussion started by: cmccabe
4 Replies

2. UNIX for Dummies Questions & Answers

Help with awk, where line length and field position are variable

I have several questions about using awk. I'm hoping someone could lend me a hand. (I'm also hoping that my questions make sense.) I have a file that contains pipe separated data. Each line has similar data but the number of fields and the field position on each line is variable. ... (3 Replies)
Discussion started by: Cheese64
3 Replies

3. UNIX for Dummies Questions & Answers

Need an awk script to calculate the percentage of value field and replace

I have a input file called file.txt with the following content: john|622.5674603562933|8|br:1;cn:3;fr:1;jp:1;us:2 andy|0.0|12|**:3;br:1;ca:2;de:2;dz:1;fr:2;nl:1 in fourth filed of input file, calulate percentage of each sub filed seperated by ; semicolon and replace value with percentage . i... (11 Replies)
Discussion started by: veeruasu
11 Replies

4. Shell Programming and Scripting

Need an awk script to calculate the percentage of value field and replace

Need an awk script to calculate the percentage of value field and replace I have a input file called file.txt with the following content: john|622.5674603562933|8|br:1;cn:3;fr:1;jp:1;us:2 andy|0.0|12|**:3;br:1;ca:2;de:2;dz:1;fr:2;nl:1 in fourth filed of input file, calulate percentage of each... (1 Reply)
Discussion started by: veeruasu
1 Replies

5. Shell Programming and Scripting

Replace a field with a character as per the field length

Hi all, I have a requirement to replace a field with a character as per the length of the field. Suppose i have a file where second field is of 20 character length. I want to replace second field with 20 stars (*). like ******************** As the field is not a fixed one, i want to do the... (2 Replies)
Discussion started by: gani_85
2 Replies

6. UNIX for Dummies Questions & Answers

awk - output field separator

In awk, how do I print all fields with a specified output field separator? I have tried the following, which does not print the output FS: echo a b c d | awk 'BEGIN{OFS = ";"}{print $0}' (3 Replies)
Discussion started by: locoroco
3 Replies

7. Shell Programming and Scripting

Flat file-make field length equal to header length

Hello Everyone, I am stuck with one issue while working on abstract flat file which i have to use as input and load data to table. Input Data- ------ ------------------------ ---- ----------------- WFI001 Xxxxxx Control Work Item A Number of Records ------ ------------------------... (5 Replies)
Discussion started by: sonali.s.more
5 Replies

8. Shell Programming and Scripting

awk output field delimiter

Dear All, 1.txt (tab in between each value in a line) a b c a b c a c d you can see below, why with ~ i can output with tab, but = cannot? # awk -F'\t' '$2 ~ /b/' 1 a b c a b c # awk -F'\t' '$2 = "b"' 1 a b c a b c a b d ... (1 Reply)
Discussion started by: jimmy_y
1 Replies

9. Shell Programming and Scripting

perl or awk, field length check

Hi Everyone, 1.txt a;1234;134;1111111 b;123;123;1111111 c;123;1334;1111111 d;1234;1234;1111111 output a;1234;134;1111111 c;123;1334;1111111 d;1234;1234;1111111 if field2 legth>3 or field3 length >3, then output. Please advice. Thanks (4 Replies)
Discussion started by: jimmy_y
4 Replies

10. Shell Programming and Scripting

creating a fixed length output from a variable length input

Is there a command that sets a variable length? I have a input of a variable length field but my output for that field needs to be set to 32 char. Is there such a command? I am on a sun box running ksh Thanks (2 Replies)
Discussion started by: r1500
2 Replies
Login or Register to Ask a Question