Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Partial average of a column with awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Partial average of a column with awk
# 1  
Old 05-06-2010
Partial average of a column with awk
Hello,

Let's assume I have 100 files FILE_${m} (0<m<101). Each of them contains 100 lines and 10 columns.
I'd like to get in a file called "result" the average value of column 3, ONLY between lines 11 and 17, in order to plot that average as a function of the parameter m.
So far I can compute with awk the average of the full column 3 (dividing by NR), but I can't get only the 7 lines that I want... any idea?
Here is my try:

Code:
#!/bin/bash
    echo "# m    AVERAGE" > result
    for ((m=1; m<=100; m=m+1))
    do
        FILE=FILE_${m}
        AVERAGE=$(awk '{for(NR=11; NR<= 17; NR++){sum +=$3};} END {print sum/7}' $FILE)                                 
        echo "$m  $AVERAGE" >> result                    
    done

Thanks in advance! Smilie
Last edited by radoulov; 05-06-2010 at 10:51 AM.. Reason: Please use code tags!
# 2  
Old 05-06-2010
Code:
AVERAGE=$(awk ' NR==11 && NR<= 17{sum +=$3; if(NR == 17){exit}} END {print sum/7}' $FILE)

# 3  
Old 05-06-2010
Thanks anbu23! I just have one question:
I tried with only one FILE_${m} containing 1.0 in each line on column 3.
So the average should be 7*1.0/7 = 1.0, but instead I get 0.142857 which is equal to 1/7.
So I replaced your solution with:
AVERAGE=$(awk ' NR==11 && NR<= 17{sum +=$3; if(NR == 17){exit}} END {print sum}' $FILE)

This time it works, but I don't understand why... any idea?
# 4  
Old 05-06-2010
Code:
NR==11 && NR<= 17

This is only true if NR==11, try:
Code:
AVERAGE=$(awk 'NR==11,NR==17{sum+=$3}NR==17{exit}END{print sum/7}' $FILE)

# 5  
Old 05-06-2010
Quote:
Originally Posted by DMini
Thanks anbu23! I just have one question:
I tried with only one FILE_${m} containing 1.0 in each line on column 3.
So the average should be 7*1.0/7 = 1.0, but instead I get 0.142857 which is equal to 1/7.
So I replaced your solution with:
AVERAGE=$(awk ' NR==11 && NR<= 17{sum +=$3; if(NR == 17){exit}} END {print sum}' $FILE)

This time it works, but I don't understand why... any idea?
Use Franklin52's code or this
Code:
AVERAGE=$(awk ' NR>=11 && NR<= 17{sum +=$3; if(NR == 17){exit}} END {print sum/7}' $FILE)

# 6  
Old 05-06-2010
Thanks!
I do not fully understand the use of the coma between NR==11 and NR==17, just by curiosity, how would you increment by 2 for example? (to sum on even lines for example)
# 7  
Old 05-06-2010
Quote:
Originally Posted by DMini
Thanks!
I do not fully understand the use of the coma between NR==11 and NR==17
It selects records in the range between the 2 conditions.
Quote:
just by curiosity, how would you increment by 2 for example? (to sum on even lines for example)
This uses the modulo operator % to select the even lines:
Code:
awk '!(NR%2){sum+=$3;i++}END{print sum/i}' $FILE

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Search for partial matches in particular column

I have a list a b c d I want to search this list to have partial matches in column 2 in data file col1 col2 col3 1 a/e aa 2 b/e aa 3 z/y aa 4 t/u bb 5 d/f aa 6 a/t aa and extract the relevant rows with header (4 Replies)
Discussion started by: jianp83
4 Replies

2. Shell Programming and Scripting

Check first column - average second column based on a condition

Hi, My input file Gene1 1 Gene1 2 Gene1 3 Gene1 0 Gene2 0 Gene2 0 Gene2 4 Gene2 8 Gene3 9 Gene3 9 Gene4 0 Condition: If the first column matches, then look in the second column. If there is a value of zero in the second column, then don't consider that record while averaging. ... (5 Replies)
Discussion started by: jacobs.smith
5 Replies

3. Shell Programming and Scripting

Get column average using ID

I have a file that looks like this: id window BV 1 1 0.5 1 2 0.2 1 3 0.1 2 1 0.5 2 2 0.1 2 3 0.2 3 1 0.4 3 2 0.6 3 3 0.8 Using awk, how would I get the average BV for window 1? Output like this: window avgBV 1 0.47 2 0.23 (10 Replies)
Discussion started by: jwbucha
10 Replies

4. UNIX for Dummies Questions & Answers

Average by specific column value, awk

Hi, I am searching for an awk-script that computes the mean values for the $2 column, but addicted to the values in the $1 column. It also should delete the unnecessary lines after computing... An example (for some reason I cant use the code tag button): cat list.txt 1 10 1 30 1 20... (2 Replies)
Discussion started by: bjoern456
2 Replies

5. Shell Programming and Scripting

Calculate the average of a column based on the value of another column

Hi, I would like to calculate the average of column 'y' based on the value of column 'pos'. For example, here is file1 id pos y c 11 1 220 aa 11 4333 207 f 11 5333 112 ee 11 11116 305 e 11 11117 310 r 11 22228 781 gg 11 ... (2 Replies)
Discussion started by: jackken007
2 Replies

6. Shell Programming and Scripting

awk/sed to extract column bases on partial match

Hi I have a log file which has outputs like the one below conn=24,196 op=1 RESULT err=0 tag=0 nentries=9 etime=3,712 dbtime=0 mem=486,183,328/2,147,483,648 Now most of the time I am only interested in the time ( the first column) and a column that begins with etime i.e... (8 Replies)
Discussion started by: pkabali
8 Replies

7. Shell Programming and Scripting

AWK - Print partial line/partial field

Hello, this is probably a simple request but I've been toying with it for a while. I have a large list of devices and commands that were run with a script, now I have lines such as: a-router-hostname-C#show ver I want to print everything up to (and excluding) the # and everything after it... (3 Replies)
Discussion started by: ippy98
3 Replies

8. Shell Programming and Scripting

AWK: how to get average based on certain column

Hi, I'm new to shell programming, can anyone help me on this? I want to do following operations - 1. Average salary for each country 2. Total salary for each city and data that looks like - salary country city 10000 zzz BN 25000 zzz BN 30000 zzz BN 10000 yyy ZN 15000 yyy ZN ... (3 Replies)
Discussion started by: shell123
3 Replies

9. Shell Programming and Scripting

Partial Column extraction/Process/Repasting changed Columns back to Source file

I have the following requirement. file1.txt (this could contain 5 million rows) ABC 1234 XYZ .... (3000 bytes) QRD 4612 GHT .... (3000 bytes) I need to create file2.txt 1234 4612 I have a EAI process to change file2.txt into file3.txt 4555 3743 Then I would have to use... (0 Replies)
Discussion started by: jostul
0 Replies

10. UNIX for Dummies Questions & Answers

Use awk to calculate average of column 3

Suppose I have 500 files in a directory and I need to Use awk to calculate average of column 3 for each of the file, how would I do that? (6 Replies)
Discussion started by: grossgermany
6 Replies
Login or Register to Ask a Question