Get the average from column and write the value at the last field


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Get the average from column and write the value at the last field
# 1  
Old 12-04-2013
Get the average from column and write the value at the last field

Dear Experts,

Kindly help me please to get the average from column 14 and to write the value at the last field., But we need to take as reference the column 16., example the first 4 lines has the same value in column 16, therefore I want ot get the average only for these lines in column 14. And the same for the rest of the file, each time when change the value in column 16 we need to find the average in column 14 and to write the value at the end.
Example
Imput file
Code:
R,6630,4918,1,2,1,2140.46,0.19,5,0.000226526,0.000159094,1800,5.9,10.9,22.7,67055157,0745                  
R,6630,4920,1,2,1,2159.54,-0.19,5,0.000208446,0.000160313,1808,4.9,10.9,28.7,67055157,0745                 
R,6630,4922,1,2,1,2129.16,0.29,5,0.000207469,0.000165304,1800,4.9,9.9,23.7,67055157,0745                   
R,6630,4924,1,2,1,2134.31,0.54,5,0.00024754,0.000167664,1792,3.0,15.8,23.7,67055157,0745                   
R,6720,5130,1,2,1,2153.83,0.62,5,0.00150284,0.000340223,-1000,4.0,5.9,10.9,67075197,0765                   
R,6720,5132,1,2,1,2118.86,0,5,0.0018297,0.00036593,-988,4.0,5.9,15.8,67075197,0765                         
R,6720,5134,1,2,1,2141.06,-0.14,5,0.00110744,0.000374551,976,3.0,5.9,16.8,67075197,0765                    
R,6720,5136,1,2,1,2139.05,0.21,5,0.00145681,0.000378479,960,4.0,5.9,14.8,67075197,0765                     
R,6720,5138,1,2,1,2119.32,0.05,5,0.0011558,0.00038862,948,4.0,5.9,15.8,67075197,0765                       
R,6720,5140,1,2,1,2146.79,0.05,5,0.000925699,0.000445295,-862,3.0,14.8,30.6,67075197,0765                  
R,6720,5240,1,2,1,2118.8,-0.13,nan,0.000251577,0.000113411,1856,7.9,8.9,21.7,65035127,1104                 
R,6720,5242,1,2,1,2104.16,-0.41,nan,0.000271981,0.000115383,-1496,4.9,9.9,17.8,65035127,1104               
R,6720,5244,1,2,1,2136.64,-0.27,nan,0.000269198,0.000118783,-1500,4.0,9.9,19.8,65035127,1104               
R,6720,5246,1,2,1,2101.84,-0.03,nan,0.000230368,0.000125439,-1504,4.0,7.9,26.7,65035127,1104               
R,6720,5248,1,2,1,2109.24,-0.03,nan,0.000206932,0.0001419,-1508,4.0,7.9,24.7,65035127,1104

desired output
Code:
R,6630,4918,1,2,1,2140.46,0.19,5,0.000226526,0.000159094,1800,5.9,10.9,22.7,67055157,0745,11.8       
R,6630,4920,1,2,1,2159.54,-0.19,5,0.000208446,0.000160313,1808,4.9,10.9,28.7,67055157,0745,11.8      
R,6630,4922,1,2,1,2129.16,0.29,5,0.000207469,0.000165304,1800,4.9,9.9,23.7,67055157,0745,11.8        
R,6630,4924,1,2,1,2134.31,0.54,5,0.00024754,0.000167664,1792,3.0,15.8,23.7,67055157,0745,11.8        
R,6720,5130,1,2,1,2153.83,0.62,5,0.00150284,0.000340223,-1000,4.0,5.9,10.9,67075197,0765,11.8        
R,6720,5132,1,2,1,2118.86,0,5,0.0018297,0.00036593,-988,4.0,5.9,15.8,67075197,0765,7.3               
R,6720,5134,1,2,1,2141.06,-0.14,5,0.00110744,0.000374551,976,3.0,5.9,16.8,67075197,0765,7.3          
R,6720,5136,1,2,1,2139.05,0.21,5,0.00145681,0.000378479,960,4.0,5.9,14.8,67075197,0765,7.3           
R,6720,5138,1,2,1,2119.32,0.05,5,0.0011558,0.00038862,948,4.0,5.9,15.8,67075197,0765,7.3             
R,6720,5140,1,2,1,2146.79,0.05,5,0.000925699,0.000445295,-862,3.0,14.8,30.6,67075197,0765,7.3        
R,6720,5240,1,2,1,2118.8,-0.13,nan,0.000251577,0.000113411,1856,7.9,8.9,21.7,65035127,1104,8.9       
R,6720,5242,1,2,1,2104.16,-0.41,nan,0.000271981,0.000115383,-1496,4.9,9.9,17.8,65035127,1104,8.9     
R,6720,5244,1,2,1,2136.64,-0.27,nan,0.000269198,0.000118783,-1500,4.0,9.9,19.8,65035127,1104,8,9     
R,6720,5246,1,2,1,2101.84,-0.03,nan,0.000230368,0.000125439,-1504,4.0,7.9,26.7,65035127,1104,8.9     
R,6720,5248,1,2,1,2109.24,-0.03,nan,0.000206932,0.0001419,-1508,4.0,7.9,24.7,65035127,1104,8.9

Thanks for your help Smilie
# 2  
Old 12-04-2013
An awk approach by joining input file:
Code:
awk -F, '
        BEGIN {
                CONVFMT = "%.1f"
        }
        NR == FNR {
                A[$1,$2,$16] += $14
                C[$1,$2,$16]++
                next
        }
        {
                $0 = $0 FS A[$1,$2,$16] / C[$1,$2,$16]
        }
        1
' file file

This User Gave Thanks to Yoda For This Post:
# 3  
Old 12-04-2013
Dear Yoda,

Thanks for your help
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to average field if matching string in another

In the awk below I am trying to get the average of the sum of $7 if the string in $4 matches in the line below it. The --- in the desired out is not needed, it is just to illustrate the calculation. The awk executes and produces the current out. I am not sure why the middle line is skipped and the... (2 Replies)
Discussion started by: cmccabe
2 Replies

2. Shell Programming and Scripting

Compute average based on field values

Im looking for a way to average the values in field 14 (when field 2 is equal to 2016) and fields 3 and 4 (when field 2 is equal to 2017). Any help is appreciated. 001001 2016 33.22 38.19 48.07 51.75 59.77 67.68 70.86 72.21 66.92 53.67 42.31 40.15 001001 2017 ... (10 Replies)
Discussion started by: ncwxpanther
10 Replies

3. Shell Programming and Scripting

awk to combine by field and average by another

In the below awk I am trying to combine all matching $4 into a single $5 (up to the -), and count the lines in $6 and average all values in $7. The awk is close but it seems to only be using the last line in the file and skipping all others. The posted input is a sample of the file that is over... (3 Replies)
Discussion started by: cmccabe
3 Replies

4. Shell Programming and Scripting

Check first column - average second column based on a condition

Hi, My input file Gene1 1 Gene1 2 Gene1 3 Gene1 0 Gene2 0 Gene2 0 Gene2 4 Gene2 8 Gene3 9 Gene3 9 Gene4 0 Condition: If the first column matches, then look in the second column. If there is a value of zero in the second column, then don't consider that record while averaging. ... (5 Replies)
Discussion started by: jacobs.smith
5 Replies

5. Shell Programming and Scripting

Combine identical lines and average the one variable field

I have the following file 299899 chrX_299716_300082 196 78.2903 299991 chrX_299982_300000 18.2538 Tajd:0.745591 FayWu:-0.245701 T2:1.45 299899 chrX_299716_300082 196 78.2903 299991 chrX_299982_300000 18.2538 Tajd:0.745591 FayWu:-0.245701 T2:0.283 311027 chrX_310892_311162 300 91.6452... (2 Replies)
Discussion started by: jfern
2 Replies

6. Shell Programming and Scripting

Get column average using ID

I have a file that looks like this: id window BV 1 1 0.5 1 2 0.2 1 3 0.1 2 1 0.5 2 2 0.1 2 3 0.2 3 1 0.4 3 2 0.6 3 3 0.8 Using awk, how would I get the average BV for window 1? Output like this: window avgBV 1 0.47 2 0.23 (10 Replies)
Discussion started by: jwbucha
10 Replies

7. Shell Programming and Scripting

Calculate the average of a column based on the value of another column

Hi, I would like to calculate the average of column 'y' based on the value of column 'pos'. For example, here is file1 id pos y c 11 1 220 aa 11 4333 207 f 11 5333 112 ee 11 11116 305 e 11 11117 310 r 11 22228 781 gg 11 ... (2 Replies)
Discussion started by: jackken007
2 Replies

8. Shell Programming and Scripting

To find sum & average of 8th field

Hi Friends, I have many files like below. total,0.7%,0.0%,0.2%,0.0%,0.2%,0.7%,98.0% total,1.9%,0.0%,0.4%,0.0%,0.0%,6.8%,90.6% total,0.9%,0.0%,0.4%,0.0%,0.0%,0.0%,98.5% total,1.4%,0.0%,0.7%,0.0%,0.2%,2.9%,94.5% total,0.7%,0.0%,0.4%,0.0%,0.0%,0.9%,97.7%... (13 Replies)
Discussion started by: SunilB2011
13 Replies

9. Shell Programming and Scripting

print running field average for a set of lines

Hi everyone, I have a program that generates logs that contains sections like this: IMAGE INPUT 81 0 0.995 2449470 0 1726 368 1 0.0635 0.3291 82 0 1.001 2448013 0 1666 365 1 0.0649 0.3235 83 0 1.009 2444822 0 1697 371 1 ... (3 Replies)
Discussion started by: euval
3 Replies

10. UNIX for Dummies Questions & Answers

calculate average of column 2

Hi I have fakebook.csv as following: F1(current date) F2(popularity) F3(name of book) F4(release date of book) 2006-06-21,6860,"Harry Potter",2006-12-31 2006-06-22,,"Harry Potter",2006-12-31 2006-06-23,7120,"Harry Potter",2006-12-31 2006-06-24,,"Harry Potter",2006-12-31... (0 Replies)
Discussion started by: onthetopo
0 Replies
Login or Register to Ask a Question