Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Averaging 3 files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Averaging 3 files
# 1  
Old 10-17-2011
Averaging 3 files
Hi,

I am trying to average the values from 3 files with the same format. They are very large files so I will describe the file and show some it of. Basically the file has 83 columns (with nearly 7000 rows). The first three columns are the same for each file while the remaining 80 are values that differ (the ones I want averaged).

The file is tab separated.

The file looks like this:

File 1 (only showing 4 of the 80 columns and 1 row):

Code:
#geneid	strand	feature_type	up.1	        up.2	        up.3	        up.4
AAC1	        -	      CDS	                -0.383672	-0.470544	-0.024423	-0.179893


File 2 (format is the same as above but values differ):

Code:
#geneid	strand	feature_type	up.1	        up.2	        up.3	        up.4
AAC1	        -	        CDS	                -0.45433	-0.560544	-0.114423	-0.174582


File 3 (format is the same but values differ):

Code:
#geneid	strand	feature_type	up.1	        up.2	        up.3	        up.4
AAC1	        -	        CDS	                -0.283672	-0.570544	-0.624423	-0.669893


Output file (average values):

Code:
#geneid	strand	feature_type	up.1	        up.2	        up.3	        up.4
AAC1	        -	        CDS	                -0.373891	-0.533877	-0.254423	-0.341456

Thanks

Kyle
# 2  
Old 10-17-2011
nawk -f kyl.awk OFS='\t' file1 file2 file3 fileN
kyl.awk:
Code:
FNR==1 { h=$0;next}
NR==2 {f3=$1 OFS $2 OFS $3}
{for(i=4;i<=NF;i++) s[i]+=$i; nf=NF;nr=NR}
END {

  printf h ORS f3 OFS
  for(i=4;i<=nf;i++) printf("%.5f%c", s[i]/nr, (i==nr)?ORS:OFS)
}

This User Gave Thanks to vgersh99 For This Post:
kylle345
# 3  
Old 10-17-2011
Sorry it does work but for only 1 row. My file contains 6000 rows. Is there something to change in the code?

Thanks
# 4  
Old 10-17-2011
Quote:
Originally Posted by kylle345
Sorry it does work but for only 1 row. My file contains 6000 rows. Is there something to change in the code?

Thanks
sorry, I misunderstood what you were after...
kyl.awk:
Code:
FNR==1 { h=$0;f++;next}
NR==2 {f3=$1 OFS $2 OFS $3}
{for(i=4;i<=NF;i++) s[FNR,i]+=$i; nf=NF;fnr=FNR}
END {
  print h
  for (i=2;i<=fnr;i++) {
     printf f3 OFS
     for(j=4;j<=nf;j++) printf("%.5f%c", s[i,j]/f, (j==nf)?ORS:OFS)
  }
}

This User Gave Thanks to vgersh99 For This Post:
kylle345
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Loop for row-wise averaging of multiple files using awk

Hello all, I need to compute a row-wise average of files with a single column based on the pattern of the filenames. I really appreciate any help on this. it would just be very difficult to do them manually as the rows are mounting to 100,000 lines. the filenames are as below with convention as... (2 Replies)
Discussion started by: ida1215
2 Replies

2. Shell Programming and Scripting

averaging specific column of multiple files

Hi all, I'm needing help again on scripting. I have weekly files with 3 columns, and I need to do monthly averaging on the values on column 3, the file naming convention is as follows: 20000105.u- 2000:year 01:month 05:day 20000112.u 20000119.u 20000126.u 20000202.u 20020209.u I need to... (15 Replies)
Discussion started by: ida1215
15 Replies

3. Shell Programming and Scripting

Averaging 3 files with multiple rows

Hi, I am trying to average the values from 3 files with the same format. They are very large files so I will describe the file and show some it of. Basically the file has 83 columns (with nearly 7000 rows). The first three columns are the same for each file while the remaining 80 are values that... (1 Reply)
Discussion started by: kylle345
1 Replies

4. Shell Programming and Scripting

averaging columns

Hi, I have a file that has 201 columns (1 for the name of each row and the other 200 are values) I want to print the average for each column the file looks like this (as an example this only has 7 columns with values) 1 2 3 4 5 6 7 abr 5 6 7 1 2 4 5 hhr 2 1 3 4 ... (8 Replies)
Discussion started by: kylle345
8 Replies

5. Shell Programming and Scripting

Averaging help in awk

Hi all, I have a data file like below, where Time is in the second column DATE TIME FRAC_DAYS_SINCE_JAN1 2011-06-25 08:03:20.000 175.33564815 2011-06-25 08:03:25.000 175.33570602... (10 Replies)
Discussion started by: gd9629
10 Replies

6. Shell Programming and Scripting

Hourly averaging using Awk

Hey all, I have a set of 5-second data as shown below. I need to find an hourly average of this data. date co2 25/06/2011 08:04:00 8.30 25/06/2011 08:04:05 8.31 25/06/2011 08:04:10 8.32 25/06/2011 08:04:15 8.33 25/06/2011 08:04:20 ... (5 Replies)
Discussion started by: gd9629
5 Replies

7. Shell Programming and Scripting

Averaging segments and including the name

Hello, I have a awk line that averages rows. So if my file looks like this: Jack 1 1 1 1 1 1 Joe 1 1 1 1 1 1 Jerry 0 0 0 0 0 0 John 1 1 1 0 0 0 The awk line below skips column 1 and then averaged the rows awk -F'\t' -v r=3... (3 Replies)
Discussion started by: phil_heath
3 Replies

8. UNIX for Dummies Questions & Answers

Averaging

Hello all, I'm trying to perform an averaging procedure which selects a selection of rows, average the corresponding value, selects the next set of rows and average the corresponding values etc. The data below illustrates what I want to do. Given two columns (day and value), I want to... (2 Replies)
Discussion started by: Muhammad Rahiz
2 Replies

9. Shell Programming and Scripting

Averaging multiple columns

Hello, I am trying to average multiple columns simultaneously while skipping the first column. I am using this awk line to average one column awk '{sum+=$3} END { print "Average = ",sum/NR}' But I want to be able to do it for multiple columns while skipping the first column. There... (4 Replies)
Discussion started by: gisele_l
4 Replies

10. Shell Programming and Scripting

Averaging segments

Hi, I have a file that I want to average. So specifically I want to average every third column for each row. Here is an example of my file 2 2 2 3 3 3 1 1 1 5 5 5 Heres what I want it to look like after averaging every third column 2 3 1 5 thanks (11 Replies)
Discussion started by: kylle345
11 Replies
Login or Register to Ask a Question