Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Averaging 3 files with multiple rows
Top Forums Shell Programming and Scripting Averaging 3 files with multiple rows Post 302565356 by kylle345 on Monday 17th of October 2011 04:27:50 PM
Old 10-17-2011
Averaging 3 files with multiple rows
Hi,

I am trying to average the values from 3 files with the same format. They are very large files so I will describe the file and show some it of. Basically the file has 83 columns (with nearly 7000 rows). The first three columns are the same for each file while the remaining 80 are values that differ (the ones I want averaged).

The file is tab separated.

The file looks like this:

File 1 (only showing 4 of the 80 columns and 1 row):


Code:
#geneid	strand	feature_type	up.1	        up.2	        up.3	        up.4
AAC1	        -	      CDS	                -0.383672	-0.470544	-0.024423	-0.179893


File 2 (format is the same as above but values differ):


Code:
#geneid	strand	feature_type	up.1	        up.2	        up.3	        up.4
AAC1	        -	        CDS	                -0.45433	-0.560544	-0.114423	-0.174582

File 3 (format is the same but values differ):



Code:
#geneid	strand	feature_type	up.1	        up.2	        up.3	        up.4
AAC1	        -	        CDS	                -0.283672	-0.570544	-0.624423	-0.669893


Output file (average values):


Code:
#geneid	strand	feature_type	up.1	        up.2	        up.3	        up.4
AAC1	        -	        CDS	                -0.373891	-0.533877	-0.254423	-0.341456

The file has many rows so there is a ton of averaging. Thanks

Kyle
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

extract multiple cloumns from multiple files; skip rows and include filenames; awk

Hello, I am trying to write a bash shell script that does the following: 1.Finds all *.txt files within my directory of interest 2. reads each of the files (25 files) one by one (tab-delimited format and have the same data format) 3. skips the first 10 rows of the file 4. extracts and... (4 Replies)
Discussion started by: manishabh
4 Replies

2. Shell Programming and Scripting

Averaging multiple columns

Hello, I am trying to average multiple columns simultaneously while skipping the first column. I am using this awk line to average one column awk '{sum+=$3} END { print "Average = ",sum/NR}' But I want to be able to do it for multiple columns while skipping the first column. There... (4 Replies)
Discussion started by: gisele_l
4 Replies

3. UNIX for Dummies Questions & Answers

Averaging the rows using 'awk'

Dear all, I have the data in the following format. I want to do average of each NR= 5 (rows) for all the 3 ($1,$2, $3) columns and want to print average result in another file in the same format. I dont know how to write code for this in 'awk', can some one help me to write a code for this in... (1 Reply)
Discussion started by: arvindr
1 Replies

4. Shell Programming and Scripting

Averaging Data From Multiple Columns, Using Header if Possible

Hi, I have a file with multiple tab delimited columns and I would like to have the average of each column: Iteration Tree No Lh HMean 1000 1 -78.834717 -78.834717 1100 1 -77.991031 -78.624046 1200 1 -79.416055 -78.761861 1300 1 -79.280494 -78.968099 1400 1 -82.846275 -80.808696 ... (4 Replies)
Discussion started by: mikey11415
4 Replies

5. Shell Programming and Scripting

Averaging 3 files

Hi, I am trying to average the values from 3 files with the same format. They are very large files so I will describe the file and show some it of. Basically the file has 83 columns (with nearly 7000 rows). The first three columns are the same for each file while the remaining 80 are values... (3 Replies)
Discussion started by: kylle345
3 Replies

6. Shell Programming and Scripting

averaging specific column of multiple files

Hi all, I'm needing help again on scripting. I have weekly files with 3 columns, and I need to do monthly averaging on the values on column 3, the file naming convention is as follows: 20000105.u- 2000:year 01:month 05:day 20000112.u 20000119.u 20000126.u 20000202.u 20020209.u I need to... (15 Replies)
Discussion started by: ida1215
15 Replies

7. Shell Programming and Scripting

Loop for row-wise averaging of multiple files using awk

Hello all, I need to compute a row-wise average of files with a single column based on the pattern of the filenames. I really appreciate any help on this. it would just be very difficult to do them manually as the rows are mounting to 100,000 lines. the filenames are as below with convention as... (2 Replies)
Discussion started by: ida1215
2 Replies

8. UNIX for Dummies Questions & Answers

Write the total number of rows in multiple files into another file

Hello Friends, I know you all are busy and inteligent too... I am stuck with one small issue if you can help me then it will be really great. My problem is I am having some files i.e. Input.txt1 Input.txt2 Input.txt3 Now my task is I need to check the total number of rows in... (4 Replies)
Discussion started by: malaya kumar
4 Replies

9. Shell Programming and Scripting

Create Multiple UNIX Files for Multiple SQL Rows output

Dear All, I am trying to write a Unix Script which fires a sql query. The output of the sql query gives multiple rows. Each row should be saved in a separate Unix File. The number of rows of sql output can be variable. I am able save all the rows in one file but in separate files. Any... (14 Replies)
Discussion started by: Rahul_Bhasin
14 Replies

10. Shell Programming and Scripting

Extracting data from specific rows and columns from multiple csv files

I have a series of csv files in the following format eg file1 Experiment Name,XYZ_07/28/15, Specimen Name,Specimen_001, Tube Name, Control, Record Date,7/28/2015 14:50, $OP,XYZYZ, GUID,abc, Population,#Events,%Parent All Events,10500, P1,10071,95.9 Early Apoptosis,1113,11.1 Late... (6 Replies)
Discussion started by: pawannoel
6 Replies

LEARN ABOUT LINUX

dh_compress

DH_COMPRESS(1)							     Debhelper							    DH_COMPRESS(1)

NAME

       dh_compress - compress files and fix symlinks in package build directories

SYNOPSIS

       dh_compress [debhelperoptions] [-Xitem] [-A] [file...]

DESCRIPTION

       dh_compress is a debhelper program that is responsible for compressing the files in package build directories, and makes sure that any
       symlinks that pointed to the files before they were compressed are updated to point to the new files.

       By default, dh_compress compresses files that Debian policy mandates should be compressed, namely all files in usr/share/info,
       usr/share/man, files in usr/share/doc that are larger than 4k in size, (except the copyright file, .html and other web files, image files,
       and files that appear to be already compressed based on their extensions), and all changelog files. Plus PCF fonts underneath
       usr/share/fonts/X11/

FILES

       debian/package.compress
	   These files are deprecated.

	   If this file exists, the default files are not compressed. Instead, the file is ran as a shell script, and all filenames that the shell
	   script outputs will be compressed. The shell script will be run from inside the package build directory. Note though that using -X is a
	   much better idea in general; you should only use a debian/package.compress file if you really need to.

OPTIONS

       -Xitem, --exclude=item
	   Exclude files that contain item anywhere in their filename from being compressed. For example, -X.tiff will exclude TIFF files from
	   compression.  You may use this option multiple times to build up a list of things to exclude.

       -A, --all
	   Compress all files specified by command line parameters in ALL packages acted on.

       file ...
	   Add these files to the list of files to compress.

CONFORMS TO

       Debian policy, version 3.0

SEE ALSO

       debhelper(7)

       This program is a part of debhelper.

AUTHOR

       Joey Hess <joeyh@debian.org>

8.9.0ubuntu2.1							    2012-06-12							    DH_COMPRESS(1)
All times are GMT -4. The time now is 08:47 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy