Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Averaging data every 30 mins using AWK

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Averaging data every 30 mins using AWK
# 15  
Old 07-26-2011
For some odd reason I'm still getting

Code:
CO2 CH4
0      0
0      0
0      0
0      0
0      0
0      0
0      0
0      0
0      0

Did you say there was a way to edit the file itself, as opposed to printing out the result on the screen?

Cheers
# 16  
Old 07-28-2011
Can you post output of:
Code:
cat -Te file.csv | head

# 17  
Old 07-28-2011
Code:
date,alarm,species,solenoid,mpv,outlet,cavp,cavt,warmbox,etalon,dastemp,co2sync,co2,ch4sync,ch4,h2osync$
25/06/2011 08:03:20 ,0,1.0000000000E+000,0.0000000000E+000,0.0000000000E+000,3.1033031768E+004,1.3998939700E+002,4.4999687195E+001,4.4999909923E+001,4.4982554694E+001,3.1751035912E+001,3.8940279934E+002,3.9515856123E+002,3.8618224512E+000,3.9114643916E+000,9.4040946797E-001$
25/06/2011 08:03:25 ,0,2.0000000000E+000,0.0000000000E+000,0.0000000000E+000,3.1036267092E+004,1.3999910014E+002,4.4999724978E+001,4.5000055343E+001,4.4982432169E+001,3.1750000000E+001,3.8935193355E+002,3.9511128829E+002,3.8489583767E+000,3.8990622477E+000,9.4172965077E-001$
25/06/2011 08:03:30 ,0,3.0000000000E+000,0.0000000000E+000,0.0000000000E+000,3.1035811523E+004,1.4000220783E+002,4.4999819829E+001,4.5000161391E+001,4.4982414246E+001,3.1795673077E+001,3.8938861262E+002,3.9514732622E+002,3.8595829527E+000,3.9062814635E+000,9.3441447742E-001$
25/06/2011 08:03:35 ,0,2.0000000000E+000,0.0000000000E+000,0.0000000000E+000,3.1036211602E+004,1.4000074680E+002,4.4999892989E+001,4.5000205994E+001,4.4982414246E+001,3.1750000000E+001,3.8945211683E+002,3.9517328822E+002,3.8841252351E+000,3.9451982461E+000,9.3417578630E-001$
25/06/2011 08:03:40 ,0,1.0000000000E+000,0.0000000000E+000,0.0000000000E+000,3.1036852113E+004,1.3999663742E+002,4.4999757503E+001,4.5000099772E+001,4.4982477978E+001,3.1793508287E+001,3.8936261353E+002,3.9511123990E+002,3.8395527261E+000,3.8931575825E+000,9.4752583244E-001$
25/06/2011 08:03:45 ,0,2.0000000000E+000,0.0000000000E+000,0.0000000000E+000,3.1032849098E+004,1.3998770158E+002,4.4999748230E+001,4.5000053406E+001,4.4982505798E+001,3.1750000000E+001,3.8931505434E+002,3.9512509568E+002,3.8351808367E+000,3.8599191928E+000,9.5367870751E-001$
25/06/2011 08:03:50 ,0,1.0000000000E+000,0.0000000000E+000,0.0000000000E+000,3.1039831891E+004,1.4001021926E+002,4.5000107359E+001,4.4999785829E+001,4.4982363833E+001,3.1750000000E+001,3.8929289686E+002,3.9508579576E+002,3.8474802185E+000,3.8609535036E+000,9.4079656257E-001$
25/06/2011 08:03:55 ,0,1.0000000000E+000,0.0000000000E+000,0.0000000000E+000,3.1034918787E+004,1.3999495360E+002,4.5000032095E+001,4.4999813527E+001,4.4982444763E+001,3.1750000000E+001,3.8929209181E+002,3.9514661440E+002,3.8572828460E+000,3.9132254247E+000,9.5624024001E-001$
25/06/2011 08:04:00 ,0,1.0000000000E+000,0.0000000000E+000,0.0000000000E+000,3.1035897944E+004,1.3999774824E+002,4.4999900818E+001,4.4999778748E+001,4.4982444763E+001,3.1750000000E+001,3.8899351438E+002,3.9493622327E+002,3.8126918495E+000,3.7581951076E+000,9.7192541150E-001$

# 18  
Old 07-28-2011
Try this:
Code:
awk 'NR>1' file.csv | awk -F"," '{a[int((NR-1)/720)]+=$13;b[int((NR-1)/720)]+=$15};END{print "CO2 CH4";for (i=0;i<NR/720;i++){print a[i]/720,b[i]/720}}'

Last edited by bartus11; 07-28-2011 at 01:18 PM..
# 19  
Old 07-29-2011
hey,

The averages were printed out to the shell so that's great, is there anyway of replacing the existing data in the csv file so the output looks like

Code:
25/06/2011 08:03:20   386.034    3.73601
25/06/2011 09:03:20   396.132    3.12456
25/06/2011 10:03:20   439.749    4.89633
25/06/2011 11:03:20   401.759    3.91748

cheers!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Script to search log file for last 15 mins data

Hi All, I have an issue which I'm trying to understand a way of doing, I have several nodes which contain syslog events which I want to force trigger an email initially (eventually leading to another method of alerting but to start with an email). Basically the syslog file will have hours worth... (6 Replies)
Discussion started by: mutley2202
6 Replies

2. Shell Programming and Scripting

Script (ksh) to get data in every 30 mins interval for the given date

Hello, Since I m new to shell, I had a hard time to sought out this problem. I have a log file of a utility which tells that batch files are successful with timestamp. Given below is a part of the log file. 2013/03/07 00:13:50 Apache/1.3.29 (Unix) configured -- resuming normal operations... (12 Replies)
Discussion started by: rpm120
12 Replies

3. Shell Programming and Scripting

How to perform averaging of values for particular timestamp using awk or anythoing else??

I have a file of the form. 16:00:26,83.33 16:05:26,83.33 16:10:26,83.33 16:15:26,83.33 16:20:26,90.26 16:25:26,83.33 16:30:26,83.33 17:00:26,83.33 17:05:26,83.33 17:10:26,83.33 17:15:26,83.33 17:20:26,90.26 17:25:26,83.33 17:30:26,83.33 For the timestamp 16:00:00 to 16:55:00, I need to... (5 Replies)
Discussion started by: Saidul
5 Replies

4. Shell Programming and Scripting

Averaging help in awk

Hi all, I have a data file like below, where Time is in the second column DATE TIME FRAC_DAYS_SINCE_JAN1 2011-06-25 08:03:20.000 175.33564815 2011-06-25 08:03:25.000 175.33570602... (10 Replies)
Discussion started by: gd9629
10 Replies

5. Shell Programming and Scripting

Hourly averaging using Awk

Hey all, I have a set of 5-second data as shown below. I need to find an hourly average of this data. date co2 25/06/2011 08:04:00 8.30 25/06/2011 08:04:05 8.31 25/06/2011 08:04:10 8.32 25/06/2011 08:04:15 8.33 25/06/2011 08:04:20 ... (5 Replies)
Discussion started by: gd9629
5 Replies

6. Shell Programming and Scripting

Averaging Data From Multiple Columns, Using Header if Possible

Hi, I have a file with multiple tab delimited columns and I would like to have the average of each column: Iteration Tree No Lh HMean 1000 1 -78.834717 -78.834717 1100 1 -77.991031 -78.624046 1200 1 -79.416055 -78.761861 1300 1 -79.280494 -78.968099 1400 1 -82.846275 -80.808696 ... (4 Replies)
Discussion started by: mikey11415
4 Replies

7. Shell Programming and Scripting

Averaging in increments using awk & head/tail

Hi, I only have a very limited understanding and experience with writing code and I was hoping I could get some help. I have a dataset of two columns (txt format, numbers in each row separated by a tab) Eg. 1 5 2 5 3 6 4 7 5 6 6 6 7 ... (5 Replies)
Discussion started by: Emred_Skye
5 Replies

8. UNIX for Dummies Questions & Answers

Averaging the rows using 'awk'

Dear all, I have the data in the following format. I want to do average of each NR= 5 (rows) for all the 3 ($1,$2, $3) columns and want to print average result in another file in the same format. I dont know how to write code for this in 'awk', can some one help me to write a code for this in... (1 Reply)
Discussion started by: arvindr
1 Replies

9. Shell Programming and Scripting

averaging column values with awk

Hello. Im just starting to learn awk so hang in there with me...I have a large text file formatted as such everything is in a single column ID001 value 1 value 2 value....n ID002 value 1 value 2 value... n I want to be able to calculate the average for values for each ID from the... (18 Replies)
Discussion started by: johnmillsbro
18 Replies

10. Shell Programming and Scripting

AWK - averaging $3 by info in $1

Hello, I have three columns of data of the format below: <name> <volume> <size> a 2 1.2 a 2 1.1 b 3 1.7 c 0.7 1.9 c 0.7 1.9 c 0.7 1.8 What I... (3 Replies)
Discussion started by: itisthus
3 Replies
Login or Register to Ask a Question