Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Averaging data every 30 mins using AWK

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Averaging data every 30 mins using AWK
# 8  
Old 07-25-2011
After you export that file to CSV format delimited by ";", try this:
Code:
awk 'NR>1' june2tmp.csv | awk -F\; '{a[int((NR-1)/720)]+=$13;b[int((NR-1)/720)]+=$15};END{print "CO2 CH4";for (i=0;i<NR/720;i++){print a[i]/720,b[i]/720}}'

# 9  
Old 07-25-2011
I exported it to a csv and changed the delimiter, but still didn't change the file at all. Did it work for you?
# 10  
Old 07-25-2011
It is not meant to change the file. It should print calculated averages on the screen.
# 11  
Old 07-25-2011
Oh my bad. It doesn't do that either, just prints 'CO2 CH4'. Is there anyway to change the file? As that is the hope in averaging to 30 minutes, that the .csv file gets edited.
# 12  
Old 07-25-2011
Well, first we need to get the proper output. When we have that, we can redirect it to some other file and move it over the original. I'll get back to you soon, cause for now I have to leave Smilie
# 13  
Old 07-25-2011
ok no worries, thanks for the help!
# 14  
Old 07-25-2011
This is what I get:
Code:
[root@rhel ~]# head -5 june2temp.csv
"date";"alarm";"species";"solenoid";"mpv";"outlet";"cavp";"cavt";"warmbox";"etalon";"dastemp";"co2sync";"co2";"ch4sync";"ch4";"h2osync"
11-06-25 08:03;0;1,00E+00;0,00E+00;0,00E+00;3,10E+04;1,40E+02;4,50E+01;4,50E+01;4,50E+01;3,18E+01;3,89E+02;3,95E+02;3,86E+00;3,91E+00;9,40E-01
11-06-25 08:03;0;2,00E+00;0,00E+00;0,00E+00;3,10E+04;1,40E+02;4,50E+01;4,50E+01;4,50E+01;3,18E+01;3,89E+02;3,95E+02;3,85E+00;3,90E+00;9,42E-01
11-06-25 08:03;0;3,00E+00;0,00E+00;0,00E+00;3,10E+04;1,40E+02;4,50E+01;4,50E+01;4,50E+01;3,18E+01;3,89E+02;3,95E+02;3,86E+00;3,91E+00;9,34E-01
11-06-25 08:03;0;2,00E+00;0,00E+00;0,00E+00;3,10E+04;1,40E+02;4,50E+01;4,50E+01;4,50E+01;3,18E+01;3,89E+02;3,95E+02;3,88E+00;3,95E+00;9,34E-01
[root@rhel ~]# awk 'NR>1' june2temp.csv | awk -F\; '{a[int((NR-1)/720)]+=$13;b[int((NR-1)/720)]+=$15};END{print "CO2 CH4";for (i=0;i<NR/720;i++){print a[i]/720,b[i]/720}}'
CO2 CH4
3 3
3 3
3.00278 3
2.45833 2.45833

Hmmm, it seems that numbers in this format are not properly processed by AWK. This code gives better results (it replaces commas with dots in fields 13 and 15):
Code:
[root@rhel ~]# awk 'NR>1' june2temp.csv | awk -F\; '{sub(",",".",$13);sub(",",".",$15);a[int((NR-1)/720)]+=$13;b[int((NR-1)/720)]+=$15};END{print "CO2 CH4";for (i=0;i<NR/720;i++){print a[i]/720,b[i]/720}}'
CO2 CH4
395 3.89621
395.335 3.87851
398.969 3.77253
322.117 3.14147

Last edited by bartus11; 07-25-2011 at 12:50 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Script to search log file for last 15 mins data

Hi All, I have an issue which I'm trying to understand a way of doing, I have several nodes which contain syslog events which I want to force trigger an email initially (eventually leading to another method of alerting but to start with an email). Basically the syslog file will have hours worth... (6 Replies)
Discussion started by: mutley2202
6 Replies

2. Shell Programming and Scripting

Script (ksh) to get data in every 30 mins interval for the given date

Hello, Since I m new to shell, I had a hard time to sought out this problem. I have a log file of a utility which tells that batch files are successful with timestamp. Given below is a part of the log file. 2013/03/07 00:13:50 Apache/1.3.29 (Unix) configured -- resuming normal operations... (12 Replies)
Discussion started by: rpm120
12 Replies

3. Shell Programming and Scripting

How to perform averaging of values for particular timestamp using awk or anythoing else??

I have a file of the form. 16:00:26,83.33 16:05:26,83.33 16:10:26,83.33 16:15:26,83.33 16:20:26,90.26 16:25:26,83.33 16:30:26,83.33 17:00:26,83.33 17:05:26,83.33 17:10:26,83.33 17:15:26,83.33 17:20:26,90.26 17:25:26,83.33 17:30:26,83.33 For the timestamp 16:00:00 to 16:55:00, I need to... (5 Replies)
Discussion started by: Saidul
5 Replies

4. Shell Programming and Scripting

Averaging help in awk

Hi all, I have a data file like below, where Time is in the second column DATE TIME FRAC_DAYS_SINCE_JAN1 2011-06-25 08:03:20.000 175.33564815 2011-06-25 08:03:25.000 175.33570602... (10 Replies)
Discussion started by: gd9629
10 Replies

5. Shell Programming and Scripting

Hourly averaging using Awk

Hey all, I have a set of 5-second data as shown below. I need to find an hourly average of this data. date co2 25/06/2011 08:04:00 8.30 25/06/2011 08:04:05 8.31 25/06/2011 08:04:10 8.32 25/06/2011 08:04:15 8.33 25/06/2011 08:04:20 ... (5 Replies)
Discussion started by: gd9629
5 Replies

6. Shell Programming and Scripting

Averaging Data From Multiple Columns, Using Header if Possible

Hi, I have a file with multiple tab delimited columns and I would like to have the average of each column: Iteration Tree No Lh HMean 1000 1 -78.834717 -78.834717 1100 1 -77.991031 -78.624046 1200 1 -79.416055 -78.761861 1300 1 -79.280494 -78.968099 1400 1 -82.846275 -80.808696 ... (4 Replies)
Discussion started by: mikey11415
4 Replies

7. Shell Programming and Scripting

Averaging in increments using awk & head/tail

Hi, I only have a very limited understanding and experience with writing code and I was hoping I could get some help. I have a dataset of two columns (txt format, numbers in each row separated by a tab) Eg. 1 5 2 5 3 6 4 7 5 6 6 6 7 ... (5 Replies)
Discussion started by: Emred_Skye
5 Replies

8. UNIX for Dummies Questions & Answers

Averaging the rows using 'awk'

Dear all, I have the data in the following format. I want to do average of each NR= 5 (rows) for all the 3 ($1,$2, $3) columns and want to print average result in another file in the same format. I dont know how to write code for this in 'awk', can some one help me to write a code for this in... (1 Reply)
Discussion started by: arvindr
1 Replies

9. Shell Programming and Scripting

averaging column values with awk

Hello. Im just starting to learn awk so hang in there with me...I have a large text file formatted as such everything is in a single column ID001 value 1 value 2 value....n ID002 value 1 value 2 value... n I want to be able to calculate the average for values for each ID from the... (18 Replies)
Discussion started by: johnmillsbro
18 Replies

10. Shell Programming and Scripting

AWK - averaging $3 by info in $1

Hello, I have three columns of data of the format below: <name> <volume> <size> a 2 1.2 a 2 1.1 b 3 1.7 c 0.7 1.9 c 0.7 1.9 c 0.7 1.8 What I... (3 Replies)
Discussion started by: itisthus
3 Replies
Login or Register to Ask a Question