Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Averaging help in awk
Top Forums Shell Programming and Scripting Averaging help in awk Post 302547372 by birei on Monday 15th of August 2011 11:19:26 AM
Old 08-15-2011
Hi,

Try next 'perl' program:
Code:
$ cat script.pl
use warnings;
use strict;

## 30min * 60min/sec / 5sec/line
my $lines_to_average = (30 * 60 / 5)+1;
# $lines_to_average = 3; #Test for ten seconds.

my $proc_lines = 0;
my ($init_date, $end_date, $total_frac_days);

while ( <> ) {
        ## Skip header.
        next if $. == 1;

        ## Remove last '\n'.
        chomp;

        ## Sum 'frac_days' while not reached to average time.
        if ( $proc_lines < $lines_to_average ) {
                my ($date,$frac_day) = (split)[1,2];
                ++$proc_lines;
                $total_frac_days += $frac_day;
                $init_date = $date if $proc_lines == 1;
                if ( $proc_lines == $lines_to_average || eof() ) {
                        $end_date = $date;
                } else {
                        next;
                }
        }

        printf "%s-%s %.8f\n", $init_date, $end_date, $total_frac_days / $lines_to_average;
        $proc_lines = $total_frac_days = 0;

}

I've made a test (changing the $lines_to_average variable) with an average of ten seconds, which results in an output of two lines: initDate-endDate<space>average
Here the script running:
Code:
$ cat infile
DATE                      TIME                      FRAC_DAYS_SINCE_JAN1      
2011-06-25                08:03:20.000              175.33564815 
2011-06-25                08:03:25.000              175.33570602
2011-06-25                08:03:30.000              175.33576389 
2011-06-25                08:03:35.000              175.33582176
$ perl script.pl infile
08:03:20.000-08:03:30.000 175.33570602
08:03:35.000-08:03:35.000 58.44527392

UPDATE:
My script with your last example of input file and an average of ten seconds.
Code:
$ cat infile
DATE                      TIME                      FRAC_DAYS_SINCE_JAN1      FRAC_HRS_SINCE_JAN1       EPOCH_TIME                ALARM_STATUS              species                   solenoid_valves           MPVPosition               OutletValve               CavityPressure            CavityTemp                WarmBoxTemp               EtalonTemp                DasTemp                   CO2_sync                  CO2_dry_sync              CH4_sync                  CH4_dry_sync              H2O_sync                  
2011-06-25                08:03:20.000              175.33564815              4208.055556               1308989000.000            0                         1.0000000000E+000         0.0000000000E+000         0.0000000000E+000         3.1033031768E+004         1.3998939700E+002         4.4999687195E+001         4.4999909923E+001         4.4982554694E+001         3.1751035912E+001         3.8940279934E+002         3.9515856123E+002         3.8618224512E+000         3.9114643916E+000         9.4040946797E-001         
2011-06-25                08:03:25.000              175.33570602              4208.056944               1308989005.000            0                         2.0000000000E+000         0.0000000000E+000         0.0000000000E+000         3.1036267092E+004         1.3999910014E+002         4.4999724978E+001         4.5000055343E+001         4.4982432169E+001         3.1750000000E+001         3.8935193355E+002         3.9511128829E+002         3.8489583767E+000         3.8990622477E+000         9.4172965077E-001         
2011-06-25                08:03:30.000              175.33576389              4208.058333               1308989010.000            0                         3.0000000000E+000         0.0000000000E+000         0.0000000000E+000         3.1035811523E+004         1.4000220783E+002         4.4999819829E+001         4.5000161391E+001         4.4982414246E+001         3.1795673077E+001         3.8938861262E+002         3.9514732622E+002         3.8595829527E+000         3.9062814635E+000         9.3441447742E-001         
2011-06-25                08:03:35.000              175.33582176              4208.059722               1308989015.000            0                         2.0000000000E+000         0.0000000000E+000         0.0000000000E+000         3.1036211602E+004         1.4000074680E+002         4.4999892989E+001         4.5000205994E+001         4.4982414246E+001         3.1750000000E+001         3.8945211683E+002         3.9517328822E+002         3.8841252351E+000         3.9451982461E+000         9.3417578630E-001         
2011-06-25                08:03:40.000              175.33587963              4208.061111               1308989020.000            0                         1.0000000000E+000         0.0000000000E+000         0.0000000000E+000         3.1036852113E+004         1.3999663742E+002         4.4999757503E+001         4.5000099772E+001         4.4982477978E+001         3.1793508287E+001         3.8936261353E+002         3.9511123990E+002         3.8395527261E+000         3.8931575825E+000         9.4752583244E-001         
2011-06-25                08:03:45.000              175.33593750              4208.062500               1308989025.000            0                         2.0000000000E+000         0.0000000000E+000         0.0000000000E+000         3.1032849098E+004         1.3998770158E+002         4.4999748230E+001         4.5000053406E+001         4.4982505798E+001         3.1750000000E+001         3.8931505434E+002         3.9512509568E+002         3.8351808367E+000         3.8599191928E+000         9.5367870751E-001         
2011-06-25                08:03:50.000              175.33599537              4208.063889               1308989030.000            0                         1.0000000000E+000         0.0000000000E+000         0.0000000000E+000         3.1039831891E+004         1.4001021926E+002         4.5000107359E+001         4.4999785829E+001         4.4982363833E+001         3.1750000000E+001         3.8929289686E+002         3.9508579576E+002         3.8474802185E+000         3.8609535036E+000         9.4079656257E-001         
2011-06-25                08:03:55.000              175.33605324              4208.065278               1308989035.000            0                         1.0000000000E+000         0.0000000000E+000         0.0000000000E+000         3.1034918787E+004         1.3999495360E+002         4.5000032095E+001         4.4999813527E+001         4.4982444763E+001         3.1750000000E+001         3.8929209181E+002         3.9514661440E+002         3.8572828460E+000         3.9132254247E+000         9.5624024001E-001         
2011-06-25                08:04:00.000              175.33611111              4208.066667               1308989040.000            0                         1.0000000000E+000         0.0000000000E+000         0.0000000000E+000         3.1035897944E+004         1.3999774824E+002         4.4999900818E+001         4.4999778748E+001         4.4982444763E+001         3.1750000000E+001         3.8899351438E+002         3.9493622327E+002         3.8126918495E+000         3.7581951076E+000         9.7192541150E-001         
2011-06-25                08:04:05.000              175.33616898              4208.068056               1308989045.000            0                         3.0000000000E+000         0.0000000000E+000         0.0000000000E+000         3.1039568209E+004         1.4001117648E+002         4.5000114441E+001         4.4999682500E+001         4.4982360253E+001         3.1750000000E+001         3.8927314658E+002         3.9508938383E+002         3.8333155741E+000         3.8801732738E+000         9.5087600370E-001
$ perl script.pl infile
08:03:20.000-08:03:30.000 175.33570602
08:03:35.000-08:03:45.000 175.33587963
08:03:50.000-08:04:00.000 175.33605324
08:04:05.000-08:04:05.000 58.44538966

Adapt it to your needs.

But agree with yazu, give better examples and how you want your output, what have you tried to solve it, etc.

Regards,
Birei
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

AWK - averaging $3 by info in $1

Hello, I have three columns of data of the format below: <name> <volume> <size> a 2 1.2 a 2 1.1 b 3 1.7 c 0.7 1.9 c 0.7 1.9 c 0.7 1.8 What I... (3 Replies)
Discussion started by: itisthus
3 Replies

2. Shell Programming and Scripting

averaging column values with awk

Hello. Im just starting to learn awk so hang in there with me...I have a large text file formatted as such everything is in a single column ID001 value 1 value 2 value....n ID002 value 1 value 2 value... n I want to be able to calculate the average for values for each ID from the... (18 Replies)
Discussion started by: johnmillsbro
18 Replies

3. UNIX for Dummies Questions & Answers

Averaging

Hello all, I'm trying to perform an averaging procedure which selects a selection of rows, average the corresponding value, selects the next set of rows and average the corresponding values etc. The data below illustrates what I want to do. Given two columns (day and value), I want to... (2 Replies)
Discussion started by: Muhammad Rahiz
2 Replies

4. UNIX for Dummies Questions & Answers

Averaging the rows using 'awk'

Dear all, I have the data in the following format. I want to do average of each NR= 5 (rows) for all the 3 ($1,$2, $3) columns and want to print average result in another file in the same format. I dont know how to write code for this in 'awk', can some one help me to write a code for this in... (1 Reply)
Discussion started by: arvindr
1 Replies

5. Shell Programming and Scripting

Averaging in increments using awk & head/tail

Hi, I only have a very limited understanding and experience with writing code and I was hoping I could get some help. I have a dataset of two columns (txt format, numbers in each row separated by a tab) Eg. 1 5 2 5 3 6 4 7 5 6 6 6 7 ... (5 Replies)
Discussion started by: Emred_Skye
5 Replies

6. Shell Programming and Scripting

Averaging data every 30 mins using AWK

A happy Monday to you all, I have a .csv file which contains data taken every 5 seconds. I want to average these 5 second data points into 30 minute averages! date co2 25/06/2011 08:04 8.31 25/06/2011 08:04 8.32 25/06/2011 08:04 8.33... (18 Replies)
Discussion started by: gd9629
18 Replies

7. Shell Programming and Scripting

Hourly averaging using Awk

Hey all, I have a set of 5-second data as shown below. I need to find an hourly average of this data. date co2 25/06/2011 08:04:00 8.30 25/06/2011 08:04:05 8.31 25/06/2011 08:04:10 8.32 25/06/2011 08:04:15 8.33 25/06/2011 08:04:20 ... (5 Replies)
Discussion started by: gd9629
5 Replies

8. Shell Programming and Scripting

Averaging 3 files

Hi, I am trying to average the values from 3 files with the same format. They are very large files so I will describe the file and show some it of. Basically the file has 83 columns (with nearly 7000 rows). The first three columns are the same for each file while the remaining 80 are values... (3 Replies)
Discussion started by: kylle345
3 Replies

9. Shell Programming and Scripting

Loop for row-wise averaging of multiple files using awk

Hello all, I need to compute a row-wise average of files with a single column based on the pattern of the filenames. I really appreciate any help on this. it would just be very difficult to do them manually as the rows are mounting to 100,000 lines. the filenames are as below with convention as... (2 Replies)
Discussion started by: ida1215
2 Replies

10. Shell Programming and Scripting

How to perform averaging of values for particular timestamp using awk or anythoing else??

I have a file of the form. 16:00:26,83.33 16:05:26,83.33 16:10:26,83.33 16:15:26,83.33 16:20:26,90.26 16:25:26,83.33 16:30:26,83.33 17:00:26,83.33 17:05:26,83.33 17:10:26,83.33 17:15:26,83.33 17:20:26,90.26 17:25:26,83.33 17:30:26,83.33 For the timestamp 16:00:00 to 16:55:00, I need to... (5 Replies)
Discussion started by: Saidul
5 Replies
All times are GMT -4. The time now is 10:26 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy