Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: calculate the average of time series data using AWK
Top Forums Shell Programming and Scripting calculate the average of time series data using AWK Post 302271378 by summer_cherry on Thursday 25th of December 2008 05:08:11 AM
Old 12-25-2008
perl:
Code:
#! /usr/bin/perl -w
open FH,"<a.txt";
while(<FH>){
	@arr=split(" ",$_);
	if($arr[2]=~m/([0-9][0-9]*)\..*/){
		$hash{$1}->{NUM}++;
		$hash{$1}->{SUM}+=$arr[1];
	}
}
close FH;
for $key (sort keys %hash){
	printf("%s.000000 %.6f %s\n",$key,$hash{$key}->{SUM}/$hash{$key}->{NUM},$hash{$key}->{NUM});
}

awk:
Code:
awk '{
	key=substr($3,1,index($3,".")-1)
	arr[key]++
	brr[key]+=$2
}
END{
	for(i in arr)
		printf("%s.000000 %.6f %s\n",i,brr[i]/arr[i],arr[i])
}
' a.txt

Last edited by summer_cherry; 12-25-2008 at 06:17 AM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Use awk to calculate average of column 3

Suppose I have 500 files in a directory and I need to Use awk to calculate average of column 3 for each of the file, how would I do that? (6 Replies)
Discussion started by: grossgermany
6 Replies

2. HP-UX

calculate average of multiple line data

I have a question as below and i need to write a shell or perl script for this query:My Input file looks like below RNo Marks 12 50 15 70 18 80 12 40 13 55 18 88 13 75... (4 Replies)
Discussion started by: smacherla
4 Replies

3. UNIX for Dummies Questions & Answers

Iterate a min/max awk script over time-series temperature data

I'm trying to iterate a UNIX awk script that returns min/max temperature data for each day from a monthly weather data file (01_weath.dat). The temperature data is held in $5. The temps are reported each minute so each day contains 1440 temperature enteries. The below code has gotten me as far as... (5 Replies)
Discussion started by: jgourley
5 Replies

4. Shell Programming and Scripting

Calculate average time using a script

Hello, I'm hoping to get some help on calculating an average time from a list of times (hour:minute:second). Here's what my list looks like right now, it will grow (I can get the full date or change the formatting of this as well): 07:55:31 09:42:00 08:09:02 09:15:23 09:27:45 09:49:26... (4 Replies)
Discussion started by: jaredhanks
4 Replies

5. Shell Programming and Scripting

AWK novice - calculate the average

Hi, I have the following data in a file for example: P1 XXXXXXX.1 YYYYYYY.1 ZZZ.1 P1 XXXXXXX.2 YYYYYYY.2 ZZZ.2 P1 XXXXXXX.3 YYYYYYY.3 ZZZ.3 P1 XXXXXXX.4 YYYYYYY.4 ZZZ.4 P1 XXXXXXX.5 YYYYYYY.5 ZZZ.5 P1 XXXXXXX.6 YYYYYYY.6 ZZZ.6 P1 XXXXXXX.7 YYYYYYY.7 ZZZ.7 P1 XXXXXXX.8 YYYYYYY.8 ZZZ.8 P2... (6 Replies)
Discussion started by: alex2005
6 Replies

6. Shell Programming and Scripting

Calculate Average AWK

I want to calculate the average line by line of some files with several lines on them, the files are identical, just want to average the 3rd columns of those files.:wall: Example file: File 1 001 0.046 0.667267 001 0.047 0.672028 001 0.048 0.656025 001 0.049 ... (2 Replies)
Discussion started by: AriasFco
2 Replies

7. Shell Programming and Scripting

Calculate average for repeated ID within a data

I have an awk script that gives the following output: Average end-to-end transmission delay 2.7 to 5.7 is 0.635392 seconds Average end-to-end transmission delay 2.1 to 5.1 is 0.66272 seconds Average end-to-end transmission delay 2.1 to 5.1 is 0.691712 seconds Average end-to-end transmission... (4 Replies)
Discussion started by: ENG_MOHD
4 Replies

8. Programming

Resample time series data with replacement any way to do this in awk or just bash script

I have some time series data that I need to resample or downsample at some specific time intervals. The firs column is time in decimal hours. I am tryiong to resample this data every 3 minutse. So I need a data value ever 0.05. Here is the example data and as you can see, there time slot for 0.1500... (3 Replies)
Discussion started by: malandisa
3 Replies

9. Shell Programming and Scripting

Average of multiple time-stamped data every half hour

Hi All, Thank you for reading through my post and helping me figure out how I would be able to perform this task. For example: I have a list of continuous output collected into a file in the format as seen below: Date...........Time........C....A......... B ==========================... (5 Replies)
Discussion started by: terrychen
5 Replies

10. Shell Programming and Scripting

Calculate Average time of one column

Hello dears, I have a log file with records like below and want to get a average of one column based on the search of one specific keyword. 2015-02-07 08:15:28 10.102.51.100 10.112.55.101 "kevin.c" POST ... (2 Replies)
Discussion started by: Newman
2 Replies

LEARN ABOUT DEBIAN

stda

STDA(1) 							   User Commands							   STDA(1)

NAME

       stda -  Simple Tools for Data Analysis (STDA)

DESCRIPTION

       STDA includes some primary tools for data analysis. You can evaluate sums, averages, integrals, derivatives, histograms or probability dis-
       tribution functions of 1-d data, and eventually plot the results. The programs are stand-alone tools (supporting the  standard  UNIX  input
       and  output  pipelines)	intended for data processing from the command line. It should be noted that all but one of the scripts use awk and
       core system utilities. For plotting you have to install Gnuplot (see http://gnuplot.info) since 'muplot' is a wrapper around  it.  In  sum-
       mary,  the  package  provides  utilities  for straightforward analysis of data series where a complex analytical approach is not needed and
       where an ultimate numerical precision with floating-point numbers is not critical. Some general examples of application cases include eval-
       uating usage statistics from server logfiles, determining a response time distribution from a series of queries to a [remote] service, pro-
       ducing a plot from multiple data files, etc.

       This software should be considered as an open project to be extended with new command-line driven utilities helpful for	performing  common
       data analysis tasks. Any contributions and suggestions are welcome.

       Following programs are included in the distribution:

       * maphimbu   - histogram builder for 1-d numerical and text data

       * mintegrate - average/sum/integral/derivative of 1-d numerical data

       * mmval	    - find minimum and maximum value in a data set

       * muplot     - plot a multi-curve figure from multiple data by using Gnuplot

       * nnum	    - produce a series of equally separated integers or floats

       * prefield   - prepare input file for 'muplot' to plot 2-d fields by arrows

EXAMPLES

       -  Evaluate  the current apache2 logfile and make an unique list of the hostnames (respectively ip-addresses) sorted by the total number of
       their http requests:

	      maphimbu -rs2 /var/log/apache2/access.log

       - On a X terminal plot the probability function and the cumulative distribution function of a sin(x) data sample:

	      nnum -3.14159 3.14159 0.00001 %.6g |awk '{ print $1, sin($1) }'  | maphimbu -d0.01 -x2 -ns1 |mintegrate -d0.01 -x1 -y3  -S  |muplot
	      lp - 1:3,4

COPYRIGHT

       Copyright (C) 2009, 2011-2012 Dimitar Ivanov <dimitar.ivanov@mirendom.net>

       License: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
       This is free software: you are free to change and redistribute it.  There is NO WARRANTY, to the extent permitted by law.

stda 1.1.1							   February 2012							   STDA(1)
All times are GMT -4. The time now is 11:30 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy