Sampling and Binning- Engineering problem Post: 302232196

Sponsored Content

Top Forums Shell Programming and Scripting Sampling and Binning- Engineering problem Post 302232196 by era on Thursday 4th of September 2008 04:18:49 AM

09-04-2008

Registered User

Code:

#!/usr/bin/perl

use strict;
use warnings;

my ($t1, $sum, $count, $block);

while (<>) {
  chomp;
  my ($timestamp, $event, $value) = split (/ /);
  my ($h, $m, $s) = split (/:/, $timestamp);
  my $t = $s + 60*$m + 3600*$h;

  if (! defined $t1 || $t >= $t1) {
    if (defined $t1) {
      print ++$block, " ", $count, " ", $sum/$count, "\n";
    }
   $t1 = $t + 5;
    $sum = $count = 0;
  }
  if ($event eq "A") {
    ++$count;
    $sum += $value;
  }
}

if ($count) {
  print ++$block, " ", $count, " ", $sum/$count, "\n";
}

Assumes sorted input. I'm not entirely sure I correctly figured out what to count and average but I imagine you can straighten it out if it's not completely correct.

I assume you really meant five-second blocks (for which the first ends at 03:38:27.999999) and so the output is not precisely as you specified. Maybe change the interval to six if you really want 03:38:22 through 03:38:28.999999 in the first block.

Last edited by era; 09-04-2008 at 05:24 AM.. Reason: Note five vs six second block size

era

View Public Profile for era

Find all posts by era

6 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

trimming and binning rows

I could not find this on the search.. I want to know how to trim a row so lets say I have a file that looks like this: bob 88888888888888 and I want to trim column 2 (lets say 4 off the front and end) bob 888888 Also, how would I bin column 2 Lets so I want to add and average...

2. Shell Programming and Scripting

data sampling

I have a requirement where I have multiple flat file sources. I need to create sample data from each source. Example: Source 1 has 10 flat files-- member, transaction,item,email,....etc Now if I get any 10 records (say first 10 records) from the member flat file, I need to find those matching...

3. Shell Programming and Scripting

Binning rows while skipping the first column

Hi I have a file that I want to bin. I am using this code: awk -F'\t' -v r=40 '{for(i=r;i<=NF;i+=r){for(j=0;j<r;j++){sum+=$(i-j)}printf "%s ", sum/r;sum=0}; printf "\n"}' file1 > file2 So basically what this code does is that it will averaging every 40 columns (creating bins of 40). But...

4. Shell Programming and Scripting

Sampling pcap file

Hi, I have a standard pcap file created using tcpdump. The file looks like 06:49:36.487629 IP 202.1.175.252 > 71.126.222.64: ICMP echo request, id 52765, seq 1280, length 40 06:49:36.489552 IP 192.120.148.227 > 71.126.222.64: ICMP echo request, id 512, seq 1280, length 40 06:49:36.491812 IP...

5. Shell Programming and Scripting

problem in binning the data

hi i have some data like this input: 1 apples oranges 234 2 oranges apples 2345 3 grapes bananas 1000000 4 melons banans 10000000 5 bananas apples 5000000 6 mangoes banans 2000000 7 apples bananas 1999999 i want to put all those which are coming between 1 and 999999 in to one bin...

6. Shell Programming and Scripting

Gnuplot 3d binning

Hello I have a text file with tens of thousands of rows The format is x y where both x and y can be anything between -100 and +100. What I would like to do is have a 3d gnuplot where there are 10,000 squared or bins and each bin will count how many rows have a value that would be...

LEARN ABOUT OPENSOLARIS

sum

sum(1)								   User Commands							    sum(1)

NAME

       sum - print checksum and block count for a file

SYNOPSIS

       sum [-r] [file]...

DESCRIPTION

       The   sum utility calculates and prints a 16-bit checksum for the named file and the number of 512-byte blocks in the file. It is typically
       used to look for bad spots, or to validate a file communicated over some transmission line.

OPTIONS

       The following options are supported:

       -r    Use an alternate (machine-dependent) algorithm in computing the checksum.

OPERANDS

       The following operands are supported:

       file    A path name of a file.  If no files are named, the standard input is used.

USAGE

       See largefile(5) for the description of the behavior of sum when encountering files greater than or equal to 2 Gbyte ( 2^31 bytes).

ENVIRONMENT VARIABLES

       See environ(5) for descriptions of the following environment variables that affect  the	execution  of	sum:  LC_CTYPE,  LC_MESSAGES,  and
       NLSPATH.

EXIT STATUS

       The following exit values are returned.

       0     Successful completion.

       >0    An error occurred.

ATTRIBUTES

       See attributes(5) for descriptions of the following attributes:

       +-----------------------------------------------------------+
       |      ATTRIBUTE TYPE		    ATTRIBUTE VALUE	   |
       |Availability		      SUNWesu			   |
       |CSI			      Enabled			   |
       +-----------------------------------------------------------+

SEE ALSO

       cksum(1), sum(1B), wc(1), attributes(5), environ(5), largefile(5)

DIAGNOSTICS

       Read error is indistinguishable from end of file on most devices. Check the block count.

NOTES

       Portable applications should use cksum(1).

       sum and usr/ucb/sum (see sum(1B)) return different checksums.

SunOS 5.11							    7 Nov 1995								    sum(1)