Sampling and Binning- Engineering problem Post: 302232196

Sponsored Content

Top Forums Shell Programming and Scripting Sampling and Binning- Engineering problem Post 302232196 by era on Thursday 4th of September 2008 04:18:49 AM

09-04-2008

Registered User

Code:

#!/usr/bin/perl

use strict;
use warnings;

my ($t1, $sum, $count, $block);

while (<>) {
  chomp;
  my ($timestamp, $event, $value) = split (/ /);
  my ($h, $m, $s) = split (/:/, $timestamp);
  my $t = $s + 60*$m + 3600*$h;

  if (! defined $t1 || $t >= $t1) {
    if (defined $t1) {
      print ++$block, " ", $count, " ", $sum/$count, "\n";
    }
   $t1 = $t + 5;
    $sum = $count = 0;
  }
  if ($event eq "A") {
    ++$count;
    $sum += $value;
  }
}

if ($count) {
  print ++$block, " ", $count, " ", $sum/$count, "\n";
}

Assumes sorted input. I'm not entirely sure I correctly figured out what to count and average but I imagine you can straighten it out if it's not completely correct.

I assume you really meant five-second blocks (for which the first ends at 03:38:27.999999) and so the output is not precisely as you specified. Maybe change the interval to six if you really want 03:38:22 through 03:38:28.999999 in the first block.

Last edited by era; 09-04-2008 at 05:24 AM.. Reason: Note five vs six second block size

era

View Public Profile for era

Find all posts by era

6 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

trimming and binning rows

I could not find this on the search.. I want to know how to trim a row so lets say I have a file that looks like this: bob 88888888888888 and I want to trim column 2 (lets say 4 off the front and end) bob 888888 Also, how would I bin column 2 Lets so I want to add and average...

2. Shell Programming and Scripting

data sampling

I have a requirement where I have multiple flat file sources. I need to create sample data from each source. Example: Source 1 has 10 flat files-- member, transaction,item,email,....etc Now if I get any 10 records (say first 10 records) from the member flat file, I need to find those matching...

3. Shell Programming and Scripting

Binning rows while skipping the first column

Hi I have a file that I want to bin. I am using this code: awk -F'\t' -v r=40 '{for(i=r;i<=NF;i+=r){for(j=0;j<r;j++){sum+=$(i-j)}printf "%s ", sum/r;sum=0}; printf "\n"}' file1 > file2 So basically what this code does is that it will averaging every 40 columns (creating bins of 40). But...

4. Shell Programming and Scripting

Sampling pcap file

Hi, I have a standard pcap file created using tcpdump. The file looks like 06:49:36.487629 IP 202.1.175.252 > 71.126.222.64: ICMP echo request, id 52765, seq 1280, length 40 06:49:36.489552 IP 192.120.148.227 > 71.126.222.64: ICMP echo request, id 512, seq 1280, length 40 06:49:36.491812 IP...

5. Shell Programming and Scripting

problem in binning the data

hi i have some data like this input: 1 apples oranges 234 2 oranges apples 2345 3 grapes bananas 1000000 4 melons banans 10000000 5 bananas apples 5000000 6 mangoes banans 2000000 7 apples bananas 1999999 i want to put all those which are coming between 1 and 999999 in to one bin...

6. Shell Programming and Scripting

Gnuplot 3d binning

Hello I have a text file with tens of thousands of rows The format is x y where both x and y can be anything between -100 and +100. What I would like to do is have a 3d gnuplot where there are 10,000 squared or bins and each bin will count how many rows have a value that would be...

LEARN ABOUT SUSE

sum

SUM(1)								   User Commands							    SUM(1)

NAME

       sum - checksum and count the blocks in a file

SYNOPSIS

       sum [OPTION]... [FILE]...

DESCRIPTION

       Print checksum and block counts for each FILE.

       -r     use BSD sum algorithm, use 1K blocks

       -s, --sysv
	      use System V sum algorithm, use 512 bytes blocks

       --help display this help and exit

       --version
	      output version information and exit

       With no FILE, or when FILE is -, read standard input.

AUTHOR

       Written by Kayvan Aghaiepour and David MacKenzie.

REPORTING BUGS

       Report sum bugs to bug-coreutils@gnu.org
       GNU coreutils home page: <http://www.gnu.org/software/coreutils/>
       General help using GNU software: <http://www.gnu.org/gethelp/>

COPYRIGHT

       Copyright (C) 2009 Free Software Foundation, Inc.  License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
       This is free software: you are free to change and redistribute it.  There is NO WARRANTY, to the extent permitted by law.

SEE ALSO

       The  full  documentation  for sum is maintained as a Texinfo manual.  If the info and sum programs are properly installed at your site, the
       command

	      info coreutils 'sum invocation'

       should give you access to the complete manual.

GNU coreutils 7.1						     July 2010								    SUM(1)