Sponsored Content
Top Forums Shell Programming and Scripting Sampling and Binning- Engineering problem Post 302232612 by Needhelp2 on Thursday 4th of September 2008 11:00:04 PM
Old 09-05-2008
hi era,

I am trying to modify the code so I can get the count and sum for every event type in one row. This with the five second block.


Wanted output:


Timestamps--CountA--CountB--SumA---SumB


Below is the modify code ( but it is not working)Smilie

..can you please help.. Smilie


thanks



#!/usr/bin/perl

use strict;
use warnings;

my ($t1, $sumA, $countA,$sumB, $countB, $block);

while (<>) {
chomp;
my ($timestamp, $event, $value) = split (/ /);
my ($h, $m, $s) = split (/:/, $timestamp);
my $t = $s + 60*$m + 3600*$h;

if (! defined $t1 || $t >= $t1) {
if (defined $t1) {
print ++$block, " ", $countA, " ", $sumA, " ", $countB, " ", $sumB, "\n";
}
$t1 = $t + 5;
$sumA = $countA = 0;
$sumB = $countB = 0;
}
if ($event eq "A") {
++$countA;
$sumA += $value;

if ($event eq "B") {
++$countB;
$sumB += $value;
}
}
}
if ($countA,$countB) {
print ++$block, " ", $countA, " ", $sumA, " ", $countB, " ", $sumB, "\n";
 

6 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

trimming and binning rows

I could not find this on the search.. I want to know how to trim a row so lets say I have a file that looks like this: bob 88888888888888 and I want to trim column 2 (lets say 4 off the front and end) bob 888888 Also, how would I bin column 2 Lets so I want to add and average... (1 Reply)
Discussion started by: phil_heath
1 Replies

2. Shell Programming and Scripting

data sampling

I have a requirement where I have multiple flat file sources. I need to create sample data from each source. Example: Source 1 has 10 flat files-- member, transaction,item,email,....etc Now if I get any 10 records (say first 10 records) from the member flat file, I need to find those matching... (2 Replies)
Discussion started by: arrivederci
2 Replies

3. Shell Programming and Scripting

Binning rows while skipping the first column

Hi I have a file that I want to bin. I am using this code: awk -F'\t' -v r=40 '{for(i=r;i<=NF;i+=r){for(j=0;j<r;j++){sum+=$(i-j)}printf "%s ", sum/r;sum=0}; printf "\n"}' file1 > file2 So basically what this code does is that it will averaging every 40 columns (creating bins of 40). But... (2 Replies)
Discussion started by: phil_heath
2 Replies

4. Shell Programming and Scripting

Sampling pcap file

Hi, I have a standard pcap file created using tcpdump. The file looks like 06:49:36.487629 IP 202.1.175.252 > 71.126.222.64: ICMP echo request, id 52765, seq 1280, length 40 06:49:36.489552 IP 192.120.148.227 > 71.126.222.64: ICMP echo request, id 512, seq 1280, length 40 06:49:36.491812 IP... (8 Replies)
Discussion started by: sajal.bhatia
8 Replies

5. Shell Programming and Scripting

problem in binning the data

hi i have some data like this input: 1 apples oranges 234 2 oranges apples 2345 3 grapes bananas 1000000 4 melons banans 10000000 5 bananas apples 5000000 6 mangoes banans 2000000 7 apples bananas 1999999 i want to put all those which are coming between 1 and 999999 in to one bin... (8 Replies)
Discussion started by: anurupa777
8 Replies

6. Shell Programming and Scripting

Gnuplot 3d binning

Hello I have a text file with tens of thousands of rows The format is x y where both x and y can be anything between -100 and +100. What I would like to do is have a 3d gnuplot where there are 10,000 squared or bins and each bin will count how many rows have a value that would be... (1 Reply)
Discussion started by: garethsays
1 Replies
RateLimiter(3pm)					User Contributed Perl Documentation					  RateLimiter(3pm)

NAME
Schedule::RateLimiter - prevent events from happening too quickly. SYNOPSIS
use Schedule::RateLimiter; # Don't let this event happen more than 5 times in a 60 second period. my $throttle = Schedule::RateLimiter->new ( iterations => 5, seconds => 60 ); # Cycle forever, but not too fast. while ( 1 ) { $throttle->event(); &do_something; } DESCRIPTION
This module provides a way to voluntarily restrict how many times a given action may take place within a specified time frame. Such a tool may be useful if you have written something which periodically polls some public resource and want to ensure that you do not overburden that resource with too many requests. Initially, one might think that solving this problem would be as simple as sleeping for the number of seconds divided by the number of iterations in between each event. However, that would only be correct if the event took no time at all. If you know exactly how much time each event is going to take then you could build an even more complicated one-liner such as this: sleep( (seconds / iterations) - single_event_time ) This module is intended to address the other cases when the exact run-time of each event is unknown and variable. This module will try very hard to allow an event to happen as many times as possible without exceeding the specified bounds. For example, suppose you want to write something that checks an 'incoming' directory once a minute for files and then does something with those files if it finds any. If it takes you two seconds to process those files, then you want to wait 58 seconds before polling the directory again. If it takes 30 seconds to process those files, then you only want to wait 30 seconds. And if it takes 3 minutes, then you want to poll the directory again immediately as soon as you are done. my $throttle = Schedule::RateLimiter->new ( seconds => 60 ); &poll_and_process while ( $throttle->event ); METHODS
" new() " Creates and returns a new Schedule::RateLimiter object. The constructor takes up to three parameters: o block (default: true) This parameter accepts a true or false value to set the default "block" behavior on future calls to event(). It makes it more convenient to turn blocking off for an entire object at a time. o iterations (default: 1) This specifies the number of times an event may take place within the given time period. This must be a positive, non-zero integer. o seconds (required) This specifies the minimum number of seconds that must transpire before we will allow (iterations + 1) events to happen. A value of 0 disables throttling. You may specify fractional time periods. example: my $throttle = Schedule::RateLimiter->new ( iterations => 2, seconds => 10 ); # Event 1 $throttle->event(); # Event 2 $throttle->event(); # Event 3 $throttle->event(); # 10 seconds will have transpired since event 1 at this point. # Event 4 $throttle->event(); # 10 seconds will have transpired since event 2 at this point. " event() " Called to signal the beginning of an event. This method will return true or false to indicate if it is ok to proceed with the event. This method uses Time::HiRes to do its calculations and sleeping, so the precision of this method will be the same as the precision of Time::HiRes on your platform. Takes one (optional) parameter: o block (default: true) If set to a false value, this method will do a non-blocking check to see if it is ok for the event to occur. If it is not ok, this method will return a false value and assume that the event did not take place. Otherwise, this method will return a true value and assume that the event did take place. example: # Stop when the code moves too fast. while ( 1 ) { if ($throttle->event( block => 0 )) { &do_something; } else { die 'I went too fast!'; } } BUGS
This module needs to keep a record of when every iteration took place, so if you are allowing a large number of iterations to happen in the given time period, this could potentially use a lot of memory. KNOWN ISSUES
If you have multiple iterations that typically happen very quickly, and you want to limit them in a long period of time, they will "clump" together. That is, they all happen at just about the same time, and then the system waits for a long period before doing the same "clump" again. That's just the nature of the best-fit algorithm. Anything that is done to try to separate single events with longer waits than necessary will potentially create a sub-optimal situation if an event in the future takes longer than expected. If you really want all of your events to start at even time periods apart from each other, then set the number of iterations to 1 and adjust the number of seconds accordingly. AUTHOR
Daniel J. Wright, <wright@pair.com> SEE ALSO
The POE module provides a more heavyweight solution to this problem as well. perl. perl v5.10.0 2003-12-04 RateLimiter(3pm)
All times are GMT -4. The time now is 06:11 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy