thanks for pointing out the existing C var, but I couldn't figure out enough about how the arrays worked to progress further anyway
you may remember: there are 2 types of loggers in use, the 1st type creates seperate directories for each day of data, the 2nd type logs to an MS Access database. You wrote scripts for 1st type (with gawk, in windoze).
I was able to figure out enough of your original scripts to modify to use with the 2nd type (a single large txt file exported from Access). I did a query to remove data in daytime before export to minimize bird count transits but have no solution for the non-access data. So the answer is yes, it would be great if you could script elimination of daytime bird data. I need to be able to change the time of "daytime" - winter would be ~ 8am to 5pm, summer ~ 6am to 9pm; that is actual time, not modified logger time of -16 hours (PC time is minus 16 hours due to daily folders being created at midnight, which does not coincide with a bat "day")
If you feel really ambitious, it would be "nice to have" additional daily stats for "large bats" and "small bats" based on the array criteria - this would enable me to claim my design of the logger was actually correct since beams are the correct width apart to give added value data
Another "nice to have" feature would be csv daily stats output that could be easily imported to spreadsheet and graphed.
There is a bit of a problem when 2 bats transit in the same second of the day (see 1st 8 rows of sample data) this is not common but does happen. I think I was able to get around it by changing the max var from 1 to 0 to try to compensate for two transits in one second - but I probably broke something else doing this!
I was getting errors so I changed CC var to X to eliminate any chance of conflict but was still getting errors. changed these lines, was it necessary?
=========================================
I found the errors- missing curly brackets.
here is working code:
Code:
# this has been changed to operate under the assumption that the 1st beam
# opens and closes before next beam opens due to spacing between beams being increased.
BEGIN { OT=0 # Time of previous measurement
MAX=0 # Max num of seconds between valid events
DAY=""; # Current day
CA=0 ; CB=0 ; CX=0 ; CD=0
# Running total of bats leaving and entering
TOTALBATS=0;
# The highest TOTALBATS has ever been
MAXBATS=0;
# Length of the patterns
L=4
# Patterns to check against
# Block 1 unBlock 0 block 0 Unblock 0
A[0]="1,1"; A[1]="1,0"; A[2]="0,1"; A[3]="0,0";
# Block 1 Block 0 unblock 1 Unblock 0
X[0]="1,1"; X[1]="0,1"; X[2]="1,0"; X[3]="0,0";
# Block 0 Unblock 0 Block 1 Unblock 1
B[0]="0,1"; B[1]="0,0"; B[2]="1,1"; B[3]="1,0";
# Block 0 block 1 unBlock 0 unblock 1
D[0]="0,1"; D[1]="1,1"; D[2]="0,0"; D[3]="1,0";
}
function print_daily(day,total,max,min,maxtime)
{
I=total; if(I<0) I=-I;
MX="no maximum"
if(maxtime > 0)
MX=sprintf("peak was at %s", strftime("%H:%M:%S",maxtime));
# printf("COUNT@%s COUNT %+d RET %d LEFT %d GUESS %d (%s)\n",
printf("Date @%s COUNT %+d IN %d OUT %d Estimate %d bats(%s)\n",
day, total, max, -min, I, MX) > "/dev/stderr";
# Reset daily counts
TOTALBATS=0; MAXBATS=0; MINBATS=0; MAXTIME=0;
MINTIME=0;
}
{ # Calculate timestamp from date string
T=mktime($1 " " $2 " " $3 " " $5 " " $6 " " $7);
T+=(60*60*16); # Add sixteen hours
$1=strftime("%Y", T); # Put these back in the strings
$2=strftime("%m", T);
$3=strftime("%d", T);
$5=strftime("%H", T);
$6=strftime("%M", T);
$7=strftime("%S", T);
# When the year, month, and/or day changes, time to print daily counts
if((DAY != $1 "-" $2 "-" $3) && (DAY != ""))
print_daily(DAY,TOTALBATS,MAXBATS,MINBATS,MAXTIME);
DAY=$1 "-" $2 "-" $3;
if($8 == "pv") # Ignore anything but PV lines.
{
# If too much time has passed since the last event, start over.
if((T-OT) > MAX) # Blank the array
for(N=0; N<(L-1); N++) C[N]="";
else # Shift elements toward the front
for(N=0; N<(L-1); N++) C[N]=C[N+1];
OT=T # Set prev time to this one.
C[L-1]=$9 "," $10; # Set the latest event in the array
# Search for events in the array.
FOUNDA=1; FOUNDB=1;
FOUNDX=1; FOUNDD=1;
for(N=0; N<L; N++)
{
if(A[N] != C[N]) FOUNDA=0;
if(B[N] != C[N]) FOUNDB=0;
if(X[N] != C[N]) FOUNDX=0;
if(D[N] != C[N]) FOUNDD=0;
}
# Count the events and mark the hour they occurred in
if(FOUNDA || FOUNDX)
{
printf("A@%s-%s-%s %s:%s:%s\n",$1,$2,$3,$5,$6,$7);
CA++; AH[$5]++;
TOTALBATS++;
}
if(FOUNDB || FOUNDD)
{
printf("B@%s-%s-%s %s:%s:%s\n",$1,$2,$3,$5,$6,$7);
CB++; BH[$5]++;
TOTALBATS--;
}
# Update our maximum daily counts
if(MAXBATS < TOTALBATS)
{
MAXBATS=TOTALBATS;
MAXTIME=T;
}
if(MINBATS > TOTALBATS)
{
MINBATS=TOTALBATS;
MINTIME=T;
}
}
}
END { # The final statistics will be printed to stderr, to easily
# seperate them from the event times printed to stdout.
# The last daily count
print_daily(DAY,TOTALBATS,MAXBATS,MINBATS,MAXTIME);
# Print the event counts
printf("A %2d\nB %2d\nT %2d\n", CA, CB, CA+CB) > "/dev/stderr";
# Print a list of hours from 1-23
STR="H";
for(N=1; N<=23; N++) STR=STR sprintf(" %2d", N);;
print STR > "/dev/stderr";
# Print hourly counts for event A
STR="A";
for(N=1; N<=23; N++)
STR=STR sprintf(" %2d", AH[sprintf("%02d", N)]);
print STR > "/dev/stderr";
# Hourly counts for event B
STR="B";
for(N=1; N<=23; N++)
STR=STR sprintf(" %2d", BH[sprintf("%02d",N)]);
print STR > "/dev/stderr";
}
===============================================
ADDITIONAL INFO:
reminder this runs from zsh, you made a batch file which calls twozsh scripts, I modified these to use the output.txt file as input (same format as small sample data above)
1) bat file:
# because windows CMD doesn't actually expand * into arguments for you,
# we need to run this in sh to do so. Bleh.
exec ./zsh.exe IpEther2.zsh * > events.txt 2> stats.txt
# operates on any dir name starting between year 2000 and 2999
# "2>" redirects sterr to stats.txt
3. 2nd zsh script
Code:
#!/bin/zsh
# this script is to be used with data exported from Ipether Access database
./gawk.exe -F "," -f batmon5.awk < output.txt # calls gawk using output.txt as the input to awk
Last edited by cmp260; 07-22-2011 at 07:53 AM..
Reason: more information
hi i am trying to perform some calculations with awk and arrays. i have this so far:
awk 'NR==FNR{ for(i=1; i<=NF; i++) {array+=$i} tot++;next}
{for(i=1; i<=NF; i++) {avg=array/tot} {diff=(array - avg)}} {for(i=1; i<=NF; i++) {printf("%5.8f\n",diff)}}' "$count".txt "$count".ttt >... (4 Replies)
I'm at wits end with this issue and my troubleshooting leads me to believe it is a problem with the file formatting of the array referenced by my script:
awk -F, '{if (NR==FNR) {a=$4","$3","$2}\
else {print a "," $0}}' WBTSassignments1.txt RNCalarms.tmp
On the WBTSassignments1.txt file... (2 Replies)
Dear All,
I am facing problem to get right output through awk program
I have file in which “B” value is appearing multiple time and I need to capture all these values.
My script is
BEGIN { FS=" " }
{
if ( substr($1,1,5) == "START" )
{
i =... (2 Replies)
Hi all,
i have a data array as follows.
array=ertfgj2345
array=456ttygkd
.
.
.
array=errdjt3235
so number or elements in the array can varies depending on how big the data input is.
now i have a variable, and it is $1 (there are $2, $3 and so on, i am only interested in $1).
... (9 Replies)
Hi,
I rather have a very complicated awk problem here, at least to me. I have two files.
File 1:
607 687 174 0 0 chr1 3000001 3000156 -194195276 - L1_Mur2 LINE L1 -4310 1567 1413 1
607 917 214 114 45 chr1 3000237 ... (19 Replies)
I am trying to map values in the input file, where 2nd column depends on the specific value in the 1st column. When 1st column is A place 1 into 2nd column, when it is B, place 2, when C place 3, otherwise no change.
My input:
U |100|MAIN ST |CLMN1|1
A |200|GREEN LN |CLMN2|2
1 |12... (4 Replies)
Hello experts,
I'm stuck with this script for three days now. Here's what i need.
I need to split a large delimited (,) file into 2 files based on the value present in the last field.
Samp: Something.csv
bca,adc,asdf,123,12C
bca,adc,asdf,123,13C
def,adc,asdf,123,12A
I need this split... (6 Replies)
Hi, I have a problem with awk array when iam trying to use awk in solaris box as below..Iam unable to figure out the problem..
Need your help. is there any alternative to make it in arrays from variable values
nawk 'BEGIN {SUBSEP=" ";
split("101880|110045 101887|110045 101896|110045... (9 Replies)
Hello All,
Can you please help me with the below.
#!/bin/bash
ARR="No Differences In Stage Between HASH_TOTALS & HASH_TOTALS_COMP For UNINUM:0722075 PROVIDER:5 EXTRACT_DT:30-SEP-12 VER_NUM:1"
ARR="No Differences In Stage Between HASH_TOTALS & HASH_TOTALS_COMP For UNINUM:0722075 PROVIDER:5... (14 Replies)
I am trying to reformat the table by filling any missing rows. The final table will have consecutive IDs in the first column. My problem is the index of the associate array in the awk script.
infile:
S01 36407 53706 88540
S02 69343 87098 87316
S03 50133 59721 107923... (4 Replies)