Shell Programming and Scripting

Hi,

sorry hadnt had the chance to look at this until now. Thank you so much for this help, it is really appreciated. I'm going to have a go with your zip file this afternoon. There is another ipcas logger in use which outputs data in a slightly different format to an access database (can export as csv). This logger does not create daily folders, but it does have included with time -as hh:mm:ss and millisewconds. I'm must say that I feel greedy asking you so much but its difficult not to since you are so helpful
If you are interested in another problem, let me know and Ill send you a copy of this file and see if its possible to do the same type of thing as the other one but also calculate speed of transit using milliseconds field?

Attaching a data logger to an access database sounds pretty crazed... good thing it can export csv. Hopefully the format's not TOO different. You'd be asking the forums in general, not just me anyway.

Hi Corona,

I just tried your program out. I am a bit unsure about the output: from looking at the code it seems that there should be 16 hours added to each event right? Thus the logfile.txt in the dir 2011-02-02: events start at about 06:26 thus these shuoud be comverted to 22:26 but it seems its actually converted to 00:15, surely that is not correct?
or am I missing something?

thanks

Be specific, post input and output data.

I should note output.txt does not have its dates converted, it's left pretty much raw.

Here's a version with the date translation removed. Does its output at least make sense?

hi,

I tested the script on a single day (28th) and then again manually using a combination of word excel and plain text to manipulate the data so I could use find/replace.
The results dont match yours exactly although they are close.
my calcs give A = 12 B = 8
wheras your script give A = 11 B = 8
here is the result of my testing (using "-" as replacement for paragraph mark so I could use word find/replace)
0,1-0,0-B-1,1-1,0-A-B-0,1-0,0-0,1-0,0-0,1-1,1-1,0-1,1-0,0-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-1,0-1,1-0,1-0,0-0,1-0,0-0,1-0,0-0,1-0,0-0,1-0,0-0,1-1,0-0,0-0,1-0,0-1,1-1,0-1,1-0,1-0,0-0,1-1,0-0,0-0,1-0,0-0,1-1,1-1,0-0,0-B-1,1-1,0-A-0,1-0,0-0,1-0,0-0,1-1,1-1,0-1,1-1,0-0,0-1,1-1,0-1,1-1,0-A-0,1-0,0-0,1-1,1-1,0-1,1-0,0-1,0-B-1,1-1,0-1,1-1,0-A-0,1-1,1-1,0-1,1-1,0-1,1-1,0-1,1-0,0-1,0-1,1-1,0-0,1-1,1-0,0-0,1-0,0-0,1-1,0-0,0-0,1-0,0-0,1-1,1-1,0-1,1-1,0-0,0-0,1-0,0-0,1-1,1-1,0-1,1-0,0-1,0-1,1-1,0-0,1-0,0-0,1-1,1-1,0-1,1-1,0-1,1-0,0-1,0-0,1-0,0-0,1-1,1-0,0-0,1-0,0-1,0-1,1-1,0-1,1-1,0-1,1-0,1-0,0-0,1-0,0-0,1-1,0-0,0-0,1-0,0-0,1-0,0-B-1,1-1,0-1,1-1,0-1,1-0,1-1,0-1,1-0,0-1,0-1,1-0,1-0,0-0,1-0,0-0,1-1,0-0,0-0,1-0,0-1,1-1,0-1,1-1,0-A-A-0,1-0,0-0,1-0,0-0,1-1,1-1,0-1,1-0,0-0,1-0,0-1,0-B-1,1-0,1-1,0-1,1-1,0-0,0-0,1-1,1-1,0-1,1-0,0-1,0-B-1,1-1,0-A-0,1-0,0-0,1-0,0-0,1-1,1-1,0-1,1-0,0-1,0-0,1-0,0-0,1-0,0-0,1-1,1-1,0-1,1-0,0-0,1-0,0-1,0-1,1-1,0-1,1-1,0-1,1-0,1-0,0-0,1-1,0-0,0-0,1-0,0-0,1-0,0-1,1-1,0-1,1-1,0-A-1,1-0,1-0,0-0,1-0,0-1,0-1,1-1,0-1,1-0,1-0,0-0,1-1,0-0,0-1,1-1,0-0,1-0,0-1,1-1,0-1,1-0,1-1,0-1,1-1,0-1,1-1,0-0,0-0,1-0,0-0,1-1,1-1,0-1,1-0,0-1,0-1,1-0,1-1,0-1,1-1,0-0,0-A-0,1-1,1-1,0-1,1-0,0-0,1-0,0-1,0-0,1-0,0-0,1-0,0-0,1-0,0-0,1-0,0-B-1,1-1,0-A-1,1-1,0-1,1-1,0-1,1-1,0-A-0,1-0,0-1,1-0,1-1,0-1,1-0,0-1,0-A-

---------- Post updated at 02:04 PM ---------- Previous update was at 12:44 PM ----------

Hi corona, I also tested 31-1-2011 and found differences in the results:
your script reports B=24 A=16
wheras manually going over the data, I get B=28 A=16
any ieas what is casusing this?
thanks

---------- Post updated at 02:48 PM ---------- Previous update was at 02:04 PM ----------

Here are stats using the output of my resilts for 31st (B=28, A=16) as you can see my cumulative total result is 11 bats.

BEAM STATE TOTAL 0 1 0 0 0 0 1 1 0 1 0 0 1 1 0 1 0 0 1 1 0 1 0 0 1 1 0 1 0 0 1 1 0 1 0 0 1 1 0 1 0 0 0 1 0 0 0 0 0 1 0 0 0 0 A
-1 0 1 -1 1 1 -1 1 0 -1 1 1 -1 0 0 -1 1 0 -1 1 1 -1 1 0 -1 1 1 -1 1 0 -1 1 1 -1 1 0 -1 1 1 -1 0 1 -1 0 0 -1 0 1 -1 1 0 -1 0 0 -1 0 1 -1 1 1 -1 0 0 -1 0 1 -1 0 0 -1 1 0 -1 1 1 -1 0 1 -1 0 0 -1 0 1 -1 1 0 -1 0 0 -1 B
-1 0 1 0 0 0 0 1 1 0 1 0 0 1 1 0 1 0 0 1 1 0 1 0 0 1 1 0 1 0 0 1 1 0 1 0 0 A
-1 0 1 -1 0 0 -1 B
-1 1 1 0 1 0 0 1 1 0 1 0 0 1 1 0 1 0 0 B
0 0 1 1 1 1 1 0 0 1 0 1 1 1 0 1 0 0 1 0 1 1 0 0 1 1 1 1 1 0 1 0 1 1 0 0 1 0 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 0 1 1 1 1 0 1 1 0 0 1 0 1 1 1 0 1 0 0 1 0 1 1 0 0 1 0 1 1 0 0 1 1 1 1 1 0 1 0 1 1 0 0 1 1 1 1 1 0 1 A
0 0 1 0 1 1 0 0 0 0 0 1 0 0 0 0 1 0 0 1 1 0 1 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 A
-1 B
-1 1 1 0 1 0 0 0 1 0 0 0 0 B
0 0 1 1 0 0 1 0 1 1 1 1 1 0 0 1 0 1 1 0 0 1 1 0 1 0 1 1 0 0 1 1 1 1 1 0 1 A
0 0 1 0 1 1 0 1 0 0 1 1 0 1 0 0 0 0 0 0 1 0 0 0 0 1 1 0 1 0 0 1 1 0 1 0 0 1 1 0 0 1 0 0 0 0 0 1 0 0 0 0 1 0 0 1 1 0 1 0 0 A
-1 0 1 -1 0 0 -1 1 1 -1 1 0 -1 B
-1 1 1 0 1 0 0 1 1 0 0 1 0 0 0 0 0 1 0 1 0 0 0 0 0 B
0 1 1 1 0 1 1 0 0 1 0 1 1 1 0 1 0 0 1 B
1 1 1 2 0 1 2 1 0 2 1 1 2 0 0 2 0 1 2 1 0 2 0 0 2 B
2 0 1 3 1 1 3 1 0 3 0 0 3 0 1 3 1 1 3 1 0 3 1 1 3 0 0 3 1 0 3 B
3 1 1 4 1 0 4 1 1 4 1 0 4 0 1 4 0 0 4 0 1 4 0 0 4 0 1 4 1 1 4 1 0 4 1 1 4 0 0 4 1 0 4 B
4 1 1 5 1 0 5 0 1 5 0 0 5 1 1 5 1 0 5 0 1 5 0 0 5 1 1 5 1 0 5 1 1 5 1 0 5 0 1 5 1 1 5 0 0 5 0 1 5 1 0 5 0 0 5 0 1 5 1 1 5 0 0 5 0 1 5 0 0 5 1 0 5 1 1 5 0 1 5 0 0 5 0 1 5 0 0 5 0 1 5 0 0 5 0 1 5 1 0 5 0 0 5 1 1 5 1 0 5 1 1 5 0 1 5 0 0 5 0 1 5 1 0 5 0 0 5 1 1 5 1 0 5 1 1 5 1 0 5 1 1 5 1 0 5 0 1 5 1 1 5 1 0 5 0 0 5 B
5 1 1 6 0 1 6 0 0 6 0 1 6 1 0 6 0 0 6 1 1 6 1 0 6 1 1 6 1 0 6 1 1 6 1 0 6 A
5 A
4 0 1 4 0 0 4 B
4 B
5 1 1 6 1 0 6 1 1 6 1 0 6 A
5 1 1 5 0 1 5 0 0 5 0 1 5 1 0 5 0 0 5 B
5 1 1 6 0 1 6 0 0 6 0 1 6 1 0 6 0 0 6 B
6 0 1 7 0 0 7 B
7 1 1 8 0 1 8 0 0 8 0 1 8 0 0 8 0 1 8 1 0 8 0 0 8 1 1 8 1 0 8 1 1 8 1 0 8 1 1 8 0 1 8 0 0 8 0 1 8 0 0 8 0 1 8 1 0 8 0 0 8 0 1 8 1 1 8 0 0 8 0 1 8 0 0 8 0 1 8 1 0 8 1 1 8 1 0 8 1 1 8 1 0 8 0 0 8 1 1 8 1 0 8 1 1 8 1 0 8 1 1 8 1 0 8 1 1 8 1 0 8 1 1 8 1 0 8 A
7 0 1 7 0 0 7 B
7 0 1 8 0 0 8 B
8 B
9 1 1 10 0 1 10 1 0 10 1 1 10 0 0 10 1 0 10 1 1 10 1 0 10 1 1 10 1 0 10 1 1 10 0 1 10 0 0 10 0 1 10 1 0 10 0 0 10 B
10 1 1 11 1 0 11 1 1 11 0 1 11 0 0 11 1 0 11 A
10 0 1 10 0 0 10 0 1 10 0 0 10 1 1 10 1 0 10 1 1 10 0 1 10 1 0 10 1 1 10 1 0 10 0 0 10 1 1 10 0 1 10 0 0 10 0 1 10 1 0 10 0 0 10 1 1 10 1 0 10 0 1 10 0 0 10 A
9 1 1 9 1 0 9 A
8 B
8 0 1 9 0 0 9 B
9 1 1 10 1 0 10 A
9 B
9 B
10 1 1 11 1 0 11 1 1 11 1 0 11 1 1 11 0 1 11 0 0 11 0 1 11 1 0 11 0 0 11 0 1 11 0 0 11 B
11 1 1 12 1 0 12 1 1 12 1 0 12 1 1 12 1 0 12 0 1 12 1 1 12 0 0 12 0 1 12 1 0 12 0 0 12 0 1 12 1 1 12 1 0 12 0 0 12 B
12 1 1 13 0 1 13 1 0 13 1 1 13 1 0 13 0 0 13 B
13 A
12 0 1 12 1 1 12 0 0 12 0 1 12 0 0 12 0 1 12 0 0 12 1 0 12 1 1 12 1 0 12 1 1 12 1 0 12 A
11 0 1 11 0 0 11 0 1 11 1 1 11 1 0 11 1 1 11 1 0 11 1 1 11 0 0 11 1 0 11

---------- Post updated at 02:48 PM ---------- Previous update was at 02:48 PM ----------

Here are stats using the output of my resilts for 31st (B=28, A=16) as you can see my cumulative total result is 11 bats.

BEAM STATE TOTAL 0 1 0 0 0 0 1 1 0 1 0 0 1 1 0 1 0 0 1 1 0 1 0 0 1 1 0 1 0 0 1 1 0 1 0 0 1 1 0 1 0 0 0 1 0 0 0 0 0 1 0 0 0 0 A
-1 0 1 -1 1 1 -1 1 0 -1 1 1 -1 0 0 -1 1 0 -1 1 1 -1 1 0 -1 1 1 -1 1 0 -1 1 1 -1 1 0 -1 1 1 -1 0 1 -1 0 0 -1 0 1 -1 1 0 -1 0 0 -1 0 1 -1 1 1 -1 0 0 -1 0 1 -1 0 0 -1 1 0 -1 1 1 -1 0 1 -1 0 0 -1 0 1 -1 1 0 -1 0 0 -1 B
-1 0 1 0 0 0 0 1 1 0 1 0 0 1 1 0 1 0 0 1 1 0 1 0 0 1 1 0 1 0 0 1 1 0 1 0 0 A
-1 0 1 -1 0 0 -1 B
-1 1 1 0 1 0 0 1 1 0 1 0 0 1 1 0 1 0 0 B
0 0 1 1 1 1 1 0 0 1 0 1 1 1 0 1 0 0 1 0 1 1 0 0 1 1 1 1 1 0 1 0 1 1 0 0 1 0 1 1 0 0 1 1 1 1 1 0 1 1 1 1 1 0 1 1 1 1 0 1 1 0 0 1 0 1 1 1 0 1 0 0 1 0 1 1 0 0 1 0 1 1 0 0 1 1 1 1 1 0 1 0 1 1 0 0 1 1 1 1 1 0 1 A
0 0 1 0 1 1 0 0 0 0 0 1 0 0 0 0 1 0 0 1 1 0 1 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 A
-1 B
-1 1 1 0 1 0 0 0 1 0 0 0 0 B
0 0 1 1 0 0 1 0 1 1 1 1 1 0 0 1 0 1 1 0 0 1 1 0 1 0 1 1 0 0 1 1 1 1 1 0 1 A
0 0 1 0 1 1 0 1 0 0 1 1 0 1 0 0 0 0 0 0 1 0 0 0 0 1 1 0 1 0 0 1 1 0 1 0 0 1 1 0 0 1 0 0 0 0 0 1 0 0 0 0 1 0 0 1 1 0 1 0 0 A
-1 0 1 -1 0 0 -1 1 1 -1 1 0 -1 B
-1 1 1 0 1 0 0 1 1 0 0 1 0 0 0 0 0 1 0 1 0 0 0 0 0 B
0 1 1 1 0 1 1 0 0 1 0 1 1 1 0 1 0 0 1 B
1 1 1 2 0 1 2 1 0 2 1 1 2 0 0 2 0 1 2 1 0 2 0 0 2 B
2 0 1 3 1 1 3 1 0 3 0 0 3 0 1 3 1 1 3 1 0 3 1 1 3 0 0 3 1 0 3 B
3 1 1 4 1 0 4 1 1 4 1 0 4 0 1 4 0 0 4 0 1 4 0 0 4 0 1 4 1 1 4 1 0 4 1 1 4 0 0 4 1 0 4 B
4 1 1 5 1 0 5 0 1 5 0 0 5 1 1 5 1 0 5 0 1 5 0 0 5 1 1 5 1 0 5 1 1 5 1 0 5 0 1 5 1 1 5 0 0 5 0 1 5 1 0 5 0 0 5 0 1 5 1 1 5 0 0 5 0 1 5 0 0 5 1 0 5 1 1 5 0 1 5 0 0 5 0 1 5 0 0 5 0 1 5 0 0 5 0 1 5 1 0 5 0 0 5 1 1 5 1 0 5 1 1 5 0 1 5 0 0 5 0 1 5 1 0 5 0 0 5 1 1 5 1 0 5 1 1 5 1 0 5 1 1 5 1 0 5 0 1 5 1 1 5 1 0 5 0 0 5 B
5 1 1 6 0 1 6 0 0 6 0 1 6 1 0 6 0 0 6 1 1 6 1 0 6 1 1 6 1 0 6 1 1 6 1 0 6 A
5 A
4 0 1 4 0 0 4 B
4 B
5 1 1 6 1 0 6 1 1 6 1 0 6 A
5 1 1 5 0 1 5 0 0 5 0 1 5 1 0 5 0 0 5 B
5 1 1 6 0 1 6 0 0 6 0 1 6 1 0 6 0 0 6 B
6 0 1 7 0 0 7 B
7 1 1 8 0 1 8 0 0 8 0 1 8 0 0 8 0 1 8 1 0 8 0 0 8 1 1 8 1 0 8 1 1 8 1 0 8 1 1 8 0 1 8 0 0 8 0 1 8 0 0 8 0 1 8 1 0 8 0 0 8 0 1 8 1 1 8 0 0 8 0 1 8 0 0 8 0 1 8 1 0 8 1 1 8 1 0 8 1 1 8 1 0 8 0 0 8 1 1 8 1 0 8 1 1 8 1 0 8 1 1 8 1 0 8 1 1 8 1 0 8 1 1 8 1 0 8 A
7 0 1 7 0 0 7 B
7 0 1 8 0 0 8 B
8 B
9 1 1 10 0 1 10 1 0 10 1 1 10 0 0 10 1 0 10 1 1 10 1 0 10 1 1 10 1 0 10 1 1 10 0 1 10 0 0 10 0 1 10 1 0 10 0 0 10 B
10 1 1 11 1 0 11 1 1 11 0 1 11 0 0 11 1 0 11 A
10 0 1 10 0 0 10 0 1 10 0 0 10 1 1 10 1 0 10 1 1 10 0 1 10 1 0 10 1 1 10 1 0 10 0 0 10 1 1 10 0 1 10 0 0 10 0 1 10 1 0 10 0 0 10 1 1 10 1 0 10 0 1 10 0 0 10 A
9 1 1 9 1 0 9 A
8 B
8 0 1 9 0 0 9 B
9 1 1 10 1 0 10 A
9 B
9 B
10 1 1 11 1 0 11 1 1 11 1 0 11 1 1 11 0 1 11 0 0 11 0 1 11 1 0 11 0 0 11 0 1 11 0 0 11 B
11 1 1 12 1 0 12 1 1 12 1 0 12 1 1 12 1 0 12 0 1 12 1 1 12 0 0 12 0 1 12 1 0 12 0 0 12 0 1 12 1 1 12 1 0 12 0 0 12 B
12 1 1 13 0 1 13 1 0 13 1 1 13 1 0 13 0 0 13 B
13 A
12 0 1 12 1 1 12 0 0 12 0 1 12 0 0 12 0 1 12 0 0 12 1 0 12 1 1 12 1 0 12 1 1 12 1 0 12 A
11 0 1 11 0 0 11 0 1 11 1 1 11 1 0 11 1 1 11 1 0 11 1 1 11 0 0 11 1 0 11

---------- Post updated at 03:01 PM ---------- Previous update was at 02:48 PM ----------

see this file
http://dl.dropbox.com/u/21773719/LogFile4.CSV


---------- Post updated at 04:13 PM ---------- Previous update was at 03:01 PM ----------

sorry about the mess of text there I didnt realize the format would not stay as columns.

HI, Ive been using this and it seems to work well still trying to get my head around the programming though. Can you tell me how to do two more things?

1) I screwed up and forgot to set the PC time to -plus 1600 hours, thus the data is all screwy when analysed (remember each "bat day" is from 4pm and continues past midnight.) Is there a simple method I can use to automate subtraction of 8 hours from every record in every direcory then add any resulting records which are before 1600 to each following days directory before I process it?
i.e. change RX<-00:01:58,pv,1,1 to read RX<-16:01:58,pv,1,1 (this would be left in its directory) but 15:00:00... changed to 23:00:00... would be moved to following days directory

2) is there a simple method of deleting or ignoring data between 8 am and 4 pm (real time) i.e. between 20:00 and 00:00 adjusted time. This is due to pidgeons going through the bat counter thus giving false counts.

Thanks for the help.
All the best.