Shell Programming and Scripting

Hi!
I have 2 files containing data that I need to process at the same time, I have problems in reading a different number of lines from the different files.
Here is an explanation of what I need to do (possibly with an awk script).

File "samples.txt" contains data in the format:
time_instant measure
File "time.txt" contains data in the format
time_instant position
I need to compute temporal averages of the measures in samples.txt, but the temporal averages must be related only to the time instants in which the position in time.txt is "in".

So, I need at the end to have:

a/b
where a=sum(measure_i*duration_measure_i)
b=total_measurement_duration

- measure_i is the second field in every line of samples.txt
- duration_measure_i is the difference between the 1st fields in two consecutive lines in samples.txt and eventually the duration of the period in which the position in time.txt was "out"

So, for example I would do for the first measurement (that is on the second line in samples.txt, since the 1st line only tells me when measurements start):
a=18.4*(906.43-903.0-0)
b=(906.43-903.0-0)

Then for the second:
a+=17.5*(912.7-906.43-2)
b+=(912.7-906.43-2)

I am quite new to awk and don't know how to read two lines at a time from samples.txt and search in time.txt the lines that are between the time instants written from samples.txt

Any suggestion? Thank you very much!!

Where the 2 in 912.7-906.43-2 comes from?

2 is the time spent "out".
I know that looking the file it should be 3, but for the periods when the position is "out", I have to consider as length of the period the final_time-initial_time (for example, in case only 1 consecutive line with "out" appeared, then the length of interval would be 0 and I would not subtract any time)
Then, 910-908=2

That's because my records on "out" periods are only instantaneous.. sorry for not explaining that in a clearer way in the previous post. :-)

Sorry, but I still don't understand how the data in times.txt is used.

First we have:

Code:

900 out 
901 out 
902 out

So, the time spent out is 2, not 0, or I'm missing something again?

What about the in values in times.txt, where are they used?

The time intervals in the two files don't coincide: in samples.txt I record my measures, then I have to search for the related time intervals of interest in time.txt.

For example, in samples.txt, measurements started at 903.0, then at 906.43 I got the first measure (2nd line). So, I have to go in time.txt and check if in the time period between 903 and 906.43 I was "in" or "out". Since I've been "in" all the time,itì's ok and I don't need to subtract anything.
The time instants preceding this interval are discarded.

Then, I go to the second measure in samples.txt (3rd line) and check if in time.txt if in the time interval between 906.43 and 912.7 I was "in" or "out". I've been "out" for 2 seconds, so I subtract this value from the time interval duration, and so on.

Time.txt records a very long time, usually with many seconds preceding and following the time instants in which measurements were taken. I refer to samples.txt, considering one sample at every iteration (i.e. the line referring to the current measure and the preceding one in order to get the time interval of interest), then I need to get the correspondent lines related to that time interval in time.txt. If the duration of "out" periods in this interval of interest is zero, it's ok and I don't need to subtract anything. In case it's non-zero, I need to subtract it from the interval duration

Hope is much clearer, thank a lot for your help anyway!



---------- Post updated at 05:11 AM ---------- Previous update was at 04:54 AM ----------

Hope this helps. Suppose you have

Code:

903.0 -
906.43 18.4
912.7 17.5
921.05 11.2

and

Code:

900 out
901 out
902 out
903 in
904 in
905 in
906 in
907 in
908 out
909 out
910 out
911 in
912 in
913 in
914 out
915 out
916 out
917 out
918 in
919 out
920 in
921 in
922 in
923 in

The result I need to get is
{18.4*(906.43-903.0-0)+17.5*(912.7-906.43-2)+11.2*(921.05-912.7-3)}/{(906.43-903.0-0)+(912.7-906.43-2)+(921.05-912.7-3)}

If I understand correctly, you could use something like this:

Code:

awk 'END { print at/bt }
NR == FNR {
  /out/ && o[++i] = $1
  if (/in/ && $2 != p2) {
    r[o[1]] = o[i]; i = 0
    }
  p1 = $1; p2 = $2; next    
  }
FNR == 1 {
  p2 == "in" || r[o[1]] = o[i] 
  p1 = $1; next
  }
{ 
  _o = 0 
  for (R in r) {
    if (R >= p1 && $1 >= r[R]) 
      _o += r[R] - R      
    }
  b = $1 - p1 - _o
  at += ($2 * b); bt += b
  }
{ p1 = $1 }' times.txt samples.txt

Very impressive! :-)
Thank you so much!!!