Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Searching for Gaps in Time

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Searching for Gaps in Time
# 1  
Old 04-19-2010
Searching for Gaps in Time
I am very new to shell scripting. We use C-Shell here and I know the issues that surround it. I hope a solution can be created using awk, sed, etc... instead of having to write a program.

I have an input file that is sorted by date and time in ascending order

RECA2010/04/19-10:50:34.876:INFO: STARTKEY[000000001]
RECA2010/04/19-10:50:44.123:INFO: ENDKEY[000000001]
RECA2010/04/19-10:51:14.876:INFO: STARTKEY[000000002]
RECA2010/04/19-10:51:15.123:INFO: ENDKEY[000000002]
RECA2010/04/19-11:00:00.876:INFO: STARTKEY[000000003]
RECA2010/04/19-11:00:01.123:INFO: ENDKEY[000000003]
RECA2010/04/19-11:01:34.876:INFO: STARTKEY[000000004]
RECA2010/04/19-11:01:44.123:INFO: ENDKEY[000000004]

The solution should produce output that states the following:

10:50 - 2 claims processed
10:52 - 10:59 - no claims processed
11:00 - 2 claims processed

Any help would be greatly appreciated.
Thanks
# 2  
Old 04-20-2010
Here's a solution in Perl:

Code:
$ 
$ 
$ cat f0
RECA2010/04/19-10:50:34.876:INFO: STARTKEY[000000001]
RECA2010/04/19-10:50:44.123:INFO: ENDKEY[000000001]
RECA2010/04/19-10:51:14.876:INFO: STARTKEY[000000002]
RECA2010/04/19-10:51:15.123:INFO: ENDKEY[000000002]
RECA2010/04/19-11:00:00.876:INFO: STARTKEY[000000003]
RECA2010/04/19-11:00:01.123:INFO: ENDKEY[000000003]
RECA2010/04/19-11:01:34.876:INFO: STARTKEY[000000004]
RECA2010/04/19-11:01:44.123:INFO: ENDKEY[000000004]
$ 
$ 
$ ##
$ perl -M"Date::Calc qw(:all)" -lne '
>   s/RECA(.*?)\..*$/$1/;
>   @x = split /[\/:-]/;
>   $x[5] = "00";
>   if ($.==1) {@start=@x} else {@end=@x}
>   $claims{Date_to_Time(@x)}++;
>   END {
>     $j = Date_to_Time(@end)-Date_to_Time(@start);
>     for ($i=0; $i <= $j; $i++) {
>       @date = Add_Delta_DHMS(@start, 0,0,0,$i);
>       if ($date[5]==0) {
>         if (defined $claims{Date_to_Time(@date)}) {
>           if ($gap ne "") {
>             if ($count==1) {
>               $gap =~ s/- .*?$/-/;
>               print "$gap 0 claims processed";
>             } else {
>               print "$gap : no claims processed";
>             }
>             $gap="";
>           }
>           printf("%4d/%02d/%02d %02d:%02d:%02d - %d claims processed\n", @date, $claims{Date_to_Time(@date)});
>           $count=0;
>         } else {
>           if ($count==0) {
>             $gap = sprintf("%4d/%02d/%02d %02d:%02d:%02d - %4d/%02d/%02d %02d:%02d:%02d", @date, @date);
>           } else {
>             $token = sprintf("%4d/%02d/%02d %02d:%02d:%02d", @date);
>             $gap =~ s/- .*?$/- $token/;
>           }
>           $count++;
>         }
>       }
>     }
>   }' f0
2010/04/19 10:50:00 - 2 claims processed
2010/04/19 10:51:00 - 2 claims processed
2010/04/19 10:52:00 - 2010/04/19 10:59:00 : no claims processed
2010/04/19 11:00:00 - 2 claims processed
2010/04/19 11:01:00 - 2 claims processed
$ 
$ 
$ cat -n f1
     1    RECA2010/04/19-10:50:34.876:INFO: STARTKEY[000000001]
     2    RECA2010/04/19-10:50:44.123:INFO: ENDKEY[000000001]
     3    RECA2010/04/19-10:51:14.876:INFO: STARTKEY[000000002]
     4    RECA2010/04/19-10:51:15.123:INFO: ENDKEY[000000002]
     5    RECA2010/04/19-10:53:15.123:INFO: ENDKEY[000000002]
     6    RECA2010/04/19-10:57:15.123:INFO: ENDKEY[000000002]
     7    RECA2010/04/19-11:00:00.876:INFO: STARTKEY[000000003]
     8    RECA2010/04/19-11:00:01.123:INFO: ENDKEY[000000003]
     9    RECA2010/04/19-11:01:34.876:INFO: STARTKEY[000000004]
    10    RECA2010/04/19-11:01:44.123:INFO: ENDKEY[000000004]
    11    RECA2010/04/20-00:13:15.123:INFO: ENDKEY[000000002]
$ 
$ 
$ perl -M"Date::Calc qw(:all)" -lne '
  s/RECA(.*?)\..*$/$1/;
  @x = split /[\/:-]/;
  $x[5] = "00";
  if ($.==1) {@start=@x} else {@end=@x}
  $claims{Date_to_Time(@x)}++;
  END {
    $j = Date_to_Time(@end)-Date_to_Time(@start);
    for ($i=0; $i <= $j; $i++) {
      @date = Add_Delta_DHMS(@start, 0,0,0,$i);
      if ($date[5]==0) {
        if (defined $claims{Date_to_Time(@date)}) {
          if ($gap ne "") {
            if ($count==1) {
              $gap =~ s/- .*?$/-/;
              print "$gap 0 claims processed";
            } else {
              print "$gap : no claims processed";
            }
            $gap="";
          }
          printf("%4d/%02d/%02d %02d:%02d:%02d - %d claims processed\n", @date, $claims{Date_to_Time(@date)});
          $count=0;
        } else {
          if ($count==0) {
            $gap = sprintf("%4d/%02d/%02d %02d:%02d:%02d - %4d/%02d/%02d %02d:%02d:%02d", @date, @date);
          } else {
            $token = sprintf("%4d/%02d/%02d %02d:%02d:%02d", @date);
            $gap =~ s/- .*?$/- $token/;
          }
          $count++;
        }
      }
    }
  }' f1
2010/04/19 10:50:00 - 2 claims processed
2010/04/19 10:51:00 - 2 claims processed
2010/04/19 10:52:00 - 0 claims processed
2010/04/19 10:53:00 - 1 claims processed
2010/04/19 10:54:00 - 2010/04/19 10:56:00 : no claims processed
2010/04/19 10:57:00 - 1 claims processed
2010/04/19 10:58:00 - 2010/04/19 10:59:00 : no claims processed
2010/04/19 11:00:00 - 2 claims processed
2010/04/19 11:01:00 - 2 claims processed
2010/04/19 11:02:00 - 2010/04/20 00:12:00 : no claims processed
2010/04/20 00:13:00 - 1 claims processed
$ 
$

tyler_durden
# 3  
Old 04-20-2010
claims.sh
Code:
#!/bin/sh

IFS='-:.'
[ "$1" ] && exec <"$1"

#  h: hour of current claim
#  m: minute of current claim
# oh: (old) hour of previous claim
# om: (old) minute of previous claim

pr_claims() {
    echo $oh:$om - $i claims processed

    # If read hit EOF, we're done
    [ $h ] || return

    # start/end hour/minute of interval between last two claims read
    local sh sm eh em

    if [ $om -ne 59 ]; then
        sm=$((om+1))
        sh=$oh
    else        
        sm=00
        if [ $oh -ne 23 ]; then
            sh=$((oh+1))
        else
            sh=00
        fi
    fi

    # If no minutes have been skipped, nothing left to do
    [ $sh -eq $h -a $sm -eq $m ] && return

    if [ $m -ne 00 ]; then
        em=$((m-1))
        eh=$h
    else
        em=59
        if [ $h -ne 00 ]; then
            eh=$((h-1))
        else
            eh=23
        fi
    fi

    printf '%0.2u:%0.2u - %0.2u:%0.2u - no claims processed\n' $sh $sm $eh $em
}


while read -r _ h m _; do
    if [ ${oh-$h}${om-$m} = $h$m ]; then
        : $((++i))
    else
        pr_claims
        i=1
    fi
    oh=$h om=$m
done
pr_claims



Code:
$ cat data
RECA2010/04/19-10:50:34.876:INFO: STARTKEY[000000001]
RECA2010/04/19-10:50:44.123:INFO: ENDKEY[000000001]
RECA2010/04/19-10:51:14.876:INFO: STARTKEY[000000002]
RECA2010/04/19-10:51:15.123:INFO: ENDKEY[000000002]
RECA2010/04/19-11:00:00.876:INFO: STARTKEY[000000003]
RECA2010/04/19-11:00:01.123:INFO: ENDKEY[000000003]
RECA2010/04/19-11:01:34.876:INFO: STARTKEY[000000004]
RECA2010/04/19-11:01:44.123:INFO: ENDKEY[000000004]

$ ./claims.sh data
10:50 - 2 claims processed
10:51 - 2 claims processed
10:52 - 10:59 - no claims processed
11:00 - 2 claims processed
11:01 - 2 claims processed

Last edited by alister; 04-20-2010 at 02:32 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Gaps and frequencies

I have this infile: >GHL8OVD01BNNCA Freq 10 TAGATGTGCCCGTGGGTTTCCCGTCAACACCGGATAGT-GCAGCA-TA >GHL8OVD01CMQVT Freq 1 TTGATGTCGTGGGTTTCCCGTCAACACCGGCAAATAGT-GCAGCA-TA >GHL8OVD01CMQVT Freq 1 TTGATGTGCCAGTTTCCCGTCTAGCAGCACTACCAGGACCTTCGC-TA >GHL8OVD01CMQVW Freq 1... (1 Reply)
Discussion started by: Xterra
1 Replies

2. Programming

Find gaps in time data and replace missing time value and column 2 value by interpolation in awk

Dear all, I am kindly seeking assistance on the following issue. I am working with data that is sampled every 0.05 hours (that is 3 minutes intervals) here is a sample data from the file 5.00000 15.5030 5.05000 15.6680 5.10000 16.0100 5.15000 16.3450 5.20000 16.7120 5.25000... (4 Replies)
Discussion started by: malandisa
4 Replies

3. Shell Programming and Scripting

Adding gaps to a string in bash

I have the following string, and want to introduce additional spaces between the two %s. This will be done by specifying the gap between the %s. Example having gap=8 will put 8 spaces between the two %s. frmt_k1d1_test="%s %s\n" I am doing the script in bash. ---------- Post updated at... (4 Replies)
Discussion started by: kristinu
4 Replies

4. Shell Programming and Scripting

perl : searching for month and storing the date and time in an array

I am writing the code in perl. I have an array in perl and each variable in the array contains the data in the below format Now I need to check the below variable w.r.t system month I need to store the date and time(Tue Aug 7 03:54:12 2012) from the below data into file if contains only 'Aug'... (5 Replies)
Discussion started by: giridhar276
5 Replies

5. Shell Programming and Scripting

Sorting and moving file sequence with gaps

Hello, I have lots of sequentially numbered files which make up an image sequence. I'm trying to do two things with it: #1: Find gaps in the sequence and move each range of sequencial files into their own subfolder. #2: Designate a starting point (file) and move every 24th file into... (4 Replies)
Discussion started by: ex_H
4 Replies

6. Shell Programming and Scripting

Searching the lines within a range of time period in a text file

Dear All, Please advice me, I have a text file with one field date and time like below given. I need to find out the lines whchi content the time stamp between Wed May 26 11:03:11 2010 and Wed May 26 11:03:52 2010 both can be included, using awk command which could be an interactive so that I... (6 Replies)
Discussion started by: chinmayadalai
6 Replies

7. Shell Programming and Scripting

searching between start and end time

Hello All, Below mentioned is my log file. I want to make a script which ask for start time and then end time and then search particular word between those lines. Like start time:2 end time: 4 and then search all values starting from cell 84 between this time. Please Help ... (2 Replies)
Discussion started by: wakhan
2 Replies

8. UNIX for Dummies Questions & Answers

Searching files by last accessed time

How can I search for files by last accessed time? I want to see files accessed in the last 24 hours, for example...or even less time, maybe in the last 3 hours? Thank you in advance, Trellot (4 Replies)
Discussion started by: Trellot
4 Replies

9. Linux

Searching for gaps in huge (2.2G) log file?

I've got a 2.2 Gig syslog file from our Cisco firewall appliance. The problem is that we've been seeing gaps in the syslog for anywhere from 10 minutes to 2 hours. Currently I've just been using 'less' and paging through the file to see if I can find any noticeable gaps. Obviously this isn't the... (3 Replies)
Discussion started by: deckard
3 Replies
Login or Register to Ask a Question