Need help looking for missing hours.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need help looking for missing hours.
# 8  
Old 06-06-2013
Rudi's 2nd awk script is great for the given problem. If at some point in the future, you're interested in processing a samfile containing data for multiple days and/or multiple streams (or if you want to know if there are multiple input lines for a given stream on a given date that have the same start time, you could try the following more complex awk script:
Code:
awk -F'[-.]' '
/DMME/ {# Note that the following tests will not catch start times that are not
        # even multiples of 15 minutes and will not catch end times like 0160
        # when paired with a start time like 0145.  If these issues are a
        # concern, I leave fixing it as an exercise for the reader.
        # However, if any start time that is a multiple of 15 mintues is not
        # present that will be caught during END processing.
        if($2 + 15 != $4 &&
                (substr($2, 3) != 45 ||
                        (((substr($2, 1, 2) + 1) % 24) "00") + 0 != $4)) {
                printf("Skipping: %s\n\t%s not 15 minutes after %s\n",
                        $0, $4, $2)
                next
        }
        if(start[$1, $2]++) {
                printf("Skipping: %s\n\t", $0) 
                printf("%d entries seen for %s stream %s @ start time %s\n",
                        start[$1, $2], substr($1, 2), substr($1, 1, 1), $2)
                next
        }
        sd[$1]++
}
END {   for(i in sd) {
                printf("Processing %d entries for %s stream %s:\n",
                        sd[i], substr(i, 2), substr(i, 1, 1))
                for(h = 0; h <= 23; h++)
                        for(m = 0; m < 60; m +=15) {
                                st = sprintf("%02d%02d", h, m)
                                if(!((i, st) in start)) {
                                        printf("\tStart time %s missing\n", st)
                                        cm++
                                }
                        }
                if(cm)  cm = 0
                else    printf("\tOK.\n")
        }
}' /home/fsan/output/samfile

With a samfile that contains data for multiple streams on one date and two dates for one of the streams (with a few errors sprinkled in for fun), the output produced is:
Code:
Skipping: A20130604.0000-0600-0016-0600_SubNetwork-DMME1ManagedElement=ces-1
	0016 not 15 minutes after 0000
Skipping: A20130605.2345-0600-2400-0600_SubNetwork-DMME1ManagedElement=ces-1
	2400 not 15 minutes after 2345
Skipping: A20130605.2345-0600-0000-0600_SubNetwork-DMME1ManagedElement=ces-1
	2 entries seen for 20130605 stream A @ start time 2345
Processing 96 entries for 20130604 stream B:
	OK.
Processing 95 entries for 20130604 stream A:
	Start time 0000 missing
Processing 96 entries for 20130605 stream A:
	OK.

This User Gave Thanks to Don Cragun For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Fill in missing hours and interpolate values using awk.

I have a time series data like this 40754,35.6931,51.3092,201610160700,21.0 40754,35.6931,51.3092,201610160800,23.0 40754,35.6931,51.3092,201610160900,24.0 40754,35.6931,51.3092,201610161000,24.0 40754,35.6931,51.3092,201610161300,25.0 40754,35.6931,51.3092,201610161400,23.0... (6 Replies)
Discussion started by: emirzaei
6 Replies

2. Red Hat

Yum - resolving missing dependencies that are not missing

I am trying to install VirtualBox on RHEL 5 but I need the 32 bit version for 32 bit Windows. When I run yum I get the following: sudo yum localinstall /auto/spvtg-it/spvss-migration/Software/VirtualBox-4.3-4.3.2_90405_el6-1.i686.rpm Loaded plugins: fastestmirror Setting up Local Package... (13 Replies)
Discussion started by: gw1500se
13 Replies

3. SuSE

How to resolve missing missing dependencies with opensuse 11.3 and 12.3?

Hello, This is a programming question as well as a suse question, so let me know if you think I should post this in programming. I have an application that I compiled under opensuse 12.2 using g77-3.3/g++3.3. The program compiles and runs just fine. I gave the application to a colleague who... (2 Replies)
Discussion started by: LMHmedchem
2 Replies

4. Shell Programming and Scripting

Get the no of hours between days

Hi, i have a date 1- 2013101511 date2 -2013101812 need toget the no of hours between them,can any one tellme the logic. (6 Replies)
Discussion started by: sandeep karna
6 Replies

5. Shell Programming and Scripting

ps -ef |grep 24 hours

I need to grep PIDs older than 24 hours (1 day) or more. ps -ef |grep ??? Please advise. (10 Replies)
Discussion started by: Daniel Gate
10 Replies

6. AIX

cron off by 5 hours

stupid question im sure, but its frustrating My cron jobs are off by 5 hours. My system time is right but all of my cron jobs are running approximately 5 hours late. Any idea why? (4 Replies)
Discussion started by: mshilling
4 Replies

7. Shell Programming and Scripting

CurrentTime-4 hours

Hi, Good Afternoon! I am writing this script on "sh" and have Variables as below. #Time in hours ex: 09 JobTime=`echo $StartTime |awk '{print $2}'|cut -f1 -d':'` SystemHours=`date +%H` How can go 4 hours back for each variable in a day? Another Question? JobStat=`dsjob -report... (5 Replies)
Discussion started by: rajubollas
5 Replies

8. What is on Your Mind?

How Many hours on Computer?

How many hours you spend on Computer in a day??? (10 Replies)
Discussion started by: malcomex999
10 Replies

9. Shell Programming and Scripting

how to list files between last 6 hours to 3 hours

Hi Frens, I want to list some files from a directory, which contains "DONE" in their name, i am receiving files every minute. In this i want to list all the files which are newer than 6 hours but older than 3 hours, of current time i dont want my list to contain the latest files which are ... (4 Replies)
Discussion started by: Prat007
4 Replies

10. Post Here to Contact Site Administrators and Moderators

Have we just had a rollback of a few hours?

Have we just had a rollback of a few hours? (1 Reply)
Discussion started by: porter
1 Replies
Login or Register to Ask a Question