Quote:
Originally Posted by
mutley2202
Quote:
Originally Posted by Don Cragun
Is this something you plan to run every 15 minutes (so you just want to see stuff added since your last run)?
Yes, I want it to run every 15 minutes and only add new events which have appeared in the last 15 minutes.
OK. Note that this means that if there is any delay in starting one of your 15 minute runs, some events that appeared just over 15 minutes before the late starting run may be missed, and if the next run starts on time, some events may be picked up by two runs.
Quote:
Originally Posted by
mutley2202
Quote:
Originally Posted by Don Cragun
Does the log file you're examining rotate? If so, is the rotation synchronized with your script, or do you need to sometimes need to examine the end of an old log file in addition to the current log file?
Yes the log file rotates at present its approximatly once a day, when the file rotates the previous one is also gziped. The rotation isn't synced with my script as its based on volume. If we can examine the old log file in the case of rotation that would be extremely helpful.
I assume that you realize that you need to grab any events logged to your log file after the last run of your script before you
gzip it; or on the next run of your script you'll need to unzip it, run your script to gather events from the end of the old log file, then rezip it, and then have your script on the start of the new log file.
Quote:
Originally Posted by
mutley2202
Quote:
Originally Posted by Don Cragun
Are you looking for the 15 minutes of data before the time on the clock when you start your script, or are you looking for the 15 minutes of data ending with the timestamp on the last entry in your log file?
15 minutes of data before the time on the clock.
As mentioned before, doing it this way means that you may miss some events and may process some events twice. I strongly suggest that instead of trying to match based on timestamps you instead keep track of the line number of the last line processed in the previous run and on the next run just start processing with the next line in that log file. Doing it this way will keep you from missing events and keep you from processing some events twice.
But, if you want to do it just based on timestamps, you can use the GNU
date utility's
-d option with an option-argument of
"now - 15 minutes" and a format of
"+%s" to get the number of seconds since the Epoch for 15 minutes ago and also use
date -d with an option-argument of the 2nd, 3rd, 5th, and 4th fields in your log file (month, day, year, and hr:min:sec) with the same format string and then select events where the seconds since the Epoch 15 minutes ago is less than the timestamp in the file. (Note that you also want to reject any events where the timestamp in the file is more than 900 seconds after your start time. Events meeting this criteria occurred after your script started and should be picked up by the next run of your script instead of by this run.)
Quote:
Originally Posted by
mutley2202
Quote:
Originally Posted by Don Cragun
What operating system and shell are you using?
GNU/Linux, shell being used is bash.
Thanks
Hope this helps.