The UNIX and Linux Forums  

Go Back   The UNIX and Linux Forums > Top Forums > UNIX for Dummies Questions & Answers
.
google unix.com



UNIX for Dummies Questions & Answers If you're not sure where to post a UNIX or Linux question, post it here. All UNIX and Linux newbies welcome !!

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Concatenating records from 2 files Powcmptr UNIX for Dummies Questions & Answers 4 04-02-2008 04:57 PM
Awk Compare Files w/Multiline Records RacerX Shell Programming and Scripting 4 12-21-2007 06:05 AM
Count No of Records in File without counting Header and Trailer Records guiguy Shell Programming and Scripting 2 06-07-2007 01:15 PM
Help comparing 2 files to find deleted records eja UNIX for Dummies Questions & Answers 2 04-03-2007 09:53 AM
How to extract duplicate records with associated header record run_eim UNIX for Dummies Questions & Answers 17 01-16-2007 11:46 PM

Closed Thread
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish Bulgarian Greek Powered by Powered by Google
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 11-14-2005
Browser_ice Browser_ice is offline
Registered User
  
 

Join Date: Nov 2005
Location: Canada
Posts: 110
Exclamation Best approach for a 10 min extract out of several log files with timestamped records

I have a task where I need to code a shell script to extract a 10 min range (10 min from now until now) extract of a log file.

I taught I could simply use a command that would say something like Start=date - 10 min but I didn't find anything. Looks like the only solution would have to code a small script that would extract the time, strip the values, sub 10 min, check for month and year crossover and convert back into a date string.

Idealy, if there was a command to extract all records from start (chronologicaly reveresed) up until matching record= 10 min back, that would be great. I seriously dought there would be such a thing.

Any suggestions ? I'm a Unix script beginner.
  #2 (permalink)  
Old 11-14-2005
Perderabo's Avatar
Perderabo Perderabo is offline Forum Staff  
Unix Daemon
  
 

Join Date: Aug 2001
Location: Ashburn, Virginia
Posts: 9,131
Post some sample data.
  #3 (permalink)  
Old 11-14-2005
Browser_ice Browser_ice is offline
Registered User
  
 

Join Date: Nov 2005
Location: Canada
Posts: 110
Sample data

sample file to have 10 min extracted. It is the result of an fgrep done on several log files on specific keywords :

/export/home/..../filea.log:2005-11-14 12:01:03,999 xxxxxxxxxxxxxxx
/export/home/..../filea.log:2005-11-14 12:01:02,123 xxxxxxxxxxxxxxx
/export/home/..../filea.log:2005-11-14 12:01:01,345 xxxxxxxxxxxxxxx
...

The logs are live and therefore always updated.

Once I have the date-10min value, wouldn't it be possible to somehow get the line# of where the 10 min matches and have a command that only gets from the start up to that line# ?
  #4 (permalink)  
Old 11-15-2005
Perderabo's Avatar
Perderabo Perderabo is offline Forum Staff  
Unix Daemon
  
 

Join Date: Aug 2001
Location: Ashburn, Virginia
Posts: 9,131
With only 10 minutes to look for, you could put all 10 timestamps in one grep statement. To reverse the lines, prepend line numbers, sort in reverse order on the line numbers, and remove the line numbers. This script uses datacalc which you can find on this site.


Code:
#! /usr/bin/ksh

date "+%Y %m %d %H %M" | read year month day hour minute
pattern=""
count=10
while((count)) ; do
        now="${year}-${month}-${day} ${hour}:${minute}"
        ((count=count-1))
        if ((minute)) ; then
                ((minute=minute-1))
        else
                minute=59
                if ((hour)) ; then
                        ((hour=hour-1))
                else
                        hour=23
                        datecalc -a $year $month $day - 1 | read year month day
                fi
        fi
        [[ -n $pattern ]] && pattern="${pattern}|"
        pattern="${pattern}${now}"
done
pattern=$(echo $pattern | tr "|" "\n")

grep -F "$pattern" | nl -ba | sort -nr | cut -f2-
exit 0

Closed Thread

Bookmarks

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On




All times are GMT -4. The time now is 06:22 AM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0