Extract loglines between two dates


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extract loglines between two dates
# 1  
Old 07-02-2009
Extract loglines between two dates

Hi all,
this is my first post, and i want to say hello to everyone in there.

I've a little problem for you all Smilie.
I have a log like this:
Code:
38714 07/02/09 00:01:36  nsrd web2consolidato:D:\ done saving to pool 'SISTMSW0060' (Q09480) 32 GB
38752 07/02/09 00:01:37  nsrd /dev/rmt/STK0.2.10.5 11:Verify label operation in progress
38752 07/02/09 00:01:39  nsrd /dev/rmt/STK0.2.10.5 11:Mount operation in progress
42504 07/02/09 00:01:48  nsrd media event cleared: Waiting for 1 writable volumes to backup pool 'index' tape(s) or disk(s) on
 bolgt03
38718 07/02/09 00:01:48  nsrd bolgt03:index:serceias6bo009 saving to pool 'index' (Q12581)
38718 07/02/09 00:01:48  nsrd bolgt03:index:efbpmb04 saving to pool 'index' (Q12581)
38718 07/02/09 00:01:48  nsrd bolgt03:index:bostn03qd saving to pool 'index' (Q12581)
38718 07/02/09 00:01:48  nsrd bolgt03:index:efbpme06 saving to pool 'index' (Q12581)
0 07/02/09 00:01:51  nsrmmdbd              pools supported: index, SISTMSW0060;
12361 07/02/09 00:01:52  nsrd [Jukebox `CRMF01', operation # 1967]. Finished with status: succeeded
42506 07/02/09 00:01:58  nsrd write completion notice: Writing to volume Q32308 complete
42506 07/02/09 00:02:08  nsrd write completion notice: Writing to volume Q09480 complete
9935 07/02/09 00:02:10  nsrmmdbd Save set ssid:3343637708 cloneid:1246485675 retention time update is before browse time, adju
sting browse time

and i need a script to extract the lines created in the last hour.
Example
I run the script at 15:31 - 07/01/09
i need every line created between the 14:31 - 07/01/09 and now.
But is not certain that there is a line in the log with that hour. The log can be unchanged in the last week or can contain several lines with the same timestamp.
The script need also to run between days and mounth changes example:
run at 00:30 07/01/09 i need every line FROM 23:30 an 06/30/09 Smilie.

Please help me... my head is near to explode.
# 2  
Old 07-02-2009
Welcome to unix.com and Please dont explode. Smilie
search with datecalc and you will get a script that can do the date calculation for you.
use the date command to get the date when you are running the script.
Try 'man date'
use awk to extract the datestamp from the logfile. use datecalc to check the difference between current date and logfile date.
~ Cheers
# 3  
Old 07-02-2009
is not that simple if the date does not appear in the file because in that minute/hour doesn't log ?

---------- Post updated at 06:16 AM ---------- Previous update was at 05:43 AM ----------

Done Smilie
Code:
#!/usr/local/bin/bash
export PATH="$PATH:/usr/local/coreutils/bin"
#07/02/09 12:35:57
ADESSO=$(cdate "+%m/%d/%y %H:%M")
UNORAFA=$(cdate -d "1 hour ago" "+%m/%d/%y %H:%M")
UNORAFA_S=$(cdate -d "1 hour ago" +%s)
echo $ADESSO
echo $UNORAFA
echo $UNORAFA_S
A="0"
while [ "$A" -lt "61" ] ; do
        echo $A
        QUACK=$(grep -n "$UNORAFA" daemon.log|tail -1)
        if [ "$QUACK" = "" ]; then
                let A=$A+1
                let UNORAFA_S=$UNORAFA_S+60
                UNORAFA=$(cdate -d "@$UNORAFA_S" "+%m/%d/%y %H:%M")
                echo devo provare $UNORAFA
        else
                echo TROVATO $UNORAFA
                echo $QUACK
                TOTAIL=$(echo $QUACK |awk -F\: '{print $1}')
                NUMLINES=$(wc -l daemon.log|awk '{print $1}')
                echo $TOTAIL
                echo $NUMLINES
                let TOTAIL=$NUMLINES-$TOTAIL
                tail -$TOTAIL daemon.log > ESTRATTO.txt
                A=61
        fi
done

# 4  
Old 07-02-2009
Quote:
Originally Posted by koma
Please help me... my head is near to explode.
See here for an example using GNU awk.

Last edited by ghostdog74; 07-02-2009 at 08:51 AM..
# 5  
Old 07-02-2009
Quote:
Originally Posted by ghostdog74
See here for an example.
it's gawk specific - not that's bad or anything...
# 6  
Old 07-02-2009
awk: Function systime is not defined.
The source line number is 4.
:|
Mhh

Code:
      A. V. Aho, B. W. Kernighan, P. J. Weinberger: The AWK Programming
      Language, Addison-Wesley, 1988.

 STANDARDS CONFORMANCE
      awk: SVID2, SVID3, XPG2, XPG3, XPG4, POSIX.2

 Hewlett-Packard Company            - 9 -      HP-UX 11i Version 1: Mar 2007

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Display dates between two dates

Hi All, I have 2 dates in mm/dd format. sdate=10/01 (October 01) edate=10/10 (October 10) I need the dates in between these 2 dates like below. 10/01 10/02 10/03 10/04 10/05 10/06 10/07 10/08 (1 Reply)
Discussion started by: jayadanabalan
1 Replies

2. UNIX for Advanced & Expert Users

How to get the Missing dates between two dates in the table?

Hi Am Using Unix Ksh ... I have a Table called date select * from date ; Date 01/02/2013 06/02/2013 I need the output as Missing Date 01/02/2013 02/02/2013 03/02/2013 04/02/2013 05/02/2013 06/02/2013 (2 Replies)
Discussion started by: Venkatesh1
2 Replies

3. Shell Programming and Scripting

Generating dates between two dates

HI, i have row like this HHH100037440313438961000201001012012073110220002 N in this i have 2 dates in pos 25-32 and 33-40 , so based upon the se two dates , i need to generated records between these two values so in the above record 20100101 and 20120731 need to genearte rows like this... (4 Replies)
Discussion started by: sathishsr
4 Replies

4. Shell Programming and Scripting

Extract the lines between two dates

Dear experts I am new bee to scripting... Pl. help me getting the lines between two strings. I am getting the oracle sql output as below. and would like to get the time from system date compare with the date in sql output file and extract the lines. if system time is 2012-07-01 19:15:00 get... (1 Reply)
Discussion started by: nmadhuhb
1 Replies

5. UNIX for Dummies Questions & Answers

How to write the dates between 2 dates into a file

Hi All, I am trying to print the dates that falls between 2 date variables into a file. Here is the example. $BUS_DATE =20120616 $SUB_DATE=20120613 Output to file abc.txt should be : 20120613,20120614,120120615,20120616 Can you pls help me accomplish this in LINUX. Thanks... (5 Replies)
Discussion started by: dsfreddie
5 Replies

6. Emergency UNIX and Linux Support

Replacing dates]] with (dates)]]

Hi guys, For my wiki site I need to fix 1400 pages that use the wrong date format, most pages (not all) use eg. 1988]] I need to change that to (1988)]] The date range goes back to 1400 so I guess I need to do the following ssh into my server, dump mysql database vi .sql dump search... (20 Replies)
Discussion started by: lawstudent
20 Replies

7. Programming

SQL: find if a set od dates falls in another set of dates

Don't know if it is important: Debian Linux / MySQL 5.1 I have a table: media_id int(8) group_id int(8) type_id int(8) expiration date start date cust_id int(8) num_runs int(8) preferred_time int(8) edit_date timestamp ON UPDATE CURRENT_TIMESTAMP id... (0 Replies)
Discussion started by: vertical98
0 Replies

8. UNIX for Dummies Questions & Answers

Extract dates from file name.

Hi All, I have files with names as us_Gec1_wk_01to01_2008.TXT ad_EngEnt_wk_01to10_2008.TXT br_EngMov_wk_01to10_2008.TXT Over here, I need to extract the dates and the year and store them in variables. How can I achieve the same in bash. In case of ad_EngEnt_wk_01to10_2008.TXT ... (3 Replies)
Discussion started by: Swapna173
3 Replies

9. Shell Programming and Scripting

data extract from dates

I have a 1 GB file.I want to take the information for last one month from that file. Say i want to extract data from May 1st. File contents below. 193.135.218.124 - - "GET /jsp/acafai/supplier/notification.htm HTTP/1.1" 200 38531 193.135.218.124 - - "GET /images/2_aca_fai.gif HTTP/1.1"... (5 Replies)
Discussion started by: Krrishv
5 Replies

10. Shell Programming and Scripting

Need script to generate all the dates in DDMMYY format between 2 dates

Hello friends, I am looking for a script or method that can display all the dates between any 2 given dates. Input: Date 1 290109 Date 2 010209 Output: 300109 310109 Please help me. Thanks. :):confused: (2 Replies)
Discussion started by: frozensmilz
2 Replies
Login or Register to Ask a Question