Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Narrow down a log the log file for certain period

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Narrow down a log the log file for certain period
# 1  
Old 09-16-2011
Bug Narrow down a log the log file for certain period
Dear friends,

Please help me out to narrow down a log file with the time range. The log file will be in the following format,

213.64.141.89 - - [26/Oct/2002:18:55:21 +0200]
213.64.248.179 - - [26/Oct/2002:21:02:19 +0200]
213.64.248.179 - - [26/Oct/2002:21:02:23 +0200]

What are shell commands to narrow the file for certain hours or days. I think this can be done with the date arithmetic. If I am right, please let me know how can do arithmetic in date values.

Thank in advance,
Tamil Pamaran
# 2  
Old 09-16-2011
Your requirement is not very clear.

For what I understood, a simple "grep" would produce the desired output.

To display a specific day:
Code:
grep '26/Oct/2002' File

To display a specific hour (7) in a day (26/Oct/2002):
Code:
grep '26/Oct/2002:07' File

Is this what you are looking for?
# 3  
Old 09-16-2011
Thanks for your response,

Basically I am looking for a mechanism to extract the log that fall under a time range.

start time
-----
-----
-----
End time

Date format is given in my first post.

Thank You
# 4  
Old 09-16-2011
You need nawk or gawk, not plain awk.
Code:
nawk -v SDATE="26/Oct/2002:18:55:21" -v EDATE="26/Oct/2002:21:02:19" 'BEGIN {
        # Set up arrays for name-to-monthnumber
        split("Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec", MON, "|");
        for(N=1; N<=12; N++) MNUM[MON[N]]=N;

        split(SDATE, D, "[:/]");
        # Convert "YYYY MM DD HH MM SS" into epoch time
        SDATE=mktime(D[3] " " MNUM[D[2]] " " D[1] " " D[4] " " D[5] " " D[6]);
        split(EDATE, D, "[:/]");
        EDATE=mktime(D[3] " " MNUM[D[2]] " " D[1] " " D[4] " " D[5] " " D[6]);
}

{       if(NF > 1)
        {
                split($(NF-1), D, "[\\[:/]");
                DATE=mktime(D[4] " " MNUM[D[3]] " " D[2] " " D[5] " " D[6] " " D[7]);

                if((DATE >= SDATE) && (DATE <= EDATE)) print;
        } }' < datafile

# 5  
Old 09-16-2011
All the nawks that I know don't have the mktime built-in function. But Corona688's solution works without mktime (just omit mktime and the surrounding parentheses) since the date values are in "yyyy mm dd hh mi ss" format which can be compared natively (you can omit all the spaces). mktime is needed if you want to compare two dates to see e.g. if they are within 10 seconds.

With a little modification:
Code:
for(N=1; N<=12; N++) MNUM[MON[N]]=sprintf("%02d", N);

Note, to compare the dates, all you need to do is arrange the values in year, month, day, hour, minute, second format. The "/"s and the ":"s don't matter as long as the digits are all the same widths and months are in numeric formats.
Last edited by binlib; 09-17-2011 at 09:30 AM.. Reason: Make month number two digits always.
These 2 Users Gave Thanks to binlib For This Post:
Corona688 tamil.pamaran
# 6  
Old 09-18-2011
Quote:
Originally Posted by binlib
All the nawks that I know don't have the mktime built-in function. But Corona688's solution works without mktime (just omit mktime and the surrounding parentheses) since the date values are in "yyyy mm dd hh mi ss" format which can be compared natively (you can omit all the spaces).
Natively, as in, alphabetically? Clever. Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Web Development

How to log http 404 error to a separate log file?

Apache Web Server: how to log http 404 error to a separate log file and i do not want to log in access.log please advice. (2 Replies)
Discussion started by: raghur77
2 Replies

2. Solaris

Log error from cron job into log file

Hi Please would it be right to log the errors from a script running in cron in the following manner: /datax/vendor/dump.sh > /datax/vendor/err.log 2>&1 (16 Replies)
Discussion started by: fretagi
16 Replies

3. Shell Programming and Scripting

Check log file size every 10 minute. Alert if log not update

How to check log size every 10min. by script (can use crontab) if log size not change with alert "Log not update" Base run on SunOS 5.8 Generic_Virtual sun4u sparc SUNW,SPARC-Enterprise logFiles="log1.log log2.log" logLocation="/usr/home/test/log/" Out put. Tue Jan 31... (3 Replies)
Discussion started by: ooilinlove
3 Replies

4. Solaris

Changing log rotation period

Hi, I would like to change log roattion period. Please suggest which configuration file needs to be changed? (1 Reply)
Discussion started by: manoj.solaris
1 Replies

5. HP-UX

Script to monitor /var/opt/resmon/log/event.log file

AM in need of some plugin/script that can monitor HP-UX file "/var/opt/resmon/log/event.log" . Have written a scrip in sh shell that is working fine for syslog.log and mail.log as having standard format, have interrogated that to Nagios and is working as I required . But same script failed to... (3 Replies)
Discussion started by: Shirishlnx
3 Replies

6. Shell Programming and Scripting

Perl's buffered I/O is causing me to miss latest log file entries in log colorizer. How to fix?

I've been finding myself using a log file colorizer written in perl to reformat and colorize the output from many different programs. Mainly, however, I use it to make the output from "tail -f" commands more readable. The base perl script I use is based on "colorlogs.pl" available from the... (1 Reply)
Discussion started by: rcsteiner
1 Replies

7. Shell Programming and Scripting

Capture all error message in Log file and send the Log file by email

Hi I have a requirement to write a script to capture all errors in a Logfile and send the file in email. If there is any error occurred the subject of email will be ERROR , If there are no error occurred the subject of email will be SUCCESS. So I created a Log file and put the Appropriate... (2 Replies)
Discussion started by: dgmm
2 Replies

8. Shell Programming and Scripting

Delete log file entries based on the Date/Timestamp within log file

If a log file is in the following format 28-Jul-10 ::: Log message 28-Jul-10 ::: Log message 29-Jul-10 ::: Log message 30-Jul-10 ::: Log message 31-Jul-10 ::: Log message 31-Jul-10 ::: Log message 1-Aug-10 ::: Log message 1-Aug-10 ::: Log message 2-Aug-10 ::: Log message 2-Aug-10 :::... (3 Replies)
Discussion started by: vikram3.r
3 Replies

9. HP-UX

narrow down reply from cmviewcl

Hi there all, I have a little problem I got a package called epdp and I got 1 called pd (wich is Data Protector) If I am writing a script to get the status of packages. how can I get only the reply from dp? Now if I do cmviewcl -v |grep pdI get epdp and pd. (2 Replies)
Discussion started by: draco
2 Replies

10. Shell Programming and Scripting

alert_oss.log oracle log file in hp-unix

Hi I have go this alert_oss.log that is basically capturing all the oracle errorlogs.Now the problem is that it is one huge file and to see log of some particular date i tried cat alert_oss.log | grep 'Mar 25 10:44:45 2007' > alert_25.txt is not giving me the required output. pls suggest ... (3 Replies)
Discussion started by: Assassin
3 Replies
Login or Register to Ask a Question