Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Search for logs traced between specific date and time from log file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Search for logs traced between specific date and time from log file
# 1  
Old 01-17-2014
Linux Search for logs traced between specific date and time from log file
HI,

I want to search for a logs which are trace between specific date and time from logs file.

My logs are generated like this :-

Code:
Tue Jun 18 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Thu Jun 20 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Fri Jun 21 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Sat Jun 22 05:00:03 EEST 2013 | file_check.sh| Message:script has files to process.
Sun Jun 23 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Mon Jun 24 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Fri Jun 28 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Sat Jun 29 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Sun Jun 30 05:00:03 EEST 2013 | file_check.sh| Message:script has files to process.
Mon Jul 1 05:00:03 EEST 2013 | file_check.sh| Message:script has files to process.
Tue Jul 2 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Wed Jul 3 05:00:01 EEST 2013 | file_check.sh| Message:script has files to process.
Thu Jul 10 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Thu Jul 16 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Fri Jan 17 04:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 04:05:01 EEST 2014 | file_check.sh| Message:script has files to process. 
Fri Jan 17 05:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 06:05:01 EEST 2014 | file_check.sh| Message:script has files to process.

I want only logs logged between "Fri Jan 17 04:00:00" and "Fri Jan 17 05:50:00"

o/p:-

Code:
Fri Jan 17 04:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 04:05:01 EEST 2014 | file_check.sh| Message:script has files to process. 
Fri Jan 17 05:00:01 EEST 2014 | file_check.sh| Message:script has files to process.

Last edited by Scott; 01-17-2014 at 11:51 AM.. Reason: Removed FONT tags; Added CODE tags. Please STOP posting from Rich Text editors, especially if posting terminal output or code
# 2  
Old 01-17-2014
Code:
$ cat file 
Tue Jun 18 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Thu Jun 20 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Fri Jun 21 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Sat Jun 22 05:00:03 EEST 2013 | file_check.sh| Message:script has files to process.
Sun Jun 23 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Mon Jun 24 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Fri Jun 28 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Sat Jun 29 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Sun Jun 30 05:00:03 EEST 2013 | file_check.sh| Message:script has files to process.
Mon Jul 1 05:00:03 EEST 2013 | file_check.sh| Message:script has files to process.
Tue Jul 2 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Wed Jul 3 05:00:01 EEST 2013 | file_check.sh| Message:script has files to process.
Thu Jul 10 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Thu Jul 16 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Fri Jan 17 04:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 04:05:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 05:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 06:05:01 EEST 2014 | file_check.sh| Message:script has files to process.

Code:
 awk '      BEGIN{
                   split("Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec",M," ")
                   for(i=1;i<=12;i++)Mon[M[i]]=sprintf("%02d",i)
                 }
function dform(v){
                   sub(substr(v,6,3),Mon[substr(v,6,3)],v)
                   gsub(":"," ",v)
                   return mktime(v)
                 }
                 {
                   now = $6" "$2" "$3" "$4
                   if(dform(now)>=dform(start) && dform(now)<=dform(end))
                   print
                 }
     ' start="2014 Jan 17 04:00:00"  end="2014 Jan 17 05:50:00"  file


Resulting
Code:
Fri Jan 17 04:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 04:05:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 05:00:01 EEST 2014 | file_check.sh| Message:script has files to process.

change start and end variable according to your need
These 3 Users Gave Thanks to Akshay Hegde For This Post:
ketanraut RavinderSingh13 srikanth38
# 3  
Old 01-20-2014
Thanks Akshay,its what i am looking for....
Could you please explain me how it works...

---------- Post updated at 04:33 PM ---------- Previous update was at 04:10 PM ----------

Hi Akshay,

my system logs are generating in 24hrs date format.so i want to check, if user input start date =04:00:00 & end=05:00:00 then logs also must serch for start=16:00:00 end=17:00:00 i.e logs are generated for PM.

Please suggest change to implement this.

thanks,
rketan
# 4  
Old 01-20-2014
Quote:
Originally Posted by ketanraut
Thanks Akshay,its what i am looking for....
Could you please explain me how it works...

---------- Post updated at 04:33 PM ---------- Previous update was at 04:10 PM ----------

Hi Akshay,

my system logs are generating in 24hrs date format.so i want to check, if user input start date =04:00:00 & end=05:00:00 then logs also must serch for start=16:00:00 end=17:00:00 i.e logs are generated for PM.

Please suggest change to implement this.

thanks,
rketan


This would be fine then
Code:
$ cat file
Thu Jul 10 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Thu Jul 16 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process.
Fri Jan 17 16:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 16:05:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 04:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 04:05:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 05:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 06:05:01 EEST 2014 | file_check.sh| Message:script has files to process.

Code:
awk '      BEGIN{
                   split("Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec",M," ")
                   for(i=1;i<=12;i++)Mon[M[i]]=sprintf("%02d",i)
                 }
function dform(v){
                   sub(substr(v,6,3),Mon[substr(v,6,3)],v)
                   gsub(":"," ",v)
                   return mktime(v)
                 }
                 {
                   now = $6" "$2" "$3" "$4
                   if(dform(now)>=dform(start) && dform(now)<=dform(end) || \
                      dform(now)>=dform(start)+12*3600 && dform(now)<=dform(end)+12*3600)
                   print
                 }
     ' start="2014 Jan 17 04:00:00"  end="2014 Jan 17 05:50:00"  file

Code:
Fri Jan 17 16:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 16:05:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 04:00:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 04:05:01 EEST 2014 | file_check.sh| Message:script has files to process.
Fri Jan 17 05:00:01 EEST 2014 | file_check.sh| Message:script has files to process.

# 5  
Old 01-20-2014
Could you please explain me how it works...
# 6  
Old 01-20-2014
Quote:
Originally Posted by ketanraut
Could you please explain me how it works...
Code:
awk '      BEGIN{
                   # Awk BEGIN block is special block in which code is executed before executing actual AWK script

                   # Split string "Jan ... Dec " into array M where delimiter is space

                   split("Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec",M," ")  

                   # Array M holds Month string 
                   # Where M[1] = Jan , M[2] = Feb ...M[12] = Dec

                   for(i=1;i<=12;i++)Mon[M[i]]=sprintf("%02d",i)

                   # Since your log month format is integer so created one more array Mon
                   # Where Mon[Jan] = 01, Mon[Feb] = 02...M[Dec] = 12

                 }
function dform(v){
                 
                   # Assume v is 2014 Jan 10 10:10:10

                   # Substitute 01 for Month Jan 
                   sub(substr(v,6,3),Mon[substr(v,6,3)],v)       

                   # After the execution of above statement v becomes  2014 01 10 10:10:10

                   # Replace colon with space in variable v
                   gsub(":"," ",v)
                   
                   # So now v becomes 2014 01 10 10 10 10
                   
                   # Function mktime returns timestamp in the same form as is returned by systime()

                   # Return timestamp 
                   return mktime(v)

                 }
                 {
                   # now = Year <space> Month <space> Day <space> Time
                   now = $6" "$2" "$3" "$4

                   # Here we are calling function dform

                   # Example dform(now) = dform(2014 Jan 10 10:10:10)

                   # Here goes comparison against input specified in start and end variable 

                   # timestamp+12*3600 is added since you want to search both AM and PM in single querry 
                 
                   # If condition satisfied then it prints line or row

                    if(dform(now)>=dform(start) && dform(now)<=dform(end) || \
                      dform(now)>=dform(start)+12*3600 && dform(now)<=dform(end)+12*3600)
                       print
 
                 }
     ' start="2014 Jan 17 04:00:00"  end="2014 Jan 17 05:50:00"  file

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Grep a log file starting from a specific time to the end of file

I have a log file which have a date and time at the start of every line. I need to search the log file starting from a specific time to the end of file. For example: Starting point: July 29 2018 21:00:00 End point : end of file My concern is what if the pattern of `July 29 2018 21:00:00`... (3 Replies)
Discussion started by: erin00
3 Replies

2. UNIX for Dummies Questions & Answers

Search specific string logfile specific date range

Hi, I have logfile like this.. === 2014-02-09 15:46:59,936 INFO RequestContext - URL: '/eyisp/sc/skins/EY/images/pickers/comboBoxPicker_Over.png', User-Agent: 'Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko': Unsupported with Accept-Encoding header === 2015-02-09... (8 Replies)
Discussion started by: kishk
8 Replies

3. UNIX for Dummies Questions & Answers

Search for a specific String in a log file for a specific date range

Hi, I have log file which rolls out every second which is as this. HttpGenRequest - -<!--OXi dbPublish--> <created="2014-03-24 23:45:37" lastMsgId="" requestTime="0.0333"> <response request="getOutcomeDetails" code="114" message="Request found no matching data" debug="" provider="undefined"/>... (3 Replies)
Discussion started by: karthikprakash
3 Replies

4. UNIX for Advanced & Expert Users

AIX idea needed to check the logs updated date and time

Hi with the help of Gabriel canepa, i have just edited filename only in his code. The help which i got and he helped is 1) I have around 22 logs and each log should be updated in the last 24 hours from the current timestamp. 2) It should check for ERROR message (not error,Error) in the log and... (2 Replies)
Discussion started by: Kalaihari
2 Replies

5. Shell Programming and Scripting

Script to Search Logs Several Directories Pulling out Only Current Date

Hi All.. I'm seeking assistance with editing a script to search log files in several directories. I'm close to what I'm seeking, but need additional guidance. The log files are always listed by current date; however, inside the log file includes dates that go back to 2011. What I'm... (6 Replies)
Discussion started by: lenaf7
6 Replies

6. Shell Programming and Scripting

Search for a specific data in a file based on a date range

Hi, Currently I am working on a script to automate the process of converting the log file from binary into text format. To achieve this, partly I am depending on my application’s utility for this conversion and the rest I am relying on shell commands to search for directory, locate the file and... (5 Replies)
Discussion started by: svajhala
5 Replies

7. Linux

search on weblogic logs with date time ranges

Hi All, The developers want me to search and capture the weblogic log, you know this big logs of htmls. They want to me to have ranges on the date and time. Like from "2010-01-20 14:04:46,186" to "2010-01-20 15:00:12,490" I can only do this, cat /usr/local/bea/logs_prod1/debug.log... (1 Reply)
Discussion started by: itik
1 Replies

8. Shell Programming and Scripting

search on weblogic logs with date time ranges 2

Hi All, The developers want me to search and capture the weblogic log, you know this big logs of htmls. They want to me to have ranges on the date and time. Like from "2010-01-20 14:04:46,186" to "2010-01-20 15:00:12,490" I can only do this, cat /usr/local/bea/logs_prod1/debug.log |... (1 Reply)
Discussion started by: itik
1 Replies

9. Shell Programming and Scripting

Get Data Between a specific Date Range from logs

I need to extract data from logs for a mentioned date range..Its quite urgent can anyone help me out with it..its to be written in unix..just thought its better to specify.. (4 Replies)
Discussion started by: sankasu
4 Replies

10. Shell Programming and Scripting

Processing a log file based on date/time input and the date/time on the log file

Hi, I'm trying to accomplish the following and would like some suggestions or possible bash script examples that may work I have a directory that has a list of log files that's periodically dumped from a script that is crontab that are rotated 4 generations. There will be a time stamp that is... (4 Replies)
Discussion started by: primp
4 Replies
Login or Register to Ask a Question