awk : Search for text between two time frame (12 hours)


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk : Search for text between two time frame (12 hours)
# 1  
Old 11-18-2013
awk : Search for text between two time frame (12 hours)

I have created the script to grep the errors from weblogic logs files and redirecting output to file.txt ...From file.txt I'm using awk command to collect the past 20 mins output...The script running from cron every 15 mins... The script working well...

Now the challenges, I'm trying to use this same script for different application but that log files format is 12 hours so need your help to use awk or sed command to collect the past 15 mins from 12 hours date format.

The log files 12 hours format (ex- Nov 18, 2013 9:50:16 AM UTC)
# 2  
Old 11-18-2013
What's your system?
# 3  
Old 11-18-2013
Redhat Linux.

---------- Post updated at 01:50 PM ---------- Previous update was at 01:48 PM ----------

awk - since its 12 hours format so I'm facing issue "AM, PM" and in case if time stamp (from=9:40" and to=10:00" then I'm not receiving any output

I tried the following but no luck...
Code:
awk '$0 >= "Nov 14, 2013 9:40:01" && $0 <= "Nov 14, 2013 9:55:01"' file

Code:
sed -n '/Nov 14, 2013 7:58:00 PM UTC/,/Nov 14, 2013 8:10:00 PM UTC/p' file

---------- Post updated at 01:51 PM ---------- Previous update was at 01:50 PM ----------

Corona688 - Thanks for the reply, please let me know if you need any other details

Last edited by Franklin52; 11-18-2013 at 05:04 PM.. Reason: Please use code tags
# 4  
Old 11-18-2013
Don't bump posts if I don't answer. I was writing. I'm certainly not shy of asking for more info if needed

Your request is possible, if awkward. <= >= compare ASCII strings alphabetically... YYYY-MM-DD HH:MM:SS dates actually do sort alphabetically, though. You need to convert Nov into month number, AM/PM into 24-hour time to accomplish this.

Code:
awk -F'[ ,:]+' 'BEGIN {
        # Build tables so MON["Jan"] becomes 1, etc.
        split("Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec", M);
        for(X in M) MON[M[X]]=X
        for(X=1; X<=11; X++) { T[X"AM"]=X ; T[X"PM"]=X+12 }
        T["12PM"]=12;        T["12AM"]=24; }

{ YYMMDD=sprintf("%04d %02d %02d %02d %02d %02d", $3, MON[$1], $2, T[$4 $7], $5, $6)); }

# YYMMDD is a timestamp directly comparable with <= >= against other YYMMDD dates.

...'


Last edited by Corona688; 11-18-2013 at 03:14 PM..
# 5  
Old 11-18-2013
Thanks Corona688.

1. so would you suggesting to covert the output (ex-file.txt) from 12 hours to 24 hours ? It seems file.txt size is 5 mb, is it any other possible way to collect the logs between two time frame without converting the format?

2. worst case, If I go for converting the time format, do I need to create two different script?
a, grep the errors from log files and redirect the output (file.txt)
b, converting 12 hours to 24 hours format ?
c, awk '$0 >= "from" && $0 <= "to"

Sorry I'm little confused, could you please explain more on this ?
# 6  
Old 11-18-2013
No, I was suggesting adapting your existing code using the code I gave you so it can handle the kind of dates in your log file. Comparing the YYYYMMDD variable and your input time instead of comparing raw lines. No need to save a new file.
# 7  
Old 11-18-2013
sorry I'm not able to catch Smilie can you provide some example please ?

My script

Code:
#!/bin/bash
#set -xv
# The script to verify the error from application logs
# Please don't edit this file

#removing the old scripting logs
rm /home/mydir/script/file.txt
rm /home/mydir/script/output

#date based on the log format
date=`date "+%b %-d, %Y"`

#Log Details
log1=application log file1*
log2=application log file2*
log3=application log file3*

#checking logs
more $log1 | grep -i "$date" | egrep -i 'error1 | error2 | error3 | error4 | error5' >> /home/mydir/script/file.txt
more $log2 | grep -i "$date" | egrep -i 'error1 | error2 | error3 | error4 | error5' >> /home/mydir/script/file.txt
more $log3 | grep -i "$date" | egrep -i 'error1 | error2 | error3 | error4 | error5' >> /home/mydir/script/file.txt

# Time stamp (based on the log format)

tot=`date "+%r %Z" | sed 's/^0//'`
to=`date "+%b %d, %Y"`

frmt=`date -d "-20 minutes" "+%r %Z" | sed 's/^0//'`
from=`date "+%b %-d, %Y"`

#collecting log output between two time stamp

awk '$0>=from && $0<=to' from="$from $frmt" to="$to $tot" /home/mydir/script/file.txt > /home/mydir/script/output

#email

email script

# END


log file format :
Code:
###<Nov 6, 2013 8:30:23 PM UTC> <Notice> <stoutdf> <host name> < <[ACTIVE] ExecuteThread:


Last edited by Franklin52; 11-19-2013 at 05:30 AM.. Reason: Please use code tags
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Need to filter the result set within 2 time frame

my sample file is like this $cat onefile 05/21/18 13:10:07 ABRT US1CPDAY Status 1 05/21/18 21:18:54 ABRT DailyBackup_VFFPRDAPENTL01 Status 6 05/21/18 21:26:24 ABRT DailyBackup_VFFPRDAPENTL02 Status 6 05/21/18 21:57:36 ABRT DailyBackup_vm-ea1ffpreng01 Status 6... (7 Replies)
Discussion started by: gotamp
7 Replies

2. Shell Programming and Scripting

Help on script to capture info on log file for a particular time frame

Hi I have a system running uname -a Linux cmovel-db01 2.6.32-38-server #83-Ubuntu SMP Wed Jan 4 11:26:59 UTC 2012 x86_64 GNU/Linux I would like to capture the contents of /var/log/syslog from 11:00AM to 11:30AM and sent to this info via email. I was thinking in set a cron entry at that... (2 Replies)
Discussion started by: fretagi
2 Replies

3. Shell Programming and Scripting

Awk:String search more than one time and capture OP

Dear All During one of mine script developemnt i am stuch at one sub part. Requiremnt is as below kindly help me. IP file: 2015-02-28 10:10:15 AL M UtranCell UtranCell=RTE001X (unavailable) 2015-02-28 10:10:15 AL M UtranCell UtranCell=RTE001Y (unavailable) 2015-02-28 10:10:15 AL M... (6 Replies)
Discussion started by: jaydeep_sadaria
6 Replies

4. UNIX for Dummies Questions & Answers

UNIX Account getting Locked Everyday between same Time Frame

I am facing an Issue with a particular Unix Account ( ie a particular Userid) getting LOCKED everyday between 7:30am and 8:00am. The Password associated with this particular Account has been setup such that it should never Expire at all but it does LOCK the Account after more than 3 failed... (5 Replies)
Discussion started by: pchegoor
5 Replies

5. Shell Programming and Scripting

awk : collecting all data between two time frame

Hi Experts , I need your help to collect the complete data between two time frame from the log files, when I try awk it's collecting the data only which is printed with time stamp for example, awk works well from "16:00 to 17:30" but its not collecting <line*> "from 17:30 to 18:00" ... (8 Replies)
Discussion started by: zenkarthi
8 Replies

6. Shell Programming and Scripting

Log search and mail it if the log is updated before 24 hours from the current time

Hi , We have around 22 logs , each has different entries. I have to automate this using shell script. The ideas which am sharing is given below 1) We use only TAIL -100 <location and name of the log> Command to check the logs. 2) We want to check whether the log was updated before 24... (13 Replies)
Discussion started by: Kalaihari
13 Replies

7. Shell Programming and Scripting

Perl : unable to frame code with Business:Hours module with example

Hi all , There are two datetimes in the below format $datetime1="2007-12-31 16:50:00"; $datetime2="2008-1-1 04:24:00"; I need to calculate time in minutes only in between the office hours.(08:00 to 18:00 ). in PERL. For the above requirement I found a module Business::Hours that... (4 Replies)
Discussion started by: scriptscript
4 Replies

8. Shell Programming and Scripting

Search for text between two time frame using sed

I have log files with time stamps. I want to search for text between two time stamp using sed even if the first tme stamp or the last time stamp are not present. For e.g. if i search between 9:30 and 9:40 then it should return text even if 9:30 or 9:40 is not there but between 9:30 and 9:40 is... (8 Replies)
Discussion started by: Azher
8 Replies

9. Shell Programming and Scripting

Shell Script to delete files within a particular time frame under multiple sub folders

Greetings! I'm looking for starting information for a shell script. Here's my scenario: I have multiple folders(100) for example: /www/test/applications/app1/logs /www/test/applications/app2/logs Within these folders there are log files files that need to be deleted after a month. ... (3 Replies)
Discussion started by: whysolucky
3 Replies

10. Shell Programming and Scripting

search string during a specific time frame

Can someone please help me with searching a string during a specific time frame. Below is the format of the time from my log file. "GET /AAM2009_wherewereheaded.wmv HTTP/1.1" 200 52307085 The search string I need is "AAM2009_wherewereheaded.wmv" I need to search the number of... (1 Reply)
Discussion started by: tadi18
1 Replies
Login or Register to Ask a Question