Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk : collecting all data between two time frame

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk : collecting all data between two time frame
# 8  
Old 01-19-2014
This would be fine, uses current row date
Code:
$ cat log
2014-01-18 16:00:49,624 DEBUG [testhandler] XMLObjectType: S=xyz@http://www.xyz.org/2000/XMLDSchema
2014-01-18 16:10:49,624 DEBUG [testhandler] XMLObjectType: S=xyz@http://www.xyz.org/2000/XMLDSchema
2014-01-18 16:30:49,624 DEBUG [testhandler] XMLObjectType: S=xyz@http://www.xyz.org/2000/XMLDSchema
2014-01-18 16:50:49,624 DEBUG [testhandler] XMLObjectType: S=xyz@http://www.xyz.org/2000/XMLDSchema
2014-01-18 17:00:49,624 DEBUG [testhandler] XMLObjectType: S=xyz@http://www.xyz.org/2000/XMLDSchema
2014-01-18 17:30:49,624 DEBUG [testhandler] XMLObjectType: S=xyz@http://www.xyz.org/2000/XMLDSchema
<line1="http://G2.rc.xyz.org">
  <line2>
    <line3>
    <line5>
  </line6>
2014-01-18 17:50:49,624 DEBUG [test123]
User: data.production@test.com
<line1="http://G2.rc.xyz.org">
  <line2>
    <line3>
    <line5>
  </line6>
  <line7>
  <line8>
2014-01-18 18:00:49,624 DEBUG [testhandler] XMLObjectType: S=xyz@http://www.xyz.org/2000/XMLDSchema
2014-01-18 18:10:49,624 DEBUG [testhandler] XMLObjectType: S=xyz@http://www.xyz.org/2000/XMLDSchema

Code:
awk -F, '
function dform(v){
                    gsub(/[-:]/," ",v)
                    return mktime(v)
                 }   
    dform($1)!=-1{
                    s = substr($1,1,10)" "start
                    e = substr($1,1,10)" "end 
                    f = dform($1)>=dform(s) && dform($1) <= dform(e) ? 1 : 0 
                 }f
         ' start="17:30:49" end="18:00:59" log

Resulting
Code:
2014-01-18 17:30:49,624 DEBUG [testhandler] XMLObjectType: S=xyz@http://www.xyz.org/2000/XMLDSchema
<line1="http://G2.rc.xyz.org">
  <line2>
    <line3>
    <line5>
  </line6>
2014-01-18 17:50:49,624 DEBUG [test123]
User: data.production@test.com
<line1="http://G2.rc.xyz.org">
  <line2>
    <line3>
    <line5>
  </line6>
  <line7>
  <line8>
2014-01-18 18:00:49,624 DEBUG [testhandler] XMLObjectType: S=xyz@http://www.xyz.org/2000/XMLDSchema

--edit---

In your requirement you didn't mention whether to consider date as well along with time or not

if date also you want to consider in query use following one, following will be helpful if log file is appended with date and time everyday.

Code:
awk -F, '
function dform(v){
                    gsub(/[-:]/," ",v)
                    return mktime(v)
                 }  
           NR==1 {
                    start = dform(start)
                     end  = dform(end)
                 }  
    dform($1)!=-1{     
                    f = dform($1)>=start && dform($1) <= end ? 1 : 0 
                 }f
        ' start="2014-01-18 17:30:49" end="2014-01-18 18:00:50" log

Last edited by Akshay Hegde; 01-19-2014 at 11:27 AM.. Reason: Note to thread poster
This User Gave Thanks to Akshay Hegde For This Post:
zenkarthi
# 9  
Old 01-19-2014
You could something like this to get entries of the last 15 minutes:
Code:
perl -MPOSIX -lane '
  BEGIN{
    $c=strftime("%Y%m%d%H%M%S",localtime(time-15*60))
  }
  if(/^....-..-/) {
    $t=$F[0].$F[1];
    $t =~ s/[-:]|,.*//g
  }
  print if ($t ge $c) .. eof()
' file

or

Code:
perl -MPOSIX -slane '
  BEGIN{
    $c=strftime("%Y%m%d%H%M%S",localtime(time-$min*60))
  }
  if(/^....-..-/) {
    $t=$F[0].$F[1];
    $t =~ s/[-:]|,.*//g
  }
  print if ($t ge $c) .. eof()
' -- -min=15

Last edited by Scrutinizer; 01-19-2014 at 12:34 PM..
This User Gave Thanks to Scrutinizer For This Post:
zenkarthi
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Need to filter the result set within 2 time frame

my sample file is like this $cat onefile 05/21/18 13:10:07 ABRT US1CPDAY Status 1 05/21/18 21:18:54 ABRT DailyBackup_VFFPRDAPENTL01 Status 6 05/21/18 21:26:24 ABRT DailyBackup_VFFPRDAPENTL02 Status 6 05/21/18 21:57:36 ABRT DailyBackup_vm-ea1ffpreng01 Status 6... (7 Replies)
Discussion started by: gotamp
7 Replies

2. Shell Programming and Scripting

Collecting all lines between two time stamp from the log

Can you help me to collect the entire logs between two time stamp. The below awk command collecting the logs only if the line has time stamp. awk '$0>=from && $0<=to' from="150318 23:19:04" to="150318 23:55:04" log file 150318 23:19:04 logentries 150318 23:29:04 logentries 150318... (11 Replies)
Discussion started by: zenkarthi
11 Replies

3. Shell Programming and Scripting

Collecting logs between two time stamps

Hi, please help me to collect the entire log files between two time stamp. for example, I am looking script to collect the entire log between "2015-03-27 15:59" to "2015-03-27 16:15" in the below sample log file. OS : RHEL 6.3 Date/Time : 24 hours format, the time is printing each log... (12 Replies)
Discussion started by: jerryknj
12 Replies

4. Programming

Find gaps in time data and replace missing time value and column 2 value by interpolation in awk

Dear all, I am kindly seeking assistance on the following issue. I am working with data that is sampled every 0.05 hours (that is 3 minutes intervals) here is a sample data from the file 5.00000 15.5030 5.05000 15.6680 5.10000 16.0100 5.15000 16.3450 5.20000 16.7120 5.25000... (4 Replies)
Discussion started by: malandisa
4 Replies

5. Shell Programming and Scripting

awk : Search for text between two time frame (12 hours)

I have created the script to grep the errors from weblogic logs files and redirecting output to file.txt ...From file.txt I'm using awk command to collect the past 20 mins output...The script running from cron every 15 mins... The script working well... Now the challenges, I'm trying to use... (27 Replies)
Discussion started by: zenkarthi
27 Replies

6. Shell Programming and Scripting

Search for text between two time frame using sed

I have log files with time stamps. I want to search for text between two time stamp using sed even if the first tme stamp or the last time stamp are not present. For e.g. if i search between 9:30 and 9:40 then it should return text even if 9:30 or 9:40 is not there but between 9:30 and 9:40 is... (8 Replies)
Discussion started by: Azher
8 Replies

7. Shell Programming and Scripting

Removing repeating lines from a data frame (AWK)

Hey Guys! I have written a code which combines lots of files into one big file(.csv). However, each of the original files had headers on the first line, and now that I've combined the files the headers are interspersed throughout the new combined data frame. For example, throughout the data... (21 Replies)
Discussion started by: gd9629
21 Replies

8. Shell Programming and Scripting

search string during a specific time frame

Can someone please help me with searching a string during a specific time frame. Below is the format of the time from my log file. "GET /AAM2009_wherewereheaded.wmv HTTP/1.1" 200 52307085 The search string I need is "AAM2009_wherewereheaded.wmv" I need to search the number of... (1 Reply)
Discussion started by: tadi18
1 Replies

9. Shell Programming and Scripting

Need help collecting addational data

I am attempting to gather data from 700 + routers. I have written a script that gets me about 70% of the information I need but I am in need of some assistance getting the remainder. I am fairly new to unix and I have not done programming since my 2nd year of high school in 1992. I have a list... (0 Replies)
Discussion started by: Garlandxj
0 Replies

10. UNIX for Dummies Questions & Answers

Collecting data from TOP to a file

Is there a way to collect data from top command to a file at regular intervals. I need this on HP-UX. I need to gather resident memory usage. I know that sar will give %cpu usage. There is a pmap command which gives memory usage in solaris. Is there a similar command in HP-UX Thanks (5 Replies)
Discussion started by: sssow
5 Replies
Login or Register to Ask a Question