Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk : collecting all data between two time frame

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk : collecting all data between two time frame
# 1  
Old 01-18-2014
awk : collecting all data between two time frame
Hi Experts ,

I need your help to collect the complete data between two time frame from the log files, when I try awk it's collecting the data only which is printed with time stamp
for example, awk works well from "16:00 to 17:30" but its not collecting <line*> "from 17:30 to 18:00"

Requirement : The script will run every 15 mins from cron to collect the past 15 mins data then it will grep the error, the error will print between line1 to line8 also

Log file : 24 hours format
OS : Linux 6.2
Log file format:
Code:
2014-01-18 16:00:49,624 DEBUG [testhandler] XMLObjectType: S=xyz@http://www.xyz.org/2000/XMLDSchema
2014-01-18 16:10:49,624 DEBUG [testhandler] XMLObjectType: S=xyz@http://www.xyz.org/2000/XMLDSchema
2014-01-18 16:30:49,624 DEBUG [testhandler] XMLObjectType: S=xyz@http://www.xyz.org/2000/XMLDSchema
2014-01-18 16:50:49,624 DEBUG [testhandler] XMLObjectType: S=xyz@http://www.xyz.org/2000/XMLDSchema
2014-01-18 17:00:49,624 DEBUG [testhandler] XMLObjectType: S=xyz@http://www.xyz.org/2000/XMLDSchema
2014-01-18 17:30:49,624 DEBUG [testhandler] XMLObjectType: S=xyz@http://www.xyz.org/2000/XMLDSchema
<line1="http://G2.rc.xyz.org">
  <line2>
    <line3>
    <line5>
  </line6>
2014-01-18 17:50:49,624 DEBUG [test123]
User: data.production@test.com
<line1="http://G2.rc.xyz.org">
  <line2>
    <line3>
    <line5>
  </line6>
  <line7>
  <line8>
2014-01-18 18:00:49,624 DEBUG [testhandler] XMLObjectType: S=xyz@http://www.xyz.org/2000/XMLDSchema
2014-01-18 18:10:49,624 DEBUG [testhandler] XMLObjectType: S=xyz@http://www.xyz.org/2000/XMLDSchema

Last edited by zenkarthi; 01-18-2014 at 12:12 PM..
# 2  
Old 01-18-2014
Code:
sed -n '/17:30/,/18:00/p' file

# 3  
Old 01-18-2014
what is expected output ?
# 4  
Old 01-18-2014
balajesuri - I think sed will not help here, in case if the log error starting from "17:05" and script collecting the details from 17:30 then what is the output....


Akshay - The expected output

Code:
2014-01-18 17:30:49,624 DEBUG [testhandler] XMLObjectType: S=xyz@http://www.xyz.org/2000/XMLDSchema
<line1="http://G2.rc.xyz.org">
  <line2>
    <line3>
    <line5>
  </line6>
2014-01-18 17:50:49,624 DEBUG [test123]
User: data.production@test.com
<line1="http://G2.rc.xyz.org">
  <line2>
    <line3>
    <line5>
  </line6>
  <line7>
  <line8>
2014-01-18 18:00:49,624 DEBUG [testhandler] XMLObjectType: S=xyz@http://www.xyz.org/2000/XMLDSchema

Last edited by zenkarthi; 01-18-2014 at 04:12 PM..
# 5  
Old 01-19-2014
Quote:
Originally Posted by zenkarthi
I think sed will not help here, in case if the log error starting from "17:05" and script collecting the details from 17:30 then what is the output
I am not able to understand here.. If you're talking about why I hardcoded the time into the sed command, then I'd say that my post is merely a nudge to help you get started Smilie
# 6  
Old 01-19-2014
@zenkranti: balajesuri's sed is producing expected output.
# 7  
Old 01-19-2014
@balajesuri,akshay, you can't use sed like that here, since it might just as well be that there is no log entry for 17:30 for example, but let's say 17:32 and then it won't work at all..
Quote:
Requirement : The script will run every 15 mins from cron to collect the past 15 mins data then it will grep the error, the error will print between line1 to line8 also
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Need to filter the result set within 2 time frame

my sample file is like this $cat onefile 05/21/18 13:10:07 ABRT US1CPDAY Status 1 05/21/18 21:18:54 ABRT DailyBackup_VFFPRDAPENTL01 Status 6 05/21/18 21:26:24 ABRT DailyBackup_VFFPRDAPENTL02 Status 6 05/21/18 21:57:36 ABRT DailyBackup_vm-ea1ffpreng01 Status 6... (7 Replies)
Discussion started by: gotamp
7 Replies

2. Shell Programming and Scripting

Collecting all lines between two time stamp from the log

Can you help me to collect the entire logs between two time stamp. The below awk command collecting the logs only if the line has time stamp. awk '$0>=from && $0<=to' from="150318 23:19:04" to="150318 23:55:04" log file 150318 23:19:04 logentries 150318 23:29:04 logentries 150318... (11 Replies)
Discussion started by: zenkarthi
11 Replies

3. Shell Programming and Scripting

Collecting logs between two time stamps

Hi, please help me to collect the entire log files between two time stamp. for example, I am looking script to collect the entire log between "2015-03-27 15:59" to "2015-03-27 16:15" in the below sample log file. OS : RHEL 6.3 Date/Time : 24 hours format, the time is printing each log... (12 Replies)
Discussion started by: jerryknj
12 Replies

4. Programming

Find gaps in time data and replace missing time value and column 2 value by interpolation in awk

Dear all, I am kindly seeking assistance on the following issue. I am working with data that is sampled every 0.05 hours (that is 3 minutes intervals) here is a sample data from the file 5.00000 15.5030 5.05000 15.6680 5.10000 16.0100 5.15000 16.3450 5.20000 16.7120 5.25000... (4 Replies)
Discussion started by: malandisa
4 Replies

5. Shell Programming and Scripting

awk : Search for text between two time frame (12 hours)

I have created the script to grep the errors from weblogic logs files and redirecting output to file.txt ...From file.txt I'm using awk command to collect the past 20 mins output...The script running from cron every 15 mins... The script working well... Now the challenges, I'm trying to use... (27 Replies)
Discussion started by: zenkarthi
27 Replies

6. Shell Programming and Scripting

Search for text between two time frame using sed

I have log files with time stamps. I want to search for text between two time stamp using sed even if the first tme stamp or the last time stamp are not present. For e.g. if i search between 9:30 and 9:40 then it should return text even if 9:30 or 9:40 is not there but between 9:30 and 9:40 is... (8 Replies)
Discussion started by: Azher
8 Replies

7. Shell Programming and Scripting

Removing repeating lines from a data frame (AWK)

Hey Guys! I have written a code which combines lots of files into one big file(.csv). However, each of the original files had headers on the first line, and now that I've combined the files the headers are interspersed throughout the new combined data frame. For example, throughout the data... (21 Replies)
Discussion started by: gd9629
21 Replies

8. Shell Programming and Scripting

search string during a specific time frame

Can someone please help me with searching a string during a specific time frame. Below is the format of the time from my log file. "GET /AAM2009_wherewereheaded.wmv HTTP/1.1" 200 52307085 The search string I need is "AAM2009_wherewereheaded.wmv" I need to search the number of... (1 Reply)
Discussion started by: tadi18
1 Replies

9. Shell Programming and Scripting

Need help collecting addational data

I am attempting to gather data from 700 + routers. I have written a script that gets me about 70% of the information I need but I am in need of some assistance getting the remainder. I am fairly new to unix and I have not done programming since my 2nd year of high school in 1992. I have a list... (0 Replies)
Discussion started by: Garlandxj
0 Replies

10. UNIX for Dummies Questions & Answers

Collecting data from TOP to a file

Is there a way to collect data from top command to a file at regular intervals. I need this on HP-UX. I need to gather resident memory usage. I know that sar will give %cpu usage. There is a pmap command which gives memory usage in solaris. Is there a similar command in HP-UX Thanks (5 Replies)
Discussion started by: sssow
5 Replies
Login or Register to Ask a Question