Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Retrieve logs for last 4 hours

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Retrieve logs for last 4 hours
# 1  
Old 11-19-2010
Retrieve logs for last 4 hours
Our log file looks like this:
Code:
2010-11-18 00:57:01,698                  ERROR
Shipment Error Log:Error Details - Account Id:3962, PO:2710015, Line:2, File:221112901, Version:V1, Desc:Order cannot not be shipped as there are no line items in New state
2010-11-18 14:59:39,960                  ERROR
Shipment Error Log:Error Details - Account Id:3962, PO:2573485, Line:2, File:221115901, Version:V1, Desc:Order cannot not be shipped as there are no line items in New state

We need to get entries which come in the last 4 hours.While trying awk command it is treating 'Shipment Error Log.." as second record and giving error while doing date comparison.

What is the command to select from the first match of condition(logtime > (currentime-4 hours)) till end of file?
Last edited by Scott; 11-23-2010 at 08:57 AM.. Reason: Code tags
# 2  
Old 11-19-2010
Hey, try this:
Quote:
awk ' BEGIN{f=0} !f { s=$0; sub(/,.+/, "", s); gsub(/[-: ]/, " ", s); t=(systime()-mktime(s)); if(t>=14400) f=1 } f' infile > outfile
# 3  
Old 11-19-2010
Thanks for your quick response.

We checked using your code.But still it copies all the logs in the file without checking the conditionSmilie
# 4  
Old 11-19-2010
try this:
Quote:
awk ' BEGIN{f=0} !f { s=$0; sub(/,.+/, "", s); gsub(/[-: ]/, " ", s); t=(systime()-mktime(s)); if(t<=14400) f=1 } f' infile > outfile
This User Gave Thanks to kevintse For This Post:
Deepthz
# 5  
Old 11-19-2010
I think it should check the date condition first. here is how i get the outfile as:

Code:
2010-11-20 14:59:39,960                  ERROR
Shipment Error Log:Error Details - Account Id:3962, PO:2573485, Line:2, File:221115901, Version:V1, Desc:Order cannot not be shipped as there are no line items in New s
2010-11-18 15:00:06,236                  ERROR
Shipment Error Log:Error Details - Account Id:7238, PO:2591168, Line:2, File:221116801, Version:V2, Desc:Order cannot not be shipped as there are no line items in New s
2010-11-19 15:05:06,250 ERROR
Shipment Error Log:Error Details - Account Id:7238, PO:2591168, Line:2, File:221116801, Version:V2, Desc:Order cannot not be shipped as there are no line items in New s
2010-11-18 20:20:25,250 ERROR
Shipment Error Log:Error Details - Account Id:7238, PO:2591168, Line:2, File:221116801, Version:V2, Desc:Order cannot not be shipped as there are no line items in New s

if you see, it is not checking the date, but only the time. Is it possible to first locate the occurance of the first record satisfying the condition i.e, sysdate-time - 4hrs and copying the logs till end of file?

---------- Post updated at 05:16 AM ---------- Previous update was at 05:00 AM ----------

Thanks Kevintse, your second command saved us! it works wonders for us now...
I just noticed that my dummy log file contained dates jumbled up, so it was not fetching the correct records. But trying with a real log file, it worked!!.

Can you please explain how does this work?
Code:
BEGIN{f=0} !f { s=$0; sub(/,.+/, "", s); gsub(/[-: ]/, " ", s); t=(systime()-mktime(s));

Last edited by Scott; 11-23-2010 at 08:57 AM.. Reason: Code tags
# 6  
Old 11-19-2010
The code check the date and time (like this: 2010 11 19 14 57 01).
But your log file looks weird, "2010-11-20 14:59:39,960" goes before "2010-11-18 15:00:06,236"....

---------- Post updated at 05:26 AM ---------- Previous update was at 05:17 AM ----------

Quote:
Originally Posted by Deepthz
I think it should check the date condition first. here is how i get the outfile as:

2010-11-20 14:59:39,960 ERROR
Shipment Error Log:Error Details - Account Id:3962, PO:2573485, Line:2, File:221115901, Version:V1, Desc:Order cannot not be shipped as there are no line items in New s
2010-11-18 15:00:06,236 ERROR
Shipment Error Log:Error Details - Account Id:7238, PO:2591168, Line:2, File:221116801, Version:V2, Desc:Order cannot not be shipped as there are no line items in New s
2010-11-19 15:05:06,250 ERROR
Shipment Error Log:Error Details - Account Id:7238, PO:2591168, Line:2, File:221116801, Version:V2, Desc:Order cannot not be shipped as there are no line items in New s
2010-11-18 20:20:25,250 ERROR
Shipment Error Log:Error Details - Account Id:7238, PO:2591168, Line:2, File:221116801, Version:V2, Desc:Order cannot not be shipped as there are no line items in New s

if you see, it is not checking the date, but only the time. Is it possible to first locate the occurance of the first record satisfying the condition i.e, sysdate-time - 4hrs and copying the logs till end of file?

---------- Post updated at 05:16 AM ---------- Previous update was at 05:00 AM ----------

Thanks Kevintse, your second command saved us! it works wonders for us now...
I just noticed that my dummy log file contained dates jumbled up, so it was not fetching the correct records. But trying with a real log file, it worked!!.

Can you please explain how does this work?
BEGIN{f=0} !f { s=$0; sub(/,.+/, "", s); gsub(/[-: ]/, " ", s); t=(systime()-mktime(s));
Quote:
awk ' BEGIN{f=0} #f is a flag indicating whether we should print the line
!f { s=$0; sub(/,.+/, "", s); #extract datetime, 2010-11-19 14:57:01
print s; gsub(/[-: ]/, " ", s); #strip '-' and ':', datetime becomes 2010 11 19 14 57 01
t=(systime()-mktime(s)); #calculate the interval
if(t<=14400) f=1 } f #if f=1, print the whole line(leaving out the action part makes awk print the whole line if the condition is true)
' infile
# 7  
Old 11-22-2010
syntax error near unexpected token `/,.+/,'
in the script while using the awk command with ssh like:
Code:
ssh user@IP awk 'BEGIN{f=0} !f { s=$0; sub(/,.+/,\"\", s); gsub(/[-: ]/,\" \", s);t=(systime()-mktime(s)); if(t<=14400) f=1 } f' /home/user/log.txt >> erroroutput.txt

it gives the error:
Code:
bash: -c: line 0: syntax error near unexpected token `/,.+/,'
bash: -c: line 0: `awk BEGIN{f=0} !f { s=$0; sub(/,.+/, \"\", s); gsub(/[-: ]/, \" \", s);t=(systime()-mktime(s)); if(t<=14400) f=1 } f /home/user/log.txt'

any help?

---------- Post updated at 02:11 AM ---------- Previous update was at 02:04 AM ----------

Quote:
Originally Posted by kevintse
The code check the date and time (like this: 2010 11 19 14 57 01).
But your log file looks weird, "2010-11-20 14:59:39,960" goes before "2010-11-18 15:00:06,236"....

---------- Post updated at 05:26 AM ---------- Previous update was at 05:17 AM ----------
in the script while using the awk command with ssh like:
Code:
ssh user@IP awk 'BEGIN{f=0} !f { s=$0; sub(/,.+/,\"\", s); gsub(/[-: ]/,\" \", s);t=(systime()-mktime(s)); if(t<=14400) f=1 } f' /home/user/log.txt >> erroroutput.txt

it gives the error:
Code:
bash: -c: line 0: syntax error near unexpected token `/,.+/,'
bash: -c: line 0: `awk BEGIN{f=0} !f { s=$0; sub(/,.+/, \"\", s); gsub(/[-: ]/, \" \", s);t=(systime()-mktime(s)); if(t<=14400) f=1 } f /home/user/log.txt'

any help?
Last edited by Scott; 11-23-2010 at 08:59 AM.. Reason: Code tags
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

If I ran perl script again,old logs should move with today date and new logs should generate.

Appreciate help for the below issue. Im using below code.....I dont want to attach the logs when I ran the perl twice...I just want to take backup with today date and generate new logs...What I need to do for the below scirpt.............. 1)if logs exist it should move the logs with extention... (1 Reply)
Discussion started by: Sanjeev G
1 Replies

2. Shell Programming and Scripting

Get the no of hours between days

Hi, i have a date 1- 2013101511 date2 -2013101812 need toget the no of hours between them,can any one tellme the logic. (6 Replies)
Discussion started by: sandeep karna
6 Replies

3. AIX

cron off by 5 hours

stupid question im sure, but its frustrating My cron jobs are off by 5 hours. My system time is right but all of my cron jobs are running approximately 5 hours late. Any idea why? (4 Replies)
Discussion started by: mshilling
4 Replies

4. Shell Programming and Scripting

Retrieve logs generated in last 10 mins from a log file using 'grep' command

HI All, I have a log file where the logs will be in the format as given below: 2011-05-25 02:32:51 INFO PROCESS STARTING 2011-05-25 02:32:52 INFO PROCESS STARTED . . . I want to retrieve only the logs which are less than 5 mins older than current time using grep... (3 Replies)
Discussion started by: rvhg16
3 Replies

5. Shell Programming and Scripting

Command to clear logs for every 6 hours in solaris

Hi Folks, I need to remove log files for six hours on Solaris. before i used to do for every 24 hours below is the code for 1 day older log files, now i tried using -mmin +360 but it says command not found. Can someone please help me out!!! part of the code: LOG_FILE=`find /home/Logdir... (1 Reply)
Discussion started by: Sendhil.Kumaran
1 Replies

6. Shell Programming and Scripting

Retrieve logs for previous 4 hours

Hi, I am in the process of configuring a script, and i intend it to retrieve logs for previous four hours, and then scan for predefined errors. I am kind of stuck on the log retrieval part where the script will run early morning like 1 AM or 2 AM, the command as posted below will give me... (4 Replies)
Discussion started by: john_prince
4 Replies

7. Shell Programming and Scripting

how to list files between last 6 hours to 3 hours

Hi Frens, I want to list some files from a directory, which contains "DONE" in their name, i am receiving files every minute. In this i want to list all the files which are newer than 6 hours but older than 3 hours, of current time i dont want my list to contain the latest files which are ... (4 Replies)
Discussion started by: Prat007
4 Replies

8. Shell Programming and Scripting

Grep yesterday logs from weblogic logs

Hi, I am trying to write a script which would go search and get the info from the logs based on yesterday timestamp and write yesterday logs in new file. The log file format is as follows: """"""""""""""""""""""""""... (3 Replies)
Discussion started by: harish.parker
3 Replies

9. Post Here to Contact Site Administrators and Moderators

Have we just had a rollback of a few hours?

Have we just had a rollback of a few hours? (1 Reply)
Discussion started by: porter
1 Replies

10. UNIX for Dummies Questions & Answers

Delete logs every 3 hours

Hi, I want to setup a cronjob that will delete logs every 2 hours. I have script that delete logs per day. but logging is too big and i want to run a conjob that will delete every 2 hours. this is my current command but it deletes on a per day basis. find . -name "*.log*" -o -name... (3 Replies)
Discussion started by: tungaw2004
3 Replies
Login or Register to Ask a Question