Sponsored Content
Top Forums Shell Programming and Scripting Extract the last 10 minutes from logfile Post 302881003 by timmywong on Thursday 26th of December 2013 05:30:08 AM
Old 12-26-2013
Extract the last 10 minutes from logfile

Any better way to extract the last 10 minutes from logfile?

Code:
  Dec 18 09:41:18 aaa 
Dec 18 09:46:29 aa 
Dec 18 09:48:39 vvv 
Dec 18 09:48:54 bbb 
Dec 18 09:54:47 bbb 
Dec 18 09:55:33 fcf
Dec 18 09:55:38 ssdf 
Dec 18 09:57:58 sdsds 
Dec 18 09:58:10 sdsd 
Dec 18 10:00:50 sdsd 
Dec 18 10:03:43 sdsds 
Dec 18 10:03:50 sds 
Dec 18 10:04:06 sdsd 
Dec 18 10:04:15 sdsd 
Dec 18 10:14:50 sds 
Dec 18 10:19:16 sdsd 
Dec 18 10:19:23 sdsd 
Dec 18 10:21:03 dsdds 
Dec 18 10:22:54 sdsd 
Dec 18 10:27:32 sdsd

Attempt 1: Too slow if the the log file contains over 10000 lines.

Code:
$ lastline=$(tail -n1 logfile) 
$ last10=$(($(date -d "$lastline" +%s) - (10 * 60))) 
$ while read line; do 
> [ $(date -d "${line:0:15}" +%s) -gt $last10 ] && printf "$line\n" 
> done < logfile

Moderator's Comments:
Mod Comment Please use CODE tags. We know that you know how to use CODE tags because you've used them before. But, several of your posts have been edited by moderators to add tags; and several more of your posts should have had tags but nobody bothered to fix them.


---------- Post updated at 06:30 PM ---------- Previous update was at 02:12 PM ----------

Python method is quite fast but only works if the log contains only dates, any suggestion on how to fix this?

Code:
Dec 18 10:19:16
Dec 18 10:19:23
Dec 18 10:21:03
Dec 18 10:22:54
Dec 18 10:27:32

Code:
from datetime import datetime  
astack=[] with open("x.txt") as f:     
for aline in f:         
astack.append(aline.strip()) 
lasttime=datetime.strptime(astack[-1], '%b %d %I:%M:%S') 
for i in astack:     
if (lasttime - datetime.strptime(i, '%b %d %I:%M:%S')).seconds <= 600:         
print i


Last edited by timmywong; 12-26-2013 at 06:34 AM.. Reason: Add CODE tags.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Convert minutes to hours, minutes, seconds

How would you convert lets say a 1000 minutes to hours, minutes, seconds (1 Reply)
Discussion started by: Vozx
1 Replies

2. Shell Programming and Scripting

Need to extract specific pattern from logfile

Log File: Attempting to contact (DESCRIPTION=(SOURCE_ROUTE=OFF)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=hostname1.com)(PORT=1521)))(CONNECT_DATA=(SID=database1)(SRVR=DEDICATED))) Attempting to contact... (2 Replies)
Discussion started by: techychap
2 Replies

3. Shell Programming and Scripting

Script to extract line from logfile

Hi , Can someone help me,I am not well versed with scripting,it is very urjent. Need a script in perl or shell for the following task. The logfile contains several entries of which one particular string to be searched and that complete line has to be removed and copied to another file to... (25 Replies)
Discussion started by: garryt21@rediff
25 Replies

4. Shell Programming and Scripting

Perl - Extract 12 hour time, convert to 24 and subtract 15 minutes?

OK, I am by no means a programmer... I have been given the task to do some automation scripts. I have got most of it working from snippets I have found on the Web. One requirement has me stumped. The initial timing file created by the user is a comma delimited in the following format.... (4 Replies)
Discussion started by: autotuner
4 Replies

5. Shell Programming and Scripting

Extract three substrings from a logfile

I have a log file like below. 66.249.73.11 - - "UCiZ7QocVqYAABgwfP8AAHAA" "US" "Mediapartners-Google" "-" www.mahashwetha.com.sg "GET... (2 Replies)
Discussion started by: Tuxidow
2 Replies

6. Shell Programming and Scripting

Checking if logfile updates after 5 minutes

I would like to create a script that will send an email to me if it does not update after 5 minutes. I have created this script but it doesn't work. #!/bin/ksh # Variable Declaration log=/opt/app/netcool/omnibus/log.txt servertime=`date | awk '{print $2,$3,$4 }' | cut -d ":" -f1,2`... (5 Replies)
Discussion started by: Dan Llave
5 Replies

7. Shell Programming and Scripting

Logfile monitoring with logfile replacement

Bonjour, I've wrote a script to monitor a logfile in realtime. It is working almost perfeclty except for two things. The script use the following technique : tail -fn0 $logfile | \ while read line ; do ... some stuff done First one, I'd like a way to end the monitoring script if a... (3 Replies)
Discussion started by: Warluck
3 Replies

8. Shell Programming and Scripting

How to extract logs between the current time and the last 15 minutes ?

I want to extract the logs between the current time stamp and 15 minutes before and sent an email to the people configured. I developed the below script but it's not working properly; can someone help me?? I have a log file containing this pattern: Constructor QuartzJob ... (3 Replies)
Discussion started by: puneetkhullar
3 Replies

9. UNIX for Beginners Questions & Answers

How to convert days hours minutes seconds to minutes?

Hi, please help with below time conversion to minutes. one column values: 2 minutes 16 seconds 420 msec 43 seconds 750 msec 0 days 3 hours 29 minutes 58 seconds 480 msec 11 seconds 150 msec I need output in minutes(total elapsed time in minutes) (2 Replies)
Discussion started by: ramu.badugula
2 Replies

10. Shell Programming and Scripting

Grep a log file for the last 5 minutes of contents every 5 minutes

Hi all, System Ubuntu 16.04.3 LTS i have the following log INFO 2019-02-07 15:13:31,099 module.py:700] default: "POST /join/8550614e-3e94-4fa5-9ab2-135eefa69c1b HTTP/1.0" 500 2042 INFO 2019-02-07 15:13:31,569 module.py:700] default: "POST /join/6cb9c452-dcb1-45f3-bcca-e33f5d450105... (15 Replies)
Discussion started by: charli1
15 Replies
All times are GMT -4. The time now is 05:42 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy