Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Retrieve logs generated in last 10 mins from a log file using 'grep' command

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Retrieve logs generated in last 10 mins from a log file using 'grep' command
# 1  
Old 05-26-2011
Retrieve logs generated in last 10 mins from a log file using 'grep' command
HI All,
I have a log file where the logs will be in the format as given below:

2011-05-25 02:32:51 INFO PROCESS STARTING
2011-05-25 02:32:52 INFO PROCESS STARTED
.
.
.

I want to retrieve only the logs which are less than 5 mins older than current time using grep command....

Any help would be appreciated....
# 2  
Old 05-26-2011
well it's not a exact solution, but this one will print entries from file based on last time entry in the log. Hope this will work for you:

Code:
 
sed -n "/`cut -c1-16 m|uniq|tail -5|head -1`/,/`cut -c1-16 m|tail -1`/p" m

here m is the filename
# 3  
Old 05-26-2011
A quick search on the site throws up:

https://www.unix.com/shell-programmin...-log-file.html

wherein you may find what you require :-)
# 4  
Old 05-26-2011
... just an example, the code is not very strict and may sometimes display more than what it is supposed to, but it may still give some idea :

The XX and YY offset (**) may need to be adjusted depending on the format of your date

**see : cut -cXX and substr(...,YY,...)

Note that the tr translation may also need to be adjusted to your needs depending on which column you want to shift (minute? hours? other ?) as well as the interval (5 min? 7min ? 10min?)...

Here an example for the last 5 min (not exactly in fact... more 4 min than 5 but you get the idea)

Code:
[ctsgnb@shell ~/sand]$ cat tst
Jul 19 18:15:23 ........
Jul 19 18:15:28 ........
Jul 19 18:17:11 ........
Jul 19 18:19:11 ........
Jul 19 18:20:23 ........
Jul 19 18:25:28 ........
Jul 19 18:26:11 ........
Jul 19 18:27:11 ........
Jul 19 18:28:23 ........
Jul 19 18:29:28 ........
Jul 19 18:30:11 ........
Jul 19 18:34:11 ........
Jul 19 18:40:23 ........
Jul 19 18:41:28 ........
Jul 19 18:42:11 ........
Jul 19 18:43:11 ........

[ctsgnb@shell ~/sand]$ date ; m=$(date | cut -c16 | tr '0123456789' '5678901234') ; echo $m
Thu May 26 08:23:22 MDT 2011
8
[ctsgnb@shell ~/sand]$ tail -r tst | awk -v M="$m" 'substr($0,12,1)==M{print;x=1}!x' | tail -r
Jul 19 18:28:23 ........
Jul 19 18:29:28 ........
Jul 19 18:30:11 ........
Jul 19 18:34:11 ........
Jul 19 18:40:23 ........
Jul 19 18:41:28 ........
Jul 19 18:42:11 ........
Jul 19 18:43:11 ........

[ctsgnb@shell ~/sand]$

---------- Post updated at 04:52 PM ---------- Previous update was at 04:38 PM ----------

A very little improvement :

Code:
m=$(date | cut -c16 | tr '0123456789' '4567890123')
tail -r tst | awk -v M="$m" 'substr($0,12,1)==M{x=1}!x' | tail -r

so that if there are more than 1 consecutive entry that have ((current_minute) - (5min)), they are all displayed instead of just the first one

Code:
[ctsgnb@shell ~/sand]$ cat tst
Jul 19 18:15:23 ........
Jul 19 18:15:28 ........
Jul 19 18:17:11 ........
Jul 19 18:19:11 ........
Jul 19 18:20:23 ........
Jul 19 18:25:28 ........
Jul 19 18:26:11 ........
Jul 19 18:27:11 ........
Jul 19 18:28:23 ........
Jul 19 18:29:28 ........
Jul 19 18:30:11 ........
Jul 19 18:34:11 ........
Jul 19 18:40:23 ........
Jul 19 18:40:23 ........
Jul 19 18:41:28 ........
Jul 19 18:42:11 ........
Jul 19 18:42:11 ........
Jul 19 18:43:11 ........

[ctsgnb@shell ~/sand]$ date ; m=$(date | cut -c16 | tr '0123456789' '4567890123') ; echo $m
Thu May 26 08:47:56 MDT 2011
1
[ctsgnb@shell ~/sand]$ tail -r tst | awk -v M="$m" 'substr($0,12,1)==M{x=1}!x' | tail -r
Jul 19 18:42:11 ........
Jul 19 18:42:11 ........
Jul 19 18:43:11 ........

[ctsgnb@shell ~/sand]$

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Grep last 5 mins from logs

Hi, system date format Thu Jun 13 12:55:18 EDT 2019 My log date format 09.148.192.60 - - "GET /akamai/sureroute-test-object.html HTTP/1.1" 404 231 can someone please help me, how to get last 5mins of logs please ? I need the command Please wrap your samples/codes in CODE TAGS,... (3 Replies)
Discussion started by: scazed
3 Replies

2. UNIX for Beginners Questions & Answers

Grep: Retrieve two strings from one file to find them anyone on line in another file

I am having trouble matching *two* strings from one file anywhere in a line of a second file, and could use some help getting this figured out. My preference would be to use grep for this because I would like to take advantage of its -A option. The latter is due to the fact that I would like both... (2 Replies)
Discussion started by: jvoot
2 Replies

3. Shell Programming and Scripting

Need logs 5 mins old

I need 5 mins old logs to be dumped into a a new file. The date formats in the two log files are Can you suggect for both formats ? bash-3.2$ uname -a SunOS myserver 5.10 Generic_150400-26 sun4v sparc sun4v ---------- Post updated 05-04-16 at 12:24 AM ---------- Previous update was... (2 Replies)
Discussion started by: mohtashims
2 Replies

4. UNIX for Dummies Questions & Answers

Script to search log file for last 15 mins data

Hi All, I have an issue which I'm trying to understand a way of doing, I have several nodes which contain syslog events which I want to force trigger an email initially (eventually leading to another method of alerting but to start with an email). Basically the syslog file will have hours worth... (6 Replies)
Discussion started by: mutley2202
6 Replies

5. Shell Programming and Scripting

Script to grep for a string in log files generated in last 15 minutes.

Dear Guru's I've a requirment to grep for a string in series of log files that are getting generated almost every minute. I'm looking to schedule a script every 15 mountes,in order to check if the error string has been generated in any of the log files generated in last 15 minutes. Please... (3 Replies)
Discussion started by: rajivatnova
3 Replies

6. AIX

Grep last 5 mins from log file in AIX

I want to grep only last 5 mins of a log file in bash I have a syslog which contains the following Mon Jul 11 20:47:42 Mon Jul 11 20:47:52 The following works in Unix but not in AIX . Please can you let me know as to what would be the AIX equivalent Code: for (( i = 5; i >=0;... (1 Reply)
Discussion started by: necro98
1 Replies

7. Shell Programming and Scripting

Script to Grep column 3 from csv file generated yesterday

Hello, Can any one please assist how to scirpt it: Every day a new log file is create and I want to process only the one generated yesterday and get the data of column 3 and 6. For example today's date is 24 then I want to get the data of log file created on 23rd. Log Files in... (7 Replies)
Discussion started by: sureshcisco
7 Replies

8. Shell Programming and Scripting

Retrieve logs for previous 4 hours

Hi, I am in the process of configuring a script, and i intend it to retrieve logs for previous four hours, and then scan for predefined errors. I am kind of stuck on the log retrieval part where the script will run early morning like 1 AM or 2 AM, the command as posted below will give me... (4 Replies)
Discussion started by: john_prince
4 Replies

9. Shell Programming and Scripting

Retrieve logs for last 4 hours

Our log file looks like this: 2010-11-18 00:57:01,698 ERROR Shipment Error Log:Error Details - Account Id:3962, PO:2710015, Line:2, File:221112901, Version:V1, Desc:Order cannot not be shipped as there are no line items in New state 2010-11-18 14:59:39,960 ERROR... (11 Replies)
Discussion started by: Deepthz
11 Replies

10. Shell Programming and Scripting

grep command to retrieve one file

The Sed/Grep command is really confusing me. I know I'm missing something that should be really easy to fix. My program displays multiple names after I ask it to display only one, How do I get it to do only one?? it looks like this: Please enter a name to display? >> John (A list then... (9 Replies)
Discussion started by: toejam
9 Replies
Login or Register to Ask a Question