Login or Register to Ask a Question and Join Our Community


UNIX for Beginners Questions & Answers

Searching exception keyword in all logs in last 5 minutes

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Searching exception keyword in all logs in last 5 minutes
# 1  
Old 01-19-2019
Searching exception keyword in all logs in last 5 minutes
Hello Folks ,

I am a new bie to the world of unix , what i am planning to do is the I have the location in server to which i am access through the putty and the location is
Code:
/mt/ttlog/avccomn/logs/201901/19

and at this location the files are listed as show
Code:
startjmsnode1.sh_03.out
startmainuiclientnode1.sh_02.out
startmaincommon_moon2.sh_01.out

Now please advise the unix command which can extract the last 5 minutes logs that i want to search the keyword Exception if it is there in last 5 minutes in logs.
# 2  
Old 01-19-2019
Quote:
Originally Posted by punpun26262626
Now please advise the unix command which can extract the last 5 minutes logs that i want to search the keyword Exception if it is there in last 5 minutes in logs.
For this we would have to know the contents of said files. Usually logs are files that are written to constantly and what would be written within these last fieve minutes we (and all others, for that matter) don't know - without looking into it.

So post a sample of the files contents and maybe we can find some pattern by which to discern "within-the-last-five-minutes"- and "older"-types of messages.

At any rate, you can use the grep utility to search for a certain pattern (like the word "exception") in a file. The following will search for either "exception" or "Exception" in a file and display all lines containing it to the screen:

Code:
grep "[Ee]xception" /path/to/file/to/examine

I hope this helps.

bakunin
# 3  
Old 01-20-2019
Serching exception in all logs of a particular directory
Hello Folks ,

I am a new bie to the world of unix , what i am planning to do is the I have the location in server to which i am access through the putty and the location is

Code:
/mt/ttlog/avccomn/logs/201901/19

and at this location contain the various log files some of them are listed as show

Code:
startjmsnode1.sh_03.out
startmainuiclientnode1.sh_02.out
startmaincommon_moon2.sh_01.out

Now please advise the unix command which can extract the last 5 minutes logs from all the different logs listed in this particular location , as i want to search the keyword Exception if it is there in last 5 minutes in all logs.

Below is the way that logs have been written as I have open them in VI editor , also please cross check the timestamp fashion along with each line

Code:
OMM 2018-11-29T02:07:40,710 DEBUG AM-EventNotificationCron  decrypt Entry
OMM 2018-11-29T02:07:40,710 DEBUG AM-EventNotificationCron  getEightByteIV Entry
OMM 2018-11-29T02:07:40,710 DEBUG AM-EventNotificationCron  getEightByteIV srciv == null (salt)
OMM 2018-11-29T02:07:40,712 DEBUG AM-EmailLogCron  (EmailLogCron.sendEmailActivity) SQL Error in abc.....
OMM 2018-11-29T02:07:40,713 DEBUG AM-abcLogCron  java.sql.SQLException: Numeric Overflow

Moderator's Comments:
Mod Comment edit by bakunin: i figured this post belongs to the same thread as the first one here. Please follow the rule "1 theme - 1 thread" and open new threads only when you have a genuinely new problem. Otherwise just use the thread you have already open. Than you for your consideration.


--- Post updated at 05:19 AM ---

folks please advise for the above problem.
Last edited by bakunin; 01-20-2019 at 01:16 AM..
# 4  
Old 01-20-2019
Try this :
Code:
#!/bin/sh
cd /mt/ttlog/avccomn/logs/201901/19
e=
for (( i = 5; i >= 0; i-- )) ; do
    e='-e /'`date +\%R -d "-$i min"`'/p '$e
done

$(sed -n $e startjmsnode1.sh_03.out > newfile)

This User Gave Thanks to sadique.manzar For This Post:
RavinderSingh13
# 5  
Old 01-20-2019
@punpun26262626: Welcome to the forum.
It is usually well received in here if you put some more effort into formulating the spec than "Now please advise". What OS / shell / tools versions do you use? What thoughts / logics lead to the desired result? Any attempts from your side on the solution?

@sadique.manzar: nice idea, with three drawbacks:
- the patterns will match more than "last 5 minutes", e.g. the day(s) before, or matching "min:sec" values (in above: /07:40/), or any other similar data.
- the key word "Exception" is requested.
- no "command substitution" for the final sed command.

Combining the two proposals this far we come up with
Code:
$ e="\("
$ DL=
$ for (( i = 5; i >= 0; i-- )) ; do  e=$(date +"$e$DL%dT%R:" -d "-$i min"); DL="\|"; done
$ e=$e"\).*[Ee]xception"
$ grep $e file
OMM 2018-11-29T02:07:40,713 DEBUG AM-abcLogCron  java.sql.SQLException: Numeric Overflow

This User Gave Thanks to RudiC For This Post:
sadique.manzar
# 6  
Old 01-21-2019
Quote:
Originally Posted by RudiC
Combining the two proposals this far we come up with
Code:
$ e="\("
$ DL=
$ for (( i = 5; i >= 0; i-- )) ; do  e=$(date +"$e$DL%dT%R:" -d "-$i min"); DL="\|"; done
$ e=$e"\).*[Ee]xception"
$ grep $e file
OMM 2018-11-29T02:07:40,713 DEBUG AM-abcLogCron  java.sql.SQLException: Numeric Overflow

Perhaps the question is if the log entries are sorted by timestamp already. If they are we could simply use:

Code:
sed -n '/[Ee]xception/p;/<timestamp more than 5 min away>/q' /path/to/log

and calculate a "threshhold timestamp" up front.

I hope this helps.

bakunin
# 7  
Old 01-21-2019
Even IF those were sorted, the approach with the single threshold timestamp would require that value to occur verbatim in the logs, so the latter need at least one entry per minute. And, wouldn't the log file need to be tacced to retrieve the last five minutes, and then quit?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to extract logs between the current time and the last 15 minutes ?

I want to extract the logs between the current time stamp and 15 minutes before and sent an email to the people configured. I developed the below script but it's not working properly; can someone help me?? I have a log file containing this pattern: Constructor QuartzJob ... (3 Replies)
Discussion started by: puneetkhullar
3 Replies

2. Shell Programming and Scripting

Need help in getting the Last 30 minutes logs from the Log File

I have a log file with the below contents : log_file_updated.txt : Jul 5 03:33:06 rsyslogd: was Jul 5 03:33:09 adcsdb1 rhsmd: This system is registered. Sep 2 02:45:48 adcsdb1 UDSAgent: 2015-07-05 04:24:48.959 INFO Worker_Thread_4032813936 Accepted connection from host <unknown>... (3 Replies)
Discussion started by: rahul2662
3 Replies

3. Shell Programming and Scripting

Searching Error Message from a log life between timestamp of every 10 minutes

HI Everyone, My task is to search error messages last 10 minutes everytime from a log file. My script, date1=`date -d '10 minutes ago' "+%H:%M:%S"` date2=`date "+%H:%M:%S"` awk -v d1="${date1}" -v d2="${date2}" '$0~d1{p=1} $0~d2{p=0} p' filename No error getting in... (3 Replies)
Discussion started by: ctscbe
3 Replies

4. Shell Programming and Scripting

Catching the exception in multiple logs

Hi folks, I have logs folder in which different type of logs are generated , I am monitoring them by the below command tail -f *.log but I want that if exception come in any of the logs then it should be catch so what i should prefix with tail -f *.log so that it imeediatley catches and... (3 Replies)
Discussion started by: punpun66
3 Replies

5. Shell Programming and Scripting

Line numbers and exception to be caught in logs

Hi Folks, I have just basic queries is that suppose I have to monitor the logs then there is a command , suppose I have to monitor the abc.log which is updating dynamically within seconds so the command will be after going to that directory is .. tail -f abc.log Now please advise what about... (1 Reply)
Discussion started by: punpun66
1 Replies

6. Shell Programming and Scripting

Monitor logs for exception and if exception come then sent an email

Hi Folks, please advise , I have logs generated on unix machine at location /ops/opt/aaa/bvg.log , now sometimes there come exception in these logs also, so I want to write such a script such that it should continuously monitor these logs and whenever any exception comes that is it try to find... (3 Replies)
Discussion started by: tuntun27272727
3 Replies

7. Shell Programming and Scripting

Searching set of string from Live Running Logs

Hey just need one simple syntax to search for the string from the Live Running Logs. The strings are placed in a $infile & everytime the script should pull each string from $infile and should provide as an input for grepping from Live running logs on a rotational basis. So here are the Contents... (14 Replies)
Discussion started by: raghunsi
14 Replies

8. OS X (Apple)

Keyword Searching

Hi all, I am in the process of building a shell script as part of a auditing utility. It will search a specified directory for keywords and output results of the file path, and line number that the word was found on. I built a test script (shown below) that does just this, but egrep apparently... (0 Replies)
Discussion started by: tmcmurtr
0 Replies

9. Shell Programming and Scripting

Searching *.gz logs

I have been trying to search for a string from close to 200 *.gz file, But i get a error. Can someone suggest a bulletproof solution Please. zgrep 20/Aug/2008:13:50:23 request.log.*.gz -bash: /usr/bin/zgrep: /bin/sh: bad interpreter: Argument list too long also zgrep 20/Aug/2008:13:50:23... (9 Replies)
Discussion started by: openspark
9 Replies

10. OS X (Apple)

keyword searching of documents

Unix based fix-it needed? Platform and feature: search programs on Apple computers (Leopard or Tiger; 10.4 and above; Spotlight) Problem: the document search feature of these programs produce hits when keyword(s) used appear anywhere in the document's content. Change required: we need to... (1 Reply)
Discussion started by: Miles
1 Replies
Login or Register to Ask a Question