Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

How to grep logs for errors and receive specific additional lines?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to grep logs for errors and receive specific additional lines?
# 8  
Old 09-21-2013
You dropped the single quote ending the awk script. Change the last line in test.sh from:
Code:
} /home/mc/servers/bukkitpvt/server.log

to:
Code:
}' /home/mc/servers/bukkitpvt/server.log

and try again.
This User Gave Thanks to Don Cragun For This Post:
nbsparks
# 9  
Old 09-21-2013
Actually, that ' is still there. the previous post of mine consisted of two things, the script "via cat" and the output of the script (due to the "set -x") at the top of the script. If you look half way through that code window, you'll see the ' is still there.
# 10  
Old 09-21-2013
The set -x shows you the command the shell is running; it does not trace commands running inside awk.

I'm grasping at straws here. Try changing the lines in my script:
Code:
!/^[1-9][0-9]{3}-[01][0-9]-[0-3][0-9]/ {
    and
/^[1-9][0-9]{3}-[01][0-9]-[0-3][0-9]/ {

to:
Code:
!/^[1-9][0-9][0-9][0-9][-][01][0-9][-][0-3][0-9]/ {
    and
/^[1-9][0-9][0-9][0-9][-][01][0-9][-][0-3][0-9]/ {

respectively.
If that doesn't work, we can try adding some printf statements in the awk script to trace what is going on in the awk script. But, trying to debug a script on a system I can't touch is frustrating when the script works perfectly on my system.
This User Gave Thanks to Don Cragun For This Post:
nbsparks
# 11  
Old 09-21-2013
Don,

I won't be offended if you're done with this thread. You gave it a solid run. I don't ever expect the amount of effort you've given on forums like this. And I certainly don't feel entitled to your time.

IF you're interested (big IF), you can gain access to the same OS by doing a virtual machine. You'd be up and running in 10 minutes or less:
But I don't expect that. You've been MORE than generous with your time.


Thanks.

---------- Post updated at 08:57 AM ---------- Previous update was at 08:47 AM ----------

I spoke too soon, Don!

Your last code change worked! This script worked:
Code:
#!/bin/bash

set -x

awk -F"[][]" '
# pc() -- print context
# print up to 11 lines of context and clear context.
function pc(    i) {
        for(i = (lc > 10) ? lc - 10 : 1; i <= lc; i++)
                print l[i]
        lc = 0
}
!/^[1-9][0-9][0-9][0-9][-][01][0-9][-][0-3][0-9]/ {
        nts = 1 # we have found a line with No Time Stamp
        if(severe) {
                pc()    # print context
                print   # print this line (which follows the SEVERE tag).
        }
        next
}
/^[1-9][0-9][0-9][0-9][-][01][0-9][-][0-3][0-9]/ {
        if(nts)
                # 1st line after one or more lines with no time stamp...
                severe = nts = lc = 0
        l[++lc] = $0
        if($2 == "SEVERE") severe = 1
}' /home/mc/servers/bukkitpvt/server.log >test.log

Thank you SO MUCH for all your hard work and effort! I can't tell you enough how much I appreciate your selfless donation of your time. I am nowhere near the level (yet?) to be able to compose a tool like this and you've made my life so much easier!
# 12  
Old 09-22-2013
To get many of the awk version to work with code like this [0-9]{3}, you need to add --posix.
Eks
Code:
awk --posix '/^[0-9-]{3}/' file

This should make the original code to work.
This User Gave Thanks to Jotne For This Post:
Don Cragun
# 13  
Old 09-22-2013
Quote:
Originally Posted by Jotne
To get many of the awk version to work with code like this [0-9]{3}, you need to add --posix.
Eks
Code:
awk --posix '/^[0-9-]{3}/' file

This should make the original code to work.
Hi Jotne,
Thanks for the information. Note, however, that adding the --posix long option to awk on a system that actually conforms to the POSIX standards is likely to give you an unknown option diagnostic rather than a working awk script.

Oh, well.
# 14  
Old 09-25-2013
I have run into a new issue. There are occasionally "[WARNING]" messages that I care about as much as "[SEVERE]" messages. Thus, I would like to be able to trigger on more than just "SEVERE". How do I trigger on more than just that?

I'm thinking of triggering on these so far:

  • SEVERE
  • java.lang.ClassCastException (or possibly just "java.lang.Class*" as there should never be any of that in a happy log file)
  • generated an exception
Of course, if I start to overlap with "SEVERE" then I may just have to use other triggers instead. Not sure.

Thanks in advance for any help you can provide.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Grep content between specific lines

cat file1 *FileHeader* Partition 0 Total Data Bytes 1416 Avg Bytes/Record 1416 Others 1 PRDX22.AUDIT_DATA_INFO Partition 4 Total Data Bytes 4615 Avg... (8 Replies)
Discussion started by: Veera_V
8 Replies

2. UNIX for Dummies Questions & Answers

Grep specific lines

Hello I have a file with nearly 90000 lines in x,y,z format but have some lines that I do not need to show. Is there anyway to delete those 3 lines after every 288 lines. Eg I keep the first 288 lines delete (289, 290 291); keep the next 288 lines after those and so on... Thanks (6 Replies)
Discussion started by: Madiouma Ndiaye
6 Replies

3. Shell Programming and Scripting

Script to grep logs for Errors

Hi Guys, I want to write a script which can grep the logs (server.log) from a file for Error String and output to a other file. Problems: How to know about the errors only between the current restart and not in previous as server.log has earlier restarts also? thanks for the help! Much... (5 Replies)
Discussion started by: ankur328
5 Replies

4. Shell Programming and Scripting

grep the output between specific lines

Symmetrix ID : 00000001234 Host Name : myown Identifiers Found : 5000000000000000 5000000000000001 Device Cap(MB) Attr Dir:P ------ ------- ---- ---- 1234 25886 (M) 8D:1, 9D:1 0123 25886 (M) 8D:1, 9D:1 1345 25886 (M) ... (5 Replies)
Discussion started by: maddy.san
5 Replies

5. Shell Programming and Scripting

using awk to get specific section of lines in logs

i have a log file that has the date and time that looks like this: Wed Jun 28 15:46:21 2012 test failed tailed passed passed not error panic what we want to focus on is the first 5 columns because they contain the date and time. the date and time can be anywhere on the line. in this... (6 Replies)
Discussion started by: SkySmart
6 Replies

6. Shell Programming and Scripting

Problems to print specific lines with awk and grep...HELP!

Hi all I have data like this: model: 1, misfit value: 0.74987 1 1.182 1.735 2.056 1.867 2 0.503 1.843 2.018 1.888 3 2.706 2.952 2.979 1.882 4 8.015 3.414 3.675 1.874 ... (1 Reply)
Discussion started by: fedora2011
1 Replies

7. Shell Programming and Scripting

Extracting specific lines of data from a file and related lines of data based on a grep value range?

Hi, I have one file, say file 1, that has data like below where 19900107 is the date, 19900107 12 144 129 0.7380047 19900108 12 168 129 0.3149017 19900109 12 192 129 3.2766666E-02 ... (3 Replies)
Discussion started by: Wynner
3 Replies

8. UNIX Desktop Questions & Answers

grep lines with two specific characters somewhere in the line

I'm having trouble with extracting certain lines from a file based on whether they have all the required fields. Original file: snt:594:Sam N This bpt:2342:Bob P That lr:123 wrp:23:Whoever Person cor:794 Desired output: snt:594:Sam N This bpt:2342:Bob P That wrp:23:Whoever Person ... (3 Replies)
Discussion started by: Chthonic
3 Replies

9. Shell Programming and Scripting

Grep for lines in web logs

I want to find the unique url in a apache logs which got 404 error . I can do something like "cat apache.log|grep 404|awk '{print $2,$3}'|grep 404 this will give me say /foo.html 404 /foo.html 404 /foo.html 404 /bar.html 404 /cat.html 404 However my output should only find... (3 Replies)
Discussion started by: gubbu
3 Replies

10. Shell Programming and Scripting

retrieving specific lines from a file - can I use grep ?

Hi there, if i had a file that looked like this my_server1 red green blue yellow blue my_server2 blue blue yellow green blue my_server3 yellow (9 Replies)
Discussion started by: hcclnoodles
9 Replies
Login or Register to Ask a Question