Sponsored Content
Top Forums Shell Programming and Scripting How to grep logs for errors and receive specific additional lines? Post 302855773 by nbsparks on Friday 20th of September 2013 08:13:08 PM
Old 09-20-2013
I threw out 10 lines as an arbitrary number because it could really vary. I would think that 10 lines would be MORE than I need most of the time. And I wouldn't want to hardcode on "Mobcatcher" as it could be any number of server plugins that could fail or error out.

The idea is for me to be able to troubleshoot and diagnose server issues with this subset of the logfile rather than looking at this logfile, noting the time, and then sifting through the main server.log file to get the context I need to troubleshoot.

---------- Post updated at 05:13 PM ---------- Previous update was at 05:11 PM ----------

I suppose that one could parse the [SEVERE] line and then look for any lines within the previous 10-20 lines that mention any of the words used in the same line as the [SEVERE] tag, but that seems overkill and unnecessary complication.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

retrieving specific lines from a file - can I use grep ?

Hi there, if i had a file that looked like this my_server1 red green blue yellow blue my_server2 blue blue yellow green blue my_server3 yellow (9 Replies)
Discussion started by: hcclnoodles
9 Replies

2. Shell Programming and Scripting

Grep for lines in web logs

I want to find the unique url in a apache logs which got 404 error . I can do something like "cat apache.log|grep 404|awk '{print $2,$3}'|grep 404 this will give me say /foo.html 404 /foo.html 404 /foo.html 404 /bar.html 404 /cat.html 404 However my output should only find... (3 Replies)
Discussion started by: gubbu
3 Replies

3. UNIX Desktop Questions & Answers

grep lines with two specific characters somewhere in the line

I'm having trouble with extracting certain lines from a file based on whether they have all the required fields. Original file: snt:594:Sam N This bpt:2342:Bob P That lr:123 wrp:23:Whoever Person cor:794 Desired output: snt:594:Sam N This bpt:2342:Bob P That wrp:23:Whoever Person ... (3 Replies)
Discussion started by: Chthonic
3 Replies

4. Shell Programming and Scripting

Extracting specific lines of data from a file and related lines of data based on a grep value range?

Hi, I have one file, say file 1, that has data like below where 19900107 is the date, 19900107 12 144 129 0.7380047 19900108 12 168 129 0.3149017 19900109 12 192 129 3.2766666E-02 ... (3 Replies)
Discussion started by: Wynner
3 Replies

5. Shell Programming and Scripting

Problems to print specific lines with awk and grep...HELP!

Hi all I have data like this: model: 1, misfit value: 0.74987 1 1.182 1.735 2.056 1.867 2 0.503 1.843 2.018 1.888 3 2.706 2.952 2.979 1.882 4 8.015 3.414 3.675 1.874 ... (1 Reply)
Discussion started by: fedora2011
1 Replies

6. Shell Programming and Scripting

using awk to get specific section of lines in logs

i have a log file that has the date and time that looks like this: Wed Jun 28 15:46:21 2012 test failed tailed passed passed not error panic what we want to focus on is the first 5 columns because they contain the date and time. the date and time can be anywhere on the line. in this... (6 Replies)
Discussion started by: SkySmart
6 Replies

7. Shell Programming and Scripting

grep the output between specific lines

Symmetrix ID : 00000001234 Host Name : myown Identifiers Found : 5000000000000000 5000000000000001 Device Cap(MB) Attr Dir:P ------ ------- ---- ---- 1234 25886 (M) 8D:1, 9D:1 0123 25886 (M) 8D:1, 9D:1 1345 25886 (M) ... (5 Replies)
Discussion started by: maddy.san
5 Replies

8. Shell Programming and Scripting

Script to grep logs for Errors

Hi Guys, I want to write a script which can grep the logs (server.log) from a file for Error String and output to a other file. Problems: How to know about the errors only between the current restart and not in previous as server.log has earlier restarts also? thanks for the help! Much... (5 Replies)
Discussion started by: ankur328
5 Replies

9. UNIX for Dummies Questions & Answers

Grep specific lines

Hello I have a file with nearly 90000 lines in x,y,z format but have some lines that I do not need to show. Is there anyway to delete those 3 lines after every 288 lines. Eg I keep the first 288 lines delete (289, 290 291); keep the next 288 lines after those and so on... Thanks (6 Replies)
Discussion started by: Madiouma Ndiaye
6 Replies

10. Shell Programming and Scripting

Grep content between specific lines

cat file1 *FileHeader* Partition 0 Total Data Bytes 1416 Avg Bytes/Record 1416 Others 1 PRDX22.AUDIT_DATA_INFO Partition 4 Total Data Bytes 4615 Avg... (8 Replies)
Discussion started by: Veera_V
8 Replies
CLIENT-LOCAL.CFG(5)						File Formats Manual					       CLIENT-LOCAL.CFG(5)

NAME
client-local.cfg - Local configuration settings for Xymon clients SYNOPSIS
~xymon/server/etc/client-local.cfg DESCRIPTION
The client-local.cfg file contains settings that are used by each Xymon client when it runs on a monitored host. It provides a convenient way of configuring clients from a central location without having to setup special configuration maintenance tools on all clients. The client-local.cfg file is currently used to configure what logfiles the client should fetch data from, to be used as the basis for the "msgs" status column; and to configure which files and directories are being monitored in the "files" status column. Note that there is a dependency between the client-local.cfg file and the hobbit-clients.cfg(5) file. When monitoring e.g. a logfile, you must first enter it into the client-local.cfg file, to trigger the Xymon client into reporting any data about the logfile. Next, you must configure hobbit-clients.cfg so the Xymon server knows what to look for in the file data sent by the client. So: client-local.cfg defines what raw data is collected by the client, and hobbit-clients.cfg defines how to analyze them. PROPAGATION TO CLIENTS
The client-local.cfg file resides on the Xymon server. When clients connect to the Xymon server to send in their client data, they will receive part of this file back from the Xymon server. The configuration received by the client is then used the next time the client runs. This method of propagating the configuration means that there is a delay of up to two poll cycles (i.e. 5-10 minutes) from a configuration change is entered into the client-local.cfg file, and until you see the result in the status messages reported by the client. FILE FORMAT
The file is divided into sections, delimited by "[name]" lines. A section name can be either an operating system identifier - linux, solaris, hp-ux, aix, freebsd, openbsd, netbsd, darwin - or a hostname. When deciding which section to send to a client, Xymon will first look for a section named after the hostname of the client; if such a section does not exist, it will look for a section named by the oper- ating system of the client. So you can configure special configurations for individual hosts, and have a default configuration for all other hosts of a certain type. Apart from the section delimiter, the file format is free-form, or rather it is defined by the tools that make use of the configuration. LOGFILE CONFIGURATION ENTRIES
A logfile configuration entry looks like this: log:/var/log/messages:10240 ignore MARK trigger Oops The log:FILENAME:SIZE line defines the filename of the log, and the maximum amount of data (in bytes) to send to the Xymon server. FILENAME is usually an explicit full-path filename on the client. If it is enclosed in backticks, it is a command which the Xymon client runs and each line of output from this command is then used as a filename. This allows scripting which files to monitor, e.g. if you have logfiles that are named with some sort of timestamp. The ignore PATTERN line (optional) defines lines in the logfile which are ignored entirely, i.e. they are stripped from the logfile data before sending it to the Xymon server. It is used to remove completely unwanted "noise" entries from the logdata processed by Xymon. "PAT- TERN" is a regular expression. The trigger PATTERN line (optional) is used only when there is more data in the log than the maximum size set in the "log:FILENAME:SIZE" line. The "trigger" pattern is then used to find particularly interesting lines in the logfile - these will always be sent to the Xymon server. After picking out the "trigger" lines, any remaining space up to the maximum size is filled in with the most recent entries from the logfile. "PATTERN" is a regular expression. COUNTING LOGENTRIES
A special type of log-handling is possible, where the number of lines matching a regular expressions are merely counted. This is linecount:FILENAME, followed by a number of lines of the form ID:PATTERN. E.g. linecount:/var/log/messages diskerrors:I/O error.*device.*hd badlogins:Failed login FILE CONFIGURATION ENTRIES
A file monitoring entry is used to watch the meta-data of a file: Owner, group, size, permissions, checksum etc. It looks like this: file:/var/log/messages[:HASH] The file:FILENAME line defines the filename of the file to monitor. As with the "log:" entries, a filename enclosed in backticks means a command which will generate the filenames dynamically. The optional [:HASH] setting defines what type of hash to compute for the file: md5, sha1 or rmd160. By default, no hash is calculated. NOTE: If you want to check multiple files using a wildcard, you must use a command to generate the filenames. Putting wildcards directly into the file: entry will not work. DIRECTORY CONFIGURATION ENTRIES
A directory monitoring entry is used to watch the size of a directory and any sub-directories. It looks like this: dir:DIRECTORYNAME The dir:DIRECTORYNAME line defines the filename of the file to monitor. As with the "log:" entries, a filename enclosed in backticks means a command which will generate the filenames dynamically. The Xymon client will run the du(1) command with the directoryname as parameter, and send the output back to the Xymon server. NOTE: If you want to check multiple directories using a wildcard, you must use a command to generate the directory names. Putting wildcards directly into the dir: entry will not work. E.g. use something like dir:`find /var/log -maxdepth 1 -type d` The "du" command used can be configured through the DU environment variable. On some systems, by default du reports data in disk blocks instead of KB (e.g. Solaris). So you may want to configure the Xymon client to use a du command which reports data in KB, e.g. by setting DU="du -k" in the hobbitclient.cfg file. NOTES
The ability of the Xymon client to calculate file hashes and monitor those can be used for file integrity validation on a small scale. How- ever, there is a significant processing overhead in calculating these every time the Xymon client runs, so this should not be considered a replacement for host-based intrusion detection systems such as Tripwire or AIDE. Use of the directory monitoring on directory structures with a large number of files and/or sub-directories can be quite ressource-inten- sive. SEE ALSO
hobbit-clients.cfg(5), hobbitd_client(8), hobbitd(8), xymon(7) Xymon Version 4.2.3: 4 Feb 2009 CLIENT-LOCAL.CFG(5)
All times are GMT -4. The time now is 06:24 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy