Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: tail | grep lagging badly
Top Forums Shell Programming and Scripting tail | grep lagging badly Post 302188107 by WasabiVengeance on Tuesday 22nd of April 2008 04:16:32 PM
Old 04-22-2008
tail | grep lagging badly
I'm trying to use tail/grep to monitor a log file. The command I cooked up is:

tail -n 50 -f output.log | grep 'type:system' | cut -f 5-

A sample line from the log file is:

1208894862 type:system session:0 severity:4 load started

the columns are tab delimited.

this works ok, except that it's inexplicably lagging badly. If I have 2 terminals open, one simply tailing the file and the other tail|greping the file as specified above, it might take 30 seconds or more for the tail|grep terminal to display the next line while the tail terminal displays it instantly.

EDIT: forgot to add this.
Also, sometimes new text does not appear until even more txt is added to the log file.
Example:
'test 1' is written to the log
tail will show this right away, tail | grep will NOT
'test 2' is written to the log
tail will show this right away, tail | grep will show both 'test1' and 'test2' now



Any hints for me? Thanks in advance
Last edited by WasabiVengeance; 04-22-2008 at 05:23 PM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Help with tail /grep needed

Hello: I'm a very newbee at UNIX/AIX. What i want to do is to tail a file from the bottom until a certain string is found and write all the lines after the found string to another file. I've tried out a lot of combination with tail and grep but doesn't find the good one. Could someone help... (4 Replies)
Discussion started by: Felix2511
4 Replies

2. Shell Programming and Scripting

tail, grep and cut

Hello all, I have some weird problem that kinda baffles me. Say I have the following test file: claudia:~/tmp$ cat testfile.txt This is a test line This is the second test line And yeah, this is the third test line Then say I want to tail the file, grep for the word "third" then... (7 Replies)
Discussion started by: sylaan
7 Replies

3. Shell Programming and Scripting

Grep or Tail in shell script

Hi, I am writing a shell script that checks catalina logs on a production system and mails me if it detects errors. It greps the logs for known errors which i have defined as variables. The problem is the logs are huge, approx 30,000 before they rotate. So I am forced to use grep instead... (3 Replies)
Discussion started by: Moxy
3 Replies

4. Shell Programming and Scripting

tail | grep

The program that is running on my machine generates log files. I want to be able to know the number of lines that contain "FT" in the most recent log file. I wrote the following, but it always returns zero. And I know the count is not zero. Any ideas? ls -rt *.log | tail -n 1 | grep -c FT (6 Replies)
Discussion started by: sdilucca
6 Replies

5. UNIX for Dummies Questions & Answers

Need help with a tail and a grep

I need to tail -f a file so I can monitor it as it is being written to. However, there is a lot of garbage in the file that I don't care about. So normally I would just pipe and grep for the string that is important to me. However, in this case, there are two things I need to grep for. I can't... (3 Replies)
Discussion started by: Silver11
3 Replies

6. UNIX for Dummies Questions & Answers

mulitple grep using tail

I have a basic tail/grep question. I have logs that are generated & kept in a directory called alert_audit. I am using "tail" to see the logs that are coming in, but I only need logs that contain the IP address 10.249.185. or 10.247.231. Here is the command I have, but it pulls all IP... (3 Replies)
Discussion started by: robertson1995
3 Replies

7. Shell Programming and Scripting

How do i use tail & grep in real time here??

Hi I have a file which is updated very frequently. Where in i wanted to use tail -f command in the script and wanted to grep for a particular word. But the issue is when i use tail -f filename|grep "word" ... it will show me blank until the word is found in the real time. if it shows... (13 Replies)
Discussion started by: nikhil jain
13 Replies

8. UNIX for Dummies Questions & Answers

Tail -f | grep > output.txt

hi guys, I perform a sort of monitoring. I have a server running and with tail -f | grep "Searchstring"I monitor the log-file for recent specific entries. This is ok and works fine. Now, in addition I want to have my search results not posted into the shell but into a file. I tried: tail... (3 Replies)
Discussion started by: LaUs3r
3 Replies

9. Debian

LM 17.3 xfce constant lagging

I'm using LM 17.3x LIVE. Have constant and sometimes, severe lagging issues. Problems started when I "upgraded" to 18.3x. I tried 5 other distros all with the same issues. Went back to 17.3 and , alas, the problem followed. Found this: sudo gedit /etc/sysctl. conf vm. swappiness = 15, but all it... (4 Replies)
Discussion started by: 69Rixter
4 Replies

10. Shell Programming and Scripting

Trouble with tail and grep

Good Morning, i ran into some trouble this morning while 'improving' my monitoring stuff. i would like to get a warning when the number of mails sent (outbound) by postfix is above a certain number. so far, so easy. to test that i simply put cat /var/log/mail.info | grep 'to=<' | grep -v -e... (1 Reply)
Discussion started by: Mike
1 Replies

LEARN ABOUT DEBIAN

logtop

LOGTOP(1)						      General Commands Manual							 LOGTOP(1)

NAME

       logtop - Realtime log line rate analyser

SYNOPSIS

       logtop [OPTIONS]

DESCRIPTION

	logtop is a System Administrator tool analyzing line rate on stdin.
	It reads on stdin and print a constantly updated result
	displaying, in columns:
	Line number, count, frequency, and the actual line.

	 $ tail -f FILE | logtop
	is the friendly version of:
	 $ watch 'tail FILE | sort | uniq -c | sort -gr'

OPTIONS

       -s, --size=K
	      Only keep K lines in memory, instead of 10000.

       -q, --quiet
	      Do not display a live view of the data, only display a top at exit.

       -l, --line-by-line=K
	      Print result line by line, in a machine friendly format, K is the number of result to print per line.

		Line by line format is : [%d %f %s	]*

		  %d : Number of occurences
		  %f : Frequency of apparition
		  %s : String (Control chars replaced by dots.

       -i, --interval=K
	      Interval between graphical updates, in seconds. Defaults to 1.

       -h, --help
	      Show summary of options.

       -v, --version
	      Show version of program.

EXAMPLES

       Here are some logtop usage examples.

       tail -f cache.log | grep -o "HIT|MISS" | logtop

       Realtime hit / miss ratio on some caching software log file.

       tail -f access.log | cut -d' ' -f1 | logtop -s 10000

       Realtime most querying IPs on your server, as long as log lines in access.log starts with the client IP.

       tail -f access.log | cut -d' ' -f7 | logtop -s 10000

       Realtime most requested web pages in a NCSA like log file.

       cat auth.log | grep -v "CRON" | grep -o ": .*" | logtop -q -s 100000

       Display a one-shot simple analyse of your auth.log.

SEE ALSO

       watch(1)

AUTHOR

       logtop was written by Julien Palard.

       This manual page was written by Julien Palard <julien@palard.fr>, for the Debian project (and may be used by others).

								  April 16, 2011							 LOGTOP(1)
All times are GMT -4. The time now is 08:28 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy