Grep or Tail in shell script

Login or Register for Dates, Times and to Reply

Thread Tools Search this Thread
# 1  
Grep or Tail in shell script


I am writing a shell script that checks catalina logs on a production system and mails me if it detects errors.
It greps the logs for known errors which i have defined as variables.
The problem is the logs are huge, approx 30,000 before they rotate.
So I am forced to use grep instead of tailing 30,000 lines.

But while testing grep I end up getting re-notified of previous errors that still exist in the logs.

Is there another way of doing this?
# 2  
Save a wc -l result from the logfile in a file, then use sed to resume where you left off
start=$(cat wherewasi)
now=$(sed -n '=$' logfile)
if [[ $now -lt $start]] ; then
echo $now > wherewasi
sed -n "$start, $now p" logfile | grep 'stuff i want to find'

# 3  
Great idea, thanks Jim
# 4  
That is quite an expensive way to do it since it causes sed to read the entire file twice per iteration. You can instead obtain the $now from the file size, and use tail -c to jump to that offset:

now=$(ls -l moxy.log | awk '{print $5}')
if [[ $now -lt $start ]] ; then
echo $now > wherewasi
tail -c +$(($start + 1)) moxy.log | grep 'stuff i want to find'

Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Trouble with tail and grep

Good Morning, i ran into some trouble this morning while 'improving' my monitoring stuff. i would like to get a warning when the number of mails sent (outbound) by postfix is above a certain number. so far, so easy. to test that i simply put cat /var/log/ | grep 'to=<' | grep -v -e... (1 Reply)
Discussion started by: Mike
1 Replies

2. UNIX for Dummies Questions & Answers

Tail -f | grep > output.txt

hi guys, I perform a sort of monitoring. I have a server running and with tail -f | grep "Searchstring"I monitor the log-file for recent specific entries. This is ok and works fine. Now, in addition I want to have my search results not posted into the shell but into a file. I tried: tail... (3 Replies)
Discussion started by: LaUs3r
3 Replies

3. Shell Programming and Scripting

How to tail -f multi logfile in 1 shell script.?

Hello, How to tail -f multi logfile from multi path in 1 shell script. File & Path /usr/home/localmode/mode110l/log/logic/number110/digit110_digit110m4_2013050210.txt /usr/home/localmode/mode103l/log/logic/number103/digit103_digit103m4_2013050210.txt... (4 Replies)
Discussion started by: ooilinlove
4 Replies

4. Shell Programming and Scripting

[linux] tail2notify - script interface between tail -f + grep and notify-send

This isn't exactly a question. Just thought I'd share something I just wrote and found useful. For those of you on modern linux boxen: you may be aware that there's a lovely little tool called notify-send that you can use to send notifications to the desktop. Any experienced shell-scripter could... (0 Replies)
Discussion started by: ryran
0 Replies

5. UNIX for Dummies Questions & Answers

mulitple grep using tail

I have a basic tail/grep question. I have logs that are generated & kept in a directory called alert_audit. I am using "tail" to see the logs that are coming in, but I only need logs that contain the IP address 10.249.185. or 10.247.231. Here is the command I have, but it pulls all IP... (3 Replies)
Discussion started by: robertson1995
3 Replies

6. UNIX for Dummies Questions & Answers

Need help with a tail and a grep

I need to tail -f a file so I can monitor it as it is being written to. However, there is a lot of garbage in the file that I don't care about. So normally I would just pipe and grep for the string that is important to me. However, in this case, there are two things I need to grep for. I can't... (3 Replies)
Discussion started by: Silver11
3 Replies

7. Shell Programming and Scripting

tail | grep

The program that is running on my machine generates log files. I want to be able to know the number of lines that contain "FT" in the most recent log file. I wrote the following, but it always returns zero. And I know the count is not zero. Any ideas? ls -rt *.log | tail -n 1 | grep -c FT (6 Replies)
Discussion started by: sdilucca
6 Replies

8. Shell Programming and Scripting

tail | grep lagging badly

I'm trying to use tail/grep to monitor a log file. The command I cooked up is: tail -n 50 -f output.log | grep 'type:system' | cut -f 5- A sample line from the log file is: 1208894862 type:system session:0 severity:4 load started the columns are tab delimited. this works ok, except... (3 Replies)
Discussion started by: WasabiVengeance
3 Replies

9. Shell Programming and Scripting

tail, grep and cut

Hello all, I have some weird problem that kinda baffles me. Say I have the following test file: claudia:~/tmp$ cat testfile.txt This is a test line This is the second test line And yeah, this is the third test line Then say I want to tail the file, grep for the word "third" then... (7 Replies)
Discussion started by: sylaan
7 Replies

10. UNIX for Dummies Questions & Answers

Help with tail /grep needed

Hello: I'm a very newbee at UNIX/AIX. What i want to do is to tail a file from the bottom until a certain string is found and write all the lines after the found string to another file. I've tried out a lot of combination with tail and grep but doesn't find the good one. Could someone help... (4 Replies)
Discussion started by: Felix2511
4 Replies

Featured Tech Videos