Visit Our UNIX and Linux User Community


Constantly updating log files (tail -f? grep? awk?)


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Constantly updating log files (tail -f? grep? awk?)
# 1  
Old 12-03-2006
Constantly updating log files (tail -f? grep? awk?)

I have a log file which is continuously added to, called log.file. I'd like to
monitor this file, and when certain lines are found, update some totals in
another file. I've played around with tail -f, grep, and awk, but can't seem
to hit the right note, so to speak.

The lines I'm interested in will have a "|" in them, will have one of these
words (stockA, stockB, stockC), and will not have the word "DEBUG". Here
are a few example lines that meet those criteria:

| 3: stockA 3.45 ---- 1408.00 90.00 159.00
|B 3: stockA 3.46 - - - ---- 1487.00 440.00 0.00
| 3: stockC 17.53 - - - ---- 1497.00 10.00 15.00

I'd like to have a 2nd file (that will then be used on a webpage) that
has a total for each stockA/stockB/etc. When a line is found that
satisfies the conditions above, I'd like to take the last number in the
line, subtract the 2nd to last number, then adjust the corresponding
total in the 2nd file. So for the above lines, if file2 looked like this:

stockA 225
stockB 23500
stockC -50

after all three lines had been taken care of, it would look like:

stockA 294
stockB 23060
stockC -45

I first tried using grep, and thought this was the right way to start:

tail -f log.file | grep "|" | grep "stockA\|stockB" | grep -v "DEBUG" >> 2nd.file

but it doesn't seem to be working exactly right. Also, i'm not sure how to
then adjust the totals in 2nd.file. I looked at using awk, but I have no experience
wtih it.

Any kind of help or pointers from a kind soul would be appreciated!
 

Previous Thread | Next Thread
Test Your Knowledge in Computers #326
Difficulty: Medium
Time on Computers is commonly referred to as GNU Time.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Joining multiple files tail on tail

I have 250 files that have 16 columns each - all numbered as follows stat.1000, stat.1001, stat.1002, stat.1003....stat.1250. I would like to join all 250 of them together tail by tail as follows. For example stat.1000 a b c d e f stat.1001 g h i j k l So that my output... (2 Replies)
Discussion started by: kayak
2 Replies

2. Shell Programming and Scripting

how to run tail -f for 3 log files from a script

hi i need to run from a bash script tail -f /var/log/access_log >> access1 tail -f /var/log/prod/prod1 >> access1 tail -f /var/log/prod/prod2 >> access1 this script purpose is to start at server boot time and should always run. what is the best way to put it on a script Thanks Dan (1 Reply)
Discussion started by: dan12341234
1 Replies

3. Shell Programming and Scripting

script to constantly read the last 500 new logs in a log file

Hello, I would like to write a bash script that would monitor a log file for a certain number of logs, let's say 500 logs and when it reaches that number to write the last log to another file. For example, I want to watch the /var/adm/messages and everytime, there is 500 new logs that are... (1 Reply)
Discussion started by: Pouchie1
1 Replies

4. Shell Programming and Scripting

Comparing 2 files with awk and updating 2nd file

file1: (unique files) 1 /pub/atomicbk/catalog/catalog.gif 693 2 /pub/atomicbk/catalog/home.gif 813 3 /pub/atomicbk/catalog/logo2.gif 12871 4 /pub/atomicbk/catalog/sleazbk.html 18338 file2: (duplicate filenames allowed) 28/Aug/1995:00:00:38 1 /pub/atomicbk/catalog/home.gif 813... (2 Replies)
Discussion started by: jontjioe
2 Replies

5. UNIX for Dummies Questions & Answers

Need help with a tail and a grep

I need to tail -f a file so I can monitor it as it is being written to. However, there is a lot of garbage in the file that I don't care about. So normally I would just pipe and grep for the string that is important to me. However, in this case, there are two things I need to grep for. I can't... (3 Replies)
Discussion started by: Silver11
3 Replies

6. Shell Programming and Scripting

Piping tail to awk to parse a log file

Hello all, I've got what I'm pretty sure is a simple problem, but I just can't seem to work past it. I'm trying to use awk to pretty up a log file, and calculate a percentage. The log file looks like this: # tail strtovrUsage 20090531-18:15:45 RSreq - 24, RSsuc - 24, RSrun - 78, RSerr -... (4 Replies)
Discussion started by: DeCoTwc
4 Replies

7. HP-UX

tape device files constantly being recreated

Hi, We've been having problems with our NetWorker Silo servers (running HP-UX) concerning the tape device files under /dev/rmt. For some reason they are constantly being removed/recreated (up to a couple of times per hour): crw-rw-rw- 1 bin bin 205 0x240300 Apr 5 19:07... (1 Reply)
Discussion started by: piper28
1 Replies

8. Shell Programming and Scripting

tail | grep

The program that is running on my machine generates log files. I want to be able to know the number of lines that contain "FT" in the most recent log file. I wrote the following, but it always returns zero. And I know the count is not zero. Any ideas? ls -rt *.log | tail -n 1 | grep -c FT (6 Replies)
Discussion started by: sdilucca
6 Replies

9. Shell Programming and Scripting

awk updating one file with another, comparing, updating

Hello, I read and search through this wonderful forum and tried different approaches but it seems I lack some knowledge and neurones ^^ Here is what I'm trying to achieve : file1: test filea 3495; test fileb 4578; test filec 7689; test filey 9978; test filez 12300; file2: test filea... (11 Replies)
Discussion started by: mecano
11 Replies

10. UNIX for Dummies Questions & Answers

Korn shell awk use for updating two files

Hi, I have two text files containing records in following format: file1 format is: name1 age1 nickname1 path1 name2 age2 nickname2 path2 file 1 example is: abcd 13 abcd.13 /home/temp/abcd.13 efgh 15 efgh.15 /home/temp/new/efgh.15 (4 Replies)
Discussion started by: alrinno
4 Replies

Featured Tech Videos