12-03-2006
Constantly updating log files (tail -f? grep? awk?)
I have a log file which is continuously added to, called log.file. I'd like to
monitor this file, and when certain lines are found, update some totals in
another file. I've played around with tail -f, grep, and awk, but can't seem
to hit the right note, so to speak.
The lines I'm interested in will have a "|" in them, will have one of these
words (stockA, stockB, stockC), and will not have the word "DEBUG". Here
are a few example lines that meet those criteria:
| 3: stockA 3.45 ---- 1408.00 90.00 159.00
|B 3: stockA 3.46 - - - ---- 1487.00 440.00 0.00
| 3: stockC 17.53 - - - ---- 1497.00 10.00 15.00
I'd like to have a 2nd file (that will then be used on a webpage) that
has a total for each stockA/stockB/etc. When a line is found that
satisfies the conditions above, I'd like to take the last number in the
line, subtract the 2nd to last number, then adjust the corresponding
total in the 2nd file. So for the above lines, if file2 looked like this:
stockA 225
stockB 23500
stockC -50
after all three lines had been taken care of, it would look like:
stockA 294
stockB 23060
stockC -45
I first tried using grep, and thought this was the right way to start:
tail -f log.file | grep "|" | grep "stockA\|stockB" | grep -v "DEBUG" >> 2nd.file
but it doesn't seem to be working exactly right. Also, i'm not sure how to
then adjust the totals in 2nd.file. I looked at using awk, but I have no experience
wtih it.
Any kind of help or pointers from a kind soul would be appreciated!
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi,
I have two text files containing records in following format:
file1 format is:
name1 age1 nickname1 path1
name2 age2 nickname2 path2
file 1 example is:
abcd 13 abcd.13 /home/temp/abcd.13
efgh 15 efgh.15 /home/temp/new/efgh.15 (4 Replies)
Discussion started by: alrinno
4 Replies
2. Shell Programming and Scripting
Hello,
I read and search through this wonderful forum and tried different approaches but it seems I lack some knowledge and neurones ^^
Here is what I'm trying to achieve :
file1:
test filea 3495;
test fileb 4578;
test filec 7689;
test filey 9978;
test filez 12300;
file2:
test filea... (11 Replies)
Discussion started by: mecano
11 Replies
3. Shell Programming and Scripting
The program that is running on my machine generates log files. I want to be able to know the number of lines that contain "FT" in the most recent log file. I wrote the following, but it always returns zero. And I know the count is not zero. Any ideas?
ls -rt *.log | tail -n 1 | grep -c FT (6 Replies)
Discussion started by: sdilucca
6 Replies
4. HP-UX
Hi,
We've been having problems with our NetWorker Silo servers (running HP-UX) concerning the tape device files under /dev/rmt. For some reason they are constantly being removed/recreated (up to a couple of times per hour):
crw-rw-rw- 1 bin bin 205 0x240300 Apr 5 19:07... (1 Reply)
Discussion started by: piper28
1 Replies
5. Shell Programming and Scripting
Hello all,
I've got what I'm pretty sure is a simple problem, but I just can't seem to work past it. I'm trying to use awk to pretty up a log file, and calculate a percentage.
The log file looks like this:
# tail strtovrUsage
20090531-18:15:45 RSreq - 24, RSsuc - 24, RSrun - 78, RSerr -... (4 Replies)
Discussion started by: DeCoTwc
4 Replies
6. UNIX for Dummies Questions & Answers
I need to tail -f a file so I can monitor it as it is being written to. However, there is a lot of garbage in the file that I don't care about. So normally I would just pipe and grep for the string that is important to me. However, in this case, there are two things I need to grep for. I can't... (3 Replies)
Discussion started by: Silver11
3 Replies
7. Shell Programming and Scripting
file1: (unique files)
1 /pub/atomicbk/catalog/catalog.gif 693
2 /pub/atomicbk/catalog/home.gif 813
3 /pub/atomicbk/catalog/logo2.gif 12871
4 /pub/atomicbk/catalog/sleazbk.html 18338
file2: (duplicate filenames allowed)
28/Aug/1995:00:00:38 1 /pub/atomicbk/catalog/home.gif 813... (2 Replies)
Discussion started by: jontjioe
2 Replies
8. Shell Programming and Scripting
Hello,
I would like to write a bash script that would monitor a log file for a certain number of logs, let's say 500 logs and when it reaches that number to write the last log to another file. For example, I want to watch the /var/adm/messages and everytime, there is 500 new logs that are... (1 Reply)
Discussion started by: Pouchie1
1 Replies
9. Shell Programming and Scripting
hi
i need to run from a bash script
tail -f /var/log/access_log >> access1
tail -f /var/log/prod/prod1 >> access1
tail -f /var/log/prod/prod2 >> access1
this script purpose is to start at server boot time
and should always run.
what is the best way to put it on a script
Thanks
Dan (1 Reply)
Discussion started by: dan12341234
1 Replies
10. Shell Programming and Scripting
I have 250 files that have 16 columns each - all numbered as follows stat.1000, stat.1001, stat.1002, stat.1003....stat.1250.
I would like to join all 250 of them together tail by tail as follows. For example
stat.1000
a b c
d e f
stat.1001
g h i
j k l
So that my output... (2 Replies)
Discussion started by: kayak
2 Replies