Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Constantly updating log files (tail -f? grep? awk?) Post 302098381 by nortonloaf on Sunday 3rd of December 2006 10:20:01 PM
Old 12-03-2006
Constantly updating log files (tail -f? grep? awk?)

I have a log file which is continuously added to, called log.file. I'd like to
monitor this file, and when certain lines are found, update some totals in
another file. I've played around with tail -f, grep, and awk, but can't seem
to hit the right note, so to speak.

The lines I'm interested in will have a "|" in them, will have one of these
words (stockA, stockB, stockC), and will not have the word "DEBUG". Here
are a few example lines that meet those criteria:

| 3: stockA 3.45 ---- 1408.00 90.00 159.00
|B 3: stockA 3.46 - - - ---- 1487.00 440.00 0.00
| 3: stockC 17.53 - - - ---- 1497.00 10.00 15.00

I'd like to have a 2nd file (that will then be used on a webpage) that
has a total for each stockA/stockB/etc. When a line is found that
satisfies the conditions above, I'd like to take the last number in the
line, subtract the 2nd to last number, then adjust the corresponding
total in the 2nd file. So for the above lines, if file2 looked like this:

stockA 225
stockB 23500
stockC -50

after all three lines had been taken care of, it would look like:

stockA 294
stockB 23060
stockC -45

I first tried using grep, and thought this was the right way to start:

tail -f log.file | grep "|" | grep "stockA\|stockB" | grep -v "DEBUG" >> 2nd.file

but it doesn't seem to be working exactly right. Also, i'm not sure how to
then adjust the totals in 2nd.file. I looked at using awk, but I have no experience
wtih it.

Any kind of help or pointers from a kind soul would be appreciated!
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Korn shell awk use for updating two files

Hi, I have two text files containing records in following format: file1 format is: name1 age1 nickname1 path1 name2 age2 nickname2 path2 file 1 example is: abcd 13 abcd.13 /home/temp/abcd.13 efgh 15 efgh.15 /home/temp/new/efgh.15 (4 Replies)
Discussion started by: alrinno
4 Replies

2. Shell Programming and Scripting

awk updating one file with another, comparing, updating

Hello, I read and search through this wonderful forum and tried different approaches but it seems I lack some knowledge and neurones ^^ Here is what I'm trying to achieve : file1: test filea 3495; test fileb 4578; test filec 7689; test filey 9978; test filez 12300; file2: test filea... (11 Replies)
Discussion started by: mecano
11 Replies

3. Shell Programming and Scripting

tail | grep

The program that is running on my machine generates log files. I want to be able to know the number of lines that contain "FT" in the most recent log file. I wrote the following, but it always returns zero. And I know the count is not zero. Any ideas? ls -rt *.log | tail -n 1 | grep -c FT (6 Replies)
Discussion started by: sdilucca
6 Replies

4. HP-UX

tape device files constantly being recreated

Hi, We've been having problems with our NetWorker Silo servers (running HP-UX) concerning the tape device files under /dev/rmt. For some reason they are constantly being removed/recreated (up to a couple of times per hour): crw-rw-rw- 1 bin bin 205 0x240300 Apr 5 19:07... (1 Reply)
Discussion started by: piper28
1 Replies

5. Shell Programming and Scripting

Piping tail to awk to parse a log file

Hello all, I've got what I'm pretty sure is a simple problem, but I just can't seem to work past it. I'm trying to use awk to pretty up a log file, and calculate a percentage. The log file looks like this: # tail strtovrUsage 20090531-18:15:45 RSreq - 24, RSsuc - 24, RSrun - 78, RSerr -... (4 Replies)
Discussion started by: DeCoTwc
4 Replies

6. UNIX for Dummies Questions & Answers

Need help with a tail and a grep

I need to tail -f a file so I can monitor it as it is being written to. However, there is a lot of garbage in the file that I don't care about. So normally I would just pipe and grep for the string that is important to me. However, in this case, there are two things I need to grep for. I can't... (3 Replies)
Discussion started by: Silver11
3 Replies

7. Shell Programming and Scripting

Comparing 2 files with awk and updating 2nd file

file1: (unique files) 1 /pub/atomicbk/catalog/catalog.gif 693 2 /pub/atomicbk/catalog/home.gif 813 3 /pub/atomicbk/catalog/logo2.gif 12871 4 /pub/atomicbk/catalog/sleazbk.html 18338 file2: (duplicate filenames allowed) 28/Aug/1995:00:00:38 1 /pub/atomicbk/catalog/home.gif 813... (2 Replies)
Discussion started by: jontjioe
2 Replies

8. Shell Programming and Scripting

script to constantly read the last 500 new logs in a log file

Hello, I would like to write a bash script that would monitor a log file for a certain number of logs, let's say 500 logs and when it reaches that number to write the last log to another file. For example, I want to watch the /var/adm/messages and everytime, there is 500 new logs that are... (1 Reply)
Discussion started by: Pouchie1
1 Replies

9. Shell Programming and Scripting

how to run tail -f for 3 log files from a script

hi i need to run from a bash script tail -f /var/log/access_log >> access1 tail -f /var/log/prod/prod1 >> access1 tail -f /var/log/prod/prod2 >> access1 this script purpose is to start at server boot time and should always run. what is the best way to put it on a script Thanks Dan (1 Reply)
Discussion started by: dan12341234
1 Replies

10. Shell Programming and Scripting

Joining multiple files tail on tail

I have 250 files that have 16 columns each - all numbered as follows stat.1000, stat.1001, stat.1002, stat.1003....stat.1250. I would like to join all 250 of them together tail by tail as follows. For example stat.1000 a b c d e f stat.1001 g h i j k l So that my output... (2 Replies)
Discussion started by: kayak
2 Replies
GREP(1) 						      General Commands Manual							   GREP(1)

NAME
grep, g - search a file for a pattern SYNOPSIS
grep [ option ... ] pattern [ file ... ] g [ option ... ] pattern [ file ... ] DESCRIPTION
Grep searches the input files (standard input default) for lines that match the pattern, a regular expression as defined in regexp(7) with the addition of a newline character as an alternative (substitute for |) with lowest precedence. Normally, each line matching the pattern is `selected', and each selected line is copied to the standard output. The options are -c Print only a count of matching lines. -h Do not print file name tags (headers) with output lines. -e The following argument is taken as a pattern. This option makes it easy to specify patterns that might confuse argument parsing, such as -n. -i Ignore alphabetic case distinctions. The implementation folds into lower case all letters in the pattern and input before interpre- tation. Matched lines are printed in their original form. -l (ell) Print the names of files with selected lines; don't print the lines. -L Print the names of files with no selected lines; the converse of -l. -n Mark each printed line with its line number counted in its file. -s Produce no output, but return status. -v Reverse: print lines that do not match the pattern. -f The pattern argument is the name of a file containing regular expressions one per line. -b Don't buffer the output: write each output line as soon as it is discovered. Output lines are tagged by file name when there is more than one input file. (To force this tagging, include /dev/null as a file name argument.) Care should be taken when using the shell metacharacters $*[^|()= and newline in pattern; it is safest to enclose the entire expression in single quotes '...'. An expression starting with '*' will treat the rest of the expression as literal characters. G invokes grep with -n and forces tagging of output lines by file name. If no files are listed, it searches all files matching *.C *.b *.c *.h *.m *.cc *.java *.cgi *.pl *.py *.tex *.ms SOURCE
/src/cmd/grep /bin/g SEE ALSO
ed(1), awk(1), sed(1), sam(1), regexp(7) DIAGNOSTICS
Exit status is null if any lines are selected, or non-null when no lines are selected or an error occurs. GREP(1)
All times are GMT -4. The time now is 06:47 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy