Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Piping tail to awk to parse a log file
Top Forums Shell Programming and Scripting Piping tail to awk to parse a log file Post 302321340 by DeCoTwc on Sunday 31st of May 2009 09:46:55 PM
Old 05-31-2009
Piping tail to awk to parse a log file
Hello all,

I've got what I'm pretty sure is a simple problem, but I just can't seem to work past it. I'm trying to use awk to pretty up a log file, and calculate a percentage.

The log file looks like this:

Code:
[root@mys-nycnm-srv01 rtaStats]# tail strtovrUsage 
20090531-18:15:45  RSreq - 24, RSsuc - 24, RSrun - 78, RSerr - 0,
20090531-18:25:45  RSreq - 17, RSsuc - 17, RSrun - 72, RSerr - 0,
20090531-18:35:45  RSreq - 19, RSsuc - 19, RSrun - 72, RSerr - 0,
20090531-18:45:45  RSreq - 16, RSsuc - 16, RSrun - 60, RSerr - 0,
20090531-18:55:45  RSreq - 14, RSsuc - 14, RSrun - 62, RSerr - 0,
20090531-19:05:46  RSreq - 27, RSsuc - 26, RSrun - 61, RSerr - 1,
20090531-19:15:46  RSreq - 37, RSsuc - 37, RSrun - 74, RSerr - 0,
20090531-19:25:46  RSreq - 48, RSsuc - 46, RSrun - 84, RSerr - 2,
20090531-19:35:46  RSreq - 54, RSsuc - 54, RSrun - 106, RSerr - 0,
20090531-19:36:52  RSreq - 0, RSsuc - 0, RSrun - 107, RSerr - 0,


If I fun something simple like:

Code:
[root@mys-nycnm-srv01 ~]# cat `which SOperf`
 tail -f /usr/local/mystro/scripts/twctools/rtaStats/strtovrUsage|awk '{print "Requests = " $4 "Success = " $7  "Running Streams = " $10 "Errors = " $13 "Percentage = " ($7 / $4 )*100 }'

[root@mys-nycnm-srv01 ~]# SOperf 
Requests = 47,Success = 47,Running Streams = 122,Errors = 0,Percentage = 100
Requests = 62,Success = 60,Running Streams = 122,Errors = 2,Percentage = 96.7742
Requests = 56,Success = 56,Running Streams = 132,Errors = 0,Percentage = 100
Requests = 50,Success = 50,Running Streams = 124,Errors = 0,Percentage = 100
Requests = 65,Success = 63,Running Streams = 138,Errors = 0,Percentage = 96.9231

It works, but it's a little simpler than what I'd like. I tried doing the following:

Code:
#!/bin/bash

printf "Time\t\t\tRequests\tSuccess\tRunning\tError\tPercentage\n"

tail   /usr/local/mystro/scripts/twctools/rtaStats/strtovrUsage |\
awk         \
'***** = $1} \
 {req = $4} \
 {suc = $7} \
 {err = $13}\
 {run = $10}\
  END       \
{print tim "\t" req "\t\t" suc "\t" run "\t" err "\t" (suc / req)*100 }'

Time                    Requests        Success Running Error   Percentage
20090531-21:35:47       65,             64,     146,    1,      98.4615

But I only get the one line of results, and it won't work with a tail -f.
If it matters, here is the information for the system I'm working on.

Code:
Linux mys-nycnm-srv01 2.6.9-34.ELsmp #1 SMP Fri Feb 24 16:54:53 EST 2006 i686 i686 i386 GNU/Linux
/usr/bin/awk: symbolic link to `../../bin/gawk'

Also, is there an effective way to work in "divide by 0" error protection into what I'm trying to do (bearing in mind my obviously low skill level?)
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

tail -f a log file redirected into a new window?

Is this possible? I am attempting to display a new xterm window and tail -f the log file within that new window. I am currently working on a solaris 8 machine if that has any different meaning than the other platforms. As you can see, I am a newbie to this forum and to UNIX. Any help would be... (2 Replies)
Discussion started by: douknownam
2 Replies

2. Shell Programming and Scripting

Bash tail monitor log file

Hi there, I have a problem here that involves bash script since I was noob in that field. Recently, I have to monitor data involve in logs so I just run command tail -f for the monitoring. The logs was generate every hour so I need to quickly change my logs every time the new hour hits according... (2 Replies)
Discussion started by: kriezo
2 Replies

3. UNIX for Dummies Questions & Answers

Constantly updating log files (tail -f? grep? awk?)

I have a log file which is continuously added to, called log.file. I'd like to monitor this file, and when certain lines are found, update some totals in another file. I've played around with tail -f, grep, and awk, but can't seem to hit the right note, so to speak. The lines I'm... (0 Replies)
Discussion started by: nortonloaf
0 Replies

4. Shell Programming and Scripting

piping output of tail running in background

Not sure why this does not work in bash: tail -f err.log |& -bash: syntax error near unexpected token `&' I am attempting to continuously read a file that is being updated by doing a "tail -f" on the file and piping the output to stdin which can then be read by the next shell command Thnx (4 Replies)
Discussion started by: anuramdas
4 Replies

5. Shell Programming and Scripting

how to tail a log file..

Hi All.. I have a log file in which all the backup information is stored. Now i have written a script which get the last line in the backup log file.. ssh -l ora${sid} ${primaryhost} "tail -1 /oracle/$ORACLE_SID/sapbackup/back$ORACLE_SID.log" However i would like to tail the line last... (4 Replies)
Discussion started by: suri.tyson
4 Replies

6. Shell Programming and Scripting

[Solved] Help piping tail to read STDIN

Hello everybody, I need some help here. I have a log file that gets updated every hour approximately. I want to make some processing on each line which is added in the log file with a program written in PERL. The problem is that I don't see anything when a line is added in the log file. I... (6 Replies)
Discussion started by: Samb95
6 Replies

7. Shell Programming and Scripting

script to tail file; problem with awk and special characters

Trying to use code that I found to send only new lines out of a log file by doing: while :; do temp=$(tail -1 logfile.out) awk "/$last/{p=1}p" logfile.out #pipe this to log analyzer program last="$temp" sleep 10 done Script works fine when logfile is basic text, but when it contains... (2 Replies)
Discussion started by: moo72moo
2 Replies

8. Shell Programming and Scripting

How to tail -f logfile. if log file is generate every 1 HR.?

Hello, How to tail -f logfile. if log file is gennerate every 1 HR. I want it works automatically all the time. never changes it by manual. Thank ls -trl CybertonTransaction.* -rw-r--r-- 1 autobot robot 617071 Jul 9 00:02 CybertonTransaction.20130709-00.log -rw-r--r-- 1 autobot ... (12 Replies)
Discussion started by: ooilinlove
12 Replies

9. Shell Programming and Scripting

Piping through grep/awk prevents file write

So, this is weird... I'm running this command: iotop -o -P -k -bt -d 5 I'd like to save the output relelvant to rsyslogd to a file, so I do this: iotop -o -P -k -bt -d 5 | grep rsyslogd >> /var/log/rsyslogd Nothing is written to the file! I can write the full output to the file: ... (2 Replies)
Discussion started by: treesloth
2 Replies

10. UNIX for Advanced & Expert Users

Alternative to tail -n -0 -F for monitoring live log file

Hello, I have been working on script which need to generate an alert based upon live logs. If string is found then an alert mail must triggered. tail -n -0 -F works fine to redirect the each latest line from live logs file to grep a pattern for matching but it seems to be not working on... (7 Replies)
Discussion started by: ketanraut
7 Replies
All times are GMT -4. The time now is 05:17 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy