Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Script to check response time from nginx logs

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script to check response time from nginx logs
# 1  
Old 01-14-2015
Script to check response time from nginx logs
Hi,

My goal is to monitor the response time from the access logs of nginx server. I am using gawk to print the needed fields - 'response time' and 'name of the service' from nginx logs.

Command: gawk '($6 ~ /cloudservice/) {print $10, $6}' access.log

Output:
0.645 /nc/cloudservice
0.041 /nc/cloudservice
1.346 /nc/cloudservice
0.020 /nc/cloudservice

So, I have the response time for this particular service. Now, I want to compare the response time with a number (say 1.000 in this case) and get the result only with response time > 1.000 and send an email with the output.

Is this possible in a single awk/gawk statement?

Thanks,
Naitik
# 2  
Old 01-14-2015
Code:
gawk '$6 ~ /cloudservice/ && $10+0 > 1.000 { print $10, $6 }' access.log | mailx -s "Subject" user@domain.com

This User Gave Thanks to Yoda For This Post:
nshah11
# 3  
Old 01-14-2015
Wow Yoda!
Thanks for the quick response. It works like a charm. Smilie

---------- Post updated at 06:26 PM ---------- Previous update was at 04:37 PM ----------

I have a question on the same - suggestions are welcome.

The access.log file is not a static file - the server log keeps on populating the file (Also, there is no log rotation on the file).

If I use this command and run it every minute, I get redundant records.

Command:
Code:
gawk '$6 ~ /cloudservice/ && $10+0 > 1.000 { print $10, $6 }' access.log | mailx -s "Subject" user@domain.com

Output:
1.346 /nc/cloudservice

I get this output everytime I run the command. How do I just get the new values in output? The access log file has a date and time stamp for each line.

I am not quite sure how to approach this. Any suggestions?

Thanks,
nshah11
# 4  
Old 01-14-2015
You could have your script "remember" the last line examined in a file eg /srv/nginx_monitor/last_line

Code:
LAST_FILE=/srv/nginx_monitor/last_line

END_LINE=$(wc -l < access.log)
START_LINE=0

if [ -f "$LAST_FILE" ]
then
    read START_LINE < "$LAST_FILE"
    # Check if file has shrunk - could have been truncated
    [ $END_LINE -lt $START_LINE ] && START_LINE=0
fi

gawk -v S=$START_LINE -v E=$END_LINE '
    # Ignore anything appended after we did count
    NR>E{exit}
    NR>=S && $6 ~ /cloudservice/ {print $10, $6}' access.log

echo "$END_LINE" > "$LAST_FILE"

This User Gave Thanks to Chubler_XL For This Post:
nshah11
# 5  
Old 01-15-2015
START_LINE is the last END_LINE, so it is rather
Code:
    (NR>S && $6 ~ /cloudservice/ && $10+0 > 1.0) {print $10, $6}' access.log

together with the first requirement.
These 2 Users Gave Thanks to MadeInGermany For This Post:
Chubler_XL nshah11
# 6  
Old 01-15-2015
Thanks MadeInGermany another out-by-one, I swear it feels like 49% of my errors are of this type.
# 7  
Old 01-15-2015
Thanks Chubler_XL and MadeInGermany for your inputs.

Seemed a little complicated to understand since I am a newbie. Currently, I am using two files and finding the difference of the contents (to find the newly appended records). Have not tested it yet - shall keep you posted once I try it out.

Thanks!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to count respon time max min avg for nginx logs?

Hi All, need your help, i want count respon time max and average my nginx logs, based on hourly or minutes per api... my nginx.log sample : 10.1.1.1 - - "POST /v2/api/find/outlet/ HTTP/1.1" 200 2667 "-" "okhttp/3.12.0" "118.215.153.47" 0.178 0.178 . 10.1.1.1 - - "POST... (4 Replies)
Discussion started by: fajar_3t3
4 Replies

2. Shell Programming and Scripting

Processes running response time

Hi All I have been asked to write scripts within our monitoring tool for a vast requirement set. One of the requirements is below: • Lowest, Highest & Average response times of the Documentum process threads serving client requests Essentially they want a view where we can see the... (4 Replies)
Discussion started by: simpsa27
4 Replies

3. UNIX for Advanced & Expert Users

AIX idea needed to check the logs updated date and time

Hi with the help of Gabriel canepa, i have just edited filename only in his code. The help which i got and he helped is 1) I have around 22 logs and each log should be updated in the last 24 hours from the current timestamp. 2) It should check for ERROR message (not error,Error) in the log and... (2 Replies)
Discussion started by: Kalaihari
2 Replies

4. Shell Programming and Scripting

Script to check logs

I have 5 log files under different directores . say for eg abc under /home/dir1 , xyz under home/dir2 . is there a script that i can run from say /home that searchers all these files for string or combination of strings and write to a file eg search file by timestamp|keyword o/p in a file (6 Replies)
Discussion started by: Nevergivup
6 Replies

5. Solaris

archive logs mount point space check script

I have the below shell script which is checking /archlog mount point space on cappire(solaris 10) server. When the space usage is above 80% it should e-mail. When i tested this script it is working as expected. -------------------------------------------------------------------------... (0 Replies)
Discussion started by: dreams5617
0 Replies

6. Shell Programming and Scripting

Calculate response time from log

I have a running log file (jboss_server.log) which rotates at midnight . I need to constantly check and calculate the time for each thread and alert if it doesnt complete within 60 minute. For example my log file has following printed . I want to run a script in cron every 30 minutes and... (2 Replies)
Discussion started by: gubbu
2 Replies

7. Web Development

[PHP] Script to Time Remote Server Response and Email

Here is a simple PHP script I wrote that times getting a link using mtime with curl and emails if it does not meet my objective. I use it in production for checking the performance of a Content Delivery Network (CDN), but you can use it for any web server. <?php $heartbeat =... (1 Reply)
Discussion started by: Neo
1 Replies

8. UNIX for Advanced & Expert Users

Response time & IO max

in HP-UX how i can measure the response time and how can i find the maximum IO (1 Reply)
Discussion started by: salhoub
1 Replies

9. UNIX for Advanced & Expert Users

Response time under packet loss

I am experiencing a problem where under a dial condition I am experiencing packet loss, which is failrly normal, but the response to the packet loss is taking bewteen 6 and 10 seconds. Could someone please advise what the industry standard is on the response time under a packet loss senario. (1 Reply)
Discussion started by: shane
1 Replies

10. UNIX for Advanced & Expert Users

ls -l : response time slow

Hi all, If I give ls , it lists files in 1 second. It I give ls -l , it takes 8 seconds There are only 55 files in the directory. Any explanation? Thanks Wilson (4 Replies)
Discussion started by: geraldwilson
4 Replies
Login or Register to Ask a Question