How to count respon time max min avg for nginx logs?


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers How to count respon time max min avg for nginx logs?
# 1  
Old 08-25-2019
How to count respon time max min avg for nginx logs?

Hi All,


need your help, i want count respon time max and average my nginx logs, based on hourly or minutes per api...



my nginx.log sample :


Code:
10.1.1.1 - - [25/Aug/2019:05:26:30 +0700] "POST /v2/api/find/outlet/ HTTP/1.1" 200 2667 "-" "okhttp/3.12.0" "118.215.153.47" 0.178 0.178 .
10.1.1.1 - - [25/Aug/2019:05:26:30 +0700] "POST /v2/api/find/outlet/ HTTP/1.1" 200 2847 "-" "okhttp/3.12.0" "189.246.151.188" 0.177 0.178 .
10.1.1.1 - - [25/Aug/2019:05:27:52 +0700] "GET /v2/api/menu/category HTTP/1.1" 401 40 "-" "okhttp/3.12.0" "139.194.84.246" 0.007 0.007 .
10.1.1.1 - - [25/Aug/2019:05:27:52 +0700] "GET /v2/api/user/point HTTP/1.1" 200 152 "-" "okhttp/3.12.0" "202.80.217.172" 0.028 0.028 .
10.1.1.1 - - [25/Aug/2019:05:27:52 +0700] "GET /v2/api/user/destination HTTP/1.1" 200 169 "-" "okhttp/3.12.0" "36.91.42.35" 0.019 0.019 .
10.1.1.1 - - [25/Aug/2019:05:28:52 +0700] "POST /v2/api/transaction/inquiry HTTP/1.1" 200 503 "-" "okhttp/3.12.0" "36.89.234.129" 0.374 0.374 .
10.1.1.1 - - [25/Aug/2019:05:28:52 +0700] "POST /v2/api/transaction/confirm HTTP/1.1" 200 874 "-" "okhttp/3.12.0" "36.89.234.129" 0.394 0.394 .
10.1.1.1 - - [25/Aug/2019:05:28:52 +0700] "GET /v2/api/user/point HTTP/1.1" 200 152 "-" "okhttp/3.12.0" "114.5.147.117" 0.024 0.024 .
10.1.1.1 - - [25/Aug/2019:05:28:52 +0700] "GET /v2/api/menu/category HTTP/1.1" 403 40 "-" "okhttp/3.12.0" "139.194.84.246" 0.003 0.003 .

my expectation sample below :


Code:
date                       |  api                               | max| avg

25/Aug/2019:05:26 /v2/api/find/outlet             2847 2757

25/Aug/2019:05:27 /v2/api/menu/category HTTP/1.1               1847 1757
25/Aug/2019:05:28 /v2/api/menu/category HTTP/1.1               1147 1257

i already trying with this awk but only qot average :


Code:
awk '/25\/Aug\/2019:18/ {c++} END{print c}' access.log

please help ...


Thanks
# 2  
Old 08-25-2019
- start with:
Code:
awk '
$4 ~ /[0-9]:[0-9]/ {
date=$4; sub(/^[^0-9]*/, "", date); sub(/:[0-9][0-9]$/, "", date);
id=date OFS $7; api[id]; apic[id]++; apit[id]+=$10; if ($10>apim[id]) apim[id]=$10;
date=$4; sub(/^[^0-9]*/, "", date); sub(/:[0-9][0-9]:[0-9][0-9]$/, " ", date);
id=date OFS $7; api[id]; apic[id]++; apit[id]+=$10; if ($10>apim[id]) apim[id]=$10;
}
END {
print "date/time" OFS "api" OFS "max" OFS "avg"
for (i in api) print i OFS apim[i] OFS apit[i] / apic[i];
}
' OFS="\t" access.log | sort -r


Last edited by Neo; 02-28-2020 at 01:18 AM..
# 3  
Old 08-25-2019
Wouldn't you need to sum up values to calculate an average?


If you want hourly averages - as indicated - , try this slight modification of rdrtx1's proposal:

Code:
awk '
$4 ~ /[0-9]:[0-9]/      {date=$4
                         sub(/^[^0-9]*/, "", date)
                         sub (/:[0-9]*:[0-9]*$/, "", date)                  # get rid of mins and secs for the hourly stuff
                         id=date OFS $7
                         apic[id]++
                         apis[id] += $10                                    # sum up values for later avg calc.
                         if ($10>apim[id]) apim[id] = $10;
}
END                     {print "date" OFS "api" OFS "max" OFS "avg"
                         for (i in apic) print i OFS apim[i] OFS apis[i] / apic[i];
                        }
' OFS="\t" file

# 4  
Old 08-25-2019
Hi rdrtx1


thanks for reply...your awk will generate per second right?

--- Post updated at 07:46 PM ---

Hi RudiC


Thanks Alot for your response...


this awk use for hour right? how to convert to minutes?


Thanks
Fajar
# 5  
Old 08-25-2019
See updated of post # 3. mins, hrs max and avg included.

Last edited by RavinderSingh13; 02-28-2020 at 02:10 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to count average and max respon time?

sorry i will revise first (1 Reply)
Discussion started by: fajar_3t3
1 Replies

2. Shell Programming and Scripting

Find min and max time taken from a log file

You have a log file as attached in sample input with various operations and time taken by each of them. Write a script to find the min and max time taken for each operation. Sample output is attached. Sample Input is given as below: operation1,83621 operation2,72321 operation3,13288... (1 Reply)
Discussion started by: Chandan_Bose
1 Replies

3. Shell Programming and Scripting

Script to check response time from nginx logs

Hi, My goal is to monitor the response time from the access logs of nginx server. I am using gawk to print the needed fields - 'response time' and 'name of the service' from nginx logs. Command: gawk '($6 ~ /cloudservice/) {print $10, $6}' access.log Output: 0.645 /nc/cloudservice... (6 Replies)
Discussion started by: nshah11
6 Replies

4. Shell Programming and Scripting

Get the min avg and max with awk

aaa: 3 ms aaa: 2 ms aaa: 5 ms aaa: 10 ms .......... to get the 3 2 5 10 ...'s min avg and max something like min: 2 ms avg: 5 ms max: 10 ms (2 Replies)
Discussion started by: yanglei_fage
2 Replies

5. AIX

Nmon max and avg for cpu and memory

Hi All, Anyone know how to capture the nmon avg and max cpu and memory for one of the AIX server for Monthly Utilization Report purposes ? Thanks. ---------- Post updated at 05:18 AM ---------- Previous update was at 05:07 AM ---------- if possible use shell script to count or sum... (6 Replies)
Discussion started by: ckwan
6 Replies

6. Shell Programming and Scripting

How to find the average,min,max ,total count?

Hi , Below is my sample data,I have this 8 column(A,B,C,D,E,F,G,H) in csv file. A , B ,C ,D ,E ,F,G ,H 4141,127337,24,15,20,69,72.0,-3 4141,128864,24,15,20,65,66.0,-1 4141,910053,24,15,4,4,5.0,-1 4141,910383,24,15,22,3,4.0,-1 4141,496969,24,15,14,6,-24.0,-18... (7 Replies)
Discussion started by: vinothsekark
7 Replies

7. Shell Programming and Scripting

Count time min/max/average for ping

I am redirecting my ping output to a file. The sample output is like this: 64 bytes from xx.xx.xx.167: icmp_seq=4490 ttl=116 3.75 ms 2011Jul12- 15 40 16 64 bytes from xx.xx.xx.167: icmp_seq=4491 ttl=116 5.29 ms 2011Jul12- 15 40 17 64 bytes from xx.xx.xx.167: icmp_seq=4492 ttl=116 4.88 ms... (6 Replies)
Discussion started by: zorrox
6 Replies

8. Shell Programming and Scripting

get min, max and average value

hi! i have a file like the attachement. I'd like to get for each line the min, max and average values. (there is 255 values for each line) how can i get that ? i try this, is it right? BEGIN {FS = ","; OFS = ";";max=0;min=0;moy=0;total=0;freq=890} $0 !~ /Trace1:/ { ... (1 Reply)
Discussion started by: riderman
1 Replies

9. UNIX for Dummies Questions & Answers

Iterate a min/max awk script over time-series temperature data

I'm trying to iterate a UNIX awk script that returns min/max temperature data for each day from a monthly weather data file (01_weath.dat). The temperature data is held in $5. The temps are reported each minute so each day contains 1440 temperature enteries. The below code has gotten me as far as... (5 Replies)
Discussion started by: jgourley
5 Replies

10. Shell Programming and Scripting

min and max value of process id

We are running a AIX 5.2 OS. Would anyone happen to know what the max value for a process id could be? Thanks jerardfjay :) (0 Replies)
Discussion started by: jerardfjay
0 Replies
Login or Register to Ask a Question