How to count respon time max min avg for nginx logs?


Login or Register for Dates, Times and to Reply

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers How to count respon time max min avg for nginx logs?
# 1  
How to count respon time max min avg for nginx logs?

Hi All,


need your help, i want count respon time max and average my nginx logs, based on hourly or minutes per api...



my nginx.log sample :


Code:
10.1.1.1 - - [25/Aug/2019:05:26:30 +0700] "POST /v2/api/find/outlet/ HTTP/1.1" 200 2667 "-" "okhttp/3.12.0" "118.215.153.47" 0.178 0.178 .
10.1.1.1 - - [25/Aug/2019:05:26:30 +0700] "POST /v2/api/find/outlet/ HTTP/1.1" 200 2847 "-" "okhttp/3.12.0" "189.246.151.188" 0.177 0.178 .
10.1.1.1 - - [25/Aug/2019:05:27:52 +0700] "GET /v2/api/menu/category HTTP/1.1" 401 40 "-" "okhttp/3.12.0" "139.194.84.246" 0.007 0.007 .
10.1.1.1 - - [25/Aug/2019:05:27:52 +0700] "GET /v2/api/user/point HTTP/1.1" 200 152 "-" "okhttp/3.12.0" "202.80.217.172" 0.028 0.028 .
10.1.1.1 - - [25/Aug/2019:05:27:52 +0700] "GET /v2/api/user/destination HTTP/1.1" 200 169 "-" "okhttp/3.12.0" "36.91.42.35" 0.019 0.019 .
10.1.1.1 - - [25/Aug/2019:05:28:52 +0700] "POST /v2/api/transaction/inquiry HTTP/1.1" 200 503 "-" "okhttp/3.12.0" "36.89.234.129" 0.374 0.374 .
10.1.1.1 - - [25/Aug/2019:05:28:52 +0700] "POST /v2/api/transaction/confirm HTTP/1.1" 200 874 "-" "okhttp/3.12.0" "36.89.234.129" 0.394 0.394 .
10.1.1.1 - - [25/Aug/2019:05:28:52 +0700] "GET /v2/api/user/point HTTP/1.1" 200 152 "-" "okhttp/3.12.0" "114.5.147.117" 0.024 0.024 .
10.1.1.1 - - [25/Aug/2019:05:28:52 +0700] "GET /v2/api/menu/category HTTP/1.1" 403 40 "-" "okhttp/3.12.0" "139.194.84.246" 0.003 0.003 .

my expectation sample below :


Code:
date                       |  api                               | max| avg

25/Aug/2019:05:26 /v2/api/find/outlet             2847 2757

25/Aug/2019:05:27 /v2/api/menu/category HTTP/1.1               1847 1757
25/Aug/2019:05:28 /v2/api/menu/category HTTP/1.1               1147 1257

i already trying with this awk but only qot average :


Code:
awk '/25\/Aug\/2019:18/ {c++} END{print c}' access.log

please help ...


Thanks
# 2  
start with:
Code:
awk '
$4 ~ /[0-9]:[0-9]/ {
  date=$4; sub(/^[^0-9]*/, "", date); sub(/:[0-9][0-9]$/, "", date);
  id=date OFS $7; api[id]; apic[id]++; apit[id]+=$10; if ($10>apim[id]) apim[id]=$10;
  date=$4; sub(/^[^0-9]*/, "", date); sub(/:[0-9][0-9]:[0-9][0-9]$/, "   ", date);
  id=date OFS $7; api[id]; apic[id]++; apit[id]+=$10; if ($10>apim[id]) apim[id]=$10;
}
END {
  print "date/time" OFS "api" OFS "max" OFS "avg"
  for (i in api) print i OFS apim[i] OFS apit[i] / apic[i];
}
' OFS="\t" access.log | sort -r


Last edited by rdrtx1; 08-25-2019 at 11:58 PM.. Reason: updated for avg. bases on total (no seconds). what the example posted was trying to do? is count per api per date going to be used? or the is the total api occurrences in the entire access file?
# 3  
Wouldn't you need to sum up values to calculate an average?


If you want hourly averages - as indicated - , try this slight modification of rdrtx1's proposal:

Code:
awk '
$4 ~ /[0-9]:[0-9]/      {date=$4
                         sub(/^[^0-9]*/, "", date)
                         sub (/:[0-9]*:[0-9]*$/, "", date)                  # get rid of mins and secs for the hourly stuff
                         id=date OFS $7
                         apic[id]++
                         apis[id] += $10                                    # sum up values for later avg calc.
                         if ($10>apim[id]) apim[id] = $10;
}
END                     {print "date" OFS "api" OFS "max" OFS "avg"
                         for (i in apic) print i OFS apim[i] OFS apis[i] / apic[i];
                        }
' OFS="\t" file

# 4  
Hi rdrtx1


thanks for reply...your awk will generate per second right?

--- Post updated at 07:46 PM ---

Hi RudiC


Thanks Alot for your response...


this awk use for hour right? how to convert to minutes?


Thanks
Fajar
# 5  
See updated of post # 3. mins, hrs max and avg included.

Last edited by rdrtx1; 08-25-2019 at 11:56 PM..
Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #106
Difficulty: Easy
A kernel is at the heart of the Linux OS; and this kernel acts as an interface between the user and underlying hardware.
True or False?

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to count average and max respon time?

sorry i will revise first (1 Reply)
Discussion started by: fajar_3t3
1 Replies

2. Shell Programming and Scripting

Find min and max time taken from a log file

You have a log file as attached in sample input with various operations and time taken by each of them. Write a script to find the min and max time taken for each operation. Sample output is attached. Sample Input is given as below: operation1,83621 operation2,72321 operation3,13288... (1 Reply)
Discussion started by: Chandan_Bose
1 Replies

3. Shell Programming and Scripting

Get min and max value in column

Gents, I have a big file file like this. 5100010002 5100010004 5100010006 5100010008 5100010010 5100010012 5102010002 5102010004 5102010006 5102010008 5102010010 5102010012 The file is sorted and I would like to find the min and max value, taking in the consideration key1... (3 Replies)
Discussion started by: jiam912
3 Replies

4. Shell Programming and Scripting

Script to check response time from nginx logs

Hi, My goal is to monitor the response time from the access logs of nginx server. I am using gawk to print the needed fields - 'response time' and 'name of the service' from nginx logs. Command: gawk '($6 ~ /cloudservice/) {print $10, $6}' access.log Output: 0.645 /nc/cloudservice... (6 Replies)
Discussion started by: nshah11
6 Replies

5. Shell Programming and Scripting

Get the min avg and max with awk

aaa: 3 ms aaa: 2 ms aaa: 5 ms aaa: 10 ms .......... to get the 3 2 5 10 ...'s min avg and max something like min: 2 ms avg: 5 ms max: 10 ms (2 Replies)
Discussion started by: yanglei_fage
2 Replies

6. AIX

Nmon max and avg for cpu and memory

Hi All, Anyone know how to capture the nmon avg and max cpu and memory for one of the AIX server for Monthly Utilization Report purposes ? Thanks. ---------- Post updated at 05:18 AM ---------- Previous update was at 05:07 AM ---------- if possible use shell script to count or sum... (6 Replies)
Discussion started by: ckwan
6 Replies

7. Shell Programming and Scripting

How to find the average,min,max ,total count?

Hi , Below is my sample data,I have this 8 column(A,B,C,D,E,F,G,H) in csv file. A , B ,C ,D ,E ,F,G ,H 4141,127337,24,15,20,69,72.0,-3 4141,128864,24,15,20,65,66.0,-1 4141,910053,24,15,4,4,5.0,-1 4141,910383,24,15,22,3,4.0,-1 4141,496969,24,15,14,6,-24.0,-18... (7 Replies)
Discussion started by: vinothsekark
7 Replies

8. Shell Programming and Scripting

Count time min/max/average for ping

I am redirecting my ping output to a file. The sample output is like this: 64 bytes from xx.xx.xx.167: icmp_seq=4490 ttl=116 3.75 ms 2011Jul12- 15 40 16 64 bytes from xx.xx.xx.167: icmp_seq=4491 ttl=116 5.29 ms 2011Jul12- 15 40 17 64 bytes from xx.xx.xx.167: icmp_seq=4492 ttl=116 4.88 ms... (6 Replies)
Discussion started by: zorrox
6 Replies

9. Shell Programming and Scripting

get min, max and average value

hi! i have a file like the attachement. I'd like to get for each line the min, max and average values. (there is 255 values for each line) how can i get that ? i try this, is it right? BEGIN {FS = ","; OFS = ";";max=0;min=0;moy=0;total=0;freq=890} $0 !~ /Trace1:/ { ... (1 Reply)
Discussion started by: riderman
1 Replies

10. UNIX for Dummies Questions & Answers

Iterate a min/max awk script over time-series temperature data

I'm trying to iterate a UNIX awk script that returns min/max temperature data for each day from a monthly weather data file (01_weath.dat). The temperature data is held in $5. The temps are reported each minute so each day contains 1440 temperature enteries. The below code has gotten me as far as... (5 Replies)
Discussion started by: jgourley
5 Replies

Featured Tech Videos