Unix/Linux Go Back    

Request per second script

Shell Programming and Scripting

Kindly Note - This is a Single User Post by Forum Member Katkota Regarding:
Request per second script.
Please Follow The Primary Link Above to View the Full Discussion.

Old Unix and Linux 6 Days Ago
Katkota Katkota is offline
Registered User
Join Date: Oct 2007
Last Activity: 20 February 2017, 10:29 PM EST
Posts: 148
Thanks: 0
Thanked 0 Times in 0 Posts
Request per second script

I'm having about 800 log files and i'm trying to write a script that report the counts of lines per second or "requests per second" in each log file and report the output which includes the timestamp for the highest lines per second count and the log file name and the highest number per second from that file and then go to the second log file and do the same thing then append the results to the same output file

I'm currently using this command to get the results from each log but doing this for 800 log files is not practical

grep "2017-16-02" logfile1 | cut -c2-18 | uniq -c

This command reports a list of lines count per 10 seconds but it's not very efficient

So the output file should look like this:

Date/time    "1st LogFileName"    "highest requests per second for this log"
Date/time    "2nd LogFileName"    "highest requests per second for this log"

Any help would be greatly appreciated

Here's the log files i'm working with:

#Fields: date time time-taken c-ip cs-username cs-auth-group x-exception-id sc-filter-result cs-categories cs(Referer) sc-status s-action cs-method
 rs(Content-Type) cs-uri-scheme cs-host cs-uri-port cs-uri-path cs-uri-query cs-uri-extension cs(User-Agent) s-ip sc-bytes cs-bytes x-virus-id x-bl
uecoat-application-name x-bluecoat-application-operation
#Remark: 1412140034 "lofnetsg1" "" "main"
2017-02-15 17:49:06 8 - - - OBSERVED "Technology/Internet" -  404 TCP_NC_MISS GET text/html;%20charset=iso-8859-1 http dci.sophosupd.c
om 80 /osxhe/d/16/d16e39472e867bf7f8407db13d3502b5.dat - dat "Sophos%20AutoUpdate/ CFNetwork/760.6.3 Darwin/15.6.0 (x86_64)" 437 243
- "none" "none"
2017-02-15 17:49:06 578 - - - OBSERVED "Web Ads/Analytics" -  200 TCP_TUNNELED CONNECT - tcp tpc.googlesyndication.com 443 / - - "Mozill
a/5.0 (Windows NT 6.1; Trident/7.0; rv:11.0) like Gecko" 21106 2873 - "none" "none"
2017-02-15 17:49:06 17 - - - OBSERVED "Web Ads/Analytics" -  200 TCP_NC_MISS GET - http t.lkqd.net 80 /t ?ev=147&tsid=132117&env=3&cb=7
&lkqdtagtype=platform-connection&tlr=0&adid=127634&asrc=15817&dealid=117723&c1=&c2=&c3=&tsia=8945&tssa=&rnd=157089284&cs=3668523497 - "Mozilla/5.0
(Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 290 1534 - "none" "none"
2017-02-15 17:49:06 669 - - - OBSERVED "Radio/Audio Streams" -  200 TCP_MISS GET audio/x-aac http primary.hls-streaming.production.str
eaming.siriusxm.com 80 /AAC_Data/broadwaysbest/HLS_broadwaysbest_64k_v3/broadwaysbest_64k_1_021560803727_00310913_v3.aac ?consumer=k2&gupId=BE4A750
32FEC2FF7293A7F5C0FAEFEA3 aac "SXMLiveAudioPlayer/1485824608 CFNetwork/808.3 Darwin/16.3.0" 79798 1119 - "none" "none"
2017-02-15 17:49:06 32 - - - OBSERVED "Web Ads/Analytics" Error
.html  200 TCP_NC_MISS GET text/javascript http beacon.krxd.net 80 /optout_check ?callback=Krux.ns._default.kxjsonp_optOutCheck - "Mozilla/5.0 (Win
dows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 635 451 - "none" "none"

Last edited by Katkota; 5 Days Ago at 10:39 PM.. Reason: Change ICODE tags to CODE tags and add CODE tags.