Sponsored Content
Full Discussion: Request per second script
Top Forums Shell Programming and Scripting Request per second script Post 302991851 by Katkota on Thursday 16th of February 2017 09:52:47 PM
Old 02-16-2017
Request per second script

Hello;
I'm having about 800 log files and i'm trying to write a script that report the counts of lines per second or "requests per second" in each log file and report the output which includes the timestamp for the highest lines per second count and the log file name and the highest number per second from that file and then go to the second log file and do the same thing then append the results to the same output file

I'm currently using this command to get the results from each log but doing this for 800 log files is not practical
Code:
grep "2017-16-02" logfile1 | cut -c2-18 | uniq -c

This command reports a list of lines count per 10 seconds but it's not very efficient

So the output file should look like this:

Code:
Date/time    "1st LogFileName"    "highest requests per second for this log"
Date/time    "2nd LogFileName"    "highest requests per second for this log"

Any help would be greatly appreciated

Here's the log files i'm working with:

Code:
#Fields: date time time-taken c-ip cs-username cs-auth-group x-exception-id sc-filter-result cs-categories cs(Referer) sc-status s-action cs-method
 rs(Content-Type) cs-uri-scheme cs-host cs-uri-port cs-uri-path cs-uri-query cs-uri-extension cs(User-Agent) s-ip sc-bytes cs-bytes x-virus-id x-bl
uecoat-application-name x-bluecoat-application-operation
#Remark: 1412140034 "lofnetsg1" "192.168.13.14" "main"
2017-02-15 17:49:06 8 10.23.53.112 - - - OBSERVED "Technology/Internet" -  404 TCP_NC_MISS GET text/html;%20charset=iso-8859-1 http dci.sophosupd.c
om 80 /osxhe/d/16/d16e39472e867bf7f8407db13d3502b5.dat - dat "Sophos%20AutoUpdate/ CFNetwork/760.6.3 Darwin/15.6.0 (x86_64)" 192.168.13.14 437 243
- "none" "none"
2017-02-15 17:49:06 578 10.4.33.22 - - - OBSERVED "Web Ads/Analytics" -  200 TCP_TUNNELED CONNECT - tcp tpc.googlesyndication.com 443 / - - "Mozill
a/5.0 (Windows NT 6.1; Trident/7.0; rv:11.0) like Gecko" 192.168.13.14 21106 2873 - "none" "none"
2017-02-15 17:49:06 17 10.17.87.27 - - - OBSERVED "Web Ads/Analytics" -  200 TCP_NC_MISS GET - http t.lkqd.net 80 /t ?ev=147&tsid=132117&env=3&cb=7
43590466414&format=0&did=2&osid=2&pubtagtype=lkqd-formats&render=html5&apt=auto&uimp=6944564170577988928&svrs=0.45&srvid=10.10.4.214&oip=134.174.21
.158&vrs=197&tsl=13827&ear=0&width=300&height=250&phost=&host=reuters.com&appname=&idfa=&aid=&bundleid=&lsid=&loclat=&loclong=&contentid=&contentti
tle=&contentlength=&contenturl=&appstoreurl=&execution=inbanner&placement=&browserid=1&adplayersize=small&pid=326&sid=150846&spid=18106&psid=&ppid=
&lkqdtagtype=platform-connection&tlr=0&adid=127634&asrc=15817&dealid=117723&c1=&c2=&c3=&tsia=8945&tssa=&rnd=157089284&cs=3668523497 - "Mozilla/5.0
(Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 192.168.13.14 290 1534 - "none" "none"
2017-02-15 17:49:06 669 10.23.112.63 - - - OBSERVED "Radio/Audio Streams" -  200 TCP_MISS GET audio/x-aac http primary.hls-streaming.production.str
eaming.siriusxm.com 80 /AAC_Data/broadwaysbest/HLS_broadwaysbest_64k_v3/broadwaysbest_64k_1_021560803727_00310913_v3.aac ?consumer=k2&gupId=BE4A750
32FEC2FF7293A7F5C0FAEFEA3 aac "SXMLiveAudioPlayer/1485824608 CFNetwork/808.3 Darwin/16.3.0" 192.168.13.14 79798 1119 - "none" "none"
2017-02-15 17:49:06 32 10.64.5.81 - - - OBSERVED "Web Ads/Analytics" Error
.html  200 TCP_NC_MISS GET text/javascript http beacon.krxd.net 80 /optout_check ?callback=Krux.ns._default.kxjsonp_optOutCheck - "Mozilla/5.0 (Win
dows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 192.168.13.14 635 451 - "none" "none"


Last edited by Katkota; 02-17-2017 at 10:39 PM.. Reason: Change ICODE tags to CODE tags and add CODE tags.
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

unix script http request

Hi everybody, I have a *.vbs file which I want to run automatically. I want to know if there is anyway to implement the given example for e.g "http://255.255.255.55/script.vbs" what I mean is does anyone know how to make an http request from a unix script?? Thanks in advance!!!!!!!!!!! (1 Reply)
Discussion started by: arksal
1 Replies

2. UNIX for Advanced & Expert Users

Using a request script

I am creating a package(Solaris10 on sparc) that needs user input. As I understand it, I need to use a request script. My problem is that the value I set in my request script is not visible in my postinstall script. Not sure if I am doing it right. Here is an example request script... (4 Replies)
Discussion started by: Tornado
4 Replies

3. UNIX for Advanced & Expert Users

How to modify BASEDIR with request script?

Here is request script: #!/bin/sh echo "Current install root path is " CONFIRM="n" while do BASEDIR="" while true do echo please input install root path then press : read BASEDIR if then echo... (6 Replies)
Discussion started by: kingpmp
6 Replies

4. Shell Programming and Scripting

Solaris request script

Hi, In the request script I need to read user input and store to variable to use it later in postinstall script. e.g. LOGDIR=/app/log echo "Please type the Log Directory : (current value: $LOGDIR)" read LOGDIR When asked, if the user enters a value the parameter is ok and I... (2 Replies)
Discussion started by: potro
2 Replies

5. Shell Programming and Scripting

Timing out lynx request in a bash script

I have a Bash script where, in a loop, I access several urls. Sometimes, if an url is not available, lynx hangs, and the script does not continue. How can I time out the lynx request when it takes more than, 10 Seconds, but continue with the other jobs... For some reason lynx does not care... (1 Reply)
Discussion started by: lowmaster
1 Replies

6. Shell Programming and Scripting

Request a script on manupilating the data. Please HELP!

Dear friends, I'm struggling to preparing a bunch of gaussian input files, say manually. It's really a time-consuming work without any techniques. I suppose that it could be done by a smart script automatically. But I lack some basic knowledge on scripting. Please help! My original input... (4 Replies)
Discussion started by: liuzhencc
4 Replies

7. Shell Programming and Scripting

Request for help with SGE submission script manipulation

Hi, I have the following SGE submission (HPC calculation) script, which is just a Bash script: #!/bin/bash -l #$ -S /bin/bash #$ -l h_rt=1:00:0 #$ -l mem=4G #$ -N XXX #$ -pe qlc 24 #$ -P XXX #$ -wd /home/uccaxxx/Scratch/222PdT/3vac/c0001/ mpirun -m $TMPDIR/machines -np $NSLOTS... (2 Replies)
Discussion started by: crunchgargoyle
2 Replies

8. Shell Programming and Scripting

Shell script request

I've a master file which will contain 100 file names, The script should read file name from a master file and format the file as below in AIX. input file Filename This is a test file Output File Filename|This is a test file Thanks in advance for file in $FileList; do (5 Replies)
Discussion started by: udayakumar
5 Replies

9. Shell Programming and Scripting

Formatting wget request within script

When using a browser and calling this url .. the data returns the proper range of information ichart dot finance dot yahoo dot com/table.csv?s=YAHOO&a=3&b=14&c=2012&d=03&e=20&f=2012&g=d&ignore.csv (geeze wont let me post url's sorry ) However in my script the formatting is messing up on... (4 Replies)
Discussion started by: harte
4 Replies
All times are GMT -4. The time now is 04:00 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy