Help with command modification


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help with command modification
# 1  
Old 03-01-2017
Help with command modification

Hello folks;
I'm using the following command to get the highest number of requests per second in a log file and it works well.

Code:
grep "2017-02-22" "LogFile.log" | cut -c1-20 | uniq -c | sort -n | tail -n1

Now i would like to also get the smallest requests per second and the amount of time the highest number stayed for. For example: let's say the results from the command was 2000 which is the highest requests per second inside the log file, i want to find out how long the 2000 requests lasted for? in another word: if the peak of 2000 happened, i'd like to find out how long it took before this went down. I hope to find an answer folks

Last edited by rbatte1; 03-02-2017 at 06:19 AM.. Reason: Changed ICODE tags to CODE tags
# 2  
Old 03-02-2017
It would be good if you can give sample data (few lines from log file) and show us the output of your original command and your expectation. Otherwise, it would be guesswork :-)
# 3  
Old 03-02-2017
Code:
#Start-Date: 2017-02-16 19:49:06
#Date: 2016-10-11 15:16:48
#Fields: date time time-taken c-ip cs-username cs-auth-group x-exception-id sc-filter-result cs-categories cs(Referer) sc-status s-action c
s-method rs(Content-Type) cs-uri-scheme cs-host cs-uri-port cs-uri-path cs-uri-query cs-uri-extension cs(User-Agent) s-ip sc-bytes cs-bytes
 x-virus-id x-bluecoat-application-name x-bluecoat-application-operation
#Remark: 1412140034 "lofnetsg1" "192.168.13.14" "main"
2017-02-16 19:49:06 116154 10.5.13.149 - - - OBSERVED "Non-Viewable/Infrastructure" -  200 TCP_TUNNELED CONNECT - tcp u-amvx4npjuy.wc.yahoo
dns.net 443 / - - "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 192.168.13.14 660
3 1036 - "none" "none"
2017-02-16 19:49:06 1 10.2.29.41 - - - OBSERVED "Technology/Internet" -  304 TCP_HIT GET application/pkix-crl http www.microsoft.com 80 /pk
i/CRL/products/Microsoft%20Windows%20Hardware%20Compatibility%20PCA(1).crl - crl "Microsoft-CryptoAPI/6.1" 192.168.13.14 568 338 - "none" "
none"
2017-02-16 19:49:06 18 10.1.15.166 - - - OBSERVED "Content Servers" http://www.foxnews.com/  304 TCP_CLIENT_REFRESH GET text/javascript;cha
rset=UTF-8 http widget-cdn.rpxnow.com 80 /translations/share/en - - "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Ge
cko) Chrome/56.0.2924.87 Safari/537.36" 192.168.13.14 487 417 - "none" "none"
2017-02-16 19:49:06 6677 172.16.121.69 - - - OBSERVED "Social Networking;Content Servers" -  200 TCP_TUNNELED CONNECT - tcp pbs.twimg.com 4
43 / - - "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 192.168.13.14 10020
3 1241 - "Twitter" "none"
2017-02-16 19:49:06 1664 10.14.16.67 - - - OBSERVED "Informational;Health" -  200 TCP_TUNNELED CONNECT - tcp www.drugs.com 443 / - - "Mozil
la/5.0 (Windows NT 6.1; Trident/7.0; rv:11.0) like Gecko" 192.168.13.14 6313 2281 - "none" "none"
2017-02-16 19:49:06 1095 172.16.121.69 - - - OBSERVED "Web Ads/Analytics" -  200 TCP_TUNNELED CONNECT - tcp as-sec.casalemedia.com 443 / -
- "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 192.168.13.14 1058 2818 -
"none" "none"
2017-02-16 19:49:06 24282 172.16.121.69 - - - OBSERVED "Web Ads/Analytics" -  200 TCP_TUNNELED CONNECT - tcp dt.adsafeprotected.com 443 / -
 - "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 192.168.13.14 3687 3007 -
 "none" "none"
2017-02-16 19:49:06 1 10.2.29.41 - - - OBSERVED "Non-Viewable/Infrastructure" -  304 TCP_HIT GET application/pkix-crl http crl.microsoft.co
m 80 /pki/crl/products/MicrosoftTimeStampPCA.crl - crl "Microsoft-CryptoAPI/6.1" 192.168.13.14 500 304 - "none" "none"
2017-02-16 19:49:06 48 10.2.50.46 - - - OBSERVED "Web Ads/Analytics" -  200 TCP_TUNNELED CONNECT - tcp x.bidswitch.net 443 / - - "Mozilla/5
.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 192.168.13.14 39 219 - "none" "none"
2017-02-16 19:49:06 26855 172.16.121.69 - - - OBSERVED "Web Ads/Analytics" -  200 TCP_TUNNELED CONNECT - tcp ping.chartbeat.net 443 / - - "
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 192.168.13.14 699 2727 - "non
e" "none"
2017-02-16 19:49:06 22 10.2.10.172 - - - OBSERVED "Web Ads/Analytics" http://player.radio.com/listen/station/985-the-sports-hub  200 TCP_NC
_MISS GET application/javascript;%20charset=utf-8 http ib.adnxs.com 80 /ttj ?id=10203641&size=300x250&pagetype=ros&promo_sizes=&cb=14872745
46795 - "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 192.168.13.14 11458 3251 -
"none" "none"
2017-02-16 19:49:06 965 10.32.14.38 - - - OBSERVED "Technology/Internet" -  200 TCP_TUNNELED CONNECT - tcp clients4.google.com 443 / - - "C
hrome WIN 56.0.2924.87 (0e9a9a6f3676ae439b78cd9b3f62b4193c3ac7d5-refs/branch-heads/2924@{#895}) channel(stable)" 192.168.13.14 1455 3073 -
"none" "none"
2017-02-16 19:49:06 939 10.7.18.97 - - - OBSERVED "Health" http://cmri.in/cmri-doctors/  200 TCP_NC_MISS GET text/html;%20charset=UTF-8 htt
p cmri.in 80 /doctor/dr-mahesh-chowdhury/ - - "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:51.0) Gecko/20100101 Firefox/51.0" 192.168.
13.14 7501 573 - "none" "none"

# 4  
Old 03-03-2017
Not very useful, because the first grep does not match any records.......

Assuming you are after records matching 2017-02-16 then I would suggest also sorting your data before running uniq -c so that it counts correctly.




Robin
# 5  
Old 03-04-2017
Thanks Robin.. This is only a small sample from the log file and for this log file i will be looking to match the record for 2017-02-16. I already included my original command to get the highest number of requests per second and it works well but you haven't provide any answer to my questions
# 6  
Old 03-05-2017
Quote:
Originally Posted by Katkota
Thanks Robin.. This is only a small sample from the log file and for this log file i will be looking to match the record for 2017-02-16. I already included my original command to get the highest number of requests per second and it works well but you haven't provide any answer to my questions
You should assume when starting a new thread that no one reading this thread has any knowledge of any of your previous threads nor that they have taken the time to search for and read all of your earlier posts to figure out what you are talking about.

As robin said, the pipeline you showed us in post #1 in this thread when fed the sample input you provided in post #3 in this thread produces absolutely no output. Based on that and your lack of a clear description of what you are trying to do in this thread, robin (and most other readers of this thread) have no way to guess at what you are trying to do. If you don't want to provide a clear explanation of what you are trying to do in this thread, I would not expect anyone to waste any more time trying to answer your questions.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

cp modification

I'm usia Raspbian, a Debian subset, and wondering what work would be involved in altering the cp command. cp at present needs a full path and file name for source and at least full filename for destination. How can I change this so the second parameter isn't needed? So if the destination... (6 Replies)
Discussion started by: MuntyScrunt
6 Replies

2. Shell Programming and Scripting

Modification to awk command

i have a php file that has this: php.code #!/usr/bin/php <?php phpinfo(); hlight_file(__FILE__); ?> I want my awk code grab whatever is inbetween and including the "<?php" and "?>". Then, it should scan all the entries between these two points. And if the entries between these... (10 Replies)
Discussion started by: SkySmart
10 Replies

3. AIX

Getting files through find command and listing file modification time upto seconds

I have to list the files of particular directory using file filter like find -name abc* something and if multiple file exist I also want time of each file up to seconds. Currently we are getting time up to minutes in AIX is there any way I can get file last modification time up to seconds. (4 Replies)
Discussion started by: Nitesh sahu
4 Replies

4. Shell Programming and Scripting

Perl command modification

below is a snippet of code from a larger perl code: my $uname = ( -e '/usr/bin/uname' ) ? '/usr/bin/uname' : '/bin/uname'; my $os = ( `$uname 2>/dev/null` ); when i run this code, it seems to be complaining about the backticks. is there any efficient way i can get rid of the backticks... (3 Replies)
Discussion started by: SkySmart
3 Replies

5. Shell Programming and Scripting

How to list the files based on the modification time using the find command?

Hi All, I need to list the files based modification time of the files from a directory, I cannot use "ls -t" as there are lot of files, which "ls" command cannot handle. New files will land there daily. So iam looking for an alternative through "find"command. All suggestions are welcomed. ... (6 Replies)
Discussion started by: Kesavan
6 Replies

6. UNIX for Dummies Questions & Answers

Modification of Two Files

I have a script that produces two output files each containing the same number of lines <file1.txt> and <file2.txt>. What I need to do is combine both files into a new file <file3.txt> where line 1 of <file1.txt> is put to the right (and on the same line) as line 1 of <file2.txt> and then the same... (5 Replies)
Discussion started by: theref
5 Replies

7. Solaris

User id modification

Dear All, Some of the users having the same user id and group id in my /etc/passwd file. Now I want to change the users to have a unique user id and group id. How can we do that. If I change this will it affect the running applications. Rgds Rj (7 Replies)
Discussion started by: jegaraman
7 Replies

8. Solaris

Command for checking modification history on file

What is the command for checking modification history on file? ---------- Post updated at 01:20 PM ---------- Previous update was at 12:35 PM ---------- Let me rephrase this. On a regular Unix file can I at least check to see the time and date history modification of the file? (6 Replies)
Discussion started by: jastanle84
6 Replies

9. Shell Programming and Scripting

help in script modification

i have the following perl script.but it searches for a given filename. i want to run the same script in my directoy which has subdirectories too and it has to display the file if sreach satisfies along with directory name. can anyone help me: perl script: my $FILE = $ARGV; for zf in... (4 Replies)
Discussion started by: a.suryakumar
4 Replies

10. UNIX for Dummies Questions & Answers

command for modification date of a file

Good morning, I would like to find all files of a certain type and display their name as well as their modification date. In order to do this, I would do the following: find ./ -name *.csv | ???????? My question: what to put after the pipe instead of the question marks? Is there a basic... (5 Replies)
Discussion started by: scampsd
5 Replies
Login or Register to Ask a Question