Shell Programming and Scripting

Hi All ,






Here is what I want to do:

Given a line:

98.70.217.222 - - [08/Jul/2012:09:14:29 +0000] "GET /liveupdate-aka.symantec.com/1340071490jtun_nav2k8enn09m25.m25?h=abcdefgh HTTP/1.1" 200 159229484 "-" "hBU1OhDsPXknMepDBJNScBj4BQcmUz5TwAAAAA" "-"

1. Get the URL component: "/liveupdate-aka.symantec.com/1340071490jtun_nav2k8enn09m25.m25?h=abcdefgh"

2. Check if the URL component excluding query string which is "/liveupdate-aka.symantec.com/1340071490jtun_nav2k8enn09m25.m25" is greater than 800 characters

I am using the command below to do this but it is checking if the entire URL component is greater than 800 characters and not excluding the Query string:

gunzip -c * |cut -d ' ' -f7|sort -n|uniq -c|grep '^.*\/[^?]*'|grep '.\{800,\}'

3. Also check is each component in the URL is greater than 200 characters

For eg. Is "/liveupdate-aka.symantec.com/1340071490jtun_nav2k8enn09m25.m25" greater than 800 characters and

is "liveupdate-aka.symantec.com" geater than 200 characters

is "1340071490jtun_nav2k8enn09m25.m25" greater than 200 characters.


It would be good if we can do all of the above in one command if possible.

Code:

$
$ cat f10
98.70.217.222 - - [08/Jul/2012:09:14:29 +0000] "GET /liveupdate-aka.symantec.com/1340071490jtun_nav2k8enn09m25.m25?h=abcdefgh HTTP/1.1" 200 159229484 "-" "hBU1OhDsPXknMepDBJNScBj4BQcmUz5TwAAAAA" "-"
$
$
$ perl -lne 's/^.*GET (.*?)\?.*/$1/; print $_, " is ",length($_)<=800?"not":"", " greater than 800 char";
             s/^\///; map {print $_, " is ", length($_)<=200?"not":"", " greater than 200 char"} split/\//
            ' f10
/liveupdate-aka.symantec.com/1340071490jtun_nav2k8enn09m25.m25 is not greater than 800 char
liveupdate-aka.symantec.com is not greater than 200 char
1340071490jtun_nav2k8enn09m25.m25 is not greater than 200 char
$
$

tyler_durden

Hi Tyler,
Thanks for the reply but I am still facing one problem as I am not even a starter in perl.

$ cat sample1.log
98.70.217.222 - - [08/Jul/2012:09:14:29 +0000] "GET /liveupdate-aka.symantec.com/1340071490jtun_nav2k8enn09m25.m25liveupdate-aka.symantec.com1340071490jtun_nav2k8enn09m25.m25liveupdate-aka.symantec.com1340071490jtun_nav2k8enn09m25?h=jshdjahsdieal HTTP/1.1" 200 159229484 "-" "hBU1OhDsPXknMepDBJNScBj4BQcmUz5TwAAAAA" "-"

$ perl -lne 's/^.*GET (.*?)\?.*/$1/; print $_, " is ",length($_)>=200?"":"", " greater than 200 char"; s/^\///; map {print $_, " is ", length($_)>=200?"":"", " greater than 200 char"} split/\//' sample1.log

/liveupdate-aka.symantec.com/1340071490jtun_nav2k8enn09m25.m25liveupdate-aka.symantec.com1340071490jtun_nav2k8enn09m25.m25liveupdate-aka.symantec.com1340071490jtun_nav2k8enn09m25 is greater than 200 char

liveupdate-aka.symantec.com is greater than 200 char

1340071490jtun_nav2k8enn09m25.m25liveupdate-aka.symantec.com1340071490jtun_nav2k8enn09m25.m25liveupdate-aka.symantec.com1340071490jtun_nav2k8enn09m25 is greater than 200 char

Now if you see the BLUE line, it's length is not greater than 200 char but still its showing this in answer set.