Sponsored Content
Top Forums Shell Programming and Scripting Checking a pattern in file and the count of characters Post 302704201 by Naks_Sh10 on Friday 21st of September 2012 07:52:31 AM
Old 09-21-2012
Checking a pattern in file and the count of characters

I am having a zipped file which has the following URL contents -

98.70.217.222 - - [08/Jul/2012:09:14:29 +0000] "GET /liveupdate-aka.symantec.com/1340071490jtun_nav2k8enn09m25.m25?h=abcdefgh HTTP/1.1" 200 159229484 "-" "hBU1OhDsPXknMepDBJNScBj4BQcmUz5TwAAAAA" "-"

In this line here is we only need to consider the components marked in BOLD above so basically:
/liveupdate-aka.symantec.com/1340071490jtun_nav2k8enn09m25.m25?h=abcdefgh : is called the URL

200: is called the response code.
h=abcdefgh : is called the query string.


I am trying to write a script which does the following:
1.) Count of each URL which have a count of 10000 or greater than 10000 that have resulted in a non successful response code( basically a non – 200, 206 or 304 response code) and do not contain the following patterns in the URL : '/F200%5E*', '/F0%5E*' and '/F100%5E*'

2.) Count of each URL excluding the query string with 800 characters in length and do not contain the following patterns in the URL : '/F200%5E*', '/F0%5E*' and '/F100%5E*'

I tried to do this with the following command:
Code:
gunzip -c * |cut -d ' ' -f7|sort -n|uniq -c|grep '^.*\/[^?]*'|grep '.\{800,\}'

it needs some changes to get the desired output.

Your help is appreciated.
Thx
Moderator's Comments:
Mod Comment Please view this code tag video for how to use code tags when posting code and data.

Last edited by vbe; 09-21-2012 at 09:32 AM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Checking for a file in file pattern before deleting it

Hi, I need a script where I have delete all the files of type abc*.* from the directory /lmn/opq (passed as parameter to script) But I need to check if there is file of type abc*.* existing in the directory or not before I use the rm abc*.* command. Thanks (1 Reply)
Discussion started by: dsrookie
1 Replies

2. Shell Programming and Scripting

how to count characters by line of file ?

Hello, Member or professional need help how to count characters by line of file Example of the file is here cdr20080817164322811681txt cdr20080817164322811txt cdr20080817164322811683txt cdr20080817164322811684txt I want to count the characters by line of file . The output that I... (4 Replies)
Discussion started by: ooilinlove
4 Replies

3. Shell Programming and Scripting

Checking existence of file using file pattern

Hi Experts:), I need to check the existense of file using patterns.How can i do it? Ex: if my current directory has a number of files of pattern (ins_*), i need to check the existense of atleast one file. pls reply me. (3 Replies)
Discussion started by: spkandy
3 Replies

4. Shell Programming and Scripting

Include special system characters in file count

Hi, I have a script that checks the length of each record/line in file - This seems to be working when there are no special systems character that are invisible or hidden. awk -v file=$file '{ if (filename==file) { k+=$5 if (length() <= 10 ){print size length(), "bytes " k} }... (2 Replies)
Discussion started by: asemota
2 Replies

5. Shell Programming and Scripting

Count characters in a csv file and add an word.

Hello, I want to add a sentence to "post column" those who are only less than 30 characters.Thank you very much for your help. "category","title","post" "Z","Zoo","test 54325 test 45363mc." "Z","Zen","rs2w3rsj 2d342dg 2d3s4f23 d23423s23h 2s34s2423g ds232d34 2342." "Z","Zet","test4444... (3 Replies)
Discussion started by: hoo
3 Replies

6. Shell Programming and Scripting

Search for a pattern in a String file and count the occurance of each pattern

I am trying to search a file for a patterns ERR- in a file and return a count for each of the error reported Input file is a free flowing file without any format example of output ERR-00001=5 .... ERR-01010=10 ..... ERR-99999=10 (4 Replies)
Discussion started by: swayam123
4 Replies

7. Shell Programming and Scripting

awk script to count characters in file 1 in file 2

I need a scripting AWK to compare 2 files. file 1 and 2 are list of keywords 1 is a b c d 2 is aa aaa b bb ccc d I want the AWK script to give us the number of times every keyword in file 1 occurs in file 2. output should be a 2 (7 Replies)
Discussion started by: anhtt
7 Replies

8. Shell Programming and Scripting

Help with listing file name containing particular text and count of lines with 10 characters.

Hi, I've 2 queries. I need to list files which doesn't contain a particular text in the content. For example say, I need to list files which doesn't contain string "abc" from all files ending with *.bad. How can I do that? Also, I want to display number of lines in a file which has atleast... (2 Replies)
Discussion started by: Gangadhar Reddy
2 Replies

9. Shell Programming and Scripting

Identify file pattern, take count of pattern, then act

Guys - Need your ideas on a section of code to finish something up. To make a long story short, I'm parsing a print output file that goes to pre-printed forms. I'm intercepting it, parsing it, formatting it, cutting it up into individual pages, grabbing the text I want in zones, building an... (3 Replies)
Discussion started by: ampsys
3 Replies

10. Shell Programming and Scripting

Pattern count in file

hi , I have a below file which contain as Use descriptive thread titles when posting Urgent. For example, do not post questions with subjects like "Help Me!", "Urgent Urgent Urgent" . or "Doubt". Post subjects like "Execution Problems with Cron" or "Help with Backup Shell Script".... (7 Replies)
Discussion started by: Jewel
7 Replies
regex(1F)                                                          FMLI Commands                                                         regex(1F)

NAME
regex - match patterns against a string SYNOPSIS
regex [-e] [ -v "string"] [ pattern template] ... pattern [template] DESCRIPTION
The regex command takes a string from the standard input, and a list of pattern / template pairs, and runs regex() to compare the string against each pattern until there is a match. When a match occurs, regex writes the corresponding template to the standard output and returns TRUE. The last (or only) pattern does not need a template. If that is the pattern that matches the string, the function simply returns TRUE. If no match is found, regex returns FALSE. The argument pattern is a regular expression of the form described in regex(). In most cases, pattern should be enclosed in single quotes to turn off special meanings of characters. Note that only the final pattern in the list may lack a template. The argument template may contain the strings $m0 through $m9, which will be expanded to the part of pattern enclosed in ( ... )$0 through ( ... )$9 constructs (see examples below). Note that if you use this feature, you must be sure to enclose template in single quotes so that FMLI does not expand $m0 through $m9 at parse time. This feature gives regex much of the power of cut(1), paste(1), and grep(1), and some of the capabilities of sed(1). If there is no template, the default is $m0$m1$m2$m3$m4$m5$m6$m7$m8$m9. OPTIONS
The following options are supported: -e Evaluates the corresponding template and writes the result to the standard output. -v "string" Uses string instead of the standard input to match against patterns. EXAMPLES
Example 1: Cutting letters out of a string To cut the 4th through 8th letters out of a string (this example will output strin and return TRUE): `regex -v "my string is nice" '^.{3}(.{5})$0' '$m0'` Example 2: Validating input in a form In a form, to validate input to field 5 as an integer: valid=`regex -v "$F5" '^[0-9]+$'` Example 3: Translating an environment variable in a form In a form, to translate an environment variable which contains one of the numbers 1, 2, 3, 4, 5 to the letters a, b, c, d, e: value=`regex -v "$VAR1" 1 a 2 b 3 c 4 d 5 e '.*' 'Error'` Note the use of the pattern '.*' to mean "anything else". Example 4: Using backquoted expressions In the example below, all three lines constitute a single backquoted expression. This expression, by itself, could be put in a menu defini- tion file. Since backquoted expressions are expanded as they are parsed, and output from a backquoted expression (the cat command, in this example) becomes part of the definition file being parsed, this expression would read /etc/passwd and make a dynamic menu of all the login ids on the system. `cat /etc/passwd | regex '^([^:]*)$0.*$' ' name=$m0 action=`message "$m0 is a user"`'` DIAGNOSTICS
If none of the patterns match, regex returns FALSE, otherwise TRUE. NOTES
Patterns and templates must often be enclosed in single quotes to turn off the special meanings of characters. Especially if you use the $m0 through $m9 variables in the template, since FMLI will expand the variables (usually to "") before regex even sees them. Single characters in character classes (inside []) must be listed before character ranges, otherwise they will not be recognized. For exam- ple, [a-zA-Z_/] will not find underscores (_) or slashes (/), but [_/a-zA-Z] will. The regular expressions accepted by regcmp differ slightly from other utilities (that is, sed, grep, awk, ed, and so forth). regex with the -e option forces subsequent commands to be ignored. In other words, if a backquoted statement appears as follows: `regex -e ...; command1; command2` command1 and command2 would never be executed. However, dividing the expression into two: `regex -e ...``command1; command2` would yield the desired result. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ SEE ALSO
awk(1), cut(1), grep(1), paste(1), sed(1), regcmp(3C), attributes(5) SunOS 5.10 12 Jul 1999 regex(1F)
All times are GMT -4. The time now is 02:32 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy