Sponsored Content
Top Forums UNIX for Advanced & Expert Users Count number of unique patterns from a log file Post 302537766 by xtechkid on Sunday 10th of July 2011 01:20:07 AM
Old 07-10-2011
Count number of unique patterns from a log file

Hello Everyone

I need your help in fixing this issue., I have a log file which has data of users logging in to an application.

I want to search for a particular pattern in the log

Code:
ISSessionValidated=N

If this key word is found , the above 8 lines will contain the name of the user who's session did not validate

for example .. the log section would be some thing like this

Code:
12:34:45 CT Session 334d3dx3z322cx User : appleInc2232
transfering tag..
transfering session..
transfering data..
level one tag completed ..
file created...
logging into main application
user session transfer created..
ISSessionValidated=N
Exception caught.. Session invalidated ..
Exiting thread 334d3dx3z322cx

I need to search the log file which has multiple sessions like this and check if the session validated = N if the keyword is found then i should check the above 8 lines and see if the user is appleInc2232 if found a count should be incremented in a file. There are over 11k sessions like this in a single log file.But the pattern is like this.

Operating system : SOLARIS 10
Single Log file size above : 4 GB
Over 700 log files with the same data is availble ( if you give a start as to how to proceed it will be helpful )
log files pattern is something like this

Code:
binded.session.01_01_2011
binded.session.01_02_2011
binded.session.01_03_2011
binded.session.01_04_2011.gz
binded.session.01_05_2011.gz
binded.session.01_06_2011.gz
---- More ---

Any help would be greatly appreciated. Thanks.

Last edited by Scott; 07-10-2011 at 06:47 AM.. Reason: Code tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to count unique number of user loged in

Hi all, I am taking my first course in unix and having some problems. I would line to create a scrip that does the following: 1) will tell me the number of users logged in. If a user is logged more than once just count it only ones. 2) want to be able to display the number of users that... (1 Reply)
Discussion started by: elchalateco
1 Replies

2. Shell Programming and Scripting

Count lines between two patterns inside a file

Hi, Im doing a script to find the number of lines included inside a file newly. These lines are in between #ifdef FLAG1 and #else or #endif or #else and #endif. I tried like this, awk '/#ifdef Flag1/,/#e/{print}' aa.c | wc -l awk '/#ifndef Flag1/,/#endif/{print}' aa.c | awk... (6 Replies)
Discussion started by: priyadarshini
6 Replies

3. Shell Programming and Scripting

Count number of Nodes created and write it to a Log file

Dear Experts, I have to count the number of AddressRecords formed in bbc.xml file using unix script file. For example: for below pasted file, I need to write an output to a log file as "No. of Address Records Created=4". Snippet of bbc.xml:- <?xml version="1.0" encoding="UTF-8" ?> -... (1 Reply)
Discussion started by: phani333
1 Replies

4. Shell Programming and Scripting

How to count Unique Values from a file.

Hi I have the following info in a file - <Cell id="25D"/> <Cell id="26A"/> <Cell id="26B"/> <Cell id="26C"/> <Cell id="27A"/> <Cell id="27B"/> <Cell id="27C"/> <Cell id="28A"/> I would like to know how would you go about counting all... (4 Replies)
Discussion started by: Prega
4 Replies

5. Shell Programming and Scripting

Unique count from flat file

Hello Guys I have a flat file with '|~|' delimited When I use to record count using below command awk -FS"+" ' {print $colno}' filename | wc -l the count is fine But when I am trying to find the unique number of record the o/p is always 1 awk -FS"+" ' {print $colno}'... (11 Replies)
Discussion started by: Pratik4891
11 Replies

6. Shell Programming and Scripting

List unique values and count instances in .csv file

I need to take the second column of a .csv file and count the number of instances of each unique value in that same second column. I'd like the output to be value,count sorted by most instances. Thanks for any guidance! Data example: 317476,317756,0 816063,318861,0 313123,319091,0... (4 Replies)
Discussion started by: batcho
4 Replies

7. Shell Programming and Scripting

count the number of occurring patterns in a file.

Hi, I have a file with a '|' pipe delimeter. I want to find number of counts for a particular pattern in particular field. Is it possible to do it in a single command? 1) want to find total number of "0" in field 4. 2) want to find total number of different records in field 4 ( similar to... (5 Replies)
Discussion started by: rudoraj
5 Replies

8. Shell Programming and Scripting

Awk to Count Multiple patterns in a huge file

Hi, I have a file that is 430K lines long. It has records like below |site1|MAP |site2|MAP |site1|MODAL |site2|MAP |site2|MODAL |site2|LINK |site1|LINK My task is to count the number of time MAP, MODAL, LINK occurs for a single site and write new records like below to a new file ... (5 Replies)
Discussion started by: reach.sree@gmai
5 Replies

9. Shell Programming and Scripting

Find and count unique date values in a file based on position

Hello, I need some sort of way to extract every date contained in a file, and count how many of those dates there are. Here are the specifics: The date format I'm looking for is mm/dd/yyyy I only need to look after line 45 in the file (that's where the data begins) The columns of... (2 Replies)
Discussion started by: ronan1219
2 Replies

10. Shell Programming and Scripting

Count number of unique values in each column of array

What is an efficient way of counting the number of unique values in a 400 column by 1000 row array and outputting the counts per column, assuming the unique values in the array are: A, B, C, D In other words the output should look like: Value COL1 COL2 COL3 A 50 51 52... (16 Replies)
Discussion started by: Geneanalyst
16 Replies
CG(1)																	     CG(1)

NAME
cg - Recursively grep for a pattern and store it. SYNOPSIS
cg [ -l ] | [ [ -i ] pattern [ files ] ] DESCRIPTION
cg does a search though text files (usually source code) recursively for a pattern, storing matches and displaying the output in a human- readable fashion. It is intended to give some of the functionaly of AT&T's cscope(1) tool, with the advantages of simplicity and not being language-specific. The script will colorize output if configured as such. It is typically run with a Perl regular expression to search for. The search can be made case insensitive by using the -i option. A list of files may also be specified with an additional argument after the pattern. Put the files pattern in quotes to make it be matched by Perl rather than by the shell. Running the script with no arguments will recall the results of the previous search. After the search, entries found can be edited using the vg(1) script. The -l option shows the last log made. SOME EXAMPLES
cg - alone recalls the previous search results. cg -i pattern - search the default list of files for all files matching the pattern (and case-insensitively). cg pattern '*.c' - search recursively for pattern in all *.c files. This automatically converts '*' to '.*' and '.' to '.' for you and does a Perl pattern match on all files in the tree. cg pattern *.c - search through the shell-expanded list of *.c files, so not done recursively (in other words, only the files your shell pass to the script as arguments). cg -l - show the last log made. COMMAND-LINE OPTIONS -i Do a case-insensitive search. -l Show the last log made. -p Toggle the default pager option. cg has a bulit-in pager function, which can be enabled or disabled by default (in .cgvgrc). If the default is enabled, this option disables the pager; if the default is disabled, this option enables it. -P Force the built-in pager to be disabled. FILES
${HOME}/.cglast Log file of the last search. ${HOME}/.cgvgrc Per-user configuration file (if the defaults are not desireable). ${HOME}/.cgvg/* Log files in $HOSTNAME.shell_pid form with the log of the last search. SEE ALSO
vg(1), perl(1), find(1), grep(1), cscope(1) AUTHOR
cg was written by Joshua Uziel <uzi@uzix.org>. 13 Mar 2002 CG(1)
All times are GMT -4. The time now is 09:12 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy