Sponsored Content
Top Forums UNIX for Advanced & Expert Users Count number of unique patterns from a log file Post 302537766 by xtechkid on Sunday 10th of July 2011 01:20:07 AM
Old 07-10-2011
Count number of unique patterns from a log file

Hello Everyone

I need your help in fixing this issue., I have a log file which has data of users logging in to an application.

I want to search for a particular pattern in the log

Code:
ISSessionValidated=N

If this key word is found , the above 8 lines will contain the name of the user who's session did not validate

for example .. the log section would be some thing like this

Code:
12:34:45 CT Session 334d3dx3z322cx User : appleInc2232
transfering tag..
transfering session..
transfering data..
level one tag completed ..
file created...
logging into main application
user session transfer created..
ISSessionValidated=N
Exception caught.. Session invalidated ..
Exiting thread 334d3dx3z322cx

I need to search the log file which has multiple sessions like this and check if the session validated = N if the keyword is found then i should check the above 8 lines and see if the user is appleInc2232 if found a count should be incremented in a file. There are over 11k sessions like this in a single log file.But the pattern is like this.

Operating system : SOLARIS 10
Single Log file size above : 4 GB
Over 700 log files with the same data is availble ( if you give a start as to how to proceed it will be helpful )
log files pattern is something like this

Code:
binded.session.01_01_2011
binded.session.01_02_2011
binded.session.01_03_2011
binded.session.01_04_2011.gz
binded.session.01_05_2011.gz
binded.session.01_06_2011.gz
---- More ---

Any help would be greatly appreciated. Thanks.

Last edited by Scott; 07-10-2011 at 06:47 AM.. Reason: Code tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to count unique number of user loged in

Hi all, I am taking my first course in unix and having some problems. I would line to create a scrip that does the following: 1) will tell me the number of users logged in. If a user is logged more than once just count it only ones. 2) want to be able to display the number of users that... (1 Reply)
Discussion started by: elchalateco
1 Replies

2. Shell Programming and Scripting

Count lines between two patterns inside a file

Hi, Im doing a script to find the number of lines included inside a file newly. These lines are in between #ifdef FLAG1 and #else or #endif or #else and #endif. I tried like this, awk '/#ifdef Flag1/,/#e/{print}' aa.c | wc -l awk '/#ifndef Flag1/,/#endif/{print}' aa.c | awk... (6 Replies)
Discussion started by: priyadarshini
6 Replies

3. Shell Programming and Scripting

Count number of Nodes created and write it to a Log file

Dear Experts, I have to count the number of AddressRecords formed in bbc.xml file using unix script file. For example: for below pasted file, I need to write an output to a log file as "No. of Address Records Created=4". Snippet of bbc.xml:- <?xml version="1.0" encoding="UTF-8" ?> -... (1 Reply)
Discussion started by: phani333
1 Replies

4. Shell Programming and Scripting

How to count Unique Values from a file.

Hi I have the following info in a file - <Cell id="25D"/> <Cell id="26A"/> <Cell id="26B"/> <Cell id="26C"/> <Cell id="27A"/> <Cell id="27B"/> <Cell id="27C"/> <Cell id="28A"/> I would like to know how would you go about counting all... (4 Replies)
Discussion started by: Prega
4 Replies

5. Shell Programming and Scripting

Unique count from flat file

Hello Guys I have a flat file with '|~|' delimited When I use to record count using below command awk -FS"+" ' {print $colno}' filename | wc -l the count is fine But when I am trying to find the unique number of record the o/p is always 1 awk -FS"+" ' {print $colno}'... (11 Replies)
Discussion started by: Pratik4891
11 Replies

6. Shell Programming and Scripting

List unique values and count instances in .csv file

I need to take the second column of a .csv file and count the number of instances of each unique value in that same second column. I'd like the output to be value,count sorted by most instances. Thanks for any guidance! Data example: 317476,317756,0 816063,318861,0 313123,319091,0... (4 Replies)
Discussion started by: batcho
4 Replies

7. Shell Programming and Scripting

count the number of occurring patterns in a file.

Hi, I have a file with a '|' pipe delimeter. I want to find number of counts for a particular pattern in particular field. Is it possible to do it in a single command? 1) want to find total number of "0" in field 4. 2) want to find total number of different records in field 4 ( similar to... (5 Replies)
Discussion started by: rudoraj
5 Replies

8. Shell Programming and Scripting

Awk to Count Multiple patterns in a huge file

Hi, I have a file that is 430K lines long. It has records like below |site1|MAP |site2|MAP |site1|MODAL |site2|MAP |site2|MODAL |site2|LINK |site1|LINK My task is to count the number of time MAP, MODAL, LINK occurs for a single site and write new records like below to a new file ... (5 Replies)
Discussion started by: reach.sree@gmai
5 Replies

9. Shell Programming and Scripting

Find and count unique date values in a file based on position

Hello, I need some sort of way to extract every date contained in a file, and count how many of those dates there are. Here are the specifics: The date format I'm looking for is mm/dd/yyyy I only need to look after line 45 in the file (that's where the data begins) The columns of... (2 Replies)
Discussion started by: ronan1219
2 Replies

10. Shell Programming and Scripting

Count number of unique values in each column of array

What is an efficient way of counting the number of unique values in a 400 column by 1000 row array and outputting the counts per column, assuming the unique values in the array are: A, B, C, D In other words the output should look like: Value COL1 COL2 COL3 A 50 51 52... (16 Replies)
Discussion started by: Geneanalyst
16 Replies
SVK::Log::Filter::Grep(3)				User Contributed Perl Documentation				 SVK::Log::Filter::Grep(3)

SYNOPSIS
SVK::Log::Filter::Grep - search log messages for a given pattern DESCRIPTION
The Grep filter requires a single Perl pattern (regular expression) as its argument. The pattern is then applied to the svn:log property of each revision it receives. If the pattern matches, the revision is allowed to continue down the pipeline. If the pattern fails to match, the pipeline immediately skips to the next revision. The pattern is applied with the /i modifier (case insensitivity). If you want case-sensitivity or other modifications to the behavior of your pattern, you must use the "(?imsx-imsx)" extended pattern (see "perldoc perlre" for details). For example, to search for log messages that match exactly the characters "foo" you might use svk log --filter "grep (?-i)foo" However, to search for "foo" without regards for case, one might try svk log --filter "grep foo" The result of any capturing parentheses inside the pattern are not available. If demand dictates, the Grep filter could be modified to place the captured value somewhere in the stash for other filters to access. If the pattern contains a pipe character ('|'), it must be escaped by preceding it with a '' character. Otherwise, the portion of the pattern after the pipe character is interpreted as the name of a log filter. STASH
/PROPERTY MODIFICATIONS Grep leaves all properties and the stash intact. perl v5.10.0 2008-08-04 SVK::Log::Filter::Grep(3)
All times are GMT -4. The time now is 11:48 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy