Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Need help with perl pattern matching

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need help with perl pattern matching
# 1  
Old 03-28-2012
Need help with perl pattern matching
My log file looks as given below, its actually a huge file around 1 GB and these are some of the line:

Code:
[22/Mar/2011:12:23:48 +0000] conn=5368758 op=10628050 msgId=64 - RESULT err=0 tag=101 nentries=1 etime=0
[22/Mar/2011:12:23:48 +0000] conn=7462122 op=-1 msgId=-1 - fd=247 slot=247 LDAPS connection from 10.13.18.12:37645 to 10.18.6.45
[22/Mar/2011:12:23:48 +0000] conn=7462122 op=-1 msgId=-1 - SSL 256-bit AES-256
[22/Mar/2011:12:23:48 +0000] conn=7462122 op=0 msgId=1 - BIND dn="" method=128 version=3
[22/Mar/2011:12:23:48 +0000] conn=7462122 op=0 msgId=1 - RESULT err=0 tag=97 nentries=0 etime=0 dn=""
[22/Mar/2011:12:23:48 +0000] conn=7462122 op=1 msgId=2 - SRCH base="ou=people,dc=abc,dc=com" scope=1 filter="(&(objectClass=shadowAccount)(uid=ora))" attrs="uid userPassword shadowLastChange shadowMax shadowMin shadowWarning shadowInactive shadowExpire shadowFlag"
[22/Mar/2011:12:23:48 +0000] conn=7462122 op=1 msgId=2 - RESULT err=0 tag=101 nentries=1 etime=0
[22/Mar/2011:12:23:48 +0000] conn=7462123 op=-1 msgId=-1 - fd=310 slot=310 LDAP connection from 10.11.3.34:58868 to 10.18.6.45
[22/Mar/2011:12:23:48 +0000] conn=7462123 op=0 msgId=1 - SRCH base="" scope=0 filter="(objectClass=*)" attrs="supportedControl supportedSALMechanisms"
[22/Mar/2011:12:23:48 +0000] conn=7462123 op=0 msgId=1 - RESULT err=0 tag=101 nentries=1 etime=0
[22/Mar/2011:12:23:48 +0000] conn=7462123 op=1 msgId=2 - UNBIND
[22/Mar/2011:12:23:48 +0000] conn=7462123 op=1 msgId=-1 - closing from 10.11.3.34:58868 - U1 - Connection closed by unbind client -

I need to find the line containg the string "LDAPS connection from" and find the IP its coming from.

For ex: From the 2nd line I want 10.13.18.12 and from the 9th line I want 10.11.3.34

Later, The I need to add up the total connections from each of these the IPs.

For eg:

Code:
10.13.18.12 - 20
10.11.3.34 - 40

I could do this in awk but it was utilizing a lot of CPU. So, I need to do this in perl. I am new to perl.

Any help would be high appreciated.
Last edited by Scrutinizer; 03-28-2012 at 04:17 AM.. Reason: Please ... Please use code tags!!
# 2  
Old 03-28-2012
Code:
perl -ne '(/LDAPS? connection from ([0-9.]+)/) && $x{$1}++; END{for(keys %x){print "$_ -> $x{$_}\n"}}' logfile

Can't guarantee if this will be quicker than awk though.
# 3  
Old 03-28-2012
Try:
Code:
awk -F'.*LDAPS connection from |:' '$2~"[.]"{A[$2]++}END{for(i in A)print i" - "A[i]}' infile

What awk did you use that was slow?
Last edited by Scrutinizer; 03-28-2012 at 04:47 AM..
# 4  
Old 03-28-2012
The perl one works but the awk one gives me a syntax error.

Thanks a lot balajesuri....
# 5  
Old 03-28-2012
To keep the forums high quality for all users, please take the time to format your posts correctly.

First of all, use Code Tags when you post any code or data samples so others can easily read your code. You can easily do this by highlighting your code and then clicking on the # in the editing menu. (You can also type code tags [code] and [/code] by hand.)

Second, avoid adding color or different fonts and font size to your posts. Selective use of color to highlight a single word or phrase can be useful at times, but using color, in general, makes the forums harder to read, especially bright colors like red.

Third, be careful when you cut-and-paste, edit any odd characters and make sure all links are working property.

Thank You.

The UNIX and Linux Forums
# 6  
Old 03-28-2012
What is your OS and version?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Perl - Use of *? in Matching Pattern

I am using Perl version 5.8.4 and trying to understand the use of regular expression. Following is my code and output. $string = "Perl is a\nScripting language"; ($start) = ($string =~ /\A(.*?) /); @lines = ($string =~ /^(.*?) /gm); print "First Word (using \\A): $start\n","Line... (4 Replies)
Discussion started by: jnrohit2k
4 Replies

2. Shell Programming and Scripting

Pattern matching in Perl

Hi, I have a list of IP, eg : 192.168.0.15 192.168.0.24 192.168.2.110 192.168.2.200 And I would like the shortest pattern who match with '192.168.0' and '192.168.2' (without the last dot and number). (7 Replies)
Discussion started by: X-Or
7 Replies

3. Shell Programming and Scripting

Pattern Matching in PERL

I have a 2 files in .gz format and it consists of 5 million lines the format of the file would be gzcat file1.gz | more abcde aerere ffgh56 .. .. 12345 gzcat file2.gz | more abcde , 12345 , 67890, ffgh56 , 45623 ,12334 whatever the string is in the file1 should be matched... (3 Replies)
Discussion started by: aravindj80
3 Replies

4. Shell Programming and Scripting

Perl Pattern matching...

I am doing a file patterhn matching for a text file in PERL I am using this,,, but it says that no file is found $filepattern = '\d{1,4}.*A0NW9693.NDM.HBIDT.*.AD34XADJ.txt'; Can anyone help me out with Perl Pattern Matching concepts and how to do pattern matching for this txt file:... (4 Replies)
Discussion started by: msrahman
4 Replies

5. Shell Programming and Scripting

Perl pattern matching!!

Hi experts, I have many occurances of the following headers in a file. I need to grep for the word changed/inserted in the header, calculate the difference between the two numbers and list the count incrementally. Headers in a file look like this: ------------------- ---------------------... (6 Replies)
Discussion started by: nmattam
6 Replies

6. Shell Programming and Scripting

Perl Pattern Matching

Hello experts, I have a file containing the following text(shortened here). File Begin ---------- < # Billboard.d3fc1302a677.imagePath=S:\\efcm_T4 < Billboard.d3fc1302a677.imagePath=S:\\efcm_T4 --- > # Billboard.d3fc1302a677.imagePath=S:\\efcm_Cassini >... (2 Replies)
Discussion started by: nmattam
2 Replies

7. Shell Programming and Scripting

Perl -Pattern Matching help..!

Hi, I got doubt in Pattern matching, could you tell me how the following differs in action ?? if ( $line1==/$line2/ ) if ( $line1=~/$line2/ ) if ( $line1=~m/$line2/) What is the significance of '~' in matching. Thanks in advance CoolBhai (5 Replies)
Discussion started by: coolbhai
5 Replies

8. Shell Programming and Scripting

Perl Pattern Matching !!! Help

Hello I got the below one from in one of this forums For Ex: Loading File System Networking in nature now i need to extract the patterns between the words File and Networking : i.e. sample output: System cmd used : cat <file> | sed 's/.*File //' | sed 's/Closing.*$//' Actually... (0 Replies)
Discussion started by: maxmave
0 Replies

9. Shell Programming and Scripting

perl pattern matching

hi i am trying to get digits inside brackes from file , whose structure is defined below CREATE TABLE TELM (SOC_NO CHAR (3) NOT NULL, TXN_AMOUNT NUMBER (17,3) SIGN_ON_TIME CHAR (8) TELLER_APP_LIMIT NUMBER (17,3) FIL01 ... (2 Replies)
Discussion started by: zedex
2 Replies

10. Shell Programming and Scripting

pattern matching + perl question

i can only find the first occurance of a pattern how do i set it to loop untill all occurances have changed. #! /usr/bin/perl use POSIX; open (DFH_FILE, "./dfh") or die "Can not read file ($!)"; foreach (<DFH_FILE>) { if ($_ !~ /^#|^$/) { chomp; ... (1 Reply)
Discussion started by: Optimus_P
1 Replies
Login or Register to Ask a Question