Pattern Matching in PERL


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Pattern Matching in PERL
# 1  
Old 01-31-2012
Pattern Matching in PERL

I have a 2 files in .gz format and it consists of 5 million lines the format of the file would be

gzcat file1.gz | more

abcde
aerere
ffgh56
..
..
12345

gzcat file2.gz | more

abcde , 12345 , 67890,
ffgh56 , 45623 ,12334


whatever the string is in the file1 should be matched with the file 2 and the complete line in file 2 need to be printed in file 3

if it can executed with the zip format only it would be helpful

tried fgrep it is failing due to source file due to memory limitations

So need some script using PERL or AWK.
# 2  
Old 01-31-2012
Would this get you what you want?
Code:
gzcat file1.gz > file1.txt
gzcat file2.gz | grep -f file1.txt - > allLinesMatched.txt

# 3  
Old 01-31-2012
awk in bash/ksh93 shell:
Code:
awk -F '[ \t,]*' 'NR==FNR{A[$1];next}{for (i=1;i<=NF;i++) if ($i in A) {print;next}}' <(gzcat file1.gz) <(gzcat file2.gz) | gzip > outfile.gz

awk only:
Code:
awk -F '[ \t,]*' -v if1=file1.gz -v if2=file2.gz -v of=outfile.gz  'BEGIN{
  cmd="gzip>"of
  while("gzcat "if1 | getline){A[$1]}
  while("gzcat "if2 | getline){for (i=1;i<=NF;i++) if ($i in A) {print | cmd ;i=NF+1}}
}'


Last edited by Scrutinizer; 01-31-2012 at 11:37 AM..
# 4  
Old 01-31-2012
It is giving a memory limitation error @ mrini
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Perl - Use of *? in Matching Pattern

I am using Perl version 5.8.4 and trying to understand the use of regular expression. Following is my code and output. $string = "Perl is a\nScripting language"; ($start) = ($string =~ /\A(.*?) /); @lines = ($string =~ /^(.*?) /gm); print "First Word (using \\A): $start\n","Line... (4 Replies)
Discussion started by: jnrohit2k
4 Replies

2. Shell Programming and Scripting

Pattern matching in Perl

Hi, I have a list of IP, eg : 192.168.0.15 192.168.0.24 192.168.2.110 192.168.2.200 And I would like the shortest pattern who match with '192.168.0' and '192.168.2' (without the last dot and number). (7 Replies)
Discussion started by: X-Or
7 Replies

3. Shell Programming and Scripting

Need help with perl pattern matching

My log file looks as given below, its actually a huge file around 1 GB and these are some of the line: conn=5368758 op=10628050 msgId=64 - RESULT err=0 tag=101 nentries=1 etime=0 conn=7462122 op=-1 msgId=-1 - fd=247 slot=247 LDAPS connection from 10.13.18.12:37645 to 10.18.6.45 conn=7462122... (5 Replies)
Discussion started by: sags007_99
5 Replies

4. Shell Programming and Scripting

Perl Pattern matching...

I am doing a file patterhn matching for a text file in PERL I am using this,,, but it says that no file is found $filepattern = '\d{1,4}.*A0NW9693.NDM.HBIDT.*.AD34XADJ.txt'; Can anyone help me out with Perl Pattern Matching concepts and how to do pattern matching for this txt file:... (4 Replies)
Discussion started by: msrahman
4 Replies

5. Shell Programming and Scripting

Perl pattern matching!!

Hi experts, I have many occurances of the following headers in a file. I need to grep for the word changed/inserted in the header, calculate the difference between the two numbers and list the count incrementally. Headers in a file look like this: ------------------- ---------------------... (6 Replies)
Discussion started by: nmattam
6 Replies

6. Shell Programming and Scripting

Perl Pattern Matching

Hello experts, I have a file containing the following text(shortened here). File Begin ---------- < # Billboard.d3fc1302a677.imagePath=S:\\efcm_T4 < Billboard.d3fc1302a677.imagePath=S:\\efcm_T4 --- > # Billboard.d3fc1302a677.imagePath=S:\\efcm_Cassini >... (2 Replies)
Discussion started by: nmattam
2 Replies

7. Shell Programming and Scripting

Perl -Pattern Matching help..!

Hi, I got doubt in Pattern matching, could you tell me how the following differs in action ?? if ( $line1==/$line2/ ) if ( $line1=~/$line2/ ) if ( $line1=~m/$line2/) What is the significance of '~' in matching. Thanks in advance CoolBhai (5 Replies)
Discussion started by: coolbhai
5 Replies

8. Shell Programming and Scripting

Perl Pattern Matching !!! Help

Hello I got the below one from in one of this forums For Ex: Loading File System Networking in nature now i need to extract the patterns between the words File and Networking : i.e. sample output: System cmd used : cat <file> | sed 's/.*File //' | sed 's/Closing.*$//' Actually... (0 Replies)
Discussion started by: maxmave
0 Replies

9. Shell Programming and Scripting

perl pattern matching

hi i am trying to get digits inside brackes from file , whose structure is defined below CREATE TABLE TELM (SOC_NO CHAR (3) NOT NULL, TXN_AMOUNT NUMBER (17,3) SIGN_ON_TIME CHAR (8) TELLER_APP_LIMIT NUMBER (17,3) FIL01 ... (2 Replies)
Discussion started by: zedex
2 Replies

10. Shell Programming and Scripting

pattern matching + perl question

i can only find the first occurance of a pattern how do i set it to loop untill all occurances have changed. #! /usr/bin/perl use POSIX; open (DFH_FILE, "./dfh") or die "Can not read file ($!)"; foreach (<DFH_FILE>) { if ($_ !~ /^#|^$/) { chomp; ... (1 Reply)
Discussion started by: Optimus_P
1 Replies
Login or Register to Ask a Question

Featured Tech Videos