Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Perl script: matching multiple lines error

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Perl script: matching multiple lines error
# 1  
Old 08-18-2013
Perl script: matching multiple lines error
Dear Perl users,

Could somebody help me how to fix my code so I can get my desired output.
Here is the data:
Code:
Pattern Gabriel
halo1
halo2
end
Pattern Andreas
halo1
halo2
end

I want to grep multiple lines between the pattern /Pattern Gabriel / and /end/.
Then I will store the output into two files (let's say Gabriel.log and Andres.log based on the name).

Code:
The Gabriel.log contains:

Code:
Pattern Gabriel
halo1
halo2
end



Code:
The Andreas.log contains:

Code:
Pattern Andreas
halo1
halo2
end

I saw there is a similar solution that I found but the desired output isn't valid yet.
here is my code so far:

Code:
#!/usr/bin/perl -w
use strict;

my @result;
my $flag = 0;
my @data = qw /Gabriel Andreas/;

while (my $line = shift @data)
{
  if (open (EXT, ">${line}.log"))
  {

  if (open (FILE, "<data_test.log"))
  {
  
  while (<FILE>) 
    {
      
      if (/Pattern\s+($line)/i) 
      {
       $flag = 1;
      } 
      elsif( /^end/ ) 
      {
       $flag = 0;
      }
      if ($flag) 
      {
        push @result, $_;
      }
    }

  print EXT "$_" for (@result);

  }
  }
}  
close (EXT);
close (FILE);

# 2  
Old 08-18-2013
If your input file is not huge, then here's on way to do it:

Code:
$ 
$ ls -1 *
input
$ 
$ cat -n input
     1    Pattern Gabriel
     2    halo1
     3    halo2
     4    end
     5    Pattern Andreas
     6    halo1
     7    halo2
     8    end
$ 
$ 
$ perl -lne 'BEGIN {$/=undef} while(/^(Pattern (\S+).*?end)/msg) {open(FH, ">", "$2.log"); print FH $1; close(FH)}' input
$ 
$ # see if the logs have been created now
$ ls -1
Andreas.log
Gabriel.log
input
$ 
$ # check the logs
$ cat Andreas.log
Pattern Andreas
halo1
halo2
end
$ 
$ cat Gabriel.log
Pattern Gabriel
halo1
halo2
end
$ 
$

# 3  
Old 08-18-2013
Hi durden_tyler,

Thanks a lot but unfortunately my file is huge. and also if your code above, I can not split into column directly (per each line), am I right?.

do you have other solution for this problem, hopefully just fixed my code because my code already huge so I do not need to change everything.

Thanks
Last edited by askari; 08-18-2013 at 02:23 PM..
# 4  
Old 08-18-2013
Quote:
Originally Posted by askari
...also if your code above, I can not split into column directly (per each line), am I right?.
...
I don't quite understand your question here.
# 5  
Old 08-18-2013
if my original data is changed and added the additional parameter below. I have used your code above before but when I want to split it (e.g. halo1 data into two column by using regexp) is it possible in the while looping directly after the pattern is match?.

Code:
Pattern Gabriel
halo1 data
halo2 data
end
Pattern Andreas
halo1 data
halo2 data
end

# 6  
Old 08-18-2013
I'll answer your second question first.

Quote:
Originally Posted by askari
if my original data is changed and added the additional parameter below. I have used your code above before but when I want to split it (e.g. halo1 data into two column by using regexp) is it possible in the while looping directly after the pattern is match?.

Code:
Pattern Gabriel
halo1 data
halo2 data
end
Pattern Andreas
halo1 data
halo2 data
end

Of course it is possible. Almost every kind of text processing is possible in Perl, if you know how to do it.

As an example:

Code:
$ 
$ 
$ cat -n input
     1    Pattern Gabriel
     2    halo1 data
     3    halo2 data
     4    end
     5    Pattern Andreas
     6    halo1 data
     7    halo2 data
     8    end
$ 
$ 
$ perl -lne 'BEGIN {$/=undef}
             while(/^(Pattern (\S+).*?end)/msg) {
               @x = split("\n",$1);
               print "Begin:",$x[0];
               for $i (1..$#x-1) {
                 print "Line = ", $x[$i];
                 ($col1, $col2) = $x[$i] =~ /^(\S+)\s+(\S+)$/;
                 print "col1 = ", $col1;
                 print "col2 = ", $col2;
                 print "=" x 20;
               }
               print "End:",$x[$#x]
             }
            ' input
Begin:Pattern Gabriel
Line = halo1 data
col1 = halo1
col2 = data
====================
Line = halo2 data
col1 = halo2
col2 = data
====================
End:end
Begin:Pattern Andreas
Line = halo1 data
col1 = halo1
col2 = data
====================
Line = halo2 data
col1 = halo2
col2 = data
====================
End:end
$ 
$ 
$

Now for your first concern about the file being huge - that is a subjective term. What may be a huge file for you may not be huge for Perl. A good idea would be to benchmark your processing and go from there.

Finally, if you want to put chunks of text into separate log files but still want to do line-by-line regex processing, then a simpler approach be to just:

- read your file line-by-line,
- start logging to the file when you encounter the start pattern
- use regexes for the following lines; do whatever you want to do with those
- and stop logging when you encounter the end pattern
# 7  
Old 08-19-2013
Hi durden_tyler,

Thanks for your code. I found the bug was the place of array variable initialization. After I put the "@result" array variable to the next line " if (open (EXT, ">${line}.log")) {",
The program already produced my desired output. Anyway thanks a lot for your code sharing.

Here is the complete code of mine.

Code:
#!/usr/bin/perl -w
use strict;


my $flag = 0;
my @data = qw /Gabriel Andreas/;

while (my $line = shift @data)
{
  if (open (EXT, ">${line}.log"))
  {
   my @result;
   
  if (open (FILE, "<data_test.log"))
  {
  
  while (<FILE>) 
    {
      
      if (/Pattern\s+($line)/i) 
      {
       $flag = 1;
      } 
      elsif( /^end/ ) 
      {
       $flag = 0;
      }
      if ($flag) 
      {
        push @result, $_;
      }
    }

  }
  print EXT "$_" for (@result);
  }
}  
close (EXT);
close (FILE)

__DATA__
Pattern Gabriel
halo1
halo2
end
Pattern Andreas
halo1
halo2
end

Last edited by askari; 08-19-2013 at 03:05 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Group Multiple Lines on SINGLE line matching pattern

Hi Guys, I am trying to format my csv file. When I spool the file using sqlplus the single row output is wrapped on three lines. Somehow I managed to format that file and finally i am trying to make the multiple line on single line. The below command is working fine but I need to pass the... (3 Replies)
Discussion started by: RJSKR28
3 Replies

2. Shell Programming and Scripting

Perl Matching multiple variables

I am trying to match mulitple (3) variables. I found the sub given below on the web which works well when all vars are defined. But there are situations where one or two will not be defined (at least one will always be defined.) Example of the variable content possibilities The sub found... (4 Replies)
Discussion started by: popeye
4 Replies

3. Shell Programming and Scripting

Help need with PERL multiple search pattern matching!

My example file is as given below: conn=1 uid=oracle conn=2 uid=db2 conn=3 uid=oracle conn=4 uid=hash conn=5 uid=skher conn=6 uid=oracle conn=7 uid=mpalkar conn=8 uid=anarke conn=9 uid=oracle conn=1 op=-1 msgId=-1 - fd=104 slot=104 LDAPS connection from 10.10.5.6 to 10.18.6.5 conn=2... (3 Replies)
Discussion started by: sags007_99
3 Replies

4. Shell Programming and Scripting

Multiple pattern matching using awk and getting count of lines

Hi , I have a file which has multiple rows of data, i want to match the pattern for two columns and if both conditions satisfied i have to add the counter by 1 and finally print the count value. How to proceed... I tried in this way... awk -F, 'BEGIN {cnt = 0} {if $6 == "VLY278" &&... (6 Replies)
Discussion started by: aemunathan
6 Replies

5. Shell Programming and Scripting

Perl Regex matching multiple lines

I need a way to extract data from X 4T Solution 21 OCT 2011 37 .00 to account 12345678 User1 user2 X 4T Solution Solution Unlimited 11 Sep 2009 248 .00 to account 87654321 user3 user4 I need it to extract 'X' '37.00' and account number 12345678. I have extracted above stuff... (3 Replies)
Discussion started by: chakrapani
3 Replies

6. Shell Programming and Scripting

Perl XML, find matching condition and grep lines and put the lines somewhere else

Hi, my xml files looks something like this <Instance Name="New York"> <Description></Description> <Instance Name="A"> <Description></Description> <PropertyValue Key="false" Name="Building A" /> </Instance> <Instance Name="B"> ... (4 Replies)
Discussion started by: tententen
4 Replies

7. Shell Programming and Scripting

perl basic multiple pattern matching

Hi everyone, and thank you for your help with this. I am VERY new with perl so all of your help is appreciated. I have tried google but as I don't know the proper terms to search for and could be daunting for a newbie scripter... I know this is very easy for most of you! Thanks! I have a... (4 Replies)
Discussion started by: sinusoid
4 Replies

8. Shell Programming and Scripting

Matching lines across multiple csv files and merging a particular field

I have about 20 CSV's that all look like this: "","","","","","","","","","","","","","","",""What I've been told I need to produce is the exact same thing, but with each file now containing the start_code from every other file where the email matches. It doesn't matter if any of the other... (1 Reply)
Discussion started by: Demosthenes
1 Replies

9. Shell Programming and Scripting

pattern matching over multiple lines and deleting the first

I've got a longish log file with content such as Uplink traffic: Downlink traffic: I want to parse the log file and remove any line that contains the string "Uplink traffic:" at the beginning of the line, but only if the line following it beginnings with the string "Downlink traffic:" (in... (7 Replies)
Discussion started by: Yorkie99
7 Replies

10. Shell Programming and Scripting

[Help] PERL Script - grep multiple lines

Hi Gurus, I need some help with the "grep" command or whatever command that you think suitable for me. I'm about to write a perl script to extract a report from the system and submit it to the end users. The input for the script will consist of 3 element. 1) Generation ID 2) Month 3) Year... (6 Replies)
Discussion started by: miskin
6 Replies
Login or Register to Ask a Question