Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Perl script: matching multiple lines error
Top Forums Shell Programming and Scripting Perl script: matching multiple lines error Post 302844327 by durden_tyler on Sunday 18th of August 2013 01:55:39 PM
Old 08-18-2013
I'll answer your second question first.

Quote:
Originally Posted by askari
if my original data is changed and added the additional parameter below. I have used your code above before but when I want to split it (e.g. halo1 data into two column by using regexp) is it possible in the while looping directly after the pattern is match?.

Code:
Pattern Gabriel
halo1 data
halo2 data
end
Pattern Andreas
halo1 data
halo2 data
end

Of course it is possible. Almost every kind of text processing is possible in Perl, if you know how to do it.

As an example:

Code:
$ 
$ 
$ cat -n input
     1    Pattern Gabriel
     2    halo1 data
     3    halo2 data
     4    end
     5    Pattern Andreas
     6    halo1 data
     7    halo2 data
     8    end
$ 
$ 
$ perl -lne 'BEGIN {$/=undef}
             while(/^(Pattern (\S+).*?end)/msg) {
               @x = split("\n",$1);
               print "Begin:",$x[0];
               for $i (1..$#x-1) {
                 print "Line = ", $x[$i];
                 ($col1, $col2) = $x[$i] =~ /^(\S+)\s+(\S+)$/;
                 print "col1 = ", $col1;
                 print "col2 = ", $col2;
                 print "=" x 20;
               }
               print "End:",$x[$#x]
             }
            ' input
Begin:Pattern Gabriel
Line = halo1 data
col1 = halo1
col2 = data
====================
Line = halo2 data
col1 = halo2
col2 = data
====================
End:end
Begin:Pattern Andreas
Line = halo1 data
col1 = halo1
col2 = data
====================
Line = halo2 data
col1 = halo2
col2 = data
====================
End:end
$ 
$ 
$

Now for your first concern about the file being huge - that is a subjective term. What may be a huge file for you may not be huge for Perl. A good idea would be to benchmark your processing and go from there.

Finally, if you want to put chunks of text into separate log files but still want to do line-by-line regex processing, then a simpler approach be to just:

- read your file line-by-line,
- start logging to the file when you encounter the start pattern
- use regexes for the following lines; do whatever you want to do with those
- and stop logging when you encounter the end pattern
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

[Help] PERL Script - grep multiple lines

Hi Gurus, I need some help with the "grep" command or whatever command that you think suitable for me. I'm about to write a perl script to extract a report from the system and submit it to the end users. The input for the script will consist of 3 element. 1) Generation ID 2) Month 3) Year... (6 Replies)
Discussion started by: miskin
6 Replies

2. Shell Programming and Scripting

pattern matching over multiple lines and deleting the first

I've got a longish log file with content such as Uplink traffic: Downlink traffic: I want to parse the log file and remove any line that contains the string "Uplink traffic:" at the beginning of the line, but only if the line following it beginnings with the string "Downlink traffic:" (in... (7 Replies)
Discussion started by: Yorkie99
7 Replies

3. Shell Programming and Scripting

Matching lines across multiple csv files and merging a particular field

I have about 20 CSV's that all look like this: "","","","","","","","","","","","","","","",""What I've been told I need to produce is the exact same thing, but with each file now containing the start_code from every other file where the email matches. It doesn't matter if any of the other... (1 Reply)
Discussion started by: Demosthenes
1 Replies

4. Shell Programming and Scripting

perl basic multiple pattern matching

Hi everyone, and thank you for your help with this. I am VERY new with perl so all of your help is appreciated. I have tried google but as I don't know the proper terms to search for and could be daunting for a newbie scripter... I know this is very easy for most of you! Thanks! I have a... (4 Replies)
Discussion started by: sinusoid
4 Replies

5. Shell Programming and Scripting

Perl XML, find matching condition and grep lines and put the lines somewhere else

Hi, my xml files looks something like this <Instance Name="New York"> <Description></Description> <Instance Name="A"> <Description></Description> <PropertyValue Key="false" Name="Building A" /> </Instance> <Instance Name="B"> ... (4 Replies)
Discussion started by: tententen
4 Replies

6. Shell Programming and Scripting

Perl Regex matching multiple lines

I need a way to extract data from X 4T Solution 21 OCT 2011 37 .00 to account 12345678 User1 user2 X 4T Solution Solution Unlimited 11 Sep 2009 248 .00 to account 87654321 user3 user4 I need it to extract 'X' '37.00' and account number 12345678. I have extracted above stuff... (3 Replies)
Discussion started by: chakrapani
3 Replies

7. Shell Programming and Scripting

Multiple pattern matching using awk and getting count of lines

Hi , I have a file which has multiple rows of data, i want to match the pattern for two columns and if both conditions satisfied i have to add the counter by 1 and finally print the count value. How to proceed... I tried in this way... awk -F, 'BEGIN {cnt = 0} {if $6 == "VLY278" &&... (6 Replies)
Discussion started by: aemunathan
6 Replies

8. Shell Programming and Scripting

Help need with PERL multiple search pattern matching!

My example file is as given below: conn=1 uid=oracle conn=2 uid=db2 conn=3 uid=oracle conn=4 uid=hash conn=5 uid=skher conn=6 uid=oracle conn=7 uid=mpalkar conn=8 uid=anarke conn=9 uid=oracle conn=1 op=-1 msgId=-1 - fd=104 slot=104 LDAPS connection from 10.10.5.6 to 10.18.6.5 conn=2... (3 Replies)
Discussion started by: sags007_99
3 Replies

9. Shell Programming and Scripting

Perl Matching multiple variables

I am trying to match mulitple (3) variables. I found the sub given below on the web which works well when all vars are defined. But there are situations where one or two will not be defined (at least one will always be defined.) Example of the variable content possibilities The sub found... (4 Replies)
Discussion started by: popeye
4 Replies

10. Shell Programming and Scripting

Group Multiple Lines on SINGLE line matching pattern

Hi Guys, I am trying to format my csv file. When I spool the file using sqlplus the single row output is wrapped on three lines. Somehow I managed to format that file and finally i am trying to make the multiple line on single line. The below command is working fine but I need to pass the... (3 Replies)
Discussion started by: RJSKR28
3 Replies

LEARN ABOUT DEBIAN

fp2hdf

FP2HDF(1)						      General Commands Manual							 FP2HDF(1)

NAME

       fp2hdf - convert floating point data to HDF

SYNOPSIS

       fp2hdf -h[elp] fp2hdf infile [infile...]  -o[utfile outfile] [-r[aster] [ras_options...]]  [-f[loat]]

DESCRIPTION

       fp2hdf converts floating point data to HDF Scientific Data Set (SDS) and/or 8-bit Raster Image Set (RIS8) format, storing the results in an
       HDF file.  The image data can be scaled about a mean value.

       Input file(s) contain a single two-dimensional or three-dimensional floating point array in either ASCII text, native  floating	point,	or
       HDF  SDS format.  If an HDF file is used for input, it must contain an SDS.  The SDS need only contain a dimension record and the data, but
       if it also contains maximum and minimum values and/or scales for each axis, these will be used.	If the	input  format  is  ASCII  text	or
       native floating point, see "Notes" below on how it must be organized.

OPTIONS

       -h[elp]
	      Print a helpful summary of usage, and exit.

       -o[utfile] outfile
	      Data from one or more input files are stored as one or more data sets and/or images in one HDF output file, outfile.

       -r[aster]
	      Store  output  as a raster image set in the output file -f[loat] Store output as a scientific data set in the the output file.  This
	      is the default if the "-r" option is not specified.

       ras_opts:

       -e[xpand] horiz vert [depth]
	      Expand float data via pixel replication to produce the image(s).	horiz and vert give the horizontal and vertical resolution of  the
	      image(s) to be produced; and optionally, depth gives the number of images or depth planes (for 3D input data).

       -i[nterp] horiz vert [depth]
	      Apply  bilinear, or trilinear, interpolation to the float data to produce the image(s).  horiz, vert, and depth must be greater than
	      or equal to the dimensions of the original dataset.

       -p[alfile] palfile
	      Store the palette with the image.  Get the palette from palfile; which may be an HDF file containing a palette, or a file containing
	      a raw palette.

       -m[ean] mean
	      If  a  floating  point mean value is given, the image will be scaled about the mean.  The new extremes (newmax and newmin), as given
	      by:

		newmax = mean + max(abs(max-mean), abs(mean-min))
		newmin = mean - max(abs(max-mean), abs(mean-min))

	      will be equidistant from the mean value.	If no mean value is given, then the mean will be:  0.5 * (max + min)

INPUT

       If the input file format is ASCII text or native floating point, it must have the following input fields:

	       format
	       nplanes
	       nrows
	       ncols
	       max_value
	       min_value
	       [plane1 plane2 plane3 ...]
	       row1 row2 row3 ...
	       col1 col2 col3 ...
	       data1 data2 data3 ...
	       ...

       Where:

       format Format designator ("TEXT", "FP32" or "FP64").

       nplanes
	      Dimension of the depth axis ("1" for 2D input).

       nrows  Dimension of the vertical axis.

       ncols  Dimension of the horizontal axis.

       max_value
	      Maximum data value.

       min_value
	      Minimum data value.

       plane1, plane2, plane3, ...
	      Scales for depth axis.

       row1, row2, row3, ...
	      Scales for the vertical axis.

       col1, col2, col3, ...
	      Scales for the horizontal axis.

       data1, data2, data3, ...
	      The data ordered by rows, left to right and top to bottom; then optionally, ordered by planes, front to back.

	      For FP32 and FP64 input format, format, nplanes, nrows, ncols, and nplanes are native integers; where format is the  integer  repre-
	      sentation  of  the appropriate 4-character string (0x46503332 for "FP32" and 0x46503634 for "FP64").  The remaining input fields are
	      composed of native 32-bit floating point values for FP32 input format, or native 64-bit floating point values for FP64 input format.

EXAMPLE

       Convert floating point data in "f1.txt" to SDS format, and store it as an SDS in HDF file "o1":

	       fp2hdf f1.txt -o o1

       Convert floating point data in "f2.hdf" to 8-bit raster format, and store it as an RIS8 in HDF file "o2":

	       fp2hdf f2.hdf -o o2 -r

       Convert floating point data in "f3.bin" to 8-bit raster format and SDS format, and store both the RIS8 and the SDS in HDF file "o3":

	       fp2hdf f3.bin -o o3 -r -f

       Convert floating point data in "f4" to a 500x600 raster image, and store the RIS8 in HDF file "o4".  Also store a  palette  from  "palfile"
       with the image:

	       fp2hdf f4 -o o4 -r -e 500 600 -p palfile

       Convert floating point data in "f5" to 200 planes of 500x600 raster images, and store the RIS8 in HDF file "o5".  Also scale the image data
       so that it is centered about a mean value of 10.0:

	       fp2hdf f5 -o o5 -r -i 500 600 200 -m 10.0

SEE ALSO

       hdf(5)

								 October 30, 1999							 FP2HDF(1)
All times are GMT -4. The time now is 06:24 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy