Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

perl : replace multiline text between two marker points

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting perl : replace multiline text between two marker points
# 1  
Old 01-19-2012
perl : replace multiline text between two marker points
Hi there

I just wondered if someone could give me some perl advice

I have a bunch of text files used for a wiki that have common headings such as

Code:
 
 
---++ Title
blah
 
---++ Summary
blah
 
---++ Details
Here is the multiline
block 
of text I 
wish 
to 
replace that will
be different
in 
every file
 
---++ More info 
blah

So basically, All of the text between the "---++ Details" marker and the "---++ More info" marker I want to strip out and replace with a static multiline piece of text which i will put in a variable ...

Does anyone know of a quick and dirty way within perl of doing this, so far in my code i am passing the filename to the script for opening, I have create a 'here doc' style variable to contain my desired mutiline block of text and that is where my mind has gone blank ...

I guess i need a way of detecting the "---++ Details" line, deleteing everything after it up to and not including the "---++ More info" line and then putting the contents of $newtext into that space ?

Any guidance would be greatly appreciated
# 2  
Old 01-19-2012
One way to do it in Perl -

Code:
$
$
$ cat -n f50
     1  ---++ Title
     2  blah
     3
     4  ---++ Summary
     5  blah
     6
     7  ---++ Details
     8  Here is the multiline
     9  block
    10  of text I
    11  wish
    12  to
    13  replace that will
    14  be different
    15  in
    16  every file
    17
    18  ---++ More info
    19  blah
$
$
$ perl -lne 'BEGIN {$x=1; $str = "My multi-line\nstatic text\nover here"}
             if (/^---\+\+ Details/) {print; $x=0}
             elsif (/^---\+\+ More info/) {print $str; $x=1}
             print if $x==1
            ' f50
---++ Title
blah
 
---++ Summary
blah
 
---++ Details
My multi-line
static text
over here
---++ More info
blah
$
$
$

tyler_durden
# 3  
Old 01-25-2012
Thanks Tyler , much appreciated

---------- Post updated at 07:16 AM ---------- Previous update was at 05:12 AM ----------

Hi one further question if i may. I am trying to create a perl script out of this so that I can put all the files I wish to edit into an array and edit them all rather than running this from the command line

but how do i tell the little function youve provided about the file I want to edit so for example


Code:
 
#!/usr/bin/perl -w
use strict;

my @array = <i will populate this array with files names>
 
my $x;
my $str = <<EOF;
 
My new
multi-line
static text

EOF
 
foreach (@array) {
  open FILE,"+>$_"
 
BEGIN { $x=1 };
if (/^---\+\+ Details/) {
   print; $x=0
} elsif (/^---\+\+ More info/) {
   print $str; $x=1
}
print if $x==1
 
close $_;
}


Ive clearly done that completely wrong as it doesnt workSmilie , but I cant seem to put your nice little function to use inside a script that passes my filenames in the array as the equivelant of the 'standard in' from your command line version?

Any advice on this would be greatly appreciated
# 4  
Old 01-25-2012
You could let Perl handle the intricacies of file manipulation. The command -

Code:
perl -i.bak -pe 'BEGIN {undef $/; $x="My multi-line\nstatic text\nover here"}
                  s/(---\+\+ Details\n).*?(\n---\+\+ More info)/$1$x$2/s
                 ' file1 file2 file3

creates backup files called "file1.bak", "file2.bak" and "file3.bak" for the original files "file1", "file2" and "file3" and then modifies them inline.

An example follows -

Code:
$
$ # check the original files
$
$ cat -n file1
   1  ---++ Title
   2  FILE1
   3
   4  ---++ Summary
   5  blah
   6
   7  ---++ Details
   8  Here is the multiline
   9  block
  10  of text I
  11  wish
  12  to
  13  replace that will
  14  be different
  15  in
  16  every file
  17
  18  ---++ More info
  19  blah
$
$ cat -n file2
   1  ---++ Title
   2  FILE2
   3
   4  ---++ Summary
   5  blah
   6
   7  ---++ Details
   8  Here is the multiline
   9  block
  10  of text I
  11  wish
  12  to
  13  replace that will
  14  be different
  15  in
  16  every file
  17
  18  ---++ More info
  19  blah
$
$ cat -n file3
   1  ---++ Title
   2  FILE3
   3
   4  ---++ Summary
   5  blah
   6
   7  ---++ Details
   8  Here is the multiline
   9  block
  10  of text I
  11  wish
  12  to
  13  replace that will
  14  be different
  15  in
  16  every file
  17
  18  ---++ More info
  19  blah
$
$ # run the Perl one-liner
$
$ perl -i.bak -pe 'BEGIN {undef $/; $x="My multi-line\nstatic text\nover here"}
                 s/(---\+\+ Details\n).*?(\n---\+\+ More info)/$1$x$2/s
                ' file1 file2 file3
$
$ # now check the modified files
$
$ cat -n file1
   1  ---++ Title
   2  FILE1
   3
   4  ---++ Summary
   5  blah
   6
   7  ---++ Details
   8  My multi-line
   9  static text
  10  over here
  11  ---++ More info
  12  blah
$
$ cat -n file2
   1  ---++ Title
   2  FILE2
   3
   4  ---++ Summary
   5  blah
   6
   7  ---++ Details
   8  My multi-line
   9  static text
  10  over here
  11  ---++ More info
  12  blah
$
$ cat -n file3
   1  ---++ Title
   2  FILE3
   3
   4  ---++ Summary
   5  blah
   6
   7  ---++ Details
   8  My multi-line
   9  static text
  10  over here
  11  ---++ More info
  12  blah
$
$ # and see if the original files were backed up
$
$ cat -n file1.bak
   1  ---++ Title
   2  FILE1
   3
   4  ---++ Summary
   5  blah
   6
   7  ---++ Details
   8  Here is the multiline
   9  block
  10  of text I
  11  wish
  12  to
  13  replace that will
  14  be different
  15  in
  16  every file
  17
  18  ---++ More info
  19  blah
$
$ cat -n file2.bak
   1  ---++ Title
   2  FILE2
   3
   4  ---++ Summary
   5  blah
   6
   7  ---++ Details
   8  Here is the multiline
   9  block
  10  of text I
  11  wish
  12  to
  13  replace that will
  14  be different
  15  in
  16  every file
  17
  18  ---++ More info
  19  blah
$
$ cat -n file3.bak
   1  ---++ Title
   2  FILE3
   3
   4  ---++ Summary
   5  blah
   6
   7  ---++ Details
   8  Here is the multiline
   9  block
  10  of text I
  11  wish
  12  to
  13  replace that will
  14  be different
  15  in
  16  every file
  17
  18  ---++ More info
  19  blah
$
$

You could use regular expressions while specifying the original files, and Perl takes care of resolving them, backing them up and editing them individually, e.g. in my case above, I could use -

Code:
perl -i.bak -pe 'BEGIN {undef $/; $x="My multi-line\nstatic text\nover here"}
               s/(---\+\+ Details\n).*?(\n---\+\+ More info)/$1$x$2/s
              ' file*

or

Code:
perl -i.bak -pe 'BEGIN {undef $/; $x="My multi-line\nstatic text\nover here"}
               s/(---\+\+ Details\n).*?(\n---\+\+ More info)/$1$x$2/s
              ' file?

tyler_durden
# 5  
Old 01-26-2012
Thanks Tyler, The situation unfortunately is that this will form part of a larger perl script that will run as part of a schedule hence the question on integrating it into a script as opposed to running it manually on the command line. The list of filenames to edit will be generated as part of another process and will be available to me to use within the same perl script as an @array
# 6  
Old 01-26-2012
Quote:
Originally Posted by rethink
...this will form part of a larger perl script that will run as part of a schedule hence the question on integrating it into a script as opposed to running it manually on the command line. The list of filenames to edit will be generated as part of another process and will be available to me to use within the same perl script as an @array
Here's a Perl program for that case -

Code:
$
$ # The Perl program to modify multi-line text in multiple files
$
$ cat -n modify_multiline.pl
     1  #!/usr/bin/perl -w
     2  use strict;
     3  my @files = qw (file1 file2 file3);   # file list to modify
     4  my $text = <<"EOF";                   # multi-line static text
     5  My multi-line
     6  static text
     7  over here
     8  EOF
     9  foreach my $old (@files) {
    10    my $new = "$old.new";
    11    open(OLD, "<", $old) or die "Can't open $old: $!";
    12    open(NEW, ">", $new) or die "Can't open $new: $!";
    13    my $on = 1;                         # set printing on at the outset
    14    while (<OLD>) {
    15      if (/^---\+\+ Details/) {         # matched pattern starts
    16        print NEW $_;                   # print this line
    17        $on = 0;                        # set printing off
    18      } elsif (/^---\+\+ More info/) {  # matched pattern ends
    19        print NEW $text;                # print static text
    20        $on = 1;                        # set printing on
    21      }
    22      print NEW $_ if $on;              # print if printing on
    23    }
    24    close(OLD) or die "Can't close $old: $!";
    25    close(NEW) or die "Can't close $new: $!";
    26    rename($old, "$old.bak") or die "Can't rename $old to $old.bak: $!";
    27    rename($new, $old) or die "Can't rename $new to $old: $!";
    28  }
$
$ # Original files before modification
$
$ cat -n file1
     1  ---++ Title
     2  FILE1
     3
     4  ---++ Summary
     5  blah
     6
     7  ---++ Details
     8  Here is the multiline
     9  block
    10  of text I
    11  wish
    12  to
    13  replace that will
    14  be different
    15  in
    16  every file
    17
    18  ---++ More info
    19  blah
$
$ cat -n file2
     1  ---++ Title
     2  FILE2
     3
     4  ---++ Summary
     5  blah
     6
     7  ---++ Details
     8  Here is the multiline
     9  block
    10  of text I
    11  wish
    12  to
    13  replace that will
    14  be different
    15  in
    16  every file
    17
    18  ---++ More info
    19  blah
$
$ cat -n file3
     1  ---++ Title
     2  FILE3
     3
     4  ---++ Summary
     5  blah
     6
     7  ---++ Details
     8  Here is the multiline
     9  block
    10  of text I
    11  wish
    12  to
    13  replace that will
    14  be different
    15  in
    16  every file
    17
    18  ---++ More info
    19  blah
$
$ # Now run the Perl program
$
$ perl modify_multiline.pl
$
$ # Check modified files
$
$ cat -n file1
     1  ---++ Title
     2  FILE1
     3
     4  ---++ Summary
     5  blah
     6
     7  ---++ Details
     8  My multi-line
     9  static text
    10  over here
    11  ---++ More info
    12  blah
$
$ cat -n file2
     1  ---++ Title
     2  FILE2
     3
     4  ---++ Summary
     5  blah
     6
     7  ---++ Details
     8  My multi-line
     9  static text
    10  over here
    11  ---++ More info
    12  blah
$
$ cat -n file3
     1  ---++ Title
     2  FILE3
     3
     4  ---++ Summary
     5  blah
     6
     7  ---++ Details
     8  My multi-line
     9  static text
    10  over here
    11  ---++ More info
    12  blah
$
$ # Check backup files
$
$ cat -n file1.bak
     1  ---++ Title
     2  FILE1
     3
     4  ---++ Summary
     5  blah
     6
     7  ---++ Details
     8  Here is the multiline
     9  block
    10  of text I
    11  wish
    12  to
    13  replace that will
    14  be different
    15  in
    16  every file
    17
    18  ---++ More info
    19  blah
$
$ cat -n file2.bak
     1  ---++ Title
     2  FILE2
     3
     4  ---++ Summary
     5  blah
     6
     7  ---++ Details
     8  Here is the multiline
     9  block
    10  of text I
    11  wish
    12  to
    13  replace that will
    14  be different
    15  in
    16  every file
    17
    18  ---++ More info
    19  blah
$
$ cat -n file3.bak
     1  ---++ Title
     2  FILE3
     3
     4  ---++ Summary
     5  blah
     6
     7  ---++ Details
     8  Here is the multiline
     9  block
    10  of text I
    11  wish
    12  to
    13  replace that will
    14  be different
    15  in
    16  every file
    17
    18  ---++ More info
    19  blah
$
$

tyler_durden
# 7  
Old 01-30-2012
thats very kind of your Tyler, thanks ..
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

UNIX multiline replace

Hi We have a database export file which needs to be formatted as below InputCreate view ABC1 as Locking ABC1 for Access select * from PQR Create view ABC2 as Locking ABC2 for access select * from PQR Create view ABC3 as Locking ABC3 for Access select * from PQR OutputCreate... (5 Replies)
Discussion started by: sheetal.arun
5 Replies

2. Shell Programming and Scripting

How to replace word with multiline text using shell scripting.

Hi all I have data files which contain data as shown below: Line 5: FIDE INST_DESC: DIAM Co Ltd/Japan => MAID Co Ltd/Japan INST_NME: DIAM Co Ltd/Japan => MAID Co Ltd/Japan Line 6: FIDE INST_DESC: DIAM DL/Pimco US Bond Open Born in the USA => MAID DL/Pimco US Bond Open Born in the... (6 Replies)
Discussion started by: Ganesh_more
6 Replies

3. UNIX for Dummies Questions & Answers

Perl one liner to replace text

Not quite a unix question but problem in a perl command. Taking a chance if someone knows about the error cat 1 a b c d perl -p -e 's/a/b/g' 1 b b c d What is the problem here?? perl -p -i -e 's/a/b/g' 1 Can't remove 1: Text file busy, skipping file. (2 Replies)
Discussion started by: analyst
2 Replies

4. Shell Programming and Scripting

Replace/Remove not specific text in perl

Hello, Consider that i have many files that have the below format: file1 900 7777 1000 5 6 23 nnnnnnnnnnnnnnnnnn 1100 kkkkkkk file2 900 1989 1000 5 3 10 kkkdfdfdffd 1100 kkkkkkk What i would like to do is on every file to search the line that starts with... (4 Replies)
Discussion started by: chriss_58
4 Replies

5. UNIX for Dummies Questions & Answers

[Solved] Vi - replace words with points (.)

Hello guys, I'm trying to replace the word "i.e." for "ie." in Vi but everytime I used the search tool for start looking for it (this is: /i.e.), it finds every word that contains the "i" and "e" word. I tried the following command: :%s/i.e./ie./g However, it doesn't work. Any help... (2 Replies)
Discussion started by: Gery
2 Replies

6. UNIX for Dummies Questions & Answers

Replace a string within 2 points and save it

I've got this xml file <file1> some text here </file1> <file2> some text here </file2> How do I change the text in element file1 to a sentence that I want, defined by variable $sentence. using ksh here. (2 Replies)
Discussion started by: alienated
2 Replies

7. Shell Programming and Scripting

grep command to replace multiline text from httpd.conf file on Fedora

Hi, I am a newbie to shell scripting and to Linux environment as well. In my project I am trying to search for following text from the httpd.conf file <Directory '/somedir/someinnerdir'> AllowOverride All </Directory> and then remove this text and again rewrite the same text. The... (1 Reply)
Discussion started by: bhushan
1 Replies

8. Shell Programming and Scripting

Multiline replace problem

I have a data of the form 0.0117843924 0. 0. 0. 0. 0.011036017 0. 0. 0. 0. 0.0103351669 0. 0. 0. 0. 4839.41211 0. 0. 0. 0. 4532.08203 0. 0. 0. 0. I would like to insert a couple of blank lines before the 4839 line, every time it appears. The numbers in the... (2 Replies)
Discussion started by: mathis
2 Replies

9. Shell Programming and Scripting

Search and replace string between 2 points

I have a file that contains the following string. connect odemgr/bank123@odsd I am liiking to write a scrupt that will change the par of the string between the "/" and the "@" anyhelp qwould be greatly appriciated. (3 Replies)
Discussion started by: whited05
3 Replies
Login or Register to Ask a Question