Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Delete lines above and below specific line of text

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Delete lines above and below specific line of text
# 1  
Old 01-23-2016
Delete lines above and below specific line of text
I'm trying to remove a specific number of lines, above and below a specific line of text, highlighted in red:

Code:
<STMTTRN>
<TRNTYPE>CREDIT
<DTPOSTED>20151205000001[-3:GMT]
<TRNAMT>10
<FITID>667800001
<CHECKNUM>667800001
<MEMO>BALANCE
</STMTTRN>
<STMTTRN>
<TRNTYPE>DEBIT
<DTPOSTED>20151207000001[-3:GMT]
<TRNAMT>-10
<FITID>667800002
<CHECKNUM>667800002
<MEMO>PAY
</STMTTRN>

That's 6 lines above and 9 below. The content of the file extends beyond those lines, so I have to specify.

I know the sed command can do this but I don't know how to proceed in this case.

Moderator's Comments:
Mod Comment Please use code tags for code and data
Last edited by Scrutinizer; 01-23-2016 at 05:37 PM.. Reason: code tags
# 2  
Old 01-23-2016
Save as filter.pl
Run as perl filter.pl bomsom.file

Substitute the highlighted for any regex you want.

Code:
#!/usr/bin/env perl
use strict;
use warnings;

my @buffer;

while( <> )
{
    push @buffer, $_;
    if( @buffer == 16 )
    {
        if( $buffer[6] =~ /<MEMO>BALANCE/ )
        {
            @buffer = ();
            next;
        }
        print shift @buffer;
    }
}
print @buffer if @buffer;

# 3  
Old 01-23-2016
Sometimes, good old ed is simpler:
Code:
#!/bin/ksh
ed -s file <<-"EOF"
	g/<MEMO>BALANCE/.-6,.+9d
	1,$p
EOF

Although written and tested using the Korn shell, this will work with any shell that uses basic Bourne shell syntax. It will also work with ex instead of ed.
# 4  
Old 01-23-2016
In case you want a throw away command line:
Code:
perl -ne 'push @b, $_; (@b==16) and $b[6]=~/<MEMO>BALANCE/?@b=():print shift @b; END{print @b if @b}' bomsom.file

However, the full script is more flexible since you could add to the work it can do; for example if you were to combine the requirement of this thread and the requirement of your other thread

Code:
cat bomsom.file

Code:
STMTTRN>
<TRNTYPE>DEBIT
<DTPOSTED>20151127000001[-3:GMT]
<TRNAMT>-8.77
<FITID>667800007
<CHECKNUM>667800007
<MEMO>PAYPAL
</STMTTRN>

<STMTTRN>
<TRNTYPE>CREDIT
<DTPOSTED>20151130000001[-3:GMT]
<TRNAMT>6.00
<FITID>667800008
<CHECKNUM>667800008
<MEMO>ENAPARK
</STMTTRN>
<STMTTRN>
<TRNTYPE>CREDIT
<DTPOSTED>20151205000001[-3:GMT]
<TRNAMT>10
<FITID>667800001
<CHECKNUM>667800001
<MEMO>BALANCE
</STMTTRN>
<STMTTRN>
<TRNTYPE>DEBIT
<DTPOSTED>20151207000001[-3:GMT]
<TRNAMT>-10
<FITID>667800002
<CHECKNUM>667800002
<MEMO>PAY
</STMTTRN>
<STMTTRN>
<TRNTYPE>DEBIT
<DTPOSTED>20151127000001[-3:GMT]
<TRNAMT>-8.77
<FITID>667800007
<CHECKNUM>667800007
<MEMO>PAYPAL
</STMTTRN>

<STMTTRN>
<TRNTYPE>CREDIT
<DTPOSTED>20151130000001[-3:GMT]
<TRNAMT>6.00
<FITID>667800008
<CHECKNUM>667800008
<MEMO>ENAPARK
</STMTTRN>

Highlighted in blue is the modification.
Code:
cat filter.pl

Code:
#!/usr/bin/env perl
use strict;
use warnings;

my @buffer;

while( <> )
{
    push @buffer, $_;
    if( @buffer == 16 )
    {
        if( $buffer[6] =~ /<MEMO>BALANCE/)
        {
            @buffer = ();
            next;
        }
        print reverse_sign( shift @buffer );
    }
}
print reverse_sign( @buffer ) if @buffer;

sub reverse_sign
{
    my $lines = \@_;
    for my $line ( @{$lines} ){
        $line =~ s/^(<TRNAMT>)(-?\d+\.\d+)$/sprintf "%s%.2f", $1, -$2/e;
    }
    return @{$lines};
}



Code:
perl filter.pl bomsom.file

Code:
<STMTTRN>
<TRNTYPE>DEBIT
<DTPOSTED>20151127000001[-3:GMT]
<TRNAMT>8.77
<FITID>667800007
<CHECKNUM>667800007
<MEMO>PAYPAL
</STMTTRN>

<STMTTRN>
<TRNTYPE>CREDIT
<DTPOSTED>20151130000001[-3:GMT]
<TRNAMT>-6.00
<FITID>667800008
<CHECKNUM>667800008
<MEMO>ENAPARK
</STMTTRN>
<STMTTRN>
<TRNTYPE>DEBIT
<DTPOSTED>20151127000001[-3:GMT]
<TRNAMT>8.77
<FITID>667800007
<CHECKNUM>667800007
<MEMO>PAYPAL
</STMTTRN>

<STMTTRN>
<TRNTYPE>CREDIT
<DTPOSTED>20151130000001[-3:GMT]
<TRNAMT>-6.00
<FITID>667800008
<CHECKNUM>667800008
<MEMO>ENAPARK
</STMTTRN>

Last edited by Aia; 01-23-2016 at 11:58 PM..
# 5  
Old 01-24-2016
Thanks a lot guys.

Aia, I ran the Perl .pl file in Terminal and it works very well combining both requirements in one script like you said. I changed the file name of course, like this:

Code:
perl filter.pl cc3.ofx

However, when I added the variable to write the changes in the file (-i) the new file is missing the last 15 lines, and just those exact lines are printed in the terminal window. I ran the command like this:
Code:
perl -i.bak filter.pl cc3.ofx

What is wrong?
# 6  
Old 01-24-2016
Simple awk? Try
Code:
awk '
NR == 1         {Above++
                 Below++
                }

                {RB[NR%Above] = $0
                }

$0 ~ Pat        {Above = NR + Below
                 B = 1
                }

NR >= Above     {print B?$0:RB[(NR+1)%Above]
                }

' Above=3 Below=5 Pat="BALANCE" file
 1 <STMTTRN>
 2 <TRNTYPE>CREDIT
 3 <DTPOSTED>20151205000001[-3:GMT]
13 <FITID>667800002
14 <CHECKNUM>667800002
15 <MEMO>PAY
16 </STMTTRN>

# 7  
Old 01-24-2016
Quote:
Originally Posted by bomsom
Thanks a lot guys.

Aia, I ran the Perl .pl file in Terminal and it works very well combining both requirements in one script like you said. I changed the file name of course, like this:

Code:
perl filter.pl cc3.ofx

However, when I added the variable to write the changes in the file (-i) the new file is missing the last 15 lines, and just those exact lines are printed in the terminal window. I ran the command like this:
Code:
perl -i.bak filter.pl cc3.ofx

What is wrong?
The flag -i works only with the <> operator. Mostly, in throw away command line executions. I am using the <> in the script trying to be flexible from where it is getting the input (from stdin, and from file given). However, it is intended to show at stdout and thus it closes $ARGV (where <> is reading) and then it finishes the job.

If you are going to use it as such, a modification must be done. The highlighted part represents the modification. It has been tested, minimally. Please, try and let me know if you need something else.

Code:
#!/usr/bin/env perl
use strict;
use warnings;

my @buffer;

while(<>) {
    push @buffer, $_;
    if(@buffer == 16) {
        if($buffer[6] =~ /<MEMO>BALANCE/) {
            @buffer = ();
            next;
        }
        print reverse_sign(shift @buffer);
    }
    if(eof) {
        print reverse_sign(@buffer) if @buffer;
    }
}

sub reverse_sign {
    my $lines = \@_;
    for my $line (@{$lines}) {
        $line =~ s/^(<TRNAMT>)(-?\d+\.\d+)$/sprintf "%s%.2f", $1, -$2/e;
    }
    return @{$lines};
}

This User Gave Thanks to Aia For This Post:
bomsom
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Printing multiple lines on the same line between specific text

This is an extract from a large file. The lines that start with fc are ports on a fabric switch. In between each fc port there is information about the port. fc2/12 is up Port description is SEIEDISCOVER-3 Speed is 4 Gbps fc2/13 is down (Administratively down) fc2/14 is up Port... (1 Reply)
Discussion started by: kieranfoley
1 Replies

2. Shell Programming and Scripting

Delete lines that contain a pattern from specific line to the end.

Gents, I am trying to delete all lines which start with "H" character, but keeping the fist header. Example In the input file I will delete all lines starting from line 8 which contents character "H" to the end of the file. I try sed '8,10000{/^H/d;}' file But as don't know the end... (1 Reply)
Discussion started by: jiam912
1 Replies

3. UNIX for Dummies Questions & Answers

How to delete specific lines (2n+3 line, n=0,1,2...296) in a file?

Dear everyone, I have a file with 900 lines (there is only numbers in one line, no string), I only need the lines 2+3n (n=0,1...296), i.e line 2, 5, 8, 11...888. I tried google but only the results such as how to delete all the odd lines or all the even lines with 'awk' command. Thanks in... (4 Replies)
Discussion started by: phamnu
4 Replies

4. Shell Programming and Scripting

A way to delete specific lines

Hi all, I hope an expert tells me that there is a way to get ride of the certain lines in my files which have no specific string on them but fixed length. It's the original pattern:... (9 Replies)
Discussion started by: GoldenFire
9 Replies

5. Shell Programming and Scripting

Delete all lines after a specific line ?

Hello. My file is like this: a b c d e f g h i I want to delete all lines after the 3rd line, means after the "c". Is there any way to do this? The lines differ between them and the lines I want to delete does not have a specific word, or the lines I want to keep (a,b,c) does not have a... (4 Replies)
Discussion started by: hakermania
4 Replies

6. Shell Programming and Scripting

How to delete all lines before a specific word?

Let's say we have a file containing: alllllsadfsdasdf qwdDDDaassss ccxxcxc#2222 dssSSSSddDDDD D1Sqn2NYOHgTI Hello Alex ssS@3 Ok, and let's say we want to delete all words from D1Sqn2NYOHgTI and back, this means to delete the words (and the lines of them) : alllllsadfsdasdf... (2 Replies)
Discussion started by: hakermania
2 Replies

7. Shell Programming and Scripting

[bash help]Adding multiple lines of text into a specific spot into a text file

I am attempting to insert multiple lines of text into a specific place in a text file based on the lines above or below it. For example, Here is a portion of a zone file. IN NS ns1.domain.tld. IN NS ns2.domain.tld. IN ... (2 Replies)
Discussion started by: cdn_humbucker
2 Replies

8. Shell Programming and Scripting

extract the lines between specific line number from a text file

Hi I want to extract certain text between two line numbers like 23234234324 and 54446655567567 How do I do this with a simple sed or awk command? Thank you. ---------- Post updated at 06:16 PM ---------- Previous update was at 05:55 PM ---------- found it: sed -n '#1,#2p'... (1 Reply)
Discussion started by: return_user
1 Replies

9. Shell Programming and Scripting

Delete specific lines from a file

Hi, I have a file ( all_users.ldif ) of the following format: cn=orcladmin, cn=Users, dc=maximus,dc=com cn=PUBLIC, cn=Users, dc=maximus,dc=com cn=portal,cn=users,dc=maximus,dc=com cn=portal_admin,cn=users,dc=maximus,dc=com cn=uddi_publisher,cn=Users,dc=maximus,dc=com... (4 Replies)
Discussion started by: itzz.me
4 Replies

10. Programming

Delete specific lines in a text file

Hi, experts, I would like to create a function that can calculate the total number of lines in a saved text file and delete specific lines in that particular file (I only want the last few lines). Hav anybody have the experience and giv me a hand in this? (9 Replies)
Discussion started by: dniz
9 Replies
Login or Register to Ask a Question