Removing multiple lines from input file, if multiple lines match a pattern.


Login or Register for Dates, Times and to Reply

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Removing multiple lines from input file, if multiple lines match a pattern.
# 1  
Removing multiple lines from input file, if multiple lines match a pattern.

GM,

I have an issue at work, which requires a simple solution. But, after multiple attempts, I have not been able to hit on the code needed.

I am assuming that sed, awk or even perl could do what I need.

I have an application that adds extra blank page feeds, for multiple reports, when sending these print jobs to CUPS.

Unfortunately, it would be very time consuming to have each of the developers go back and remove these extra page feeds, through hundreds of different reports, not to mention that would require QA, Release Management, and official patches for all of our customers.

Here's what I've done already.

I am intercepting the raw print jobs when they come into CUPS, as plain text PostScript files.

I have identified the actual lines that need to be removed from each job.

If I edit the file manually, and remove the lines manually, the subsequent printout works perfectly, with the blank pages suppressed.

But, I've been trying to use normal KSH or BASH to programatically remove the offending lines.

Unfortunately, in doing so, the script removes ALL of the necessary hidden formatting / special characters.

If I edit the file by hand, and just remove these offending lines, the document prints perfectly.

If I run the PostScript file through my "filter" script, it removes all of the special characters, and the postscript data itself is sent to the printer.

Let's say hypothetically that the following 4 lines are found. These same 4 lines can occur at different places inside of the postscript job, but normally, they will have data inserted between rows.

If these same 4 lines occur, sequentially, right after each other, without any other data, then I want to remove them:

abc
def
ghi
jkl

It is important that I must not remove any special formatting or characters from any previous or future lines within the file.

Thanks in advance,

JCF
# 2  
A real although abbreviated sample would be utmost helpful.
# 3  
Not sure if I understood it exactly - did you try something like:
Code:
grep -vE "abc|def|ghi|jkl" infile
# or more precise
grep -vE "^(abc|def|ghi|jkl)$" infile

? Should the file be edited in place or is a temp file ok?
# 4  
Zaxxon reply

Hi Zaxxon,

Unfortunately, you did not understand the issue.

I am not looking for 4 different strings, each on it's own line. Grep -e, or egrep, would work fine for this.

I am searching for these 4 lines, together, when they appear back to back.

In order for the condition to be true, all 4 lines must exist, exactly as seen below.

In the pattern match, I need to search for something like this:
Code:
  sed -e s/"abc\ndef\nghi\njkl\n"/""/g

I also tried:
Code:
sed -e s/"abc\rdef\rghi\rjkl\r"/""/g

I also thought about using "tr" to delete the matching strings, but I'm still having an issue matching the 4 lines, to include their special characters. i.e. Line Feeds or Carriage Returns.

Unfortunately, either I'm using the wrong carriage return characters, or something is wrong with my systax.

Basically, each time the above 4 lines occur, back to back, on separate lines, I need to remove all 4 lines.

There will be times when the 4 lines will appear, where they have some other entries in the middle, such as:

Code:
     abc
          e2c422 a12652 
     def
     ghi
     jkl

Note that if there are ANY characters or data of any kind between, or in the middle of the 4 line pattern, those are valid data lines, and must not be removed.

Only when the 4 lines appear, back to back, with NOTHING else between them, or appended to them, do they need to be removed.

I hope this helps to clarify the issue.

JCF

Moderator's Comments:
Mod Comment Use code tags, thanks.

Last edited by zaxxon; 09-28-2015 at 12:04 PM.. Reason: code tags
# 5  
Try
Code:
awk '
$1 == "abc"     {getline L1
                 getline L2
                 getline L3
                 if (L1!="def" ||
                     L2!="ghi" ||
                     L3!="jkl") {print
                                 print L1
                                 print L2
                                 print L3
                                }
                 next
                }
1
' file

This User Gave Thanks to RudiC For This Post:
# 6  
Try:
Code:
SEARCH="abc
def
ghi
jkl"
sed s,"$SEARCH","",g

hth
This User Gave Thanks to sea For This Post:
# 7  
Thank you for the potential solutions.

Unfortunately, this has been a very hectic morning so far (Mondays), and I have not had a chance to test either of the proposed solutions yet.

I promise, I will try to test with both solutions very, very soon.

Thanks again for your help and input.

JCF
Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #17
Difficulty: Easy
Microsoft first developed the Java programming language.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing carriage returns from multiple lines in multiple files of different number of columns

Hello Gurus, I have a multiple pipe separated files which have records going over multiple Lines. End of line separator is \n and records going over multiple lines have <CR> as separator. below is example from one file. 1|ABC DEF|100|10 2|PQ RS T|200|20 3| UVWXYZ|300|30 4| GHIJKL|400|40... (7 Replies)
Discussion started by: dJHa
7 Replies

2. Shell Programming and Scripting

Remove multiple lines that match pattern

Not sure how I can accomplish this. I would like to remove all interfaces that have the commands I would like to see: switchport port-security, spanning-tree portfast. One line is no problem. interface FastEthernet0/8 spanning-tree portfast interface FastEthernet0/9 spanning-tree... (4 Replies)
Discussion started by: mrlayance
4 Replies

3. Shell Programming and Scripting

Match Pattern and print pattern and multiple lines into one line

Hello Experts , require help . See below output: File inputs ------------------------------------------ Server Host = mike id rl images allocated last updated density vimages expiration last read <------- STATUS ------->... (4 Replies)
Discussion started by: tigerhills
4 Replies

4. Shell Programming and Scripting

Extract a pattern from multiple lines in a file

I have a file that has some lines starts with * I want to get these lines, then get the word between "diac" and "lex". ex. file: ;;WORD AlAx *0.942490 diac:Al>ax lex:>ax_1 bw:Al/DET+>ax/NOUN+ gloss:brother pos:noun prc3:0 prc2:0 prc1:0 prc0:Al_det per:na asp:na vox:na mod:na gen:m num:s... (4 Replies)
Discussion started by: Viernes
4 Replies

5. Shell Programming and Scripting

Awk match multiple columns in multiple lines in single file

Hi, Input 7488 7389 chr1.fa chr1.fa 3546 9887 chr5.fa chr9.fa 7387 7898 chrX.fa chr3.fa 7488 7389 chr21.fa chr3.fa 7488 7389 chr1.fa chr1.fa 3546 9887 chr9.fa chr5.fa 7898 7387 chrX.fa chr3.fa Desired Output 7488 7389 chr1.fa chr1.fa 2 3546 9887 chr5.fa chr9.fa 2... (2 Replies)
Discussion started by: jacobs.smith
2 Replies

6. Shell Programming and Scripting

shell script: grep multiple lines after pattern match

I have sql file containing lot of queries on different database table. I have to filter specific table queries. Let say i need all queries of test1,test2,test3 along with four lines above it and sql queries can be multi lines or in single line. Input file contains. set INSERT_ID=1; set... (1 Reply)
Discussion started by: mirfan
1 Replies

7. Shell Programming and Scripting

Perl: Printing Multiple Lines after pattern match

Hello People, Need some assistance/guidance. OUTLINE: Two files (File1 and File2) File1 has some ids such as 009463_3922_1827 897654_8764_5432 File2 has things along the lines of: Query= 009463_3922_1827 length=252 (252 letters) More stufff here ... (5 Replies)
Discussion started by: Deep9000
5 Replies

8. UNIX for Dummies Questions & Answers

removing multiple lines of text in a file

Hi, I'm trying to remove multiple lines of text based off a series of different words and output it to a new file The document contains a ton of data but i want to delete any line that has the following mx1.rr.biz.com or ns2.ri.biz.com i tried using grep -v filename "mx1.rr.biz.com" >... (3 Replies)
Discussion started by: spartan22
3 Replies

9. Shell Programming and Scripting

removing pattern which is spread in multiple lines

I have several huge files wich contains oracle table creation scripts as follows: I would need to remove the pattern colored in red above. Any sed/awk/pearl code will be of much help. Thanks (2 Replies)
Discussion started by: sabyasm
2 Replies

10. Shell Programming and Scripting

Concatenating multiple lines to one line if match pattern

Hi all, I've been working on a script which I have hit a road block now. I have written a script using sed to extract the below data and pumped into another file: Severity............: MAJORWARNING Summary: System temperature is out of normal range. Severity............: MAJORWARNING... (13 Replies)
Discussion started by: phixsius
13 Replies

Featured Tech Videos