sed pattern to delete lines containing a pattern, except the first occurance


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting sed pattern to delete lines containing a pattern, except the first occurance
# 1  
Old 06-23-2011
sed pattern to delete lines containing a pattern, except the first occurance

Hello sed gurus. I am using ksh on Sun and have a file created by concatenating several other files. All files contain header rows. I just need to keep the first occurrence and remove all other header rows.

Code:
header for file
1111
2222
3333
header for file
1111
2222
3333
header for file
1111
2222
3333

Should look like:
Code:
header for file
1111
2222
3333
1111
2222
3333
1111
2222
3333

I know how to remove lines matching a pattern, but I am brain-locking on the pattern needed to skip the first occurrence.

Removes lines starting with "header"
Code:
sed '/^header/d' filename

Thanks for any help!
Gary
# 2  
Old 06-23-2011
an awk one:

Code:
 
awk '/header/&&c++>0 {next} 1' input_file

This User Gave Thanks to panyam For This Post:
# 3  
Old 06-23-2011
Code:
sed '1!{/^header/d;}' inputfile

or

Code:
sed '2,${/^header/d;}' inputfile

This User Gave Thanks to ctsgnb For This Post:
# 4  
Old 06-23-2011
Code:
awk ' /^header/ && FNR > 1 {next} {print $0} ' infile > newfile

This User Gave Thanks to jim mcnamara For This Post:
# 5  
Old 06-23-2011
Thanks ctsgnb I could net get that syntax for specifying the range to operate on.
Thanks to all for the different methods!
Gary
# 6  
Old 10-06-2011
Quote:
Originally Posted by panyam
an awk one:

Code:
 
awk '/header/&&c++>0 {next} 1' input_file

Is there a way to do this for multiple regex's? I have got part of it with sed:

Code:
sed '/pattern1/d;/pattern2/d;/pattern3/d;/pattern4/d' file > filetest

However, I didn't figure out how to skip the first instance of each pattern...

This one seems to double the data in the file...:

Code:
 awk '/pattern1/&&c++>0 {next} 1;/pattern2/&&c++>0 {next} 1' file > testfile

Smilie

Last edited by herot; 10-06-2011 at 02:23 PM..
# 7  
Old 10-06-2011
Quote:
Originally Posted by panyam
an awk one:

Code:
 
awk '/header/&&c++>0 {next} 1' input_file

awk '/header/&&c++ {next} 1'
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

[sed] Finding and sticking the pattern to the beginning of successive lines up to the next pattern

I have a file like below. 2018.07.01, Sunday 09:27 some text 123456789 0 21 0.06 0.07 0.00 2018.07.02, Monday 09:31 some text 123456789 1 41 0.26 0.32 0.00 09:39 some text 456789012 1 0.07 0.09 0.09 09:45 some text 932469494 1 55 0.29 0.36 0.00 16:49 some text 123456789 0 48 0.12 0.15 0.00... (9 Replies)
Discussion started by: father_7
9 Replies

2. Shell Programming and Scripting

How to delete all lines before a particular pattern when the pattern is defined in a variable?

I have a file Line 1 a Line 22 Line 33 Line 1 b Line 22 Line 1 c Line 4 Line 5 I want to delete all lines before last occurrence of a line which contains something which is defined in a variable. Say a variable var contains 'Line 1', then I need the following in the output. ... (21 Replies)
Discussion started by: Soham
21 Replies

3. Shell Programming and Scripting

sed -- Find pattern -- print remainder -- plus lines up to pattern -- Minus pattern

The intended result should be : PDF converters 'empty line' gpdftext and pdftotext?xml version="1.0"?> xml:space="preserve"><note-content version="0.1" xmlns:/tomboy/link" xmlns:size="http://beatniksoftware.com/tomboy/size">PDF converters gpdftext and pdftotext</note-content>... (9 Replies)
Discussion started by: Klasform
9 Replies

4. Shell Programming and Scripting

sed search pattern and delete lines

Hello, i have a question. My problem is that i have a file like: TEST JOHN ADAM MICHAEL SEBASTIAN ANDY i want find for MICHAEL and want delete lines like this: TEST (4 Replies)
Discussion started by: eightball
4 Replies

5. Shell Programming and Scripting

Sed delete blank lines upto first pattern match

Hi Im trying to do the following in sed. I want to delete any blank line at the start of a file until it matches a pattern and then stops. for example: Input output: I have got it to work within a range of two patterns with the following: sed '/1/,/pattern/{/^]*$/d}' The... (2 Replies)
Discussion started by: duonut
2 Replies

6. Shell Programming and Scripting

sed/awk : how to delete lines based on IP pattern ?

Hi, I would like to delete lines in /etc/hosts on few workstations, basically I want to delete all the lines for a list of machines like this : for HOST in $(cat stations.lst |uniq) do # echo -n "$HOST" if ping -c 1 $HOST > /dev/null 2>&1 then HOSTNAME_val=`rsh $HOST "sed... (3 Replies)
Discussion started by: albator1932
3 Replies

7. Shell Programming and Scripting

copy, then delete lines in file with sed using a pattern

I need to copy lines to a new file from files with sed using a pattern in char postions 1-3. Then after the copy, I need to delete those same lines from the input files. For example, string "ABC" in pos 1-3 (6 Replies)
Discussion started by: laksjfhoius9123
6 Replies

8. Shell Programming and Scripting

Need 10 lines before the first occurance of a pattern

Here is the text file: 1This is a text file 2this is a text file 3This is a text file 4this is a text file 5This is a text file 6this is a text file 7This is a text file 8this is a text file 9This is a text file 10this is a text file 11This is a text file 12this is a text file 13This... (4 Replies)
Discussion started by: Johny001
4 Replies

9. Shell Programming and Scripting

sed delete pattern skipping first n lines of file.

I have files of more than 10K lines that I need to delete lines that contain a pattern, but I want to keep the first few lines intact. Can this be done with sed? (7 Replies)
Discussion started by: tkg
7 Replies

10. Shell Programming and Scripting

SED: match pattern & delete matched lines

Hi all, I have the following data in a file x.csv: > ,this is some text here > ,,,,,,,,,,,,,,,,2006/11/16,0.23 > ,,,,,,,,,,,,,,,,2006/12/16,0.88 < ,,,,,,,,,,,,,,,,this shouldnt be deleted I need to use SED to match anything with a > in the line and delete that line, can someone help... (7 Replies)
Discussion started by: not4google
7 Replies
Login or Register to Ask a Question