Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

sed pattern to delete lines containing a pattern, except the first occurance

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting sed pattern to delete lines containing a pattern, except the first occurance
# 1  
Old 06-23-2011
sed pattern to delete lines containing a pattern, except the first occurance
Hello sed gurus. I am using ksh on Sun and have a file created by concatenating several other files. All files contain header rows. I just need to keep the first occurrence and remove all other header rows.

Code:
header for file
1111
2222
3333
header for file
1111
2222
3333
header for file
1111
2222
3333

Should look like:
Code:
header for file
1111
2222
3333
1111
2222
3333
1111
2222
3333

I know how to remove lines matching a pattern, but I am brain-locking on the pattern needed to skip the first occurrence.

Removes lines starting with "header"
Code:
sed '/^header/d' filename

Thanks for any help!
Gary
# 2  
Old 06-23-2011
an awk one:

Code:
 
awk '/header/&&c++>0 {next} 1' input_file

This User Gave Thanks to panyam For This Post:
gary_w
# 3  
Old 06-23-2011
Code:
sed '1!{/^header/d;}' inputfile

or

Code:
sed '2,${/^header/d;}' inputfile

This User Gave Thanks to ctsgnb For This Post:
gary_w
# 4  
Old 06-23-2011
Code:
awk ' /^header/ && FNR > 1 {next} {print $0} ' infile > newfile

This User Gave Thanks to jim mcnamara For This Post:
gary_w
# 5  
Old 06-23-2011
Thanks ctsgnb I could net get that syntax for specifying the range to operate on.
Thanks to all for the different methods!
Gary
# 6  
Old 10-06-2011
Quote:
Originally Posted by panyam
an awk one:

Code:
 
awk '/header/&&c++>0 {next} 1' input_file

Is there a way to do this for multiple regex's? I have got part of it with sed:

Code:
sed '/pattern1/d;/pattern2/d;/pattern3/d;/pattern4/d' file > filetest

However, I didn't figure out how to skip the first instance of each pattern...

This one seems to double the data in the file...:

Code:
 awk '/pattern1/&&c++>0 {next} 1;/pattern2/&&c++>0 {next} 1' file > testfile

Smilie
Last edited by herot; 10-06-2011 at 02:23 PM..
# 7  
Old 10-06-2011
Quote:
Originally Posted by panyam
an awk one:

Code:
 
awk '/header/&&c++>0 {next} 1' input_file

awk '/header/&&c++ {next} 1'
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

[sed] Finding and sticking the pattern to the beginning of successive lines up to the next pattern

I have a file like below. 2018.07.01, Sunday 09:27 some text 123456789 0 21 0.06 0.07 0.00 2018.07.02, Monday 09:31 some text 123456789 1 41 0.26 0.32 0.00 09:39 some text 456789012 1 0.07 0.09 0.09 09:45 some text 932469494 1 55 0.29 0.36 0.00 16:49 some text 123456789 0 48 0.12 0.15 0.00... (9 Replies)
Discussion started by: father_7
9 Replies

2. Shell Programming and Scripting

How to delete all lines before a particular pattern when the pattern is defined in a variable?

I have a file Line 1 a Line 22 Line 33 Line 1 b Line 22 Line 1 c Line 4 Line 5 I want to delete all lines before last occurrence of a line which contains something which is defined in a variable. Say a variable var contains 'Line 1', then I need the following in the output. ... (21 Replies)
Discussion started by: Soham
21 Replies

3. Shell Programming and Scripting

sed -- Find pattern -- print remainder -- plus lines up to pattern -- Minus pattern

The intended result should be : PDF converters 'empty line' gpdftext and pdftotext?xml version="1.0"?> xml:space="preserve"><note-content version="0.1" xmlns:/tomboy/link" xmlns:size="http://beatniksoftware.com/tomboy/size">PDF converters gpdftext and pdftotext</note-content>... (9 Replies)
Discussion started by: Klasform
9 Replies

4. Shell Programming and Scripting

sed search pattern and delete lines

Hello, i have a question. My problem is that i have a file like: TEST JOHN ADAM MICHAEL SEBASTIAN ANDY i want find for MICHAEL and want delete lines like this: TEST (4 Replies)
Discussion started by: eightball
4 Replies

5. Shell Programming and Scripting

Sed delete blank lines upto first pattern match

Hi Im trying to do the following in sed. I want to delete any blank line at the start of a file until it matches a pattern and then stops. for example: Input output: I have got it to work within a range of two patterns with the following: sed '/1/,/pattern/{/^]*$/d}' The... (2 Replies)
Discussion started by: duonut
2 Replies

6. Shell Programming and Scripting

sed/awk : how to delete lines based on IP pattern ?

Hi, I would like to delete lines in /etc/hosts on few workstations, basically I want to delete all the lines for a list of machines like this : for HOST in $(cat stations.lst |uniq) do # echo -n "$HOST" if ping -c 1 $HOST > /dev/null 2>&1 then HOSTNAME_val=`rsh $HOST "sed... (3 Replies)
Discussion started by: albator1932
3 Replies

7. Shell Programming and Scripting

copy, then delete lines in file with sed using a pattern

I need to copy lines to a new file from files with sed using a pattern in char postions 1-3. Then after the copy, I need to delete those same lines from the input files. For example, string "ABC" in pos 1-3 (6 Replies)
Discussion started by: laksjfhoius9123
6 Replies

8. Shell Programming and Scripting

Need 10 lines before the first occurance of a pattern

Here is the text file: 1This is a text file 2this is a text file 3This is a text file 4this is a text file 5This is a text file 6this is a text file 7This is a text file 8this is a text file 9This is a text file 10this is a text file 11This is a text file 12this is a text file 13This... (4 Replies)
Discussion started by: Johny001
4 Replies

9. Shell Programming and Scripting

sed delete pattern skipping first n lines of file.

I have files of more than 10K lines that I need to delete lines that contain a pattern, but I want to keep the first few lines intact. Can this be done with sed? (7 Replies)
Discussion started by: tkg
7 Replies

10. Shell Programming and Scripting

SED: match pattern & delete matched lines

Hi all, I have the following data in a file x.csv: > ,this is some text here > ,,,,,,,,,,,,,,,,2006/11/16,0.23 > ,,,,,,,,,,,,,,,,2006/12/16,0.88 < ,,,,,,,,,,,,,,,,this shouldnt be deleted I need to use SED to match anything with a > in the line and delete that line, can someone help... (7 Replies)
Discussion started by: not4google
7 Replies
Login or Register to Ask a Question