sed - combination of line deletion and pattern matching


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers sed - combination of line deletion and pattern matching
# 1  
Old 08-05-2012
sed - combination of line deletion and pattern matching

I want to delete all the blank lines from a file before a certain line number. e.g.

Input file (n: denotes line number)
Code:
1: a
2: 
3: b
4: c
5: 
6: d

I want to delete all blank lines before line number 3, such that my output is:

Code:
a
b
c

d

I see that
Code:
 sed '/^$/d' in_file

works fine but works on the whole file.
Code:
 sed '1,3 /^$/d' 1

is incorrect. How can I achieve this?
# 2  
Old 08-05-2012
Code:
awk 'NR>3 || ! /^$/ {print}' in_file

---------- Post updated at 03:12 AM ---------- Previous update was at 03:00 AM ----------

You can also do it in sed with:
Code:
sed -n '1,3p;n
/^$/d
p' in_file

# 3  
Old 08-05-2012
Try the sed version
Code:
sed '1,3 {/^$/d}' infile

# 4  
Old 08-05-2012
Quote:
Originally Posted by jawsnnn
Code:
 sed '1,3 /^$/d' 1

is incorrect. How can I achieve this?
You got it right - almost - but missed by a hairs width.

Any command (like "s", etc.) can be preceeded by a range clause. A range clause can be either a range of lines, determined by line numbers or a starting/ending regexp, or it can be a single line number or regexp. In the second case the command would either apply to the single line number or to all lines matching the regexp:

Code:
1,3 s/x/y/
/^aa/,/^bb/ s/x/y/
1 s/x/y/
/^abc/ s/x/y/

The first line would apply "s/x/y/" only to lines 1-3, the second line would apply it to all lines from a line starting with "aa" to a line starting with "bb".

The third line would apply the command only to line number 1 and the fourth to all lines starting with "abc".

This you did correctly, but range clauses cannot be nested, and

Code:
/^$/d

is also a range clause: one, that applies the command "d" only to lines of the form "/^$/", like in my fourth example. To correct your code you simply have to correctly nest the commands:

Code:
1,3 {
     /^$/ {
          d
     }
}

or, in short form:

Code:
1,3 {;/^$/d;}


I hope this helps.

bakunin
This User Gave Thanks to bakunin For This Post:
# 5  
Old 08-05-2012
Quote:
Originally Posted by bakunin
You got it right - almost - but missed by a hairs width.

Any command (like "s", etc.) can be preceeded by a range clause. A range clause can be either a range of lines, determined by line numbers or a starting/ending regexp, or it can be a single line number or regexp. In the second case the command would either apply to the single line number or to all lines matching the regexp:

Code:
1,3 s/x/y/
/^aa/,/^bb/ s/x/y/
1 s/x/y/
/^abc/ s/x/y/

The first line would apply "s/x/y/" only to lines 1-3, the second line would apply it to all lines from a line starting with "aa" to a line starting with "bb".

The third line would apply the command only to line number 1 and the fourth to all lines starting with "abc".

This you did correctly, but range clauses cannot be nested, and

Code:
/^$/d

is also a range clause: one, that applies the command "d" only to lines of the form "/^$/", like in my fourth example. To correct your code you simply have to correctly nest the commands:

Code:
1,3 {
     /^$/ {
          d
     }
}

or, in short form:

Code:
1,3 {;/^$/d;}


I hope this helps.

bakunin
All of the sed documentation I've seen says that the "{" command format is:
Quote:
[2addr] {function
function
. . .
}
The use you're showing here should be documented as:
[QUOTE][2addr] {command
command
. . .
}[/QOTE]
because "function" doesn't include an address. I wonder if this is something that all implementation of sed do (but never documented) {and should be included in a future version of the POSIX and UNIX standards}, or if it is only provided by some implementations of sed?

Last edited by Don Cragun; 08-05-2012 at 08:42 AM..
# 6  
Old 08-05-2012
Quote:
Originally Posted by Don Cragun
All of the sed documentation I've seen says that the "{" command format is:
Code:
[2addr] {function
function
. . .
}

because "function" doesn't include an address. I wonder if this is something that all implementation of sed do (but never documented) {and should be included in a future version of the POSIX and UNIX standards}, or if it is only provided by some implementations of sed?
"function" is probably meant recursively. A function itself is a list of commands and may contain other functions as well. This is a principle in probably any programming language.

To be honest i haven't read the POSIX standard papers as intensively as you seem to have done, so i can only guess. What i have explained above is true for all the sed-variants i have ran across in my nearly 30 years of Unix, so even if my explanation is not completely POSIXly-correct it seems to cover the truth from a practical point of view.

Still, i would be interested if you could find out how it is really supposed to work as i would prefer to be sure instead of almost sure. It seems like you could shed some light on it.

I hope this helps.

bakunin
# 7  
Old 08-05-2012
Quote:
Originally Posted by bakunin
"function" is probably meant recursively. A function itself is a list of commands and may contain other functions as well. This is a principle in probably any programming language.

To be honest i haven't read the POSIX standard papers as intensively as you seem to have done, so i can only guess. What i have explained above is true for all the sed-variants i have ran across in my nearly 30 years of Unix, so even if my explanation is not completely POSIXly-correct it seems to cover the truth from a practical point of view.

Still, i would be interested if you could find out how it is really supposed to work as i would prefer to be sure instead of almost sure. It seems like you could shed some light on it.

I hope this helps.

bakunin
Check your local sed(1) man page... I bet it'll say something like:
Quote:
The form of a sed command is as follows:
[address[,address]]function[arguments]
And later give a list of sed functions and the number of addresses that need to be supplied with them to form a command.

I'll post a question to the POSIX mailing list to see if I can find an example where:
Quote:
[2addr]{command
command
...
}
doesn't work.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Sed: printing lines AFTER pattern matching EXCLUDING the line containing the pattern

'Hi I'm using the following code to extract the lines(and redirect them to a txt file) after the pattern match. But the output is inclusive of the line with pattern match. Which option is to be used to exclude the line containing the pattern? sed -n '/Conn.*User/,$p' > consumers.txt (11 Replies)
Discussion started by: essem
11 Replies

2. Shell Programming and Scripting

Replacing lines matching a multi-line pattern (sed/perl/awk)

Dear Unix Forums, I am hoping you can help me with a pattern matching problem. What am I trying to do? I want to replace multiple lines of a text file (that match a multi-line pattern) with a single line of text. These patterns can span several lines and do not always have the same number of... (10 Replies)
Discussion started by: thefang
10 Replies

3. UNIX for Dummies Questions & Answers

Sed: Adding new line after matching pattern

Hi I just wanted to add a new line after every matching pattern: The method doing this doesn't matter, however, I have been using sed and this is what I tried doing, knowing that I am a bit off: sed 'Wf a\'/n'/g' Basically, I want to add a new line after occurrence of Wf. After the line Wf... (5 Replies)
Discussion started by: MIA651
5 Replies

4. Shell Programming and Scripting

Pattern Matching and text deletion using VI

Can someone please assist me, I'm trying to get vi to remove all the occurences of the text in a file i.e. "DEVICE=/dev/mt??". The "??" represents a number variable. Is there a globel search and delete command that I can use? Thank You in Advance. (3 Replies)
Discussion started by: roadrunner
3 Replies

5. Shell Programming and Scripting

How to use sed to modify a line above or below matching pattern?

I couldn't figure out how to use sed or any other shell to do the following. Can anyone help? Thanks. If seeing a string (e.g., TODAY) in the line, replace a string in the line above (e.g, replace "Raining" with "Sunny") and replace a string in the line below (e.g., replace "Reading" with... (7 Replies)
Discussion started by: sprinner
7 Replies

6. Shell Programming and Scripting

SED Question: Search and Replace start of line to matching pattern

Hi guys, got a problem here with sed on the command line. If i have a string as below: online xx:wer:xcv: sdf:/asdf/http:https-asdfd How can i match the pattern "http:" and replace the start of the string to the pattern with null? I tried the following but it doesn't work: ... (3 Replies)
Discussion started by: DrivesMeCrazy
3 Replies

7. Shell Programming and Scripting

sed - matching pattern one but not pattern two

All, I have the following file: -------------------------------------- # # /etc/pam.d/common-password - password-related modules common to all services # # This file is included from other service-specific PAM config files, # and should contain a list of modules that define the services... (2 Replies)
Discussion started by: RobertBerrie
2 Replies

8. Shell Programming and Scripting

sed find matching pattern delete next line

trying to use sed in finding a matching pattern in a file then deleting the next line only .. pattern --> <ad-content> I tried this but it results are not what I wish sed '/<ad-content>/{N;d;}' akv.xml > akv5.xml ex, <Celebrant2First>Mickey</Celebrant2First> <ad-content> Minnie... (2 Replies)
Discussion started by: aveitas
2 Replies

9. Shell Programming and Scripting

comment/delete a particular pattern starting from second line of the matching pattern

Hi, I have file 1.txt with following entries as shown: 0152364|134444|10.20.30.40|015236433 0233654|122555|10.20.30.50|023365433 ** ** ** In file 2.txt I have the following entries as shown: 0152364|134444|10.20.30.40|015236433 0233654|122555|10.20.30.50|023365433... (4 Replies)
Discussion started by: imas
4 Replies

10. UNIX for Dummies Questions & Answers

vi part-pattern matching and deletion

Hi, I have a log file which shows the files which has been changed over the last week. They follow this pattern: old_file_version_number@@new_file_version_number Now I need to know how to delete from each line parts starting from @@. I would be issuing the command inside vi(m). So... (5 Replies)
Discussion started by: vino
5 Replies
Login or Register to Ask a Question