Remove a range of lines from a file using sed


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Remove a range of lines from a file using sed
# 1  
Old 01-19-2012
Remove a range of lines from a file using sed

Hi

I am having some issue editing a file in sed.

What I want to do is, in a loop pass a variable to a sed command. Sed should then search a file for a line that matches that variable, then remove all lines below until it reaches a line starting with a constant.

I have managed to write a sed command that will give me the block of text, but I cannot figure out to return the contents of the excluding the text I have searched for.

Code I have so far is.
Code:
sed -n '/'"${var[$count]}"'/,/'"Con       ST"'q/p'  < input | head -5> output

Moderator's Comments:
Mod Comment Please use next time code tags for your code and data

Last edited by vbe; 01-19-2012 at 12:21 PM..
# 2  
Old 01-19-2012
You have the logic backwards because instead of printing out the unwanted lines just delete them...
Code:
sed '/'"${var[$count]}"'/,/'"Con       ST"'q/d' file

# 3  
Old 01-19-2012
Running sed many times to process one file is extremely wasteful and slow. What you want to do, if I understand your intention, can be done with one single execution of awk.

Code:
# Array of lines to skip, initialized in the BASH way
ARRAY=( 3 4 5 )
# Feed the array into awk in the X variable.
# Then split it into the array L[1]=3, L[2]=4, L[3]=5
# Then turn that into the array S[3]=1, S[4]=1, S[5]=1.
# Then we can use the simple statement !S[NR] to check if the
# line should be skipped or not and print it accordingly.
awk -v X="${ARRAY[*]}" 'BEGIN { split(X,L," "); for(Z in L) S[L[Z]]=1; }; !S[NR]' filename

# 4  
Old 01-25-2012
Thanks for the feedback. The command is nearly working. I am still having an issue getting sed to stop searching when it finds the second string in the range.

What it should do is.
1. Search the file for the first string in the range. This string will have a unique value for each occurrence in the file. So I am using a variable.
2. Take the lines from the file under the above string until it gets to the second occurrence of the second string in the range. The second string is no unique in the file. This is why I want sed to stop searching when it finds it for the second time after it has found the first string.

File looks like this

1st String. Remove 1
Data
Data
2nd String
2nd String
1st String. Do not remove
Data
Data
2nd String
2nd String
1st String Remove 2
Data
Data
2nd String
2nd String

When I am finished I would want the file to look like

1st String
Data
Data
2nd String
2nd String

When the command runs it removes everything from the file under 1st String. Remove 1

Is this even possible to do in sed? Or should I be looking into using awk. I used sed as I have no experience in Awk and had some in sed.
The code that I am using is as in the second post above

Code:
 sed '/'"${var[$count]}"'/,/'"Con       ST"'q/d' file

# 5  
Old 01-25-2012
I'd still suggest awk, since it allows you to do rational if/then/else statements in code blocks instead of being completely restricted to inscrutiable line-matching regular expressions. It has those, too, but you get to do what you want with them Smilie

But your outline here is still incomplete -- we still have no idea what the contents of your array are, hence, what relation your output has to your input..
# 6  
Old 01-25-2012
In the example above the array would have 2 elements.
1st = Remove 1
2nd = Remove 2

These values come from another file that identifies the String 1 records that need to be removed. So on the separate file there would be no line that is equal to "Do not remove"

The array values are read using a loop.

So on the first pass through of the file the block of text
1st String. Remove 1
Data
Data
2nd String
2nd String
Should be removed. On the second pass the block of text
1st String Remove 2
Data
Data
2nd String
2nd String
Should be removed.


Leaving the block of text below untouched.
1st String. Do not remove
Data
Data
2nd String
2nd String
# 7  
Old 01-25-2012
corona688 can you look at my post as well


https://www.unix.com/shell-programmin...ther-file.html

THanks
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Sed/awk to delete a regex between range of lines

Hi Guys I am looking for a solution to one problem to remove parentheses in a range of lines. Input file module bist_logic_inst(a, ab , dhd, dhdh , djdj, hdh, djjd, jdj, dhd, dhp, dk ); input a; input ab; input dhd; input djdj; input dhd; output hdh; output djjd; output jdj;... (5 Replies)
Discussion started by: kshitij
5 Replies

2. Shell Programming and Scripting

sed or awk to remove specific column to one range

I need to remove specific column to one range source file 3 1 000123456 2 2 000123569 3 3 000123564 12 000123156 15 000125648 128 000125648 Output required 3 000123456 2 000123569 3 000123564 12 000123156 15 000125648 128 000125648 (6 Replies)
Discussion started by: ranjancom2000
6 Replies

3. Shell Programming and Scripting

How to remove repetitive lines in a file with sed?

Hello, My goal is the make all x times repeated lines into a single line. I need to attain the expected output with sed -i , I need to overwrite the MyFile MyFile: Hello World Welcome Hello World Welcome Back This is my test Expected output: Hello World Welcome Welcome Back This is... (6 Replies)
Discussion started by: baris35
6 Replies

4. Shell Programming and Scripting

sed to remove all lines in file that are not .vcf.gz extention

I am trying to use sed to remove all lines in a file that are nor vcf.gz. The sed below runs but returns all the files with vcf.gz in them, rather then just the ones that end in only that extention. Thank you :). file ... (9 Replies)
Discussion started by: cmccabe
9 Replies

5. Shell Programming and Scripting

Using sed in a loop/to remove lines contained in variable from file

I've tried numerous commands, but I am not sure how to use sed in a loop. This is what I have: VARZ contains CARD_FILE_LIST and it also contains CARD_FILE_LIST2 so echo "$VARZ" CARD_FILE_LIST CARD_FILE_LIST2 I have a file with 60 lines in /tmp/testfile it and I want those lines deleted... (3 Replies)
Discussion started by: newbie2010
3 Replies

6. Shell Programming and Scripting

sed filtering lines by range fails 1-line-ranges

The following is part of a larger project and sed is (right now) a given. I am working on a recursive Korn shell function to "peel off" XML tags from a larger text. Just for context i will show the complete function (not working right now) here: function pGetXML { typeset chTag="$1" typeset... (5 Replies)
Discussion started by: bakunin
5 Replies

7. Shell Programming and Scripting

Sed print range of lines between line number and pattern

Hi, I have a file as below This is the line one This is the line two <\XMLTAG> This is the line three This is the line four <\XMLTAG> Output of the SED command need to be as below. This is the line one This is the line two <\XMLTAG> Please do the need to needful to... (4 Replies)
Discussion started by: RMN
4 Replies

8. Shell Programming and Scripting

grep/sed to remove lines in file

Hi, I have a file with values, file1: BELL-1180-1180-81|577:1017| BELL-1180-1180-81|jm10i-auto-stub1/577:102| BELL-1180-1180-81|jm10i-auto-stub1/577:101| BELL-1180-1180-81|jm10i-auto-stub1/577:1700| BELL-1180-1180-81|jm10i-auto-stub1/577:1699| I need to remove the lines which has... (9 Replies)
Discussion started by: giri_luck
9 Replies

9. Shell Programming and Scripting

Remove strings within range using sed

Hey folks I have a big file that contains junk data between the tags <point> and </point> and I need to delete it (including `<point>' and `</point>'). i.e. a = 1 <point> 123123 2342352 234231 234256 </point> print a needs to become a = 1 print a I'm certain that this is a... (10 Replies)
Discussion started by: ksk
10 Replies

10. Shell Programming and Scripting

remove range part of a file with sed

Hello, I would like to remove a range from a file with sed or any script command that is appropriate The section start by and finish by and I would like to keep line Could you tell me which command I should type ? Thanks a lot, Franck My input file is like this... (1 Reply)
Discussion started by: mfranck
1 Replies
Login or Register to Ask a Question

Featured Tech Videos