Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Select everything between first and last occurrence of same pattern

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Select everything between first and last occurrence of same pattern
# 15  
Old 06-21-2011
Quote:
Originally Posted by usha rao
Code:
 
nawk 'NR==FNR{if(/xyz/){last=NR;if(!first)first=NR};next} FNR>=first && FNR<=last' test.txt test.txt

Here nawk is doing two iteration:
the first to discover where the first and last occurrence are, the second to actually print the lines in between.

Can anyone help me with the sed substitute for selecting everything between the first and last occurrence of a pattern in just one iteration.

Thanks in advance.
Usha

The following sed script should do the same job as that nawk but in one pass. It will ignore any lines before the first occurence of the pattern. Beginning with the first occurrence of pattern inclusive, append the line in the pattern space to the hold space. When a matching line is found, we output the accumulated chunk and clear the hold space. Finally, the next line in the file is read into the pattern space before returning to the top of the loop:
Code:
#n

/xyz/! d

:top
H
/xyz/ {
    s/.*//
    x
    s/\n//
    p
}
n
b top

If that is saved in a file named first-last.sed, you can invoke it thusly: sed -f first-last.sed test.txt

If for some reason you'd rather not store the script in a file, but prefer to have it inline:
Code:
sed -n '/xyz/!d; :top
H; /xyz/ {s/.*//; x; s/\n//; p;}; n; b top' test.txt

To ensure maximum portability, this sed script uses only POSIX-compliant syntax (e.g. labels are terminated by a newline).

Regards,
Alister
Last edited by alister; 06-21-2011 at 12:52 PM..
# 16  
Old 06-21-2011
Hello Alister,

For some reason the code you provided is not working. I would like to know ( the final sed one liner which will work ) the sed version too.


Code:
 
sed -n '/xyz/!d; :top
> H; /xyz/ {s/.*//; x; s/\n//; p;}; n; b top'  rem
xyz
xyz
xyz

# 17  
Old 06-21-2011
Quote:
Originally Posted by panyam
Hello Alister,

For some reason the code you provided is not working. I would like to know ( the final sed one liner which will work ) the sed version too.


Code:
 
sed -n '/xyz/!d; :top
> H; /xyz/ {s/.*//; x; s/\n//; p;}; n; b top'  rem
xyz
xyz
xyz


What platform are you using? Also, please provide the entire input data file.


---------- Post updated at 11:18 AM ---------- Previous update was at 11:12 AM ----------

I tested on old versions of GNU/Linux, OpenBSD, and OS X. It worked fine on all of them.

Regards,
Alister
# 18  
Old 06-21-2011
Code:
/home/ravi> uname -a
HP-UX avalon B.11.11 U 9000/800 3547052374 unlimited-user license
 
/home/ravi>cat rem
1
xyz
2
3
4
5
6
xyz
8
9
10
xyz
11
12
13

/home/ravi>sed -n '/xyz/!d; :top
> H; /xyz/ {s/.*//; x; s/\n//; p;}; n; b top' rem
xyz
xyz
xyz

Regards
Ravi
# 19  
Old 06-21-2011
Quote:
Originally Posted by panyam
Code:
/home/ravi>sed -n '/xyz/!d; :top
> H; /xyz/ {s/.*//; x; s/\n//; p;}; n; b top' rem
xyz
xyz
xyz

Regards
Ravi
Looks like it's not looping properly. Try the following:
Code:
sed -n '/xyz/!d; :top
H; /xyz/ {s/.*//; x; s/\n//; p;}; n;
b top
' rem

If that doesn't work, I'd be curious to know if the script file version works (using the code that begins with #n in a file and executing it with sed's -f option).

Regards,
Alister
# 20  
Old 06-21-2011
Quote:
Originally Posted by yazu
Code:
sed -n '/one/,/one/p'

this code will not give the wanted output since it will just toggle between on and off as soon as it find an occurence of the pattern and may skip some wanted lines.

Code:
# echo "one
> bla
> one
> bla
> bla
> one" | sed -n '/one/,/one/p'
one
bla
one
one
#

---------- Post updated at 07:37 PM ---------- Previous update was at 07:16 PM ----------

... yet another awk:

Code:
awk '{A[NR]=$0}/one/{m=x?m:NR;x=NR}END{while(m<=x)print A[m++]}' input

Last edited by ctsgnb; 06-22-2011 at 08:02 AM.. Reason: Awk code fixed (error of logic)
# 21  
Old 06-21-2011
Quote:
Originally Posted by ctsgnb
... yet another awk:

Code:
awk '/one/{f=1;x=NR}f{A[++c]=$0}END{while(++d<x)print A[d]}' input

@ctsgnb,

The command also prints the line after the last regex.
Shouldn't it be?
Code:
awk '/one/{f=1;x=NR}f{A[++c]=$0}END{while(++d<x-1)print A[d]}' input

Another approach:
Code:
awk '/xyz/{if(s){print s;s=x}print $0;f=1;next}f{s=s?s RS $0:$0}' file

These 2 Users Gave Thanks to Franklin52 For This Post:
ctsgnb usha rao
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to extract and print first occurrence of pattern in each line

I am trying to use awk to extract and print the first ocurrence of NM_ and NP_ with a : before in each line. The input file is tab-delimeted, but the output does not need to be. The below does execute but prints all the lines in the file not just the patterns. Thank you :). file tab-delimeted ... (2 Replies)
Discussion started by: cmccabe
2 Replies

2. Shell Programming and Scripting

awk command to get file content until 2 occurrence of pattern match

AWK command to get file content until 3 occurrence of pattern match, INPUT FILE: JMS_BODY_FIELD:JMSText = <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <custOptIn xmlns="http://com/walm/ta/cu/ccs/xml2"> <person>Romi</person> <appName>SAP</appName> </custOptIn> ... (4 Replies)
Discussion started by: prince1987
4 Replies

3. Shell Programming and Scripting

UNIX help to print 50 lines after every 3rd occurrence pattern till end of file

I need help with extract/print lines till stop pattern. This needs to happen after every 3rd occurrence of start pattern and continue till end of file. Consider below is an example of the log file. my start pattern will be every 3rd occurrence of ERROR_FILE_NOT_FOUND and stop pattern will be... (5 Replies)
Discussion started by: NSS
5 Replies

4. Shell Programming and Scripting

Match from one pattern to second occurrence of second pattern

Given an XML file that contains (NOT "consists of"): </dict> <key>system.</key> <dict> <key>rule</key> <string>default</string> </dict> <key>system.burn</key> ... (9 Replies)
Discussion started by: jnojr
9 Replies

5. Shell Programming and Scripting

Changing the first occurrence after matching a pattern

Hi, I got a file which looks like this: Value A Status: - Other: - Value B Status: - Other: - Value C Status: - Other: - I would like to change only the first line which includes the "Status:" string after matching the line containing "Value B", so the output file should look... (5 Replies)
Discussion started by: wenclu
5 Replies

6. Shell Programming and Scripting

Insert new pattern in newline after the nth occurrence of a line pattern - Bash in Ubuntu 12.04

Hi, I am getting crazy after days on looking at it: Bash in Ubuntu 12.04.1 I want to do this: pattern="system /path1/file1 file1" new_pattern=" data /path2/file2 file2" file to edit: data.db - I need to search in the file data.db for the nth occurrence of pattern - pattern must... (14 Replies)
Discussion started by: Phil3759
14 Replies

7. Shell Programming and Scripting

print only the first occurrence of a pattern

Hi, I have a file as below select or create proc /*comments*/ /*comments*/ /*comments*/ /*comments*/ ( variables4 datatypes1, variables1 datatypes2, variables2 datatypes3, variables3 datatypes2 ) some text some text ( sometext some text ) some text some text (3 Replies)
Discussion started by: manasa_vs
3 Replies

8. UNIX for Advanced & Expert Users

Unix: list out Pattern occurrence (count)

Need to search a pattern occurrence (count) in a specified file. Below is the details $ cat fruits apple apple ball ball apple ball ball ball apple apple apple cat cat cat cat cat apple apple Note: If I'll use the grep command with -c option then it'll count the 1st occurrence in... (6 Replies)
Discussion started by: tp2115
6 Replies

9. UNIX for Dummies Questions & Answers

line number of the i-th occurrence of a pattern

Hi all, is there a simple way to obtain the line number of the i-th occurrence of a pattern? I have OCCURRENCE=`grep -io "${STRING_NAME}" ${1}-${8}${EXT}.out_bis| wc -l` which tells me how many occurency I have. I would like to go through them and determine the line number and assign... (6 Replies)
Discussion started by: f_o_555
6 Replies

10. Shell Programming and Scripting

Count the number of occurrences of a pattern between each occurrence of a different pattern

I need to count the number of occurrences of a pattern, say 'key', between each occurrence of a different pattern, say 'lu'. Here's a portion of the text I'm trying to parse: lu S1234L_149_m1_vg.6, part-att 1, vdp-att 1 p-reserver IID 0xdb registrations: key 4156 4353 0000 0000 ... (3 Replies)
Discussion started by: slipstream
3 Replies
Login or Register to Ask a Question