Unix/Linux Go Back    


Shell Programming and Scripting BSD, Linux, and UNIX shell scripting — Post awk, bash, csh, ksh, perl, php, python, sed, sh, shell scripts, and other shell scripting languages questions here.

Extract line between two patterns

Shell Programming and Scripting


Closed    
 
Thread Tools Search this Thread Display Modes
    #1  
Old Unix and Linux 12-02-2012   -   Original Discussion by rajamohan
rajamohan rajamohan is offline
Registered User
 
Join Date: Mar 2008
Last Activity: 19 February 2014, 6:43 AM EST
Posts: 27
Thanks: 3
Thanked 1 Time in 1 Post
Extract line between two patterns

Hi All,
I need a script to extract a lines between two patterns.I have done this using grep,cut,tail and head.But its very slow, because my input file contain more than a lakh.


Code:
COMMAND:XXXXXXXXXXXXXXXXXXXX
yyyyy
zzzzzz
REQUESTSTRING:aaaaaaaaaaaaaaa;11111
222222
333333
SEARCHPATTERN
44444444444444
55555555
66666

COMMAND:XXXXXXXXXXXXXXXXXXXX
yyyyy
zzzzzz
REQUESTSTRING:bbbbb SEARCHPATTERN bbbbbbbb;11111
222222
333333
44444444444444
55555555
66666

COMMAND:XXXXXXXXXXXXXXXXXXXX
yyyyy
zzzzzz
REQUESTSTRING:cccccccccccc;11111
222222
333333
SEARCHPATTERN
44444444444444
55555555
66666

COMMAND:XXXXXXXXXXXXXXXXXXXX
yyyyy
zzzzzz
REQUESTSTRING:dddddddddddd;111111
222222
333333
44444444444444
55555555
66666

From the above sample,

Extract single COMMAND and search for SEARCHPATTERN, if SEARCHPATTERN available print the REQUESTSTRING line before ";" .

Note:
The search pattern may present in separate line or in REQUESTSTRING line itself.

Expected output is:


Code:
REQUESTSTRING:aaaaaaaaaaaaaaa
REQUESTSTRING:bbbbb SEARCHPATTERN bbbbbbbb
REQUESTSTRING:ccccccccccccccc

My sample code using grep..


Code:
grep -n -e "COMMAND" input.txt > tmp.txt
startline=0
while read line
do
	if [ "$startline" = "0" ]
	then
		read line
	fi
	endline=`echo $line |cut -d ":" -f 1`
	endline=`expr $endline - 1`
#	echo "Startline:$startline,Endline:$endline"
	noofline=`expr $endline - $startline`
	head -$endline input.txt > t1.txt
	tail -$noofline t1.txt > t2.txt 
	grep "SEARCHPATTERN " t2.txt > /dev/dull
	if [ "$?" = "0" ] 
	then
		grep "REQUESTSTRING " t2.txt | cut -d ";" -f 1 
	fi
	startline=$endline
done < tmp.txt

Sponsored Links
    #2  
Old Unix and Linux 12-02-2012   -   Original Discussion by rajamohan
elixir_sinari's Unix or Linux Image
elixir_sinari elixir_sinari is offline Forum Advisor  
Registered User
 
Join Date: Mar 2012
Last Activity: 9 August 2017, 6:59 AM EDT
Location: India
Posts: 1,413
Thanks: 101
Thanked 497 Times in 474 Posts
With assumptions:

Code:
awk '/SEARCHPATTERN/ && NF==2{split($2,a,/;/);print FS a[1]}' FS='REQUESTSTRING:' RS= file

Sponsored Links
    #3  
Old Unix and Linux 12-02-2012   -   Original Discussion by rajamohan
Don Cragun's Unix or Linux Image
Don Cragun Don Cragun is offline Forum Staff  
Administrator
 
Join Date: Jul 2012
Last Activity: 23 November 2017, 11:06 PM EST
Location: San Jose, CA, USA
Posts: 10,680
Thanks: 575
Thanked 3,738 Times in 3,190 Posts
I believe this does what you want:

Code:
awk -F';' '/^COMMAND:/{
        found=1
        if(saved) print saved
}
/^REQUESTSTRING:/{
        if(found) saved = $1
}' input

----------------------------
Oops. I looked too closely at the expected output and missed the part about needing SEARCHPATTERN. The above script happens to work with the sample input, but is not correct in the general case.

What I should have said is something more like:

Code:
awk -F';' '/^COMMAND:/{
        if(fc && fs && saved) print saved
        fc = 1
        saved = ""
        fs = 0
}
/^REQUESTSTRING:/{
        if(fc) saved = $1
}
/SEARCHPATTERN/{
        fs = 1
}
END {   if(fc && fs && saved) print saved}' input

but elixir_sinari's proposal is shorter and should work fine as long as the input file doesn't violate the assumptions made. And, given the sample input, the assumptions look perfectly reasonable to me.

Last edited by Don Cragun; 12-02-2012 at 07:59 AM..
The Following User Says Thank You to Don Cragun For This Useful Post:
rajamohan (12-03-2012)
    #4  
Old Unix and Linux 12-02-2012   -   Original Discussion by rajamohan
michaelrozar17's Unix or Linux Image
michaelrozar17 michaelrozar17 is offline
Registered User
 
Join Date: Jul 2010
Last Activity: 22 September 2016, 3:57 AM EDT
Location: IN
Posts: 894
Thanks: 15
Thanked 183 Times in 174 Posts
Through Sed..

Code:
 sed -n 'H;/^$/{x;/COMMAND.*SEARCHPATTERN/s/.*\(REQUESTSTRING[^;]*\);.*/\1/p}' inputfile

The Following User Says Thank You to michaelrozar17 For This Useful Post:
rajamohan (12-03-2012)
Sponsored Links
    #5  
Old Unix and Linux 12-03-2012   -   Original Discussion by rajamohan
rajamohan rajamohan is offline
Registered User
 
Join Date: Mar 2008
Last Activity: 19 February 2014, 6:43 AM EST
Posts: 27
Thanks: 3
Thanked 1 Time in 1 Post
Quote:
Originally Posted by Don Cragun View Post
I believe this does what you want:

Code:
awk -F';' '/^COMMAND:/{
        found=1
        if(saved) print saved
}
/^REQUESTSTRING:/{
        if(found) saved = $1
}' input

----------------------------
Oops. I looked too closely at the expected output and missed the part about needing SEARCHPATTERN. The above script happens to work with the sample input, but is not correct in the general case.

What I should have said is something more like:

Code:
awk -F';' '/^COMMAND:/{
        if(fc && fs && saved) print saved
        fc = 1
        saved = ""
        fs = 0
}
/^REQUESTSTRING:/{
        if(fc) saved = $1
}
/SEARCHPATTERN/{
        fs = 1
}
END {   if(fc && fs && saved) print saved}' input

but elixir_sinari's proposal is shorter and should work fine as long as the input file doesn't violate the assumptions made. And, given the sample input, the assumptions look perfectly reasonable to me.
Thanks a lot.. Don Cragun.. its working good..

---------- Post updated at 01:55 AM ---------- Previous update was at 01:53 AM ----------

Quote:
Originally Posted by michaelrozar17 View Post
Through Sed..

Code:
 sed -n 'H;/^$/{x;/COMMAND.*SEARCHPATTERN/s/.*\(REQUESTSTRING[^;]*\);.*/\1/p}' inputfile

Thanks a lot michaelrozar17.
Its very simple and working good..
Sponsored Links
Closed

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Search and extract matching patterns lucasvs UNIX for Dummies Questions & Answers 9 02-28-2012 05:04 AM
PERL: extract lines between two patterns mnithink Shell Programming and Scripting 11 01-23-2012 11:58 PM
Extract patterns and copy them in different files shoaibjameel123 Shell Programming and Scripting 5 08-26-2011 12:50 AM
Extract certain patterns from file. shoaibjameel123 Shell Programming and Scripting 2 07-08-2011 10:56 AM
To extract the string between two patterns aajan Shell Programming and Scripting 6 09-17-2007 03:41 AM



All times are GMT -4. The time now is 01:39 AM.