Shell Programming and Scripting

BSD, Linux, and UNIX shell scripting — Post awk, bash, csh, ksh, perl, php, python, sed, sh, shell scripts, and other shell scripting languages questions here.

Extract line between two patterns

👤 Login to reply

    #1  
Old 12-02-2012
rajamohan rajamohan is offline
Registered User
 
Extract line between two patterns

Hi All,
I need a script to extract a lines between two patterns.I have done this using grep,cut,tail and head.But its very slow, because my input file contain more than a lakh.

Code:
COMMAND:XXXXXXXXXXXXXXXXXXXX
yyyyy
zzzzzz
REQUESTSTRING:aaaaaaaaaaaaaaa;11111
222222
333333
SEARCHPATTERN
44444444444444
55555555
66666

COMMAND:XXXXXXXXXXXXXXXXXXXX
yyyyy
zzzzzz
REQUESTSTRING:bbbbb SEARCHPATTERN bbbbbbbb;11111
222222
333333
44444444444444
55555555
66666

COMMAND:XXXXXXXXXXXXXXXXXXXX
yyyyy
zzzzzz
REQUESTSTRING:cccccccccccc;11111
222222
333333
SEARCHPATTERN
44444444444444
55555555
66666

COMMAND:XXXXXXXXXXXXXXXXXXXX
yyyyy
zzzzzz
REQUESTSTRING:dddddddddddd;111111
222222
333333
44444444444444
55555555
66666

From the above sample,

Extract single COMMAND and search for SEARCHPATTERN, if SEARCHPATTERN available print the REQUESTSTRING line before ";" .

Note:
The search pattern may present in separate line or in REQUESTSTRING line itself.

Expected output is:

Code:
REQUESTSTRING:aaaaaaaaaaaaaaa
REQUESTSTRING:bbbbb SEARCHPATTERN bbbbbbbb
REQUESTSTRING:ccccccccccccccc

My sample code using grep..

Code:
grep -n -e "COMMAND" input.txt > tmp.txt
startline=0
while read line
do
	if [ "$startline" = "0" ]
	then
		read line
	fi
	endline=`echo $line |cut -d ":" -f 1`
	endline=`expr $endline - 1`
#	echo "Startline:$startline,Endline:$endline"
	noofline=`expr $endline - $startline`
	head -$endline input.txt > t1.txt
	tail -$noofline t1.txt > t2.txt 
	grep "SEARCHPATTERN " t2.txt > /dev/dull
	if [ "$?" = "0" ] 
	then
		grep "REQUESTSTRING " t2.txt | cut -d ";" -f 1 
	fi
	startline=$endline
done < tmp.txt

Sponsored Links
    #2  
Old 12-02-2012
elixir_sinari's Unix or Linux Image
elixir_sinari elixir_sinari is offline Forum Advisor  
Registered User
 
With assumptions:
Code:
awk '/SEARCHPATTERN/ && NF==2{split($2,a,/;/);print FS a[1]}' FS='REQUESTSTRING:' RS= file

Sponsored Links
    #3  
Old 12-02-2012
Don Cragun's Unix or Linux Image
Don Cragun Don Cragun is online now Forum Staff  
Administrator
 
I believe this does what you want:
Code:
awk -F';' '/^COMMAND:/{
        found=1
        if(saved) print saved
}
/^REQUESTSTRING:/{
        if(found) saved = $1
}' input

----------------------------
Oops. I looked too closely at the expected output and missed the part about needing SEARCHPATTERN. The above script happens to work with the sample input, but is not correct in the general case.

What I should have said is something more like:
Code:
awk -F';' '/^COMMAND:/{
        if(fc && fs && saved) print saved
        fc = 1
        saved = ""
        fs = 0
}
/^REQUESTSTRING:/{
        if(fc) saved = $1
}
/SEARCHPATTERN/{
        fs = 1
}
END {   if(fc && fs && saved) print saved}' input

but elixir_sinari's proposal is shorter and should work fine as long as the input file doesn't violate the assumptions made. And, given the sample input, the assumptions look perfectly reasonable to me.

Last edited by Don Cragun; 12-02-2012 at 06:59 AM..
The Following User Says Thank You to Don Cragun For This Useful Post:
rajamohan (12-03-2012)
    #4  
Old 12-02-2012
michaelrozar17's Unix or Linux Image
michaelrozar17 michaelrozar17 is offline
Registered User
 
Through Sed..
Code:
 sed -n 'H;/^$/{x;/COMMAND.*SEARCHPATTERN/s/.*\(REQUESTSTRING[^;]*\);.*/\1/p}' inputfile

The Following User Says Thank You to michaelrozar17 For This Useful Post:
rajamohan (12-03-2012)
Sponsored Links
    #5  
Old 12-03-2012
rajamohan rajamohan is offline
Registered User
 
Quote:
Originally Posted by Don Cragun View Post
I believe this does what you want:
Code:
awk -F';' '/^COMMAND:/{
        found=1
        if(saved) print saved
}
/^REQUESTSTRING:/{
        if(found) saved = $1
}' input

----------------------------
Oops. I looked too closely at the expected output and missed the part about needing SEARCHPATTERN. The above script happens to work with the sample input, but is not correct in the general case.

What I should have said is something more like:
Code:
awk -F';' '/^COMMAND:/{
        if(fc && fs && saved) print saved
        fc = 1
        saved = ""
        fs = 0
}
/^REQUESTSTRING:/{
        if(fc) saved = $1
}
/SEARCHPATTERN/{
        fs = 1
}
END {   if(fc && fs && saved) print saved}' input

but elixir_sinari's proposal is shorter and should work fine as long as the input file doesn't violate the assumptions made. And, given the sample input, the assumptions look perfectly reasonable to me.
Thanks a lot.. Don Cragun.. its working good..

---------- Post updated at 01:55 AM ---------- Previous update was at 01:53 AM ----------

Quote:
Originally Posted by michaelrozar17 View Post
Through Sed..
Code:
 sed -n 'H;/^$/{x;/COMMAND.*SEARCHPATTERN/s/.*\(REQUESTSTRING[^;]*\);.*/\1/p}' inputfile

Thanks a lot michaelrozar17.
Its very simple and working good..
Sponsored Links
👤 Login to reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Search and extract matching patterns lucasvs UNIX for Dummies Questions & Answers 9 02-28-2012 04:04 AM
PERL: extract lines between two patterns mnithink Shell Programming and Scripting 11 01-23-2012 10:58 PM
Extract patterns and copy them in different files shoaibjameel123 Shell Programming and Scripting 5 08-25-2011 11:50 PM
Extract certain patterns from file. shoaibjameel123 Shell Programming and Scripting 2 07-08-2011 09:56 AM
To extract the string between two patterns aajan Shell Programming and Scripting 6 09-17-2007 02:41 AM



All times are GMT -4. The time now is 05:02 AM.

Unix & Linux Forums Content Copyright©1993-2018. All Rights Reserved.
×
UNIX.COM Login
Username:
Password:  
Show Password





Not a Forum Member?
Forgot Password?