Home Man
Search
Today's Posts
Register

BSD, Linux, and UNIX shell scripting — Post awk, bash, csh, ksh, perl, php, python, sed, sh, shell scripts, and other shell scripting languages questions here.

Extract line between two patterns

Tags
shell scripts

Login to Reply

 
Thread Tools Search this Thread
# 1  
Old 12-02-2012
Extract line between two patterns

Hi All,
I need a script to extract a lines between two patterns.I have done this using grep,cut,tail and head.But its very slow, because my input file contain more than a lakh.

Code:
COMMAND:XXXXXXXXXXXXXXXXXXXX
yyyyy
zzzzzz
REQUESTSTRING:aaaaaaaaaaaaaaa;11111
222222
333333
SEARCHPATTERN
44444444444444
55555555
66666

COMMAND:XXXXXXXXXXXXXXXXXXXX
yyyyy
zzzzzz
REQUESTSTRING:bbbbb SEARCHPATTERN bbbbbbbb;11111
222222
333333
44444444444444
55555555
66666

COMMAND:XXXXXXXXXXXXXXXXXXXX
yyyyy
zzzzzz
REQUESTSTRING:cccccccccccc;11111
222222
333333
SEARCHPATTERN
44444444444444
55555555
66666

COMMAND:XXXXXXXXXXXXXXXXXXXX
yyyyy
zzzzzz
REQUESTSTRING:dddddddddddd;111111
222222
333333
44444444444444
55555555
66666

From the above sample,

Extract single COMMAND and search for SEARCHPATTERN, if SEARCHPATTERN available print the REQUESTSTRING line before ";" .

Note:
The search pattern may present in separate line or in REQUESTSTRING line itself.

Expected output is:

Code:
REQUESTSTRING:aaaaaaaaaaaaaaa
REQUESTSTRING:bbbbb SEARCHPATTERN bbbbbbbb
REQUESTSTRING:ccccccccccccccc

My sample code using grep..

Code:
grep -n -e "COMMAND" input.txt > tmp.txt
startline=0
while read line
do
	if [ "$startline" = "0" ]
	then
		read line
	fi
	endline=`echo $line |cut -d ":" -f 1`
	endline=`expr $endline - 1`
#	echo "Startline:$startline,Endline:$endline"
	noofline=`expr $endline - $startline`
	head -$endline input.txt > t1.txt
	tail -$noofline t1.txt > t2.txt 
	grep "SEARCHPATTERN " t2.txt > /dev/dull
	if [ "$?" = "0" ] 
	then
		grep "REQUESTSTRING " t2.txt | cut -d ";" -f 1 
	fi
	startline=$endline
done < tmp.txt

# 2  
Old 12-02-2012
With assumptions:
Code:
awk '/SEARCHPATTERN/ && NF==2{split($2,a,/;/);print FS a[1]}' FS='REQUESTSTRING:' RS= file

# 3  
Old 12-02-2012
I believe this does what you want:
Code:
awk -F';' '/^COMMAND:/{
        found=1
        if(saved) print saved
}
/^REQUESTSTRING:/{
        if(found) saved = $1
}' input

----------------------------
Oops. I looked too closely at the expected output and missed the part about needing SEARCHPATTERN. The above script happens to work with the sample input, but is not correct in the general case.

What I should have said is something more like:
Code:
awk -F';' '/^COMMAND:/{
        if(fc && fs && saved) print saved
        fc = 1
        saved = ""
        fs = 0
}
/^REQUESTSTRING:/{
        if(fc) saved = $1
}
/SEARCHPATTERN/{
        fs = 1
}
END {   if(fc && fs && saved) print saved}' input

but elixir_sinari's proposal is shorter and should work fine as long as the input file doesn't violate the assumptions made. And, given the sample input, the assumptions look perfectly reasonable to me.

Last edited by Don Cragun; 12-02-2012 at 06:59 AM..
The Following User Says Thank You to Don Cragun For This Useful Post:
rajamohan (12-03-2012)
# 4  
Old 12-02-2012
Through Sed..
Code:
 sed -n 'H;/^$/{x;/COMMAND.*SEARCHPATTERN/s/.*\(REQUESTSTRING[^;]*\);.*/\1/p}' inputfile

The Following User Says Thank You to michaelrozar17 For This Useful Post:
rajamohan (12-03-2012)
# 5  
Old 12-03-2012
Quote:
Originally Posted by Don Cragun
I believe this does what you want:
Code:
awk -F';' '/^COMMAND:/{
        found=1
        if(saved) print saved
}
/^REQUESTSTRING:/{
        if(found) saved = $1
}' input

----------------------------
Oops. I looked too closely at the expected output and missed the part about needing SEARCHPATTERN. The above script happens to work with the sample input, but is not correct in the general case.

What I should have said is something more like:
Code:
awk -F';' '/^COMMAND:/{
        if(fc && fs && saved) print saved
        fc = 1
        saved = ""
        fs = 0
}
/^REQUESTSTRING:/{
        if(fc) saved = $1
}
/SEARCHPATTERN/{
        fs = 1
}
END {   if(fc && fs && saved) print saved}' input

but elixir_sinari's proposal is shorter and should work fine as long as the input file doesn't violate the assumptions made. And, given the sample input, the assumptions look perfectly reasonable to me.
Thanks a lot.. Don Cragun.. its working good..

---------- Post updated at 01:55 AM ---------- Previous update was at 01:53 AM ----------

Quote:
Originally Posted by michaelrozar17
Through Sed..
Code:
 sed -n 'H;/^$/{x;/COMMAND.*SEARCHPATTERN/s/.*\(REQUESTSTRING[^;]*\);.*/\1/p}' inputfile

Thanks a lot michaelrozar17.
Its very simple and working good..
Login to Reply

« Previous Thread | Next Thread »
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Extract lines between patterns squrcles Shell Programming and Scripting 10 01-18-2016 12:15 PM
Extract all the sentences that matched two patterns my_Perl Shell Programming and Scripting 1 06-23-2015 08:45 PM
Extract Lines Between 2 patterns if exist... DSommers Debian 3 09-29-2014 05:44 PM
Find matched patterns and print them with other patterns not the whole line redse171 Shell Programming and Scripting 3 09-24-2014 01:12 PM
Extract all the lines in between of 2 patterns and merge them prvnrk Shell Programming and Scripting 4 11-24-2013 04:10 AM
Need to extract text repetitively between two patterns Vignesh58 Shell Programming and Scripting 2 02-21-2013 06:53 AM
PERL: extract lines between two patterns mnithink Shell Programming and Scripting 11 01-23-2012 10:58 PM
Extract patterns and copy them in different files shoaibjameel123 Shell Programming and Scripting 5 08-25-2011 11:50 PM
Extract certain patterns from file. shoaibjameel123 Shell Programming and Scripting 2 07-08-2011 09:56 AM
To extract the string between two patterns aajan Shell Programming and Scripting 6 09-17-2007 02:41 AM


All times are GMT -4. The time now is 05:20 AM.

Unix & Linux Forums Content Copyright©1993-2018. All Rights Reserved.
UNIX.COM Login
Username:
Password:  
Show Password