Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Grep lines before a pattern having some other pattern

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Grep lines before a pattern having some other pattern
# 1  
Old 02-05-2015
Grep lines before a pattern having some other pattern
Hi All,

I am trying to fetch lines before a pattern, I got to know about -B flag in grep but we have to pass the number to get those lines before some pattern say (X), now what if I want to get line/s with some other pattern say (Y) before X pattern? How to get about it? please help.

Input: test.xml
Code:
 
<EXTENSION NAME ="File Writer" SESSNAME ="NATIONAL_ARCHIVES" SUBTYPE ="File Writer" STAGETYPE ="Target Definition" TYPE ="WRITER">
<CONNREF CNXREFNAME ="CONN" CONNNAME ="" CONNNUMBER ="1" CONNSUBTYPE ="" CONNTYPE ="" VARIABLE =""/>
<ELEMENT NAME ="Merge Type" VALUE ="No Merge"/>
<ELEMENT NAME ="Merge File Directory" VALUE ="$DestFileDir&#x1D;"/>
<ELEMENT NAME ="Merge File Name" VALUE ="national_archives.out"/>
<ELEMENT NAME ="Append if Exists" VALUE ="NO"/>
<ELEMENT NAME ="Create Target Directory" VALUE ="NO"/>
<ELEMENT NAME ="Header Options" VALUE ="No Header"/>
<ELEMENT NAME ="Header Command" VALUE =""/>
<ELEMENT NAME ="Footer Command" VALUE =""/>
<ELEMENT NAME ="Output Type" VALUE ="File"/>
<ELEMENT NAME ="Merge Command" VALUE =""/>
</SESSNAME>
<SESSNAME NAME ="Relational Lookup" SINSTANCENAME ="LKP_CUSTOMER_EQUIPMENT_DTL_FACT" SUBTYPE ="Relational Lookup" STAGETYPE ="Lookup Procedure" TYPE ="LOOKUPEXTENSION">
<CONNREF CNXREFNAME ="DB CONN" CONNNAME ="XXX@YYY" CONNNUMBER ="1" CONNSUBTYPE ="Oracle" CONNTYPE ="Relational" VARIABLE =""/>
</SESSNAME>
<SESSNAME NAME ="Relational Reader" SINSTANCENAME ="SQ_EMPLOYEE" SUBTYPE ="Relational Reader" STAGETYPE ="Source Qualifier" TYPE ="READER">
<CONNREF CNXREFNAME ="DB CONN" CONNNAME ="YYY@XXX" CONNNUMBER ="1" CONNSUBTYPE ="Oracle" CONNTYPE ="Relational" VARIABLE =""/>
</SESSNAME>
<SESSNAME DDQINSTNAME ="SQ_EMPLOYEE" DSQINSTTYPE ="Source Qualifier" NAME ="Relational Reader" SINSTANCENAME ="EMPLOYEE" SUBTYPE ="Relational Reader" STAGETYPE ="Source Definition" TYPE ="READER"/>
<ELEMENT NAME ="General Options" VALUE =""/>
<ELEMENT NAME ="Write Backward Compatible Session Log File" VALUE ="NO"/>
<ELEMENT NAME ="Session Log File Name" VALUE ="employee.log"/>
<ELEMENT NAME ="Session Log File directory" VALUE ="$SessLogDir&#x1D;"/>
</SESSION>
<SESSION DESCRIPTION ="This session will load data into EMPLOYEE table." ISVALID ="YES" MAPNAME ="employee_job" NAME ="sess_employee_job" REUSABLE ="YES" SORTORDER ="Binary" VERSIONNUMBER ="8">
<SESSTRANSFORMATIONINST ISREPARTITIONPOINT ="NO" PIPELINE ="1" SINSTANCENAME ="LKP_DEPARTMENT" STAGE ="2" TRANSFORMATIONNAME ="LKP_DEPARTMENT" STAGETYPE ="Lookup Procedure">
<PARTITION DESCRIPTION ="" NAME ="Partition #1"/>
<ELEMENT NAME ="CONN Information" VALUE ="DATA@DB"/>
</SESSTRANSFORMATIONINST>
<SESSTRANSFORMATIONINST ISREPARTITIONPOINT ="NO" PIPELINE ="1" SINSTANCENAME ="FLT_DEPT_NAMES" STAGE ="2" TRANSFORMATIONNAME ="FLT_DEPT_NAMES" STAGETYPE ="Filter">
<PARTITION DESCRIPTION ="" NAME ="Partition #1"/>
</SESSTRANSFORMATIONINST>
<SESSTRANSFORMATIONINST ISREPARTITIONPOINT ="NO" PIPELINE ="0" SINSTANCENAME ="DEPARTMENT_DIM" STAGE ="0" TRANSFORMATIONNAME ="DEPARTMENT_DIM" STAGETYPE ="Source Definition"/>
<SESSTRANSFORMATIONINST ISREPARTITIONPOINT ="YES" PARTITIONTYPE ="PASS THROUGH" PIPELINE ="1" SINSTANCENAME ="SQ_STATE_DIM" STAGE ="2" TRANSFORMATIONNAME ="SQ_STATE_DIM" STAGETYPE ="Source Qualifier"/>
<SESSTRANSFORMATIONINST ISREPARTITIONPOINT ="YES" PARTITIONTYPE ="PASS THROUGH" PIPELINE ="1" SINSTANCENAME ="TEST_file" STAGE ="3" TRANSFORMATIONNAME ="TEST_file" STAGETYPE ="Target Definition">
<FLATFILE CODEPAGE ="MS1252" CONSECDELIMITERSASONE ="NO" DELIMITED ="YES" DELIMITERS ="," ESCAPE_CHARACTER ="" KEEPESCAPECHAR ="NO" LINESEQUENTIAL ="NO" MULTIDELIMITERSASAND ="NO" NULLCHARTYPE ="ASCII" NULL_CHARACTER ="*" PADBYTES ="1" QUOTE_CHARACTER ="NONE" REPEATABLE ="NO" ROWDELIMITER ="0" SKIPROWS ="0" STRIPTRAILINGBLANKS ="NO"/>
<ELEMENT NAME ="Thread Record" VALUE ="RMS"/>
</SESSTRANSFORMATIONINST>
<SESSION DESCRIPTION ="" ISVALID ="YES" MAPNAME ="test_job" NAME ="sess_test_job" REUSABLE ="NO" SORTORDER ="Binary" VERSIONNUMBER ="2">
<SESSTRANSFORMATIONINST ISREPARTITIONPOINT ="YES" PARTITIONTYPE ="PASS THROUGH" PIPELINE ="2" SINSTANCENAME ="TEST1" STAGE ="4" TRANSFORMATIONNAME ="TEST1" STAGETYPE ="Target Definition">
<ELEMENT NAME ="Target Table Name" VALUE ="TEST2"/>
</SESSTRANSFORMATIONINST>
<SESSTRANSFORMATIONINST ISREPARTITIONPOINT ="NO" PIPELINE ="0" SINSTANCENAME ="TEST" STAGE ="0" TRANSFORMATIONNAME ="AGG_TEST" STAGETYPE ="Source Definition">
<ELEMENT NAME ="Source Table Name" VALUE ="AGG_TEST2"/>
</SESSTRANSFORMATIONINST>

Code:
 
grep -B10 "Thread" test.xml | grep "SESSION DESCRIPTION"

results into
Code:
 
    <SESSION DESCRIPTION ="This session will load data into EMPLOYEE table." ISVALID ="YES" MAPNAME ="employee_job" NAME ="sess_employee_job" REUSABLE ="YES" SORTORDER ="Binary" VERSIONNUMBER ="8">
<SESSION DESCRIPTION ="" ISVALID ="YES" MAPNAME ="test_job" NAME ="sess_test_job" REUSABLE ="NO" SORTORDER ="Binary" VERSIONNUMBER ="2">

But the problem I am facing is that I don't know where "SESSION DESCRIPTION" will be present? will it be at the 15th line, 10th line above the "Thread" pattern? Also I've to capture the first "SESSION DESCRIPTION" pattern when multiple are present. So how to decide the NUM for -B or is there a way to capture the first pattern encountered before the line grep-ed? Smilie

I hope I've explained properly!

-dips
# 2  
Old 02-05-2015
No. Not clear.

I don't think your grep pipe will result in what you post, as SESSION DESCRIPTIONis in line 26 while Thread is in 38.

I'd propose to look into awk to get close to what you need.
# 3  
Old 02-05-2015
Just doing something like
Code:
grep "\(Thread\|SESSION DESCRIPTION\)" test.xml

may help, then dealing with a simpler stream containing only those lines.
This User Gave Thanks to Walter Misar For This Post:
dips_ag
# 4  
Old 02-06-2015
Rudic & Walter, thanks for your time!

I was afraid that my explanation might seem like blabbering!! Smilie

But what Walter suggested, I think I can take up from there and work with that.

-dips
# 5  
Old 02-06-2015
If I understand what you're trying to do, I think you want something like:
Code:
awk '/SESSION DESCRIPTION/,/Thread/' test.xml

If you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk, /usr/xpg6/bin/awk, or nawk.

Note, however, that this will print all lines from a line containing SESSION DESCRIPTION up through the next line containing Thread even if there is no line containing Thread after a line containing SESSION DESCRIPTION. It isn't clear to me if you want that or not. If you only want sets of lines that contain both strings, the code is slightly more complicated.
# 6  
Old 02-07-2015
Hi.

I could not tell if only the 2 lines matching 2 patterns were desired, or if the entire section between those two lines was wanted. So here is a solution for both, using easily accessible grep-like utilities:
Code:
#!/usr/bin/env bash

# @(#) s1	Demonstrate search for previous secondary pattern, peg, cgrep.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
LC_ALL=C ; LANG=C ; export LC_ALL LANG
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C peg cgrep

FILE=${1-data1}

pl " Input data file $FILE, numbered:"
cat -n $FILE

p=$( basename $0 ) t1="$Revision: 1.11 $" v=${t1//[!0-9.]/}
[[ $# > 0 ]] && [[ "$1" =~ -version ]] &&  { echo "$p (local) $v" ; exit 0 ; }

pl " Results, secondary pattern and main pattern ONLY:"
peg -z '/header/' 'time' $FILE

pl " Results, cgrep, ENTIRE section, secondary pattern THROUGH main pattern:"
cgrep -w 'header' 'time' $FILE

exit 0
producing:
Code:
$ ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian 5.0.8 (lenny, workstation) 
bash GNU bash 3.2.39
peg (local) 3.10
cgrep ATT cgrep 8.15

-----
 Input data file data1, numbered:
     1	header 1
     2	a
     3	b
     4	time
     5	  c
     6		header 2
     7	d
     8	time
     9	e
    10	f
    11	 header 3
    12	time

-----
 Results, secondary pattern and main pattern ONLY:
**** (1) header 1
time
**** (6) 	header 2
time
**** (11)  header 3
time

-----
 Results, cgrep, ENTIRE section, secondary pattern THROUGH main pattern:
header 1
a
b
time
========================================
	header 2
d
time
========================================
 header 3
time

peg: http://www.cpan.org/authors/id/A/AD/ADAVIES/peg-3.14

cgrep: cgrep | SourceForge.net

Both codes have many more features than noted here. Regarding peg, because it is in perl, it can be taken to almost any platform that has perl. The cgrep code is c, which means that it must be compiled. I had no trouble compiling it on 32-bit and 64-bit systems with gcc.

Best wishes ... cheers, drl
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Grep all lines with the pattern .sh

Linux version : Oracle Linux 6.5 Shell : bash In the the below text file (someString.text), I want to grep all lines with .sh in it. ie. Only the lines mysript.sh and anotherscript.sh should be returned. My below attempts failed. I gather that in regular expression world, dot (.) is the... (3 Replies)
Discussion started by: John K
3 Replies

2. Shell Programming and Scripting

sed -- Find pattern -- print remainder -- plus lines up to pattern -- Minus pattern

The intended result should be : PDF converters 'empty line' gpdftext and pdftotext?xml version="1.0"?> xml:space="preserve"><note-content version="0.1" xmlns:/tomboy/link" xmlns:size="http://beatniksoftware.com/tomboy/size">PDF converters gpdftext and pdftotext</note-content>... (9 Replies)
Discussion started by: Klasform
9 Replies

3. UNIX for Dummies Questions & Answers

Grep -v lines starting with pattern 1 and not matching pattern 2

Hi all! Thanks for taking the time to view this! I want to grep out all lines of a file that starts with pattern 1 but also does not match with the second pattern. Example: Drink a soda Eat a banana Eat multiple bananas Drink an apple juice Eat an apple Eat multiple apples I... (8 Replies)
Discussion started by: demmel
8 Replies

4. AIX

Grep a pattern and print following n lines

Hi all, I am struck with the below requirement. I need to grep a particular pattern in a file and then print next n lines of it for further processing. I have used the below code grep -A 3 "pattern" filename But it is throwing error as below. grep: illegal option -- A Can... (14 Replies)
Discussion started by: ssk250
14 Replies

5. Shell Programming and Scripting

Grep pattern and display all lines below

Hi I need to grep for a patter and display all lines below the pattern. For ex: say my file has the below lines file1 file2 file3 file4 file5 I NEED to grep for patter file3 and display all lines below the pattern. do we have an option to get this data. Let me know if you require... (5 Replies)
Discussion started by: venkidhadha
5 Replies

6. Shell Programming and Scripting

sed and awk usage to grep a pattern 1 and with reference to this grep a pattern 2 and pattern 3

Hi , I have a file where i have modifed certain things compared to original file . The difference of the original file and modified file is as follows. # diff mir_lex.c.modified mir_lex.c.orig 3209c3209 < if(yy_current_buffer -> yy_is_our_buffer == 0) { --- >... (5 Replies)
Discussion started by: breezevinay
5 Replies

7. Shell Programming and Scripting

Grep the word from pattern line and update in subsequent lines till next pattern line reached

Hi, I have got the below requirement. please suggest. I have a file like, Processing Item is: /data/ing/cfg2/abc.txt /data/ing/cfg3/bgc.txt Processing Item is: /data/cmd/for2/ght.txt /data/kernal/config.klgt.txt I want to process the above file to get the output file like, ... (5 Replies)
Discussion started by: rbalaj16
5 Replies

8. Shell Programming and Scripting

grep: get last 3 lines containing PATTERN from many files

Hi all, I am looking for a quick solution for this: I have many log files of an iterative program, and I would like to display the parameters of the last three iteration from each of those files. Relevant lines have the keyword: ITER I am using: tac ~/modeling*/fitting.log | grep -m 3 -e... (2 Replies)
Discussion started by: pnemeth
2 Replies

9. UNIX for Dummies Questions & Answers

Grep with 8 lines before and after pattern.

OK. I have a file I'd like to be able to grep, but on top of returning the line where the pattern matches, I'd like to be able to get the previous 8 lines and the following 8 lines. Is there a way to do this? (2 Replies)
Discussion started by: mrwatkin
2 Replies

10. Solaris

How to grep (say)last-3 and next-3 lines of Desired Pattern

Hi All, OS-Type=Sun-OS 5.8 Sparc9 Processor Can I grep the previous 4 lines and next 4 lines of a matched pattern(context grep)? For example here we need to monitor logs of live traffic.The data obtained from "tail -f LiveTrafficData.log" looks something like this:-... (3 Replies)
Discussion started by: Sujan Banerjee
3 Replies
Login or Register to Ask a Question