Print all lines between two keyword if a specific pattern exist


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Print all lines between two keyword if a specific pattern exist
# 1  
Old 08-14-2015
Print all lines between two keyword if a specific pattern exist

I have input file as below I need to check for a pattern and if it is there in file then I need to print all the lines below BEGIN and END keyword. Could you please help me how to get this in AIX using sed or awk.

Input file:
Code:
ABC
******** BEGIN *****
My name is Amit.
I am learning unix.
***** END *****
***** BEGIN *****
My name is Rahul.
***** END *****
XYZ

If I am looking for Amit then Output should be
Code:
******** BEGIN *****
My name is Amit.
I am learning unix.
***** END *****

If I am looking for Rahul then Output should be
Code:
***** BEGIN *****
My name is Rahul.
***** END *****

If I am looking for XYZ then Output should be nothing.


From below command I am able to get all the data between two keyword but not able to grep the data.
Code:
sed -n '/BEGIN/,/END/p'

# 2  
Old 08-14-2015
Code:
$ awk ' /BEGIN/,/END/ { str = str ?  str "\n" $0 : $0 } /END/ { if( str ~ /Amit/ ) { print str }; str = "" } ' file
******** BEGIN *****
My name is Amit.
I am learning unix.
***** END *****

Code:
$ sed -n -e '/BEGIN/,/END/{H;/BEGIN/h;}'  -e '/END/{g;/Amit/p;} ' file
******** BEGIN *****
My name is Amit.
I am learning unix.
***** END *****

Code:
sed -n "/BEGIN/h;/BEGIN/!H; /END/ {x;/Amit/p;}" file
******** BEGIN *****
My name is Amit.
I am learning unix.
***** END *****


Last edited by anbu23; 08-14-2015 at 03:30 AM..
These 2 Users Gave Thanks to anbu23 For This Post:
# 3  
Old 08-14-2015
While not using one of the requested tools:
Code:
perl -00 -ne 'print $& if m{^\N+\s+BEGIN\s+\N+\n.*?Amit.*?\n\N+\s+END\s+\N+\n}ms;'

# 4  
Old 08-14-2015
Another awk solution, that doesn't need two times the /END/ expression:
Code:
awk '/BEGIN/ {block=1} block {str=str sep $0; sep=RS} /END/ {block=0; if (str~/Amit/) {print str} str=sep=""}' file

This User Gave Thanks to MadeInGermany For This Post:
# 5  
Old 08-14-2015
Quote:
Originally Posted by Amit Joshi
From below command I am able to get all the data between two keyword but not able to grep the data.
Code:
sed -n '/BEGIN/,/END/p'

This command prints everything between a line containing "BEGIN" and one containing "END", regardless of what is in between.

Your problem is, if i have understood correctly, is to only print these lines if some condition (a line containing a third word, like "Amit") is met. Here is how you solve these problems with sed:

The first thing you need is: you have to store the text in question somewhere until you decide if you print it or not. For this there is the "hold space". This is a text buffer you can manipulate separately from the "pattern space". It will maintain its content across the processing of lines. See the man page of "sed", the commands "g", "h", "x", "G" and "H". The principle being when you encounter a line with "BEGIN" you start a new cycle: put the encountered line in the hold space. From here on you append every line to there and accumulate the text this way until you encounter a line with "END" in it. When you finally encounter a line with "END" in it you clean out the hold space and start over again.

The second task is to decide if the text should be printed or not: When you encounter a line with END this ends the cycle: move all text accumulated in the hold space back to the pattern space, search the text there for your search string "Amit" and either print the whole text or discard it.

Code:
sed -n '/BEGIN/,/END/H        # from "BEGIN" to "END" execute "H" (append to hold space)
        /END/ {               # for every line containing "END" do:
                 g            # replace the pattern space with the content of the hold space
                              # (that makes the accumulated text available again)
                 /Amit/p      # if in this text is "Amit" somewhere, print it
                 s/.*//       # delete the text
                 x            # exchange pattern- and hold space
                              # (so hold space will be empty for the next cycle)
              }' /path/to/your/file

Or, the same in one line:

Code:
sed -n '/BEGIN/,/END/ H;/END/ {;g;/Amit/p;s/.*//;x;}' /path/to/your/file

I hope this helps.

bakunin

/PS: only now i saw that Anbu23 has already posted a sed solution which works the same way. My script and his second solution are quite similar but his way of cleaning the hold space is better than mine, so i suggest you use his.

Last edited by bakunin; 08-14-2015 at 08:49 AM..
This User Gave Thanks to bakunin For This Post:
# 6  
Old 08-14-2015
Hi.

There are grep-like utilities that are designed to handle these kinds of situations. Here is one called cgrep:
Code:
#!/usr/bin/env bash

# @(#) s1	Demonstrate text block extraction with enclosed pattern, cgrep.
# For cgrep source see:
# http://sourceforge.net/projects/cgrep/
# Verified existence Fri Aug 14 08:08:53 CDT 2015

# Support data and functions.
LC_ALL=C ; LANG=C ; export LC_ALL LANG
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C cgrep

NAME=${1?" Need a name"}
shift
FILE=${1-data1}

pl " Input data file $FILE:"
cat $FILE

pl " Results for searching for \"$NAME\":"
cgrep -D -F -w "BEGIN" +w "END" "$NAME" $FILE

exit 0

producing:
Code:
./s1 Amit

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian 5.0.8 (lenny, workstation) 
bash GNU bash 3.2.39
cgrep ATT cgrep 8.15

-----
 Input data file data1:
ABC
******** BEGIN *****
My name is Amit.
I am learning unix.
***** END *****
***** BEGIN *****
My name is Rahul.
***** END *****
XYZ

-----
 Results for searching for "Amit":
******** BEGIN *****
My name is Amit.
I am learning unix.
***** END *****

and
Code:
./s1 Rahul
...
-----
 Results for searching for "Rahul":
***** BEGIN *****
My name is Rahul.
***** END *****

and
Code:
./s1
...
./s1: line 16: 1:  Need a name

The source for cgrep can be found at the site mentioned in the comments in the demonstration script above.

Best wishes ... cheers, drl
# 7  
Old 08-14-2015
Hi !!

[QUOTE=anbu23;302952135]
Code:
$ awk ' /BEGIN/,/END/ { str = str ?  str "\n" $0 : $0 } /END/ { if( str ~ /Amit/ ) { print str }; str = "" } ' file
******** BEGIN *****
My name is Amit.
I am learning unix.
***** END *****

Could you explain the part
Code:
str = str ?  str "\n" $0

of your command line please, it's confuse for me.
I've understood that for each block /BEGIN/,/END/ you do this { str = str ? str "\n" $0 : $0 }, i know the result but it's not clear for me
Thx.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to print lines from a files with specific start and end patterns and pick only the last lines?

Hi, I need to print lines which are matching with start pattern "SELECT" and END PATTERN ";" and only select the last "select" statement including the ";" . I have attached sample input file and the desired input should be as: INPUT FORMAT: SELECT ABCD, DEFGH, DFGHJ, JKLMN, AXCVB,... (5 Replies)
Discussion started by: nani2019
5 Replies

2. Shell Programming and Scripting

Print lines present in first that match second variable. And ones that does not exist in second.

I have multi line input(var1) and reference(var2) variables. How to capture lines not present in var2 but present in var1? How to capture lines present var2 but not in var1? # configuration from server var1=""" Custom JAX-RS Custom Shared Web 2.0 """ # required configuration... (6 Replies)
Discussion started by: kchinnam
6 Replies

3. Shell Programming and Scripting

sed -- Find pattern -- print remainder -- plus lines up to pattern -- Minus pattern

The intended result should be : PDF converters 'empty line' gpdftext and pdftotext?xml version="1.0"?> xml:space="preserve"><note-content version="0.1" xmlns:/tomboy/link" xmlns:size="http://beatniksoftware.com/tomboy/size">PDF converters gpdftext and pdftotext</note-content>... (9 Replies)
Discussion started by: Klasform
9 Replies

4. Shell Programming and Scripting

Match Pattern and print pattern and multiple lines into one line

Hello Experts , require help . See below output: File inputs ------------------------------------------ Server Host = mike id rl images allocated last updated density vimages expiration last read <------- STATUS ------->... (4 Replies)
Discussion started by: tigerhills
4 Replies

5. UNIX for Dummies Questions & Answers

How to Detect Specific Pattern and Print the Specific String after It?

I'm still beginner and maybe someone can help me. I have this input: the great warrior a, b, c and what i want to know is, with awk, how can i detect the string with 'warrior' string on it and print the a, b, and c seperately, become like this : Warrior Type a b c Im still very... (3 Replies)
Discussion started by: radynaraya
3 Replies

6. Shell Programming and Scripting

Need one liner to search pattern and print everything expect 6 lines from where pattern match made

i need to search for a pattern from a big file and print everything expect the next 6 lines from where the pattern match was made. (8 Replies)
Discussion started by: chidori
8 Replies

7. Shell Programming and Scripting

How to print Specific keyword, by using awk?

How to print Specific keyword, by using awk.? prime:root:I want output. 78 1457 10000 10000 5985 307 10000 10000 10000 10000 3760 692 6656 157 696 (4 Replies)
Discussion started by: ooilinlove
4 Replies

8. Shell Programming and Scripting

Print Specific lines when found specific character

Hello all, I have thousand file input like this: file1: $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ $$ | | | |$$ $$ UERT | TTYH | TAFE | FRFG |$$ $$______|______|________|______|$$ $$ | | | |$$ $$ 1 | DISK | TR1311 | 1 |$$ $$ 1 |... (4 Replies)
Discussion started by: attila
4 Replies

9. Shell Programming and Scripting

print first few lines, then apply regex on a specific column to print results.

abc.dat tty cpu tin tout us sy wt id 0 0 7 3 19 71 extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 0.0 133.2 0.0 682.9 0.0 1.0 0.0 7.2 0 79 c1t0d0 0.2 180.4 0.1 5471.2 3.0 2.8 16.4 15.6 15 52 aaaaaa1-xx I want to skip first 5 line... (4 Replies)
Discussion started by: kchinnam
4 Replies

10. Shell Programming and Scripting

Extract lines of text based on a specific keyword

I regularly extract lines of text from files based on the presence of a particular keyword; I place the extracted lines into another text file. This takes about 2 hours to complete using the "sort" command then Kate's find & highlight facility. I've been reading the forum & googling and can find... (4 Replies)
Discussion started by: DionDeVille
4 Replies
Login or Register to Ask a Question