Grab contents between two matched patterns


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Grab contents between two matched patterns
# 8  
Old 02-05-2013
Did you replace "1.pattern" with a reasonable string to identify the start of your table, and "2.pattern" with sth. to recognize its end? Post the command you applied!
# 9  
Old 02-05-2013
The table looks like this, I want to grab those data and perform further awk operation (if column 6 greater than certain value then output column 2)


Code:
    N   Batch    Mn(I)   RMSdev  I/rms  Rmerge    Number  Nrej Cm%poss  AnoCmp MaxRes CMlplc SmRmerge SmMaxRes  $$ $$
    1       1    148.4     32.3   4.59   0.147       168     0     0.3      -     3.5   0.00    0.066     3.17
    2       2    526.8     80.3   6.56   0.062       808     0     2.1      -     3.0   0.02    0.066     3.17
    3       3    478.7     57.5   8.32   0.064      1020     0     4.1     0.0    3.2   0.04    0.066     3.17
$$
    N   Batch    Mn(I)   RMSdev  I/rms  Rmerge    Number  Nrej Cm%poss  AnoCmp MaxRes CMlplc SmRmerge SmMaxRe

I tried command

Code:
awk '/N   Batch    Mn\(I\)   RMSdev  I\/rms  Rmerge    Number  Nrej Cm\%poss  AnoCmp MaxRes CMlplc SmRmerge SmMaxRes  \$\$ \$\$/ {on=1} /N   Batch    Mn\(I\)   RMSdev  I\/rms  Rmerge    Number  Nrej Cm\%poss  AnoCmp MaxRes CMlplc SmRmerge SmMaxRes/ {on=0} on' myfile

it returned empty

If I tried

Code:
awk '/N   Batch    Mn\(I\)   RMSdev  I\/rms  Rmerge    Number  Nrej Cm\%poss  AnoCmp MaxRes CMlplc SmRmerge SmMaxRes  \$\$ \$\$/, /N   Batch    Mn\(I\)   RMSdev  I\/rms  Rmerge    Number  Nrej Cm\%poss  AnoCmp MaxRes CMlplc SmRmerge SmMaxRes/' myfile

it returned a single line but no data

Code:
N   Batch    Mn(I)   RMSdev  I/rms  Rmerge    Number  Nrej Cm%poss  AnoCmp MaxRes CMlplc SmRmerge SmMaxRes  $$ $$

I just have no clue why it wouldn't work
# 10  
Old 02-05-2013
Your awk statement does exactly what you tell it to do:
If "pattern $$ $$" is matched, switch "on" to TRUE (= 1).
If "pattern" is matched, switch "on" to FALSE (= 0).
If "on" == TRUE, print the actual line.

The problem you encounter is that you switch on and off in adjacent actions on the same input line, so the script has no chance to print anything. So the task resolves to scrutinize the data to find patterns that can't be intermixed to switch printing on and off AND carefully formulate regexes that uniquely identify either.

BTW - my earlier statement that awk does not recognize sed like address ranges was wrong - apologies for that. So your original ansatz was OK in priciple (syntax error mayhap due to the comma after the awk program) but suffered from above - find the right patterns. Try this:
Code:
$ awk '/N   Batch  --- looooong pattern ---  \$\$ \$\$/, /^\$\$/' file

You might want to abbreviate the long pattern by replacing non-relevant part with wildcads like .*.

Last edited by RudiC; 02-05-2013 at 06:58 AM..
# 11  
Old 02-05-2013
your code worked! many thanks for that. Perhaps I do not truly understand how the syntax
Code:
awk '/pattern1/, /pattern2/' file

works could you explain why adding the ^ character has made the difference? is the {on=1} and {on=0} implicit in that command line?

also would it be possible to not print the N Batch...... line in the output?
# 12  
Old 02-05-2013
1) T'was the same error you encountered before: $$ was present at the end of the starting pattern and thus closed the address range immediately, printing exactly that one line. Adding the "^" told awk to look for $$ as the first chars in the line, making it ignore those at the end and thus extending the range.

2) I would not phrase it like so, but yes, you could imagine implicit ons and offs in that command.

3) Yes, by carefully rearranging the sequence of commands in the awk program:
Code:
...
/pattern2/ {on=0}
on
/pattern1/ {on=1}
...

# 13  
Old 02-05-2013
Why does that work? (what does it do?)

and why doesn't it work when you type it in one single line?
# 14  
Old 02-05-2013
You want to leave out the switching line - so switch off before print and switch on after print.
I don't see a reason why it should not work on a single line except that awk might have difficulties and needed some help to separate the pattern {action} pairs.
This User Gave Thanks to RudiC For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to print two matched patterns only from each line?

My input looks like this. # Lot Of CODE Before AppType_somethinglese=$(cat << EOF AppType_test1='test-tool/blatest-tool-ear' AppType_test2='test/blabla-ear' # Lot Of CODE After I want to print text betwen 1) _ and = and 2)/ and ' from each line and exclude lines with "EOF". Output... (2 Replies)
Discussion started by: kchinnam
2 Replies

2. Shell Programming and Scripting

Extract all the sentences that matched two patterns

Hi I have two lists of patterns named A and B consisting of around 200 entries in each and I want to extract all the sentences from a big text file which match atleast one pattern from both A and B. For example, pattern list A consists of : ama ani ahum mari ... ... and pattern... (1 Reply)
Discussion started by: my_Perl
1 Replies

3. Shell Programming and Scripting

Find matched patterns and print them with other patterns not the whole line

Hi, I am trying to extract some patterns from a line. The input file is space delimited and i could not use column to get value after "IN" or "OUT" patterns as there could be multiple white spaces before the next digits that i need to print in the output file . I need to print 3 patterns in a... (3 Replies)
Discussion started by: redse171
3 Replies

4. Shell Programming and Scripting

Matched multiple patterns that could be in a same line

Hi, I need help to match pattern started with "RW" in file 1 and with pattern in $1 in file 2 as follows:- File 1 BH /TOTAL=466(423); /POSITIVE=300(257); /UNKNOWN=25(25); BH /F_P=141(141); /F_N=136; /P=4; CC /TAX=!?; /MAX-R=2; CC /VER=2; RW P9610, AR_BSU , T; PAE25, AE_E57... (10 Replies)
Discussion started by: redse171
10 Replies

5. Shell Programming and Scripting

Grab contents between two patterns

Hi, What is the best approach to grab contents between Changes Dependencies from the following example snippy Changes in packages about to be updated: bash-3.2-32.el5_9.1.x86_64 * Thu Jun 27 22:00:00 2013 Roman Rakus <rrakus@redhat.com> - 3.2-32.1 - Fixed a bug that caused... (2 Replies)
Discussion started by: ashokvpp
2 Replies

6. Shell Programming and Scripting

Grab nth occurence in between two patterns using awk or sed

Hi , I have an issue where I want to parse through the output from a file and I want to grab the nth occurrence of text in between two patterns preferably using awk or sed ! TICKET NBR : 1 !GSI : 102 ! 3100.2.112.1 11/06/2013 15:56:29 ! 3100.2.22.3 98 ! 3100.2.134.2... (8 Replies)
Discussion started by: OTNA
8 Replies

7. Shell Programming and Scripting

Grab the contents with in special character

I have a file which contains below kind of lines 2013-05-21 00:00:03 INFO moved to unprocessed, as doesn't exist in masklist and modified at 2013-05-20@21:21:21.000000000. 2013-05-21 00:00:03 INFO moved to unprocessed, as doesn't exist in masklist and modified at... (1 Reply)
Discussion started by: manas_ranjan
1 Replies

8. UNIX for Dummies Questions & Answers

grep to show patterns being matched (-f option)

I have a list of patterns (regexes) in a file and use with `grep -f <file_with_list_of_regexes.txt> input.txt` to search in my input for those patterns. grep is doing a fantastic job at it and finds me the matching input text but I also want to see in the output the regex (from... (1 Reply)
Discussion started by: mirage
1 Replies

9. Shell Programming and Scripting

How to group matched patterns in different files

Hi, I have a master file that i need to split into multiple files based on matched patterns. sample of my data as follows:- scaff_1 a e 123 130 c_scaff_100 scaff_1 a e 132 138 c_scaff_101 scaff_1 a e 140 150 ... (2 Replies)
Discussion started by: redse171
2 Replies

10. Shell Programming and Scripting

How to get only matched contents?

Hi, I have an array. @arr=("abcdefgh","ppppppp","rrr"); $tofind="rrr";#string to find. I want to match this string and retrieve only matched contents. In this case rrr is found in 2nd position in an array i want to print only rrr. If the string is matched i have to retrieve only... (1 Reply)
Discussion started by: vanitham
1 Replies
Login or Register to Ask a Question