Extract data from records that match pattern


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extract data from records that match pattern
# 1  
Old 03-22-2010
Extract data from records that match pattern

Hi Guys,

I have a file as follows:

Code:
a b c  1 2 3 4 
pp gg gh hh 1 2 fm 3 4 
g h i j k l m 1 2 3 4 
d e f g h j i  k l 1 2 3 f 3 4 
r t y u i o p d p re 1 2 3 f 4 
 
t y w e q w r a s p a 1 2 3 4

I am trying to extract all the 2's from each row. 2 is just an example not a fixed number. The number of fields starting from the beginning of the each row are variable so I am trying to use the end of each row as the starting address to extract the 2 as the number of fields stay constant starting from the end of the row and moving backwards except some rows have an "f" or an "fm". When there is an "f" or an "fm" in the row, I want to extract the 3rd field from the end of the row (which will give me 2). When there is no "f" or "fm" in a row, I want to extract the 2nd field from the end of the row.

I also have blanks in the file and I want to extract those too. Does not matter which field I select for a blank line, any field will do as long as I get a blank.

Here is what I am trying:

Code:
 
awk 'if (/ f / || / fm /) {print $(NF - 2)}; else {print $(NF - 3)}' test.txt

It's giving me a syntax error on the else statement.

Also I know this won't work for a blank line as the NF will be zero.

Is there any easier way to do this?

Thanks.
# 2  
Old 03-22-2010
Code:
awk '/^[ ]*$/ { print " "; next };/ f / || / fm / { print $(NF - 3); next }; { print $(NF - 2) }' test.txt


Last edited by Scott; 04-23-2010 at 07:38 PM.. Reason: Added code tags
# 3  
Old 03-22-2010
Code:
 nawk 'NF { print (/ f / || / fm /)?$(NF-2):$(NF-3)}' myFile

# 4  
Old 03-22-2010
Quote:
Originally Posted by vgersh99
Code:
 nawk 'NF { print (/ f / || / fm /)?$(NF-2):$(NF-3)}' myFile

I'm not a nawk expert, but this stops processing after a blank/empty line.

I must admit tho, it's an interesting construction. I learned something.
# 5  
Old 03-22-2010
No, it doesn't. For empty lines, the pattern (in this case the value of NF) will evaluate to a false value, and its associated action (the code inside the curly braces) will not be executed. AWK will then proceed to the next line and re-evalute the pattern, until it reaches the end of the file. An explicit exit statement would be needed to quit before EOF.

Regards,
Alister

Last edited by alister; 03-22-2010 at 09:54 PM..
# 6  
Old 03-23-2010
Quote:
Originally Posted by alister
No, it doesn't. For empty lines, the pattern (in this case the value of NF) will evaluate to a false value, and its associated action (the code inside the curly braces) will not be executed. AWK will then proceed to the next line and re-evalute the pattern, until it reaches the end of the file. An explicit exit statement would be needed to quit before EOF.

Regards,
Alister
On my Solaris 10 system it did stop executing after the blank line
# 7  
Old 03-23-2010
Hi, sb008:

I'm not familiar with the Solaris userland, but that's odd and unexpected behavior, relative to every version of AWK that I've used over the past decade (nawk aka one true awk, gawk, mawk) and to the posix standard.

Regards,
Alister
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Help with pattern match and Extract

Hi All, I am having a file like below . Basically when SB comes in the text with B. I have to take the word till SB. When there only B I should take take till B. Tried for cut it by demilter but not able to build the logic SB- CD B_RESTO SB_RESTO CRYSTALS BOILERS -->There SB and B so I... (6 Replies)
Discussion started by: arunkumar_mca
6 Replies

2. Shell Programming and Scripting

Match and extract data using two files

Hello, Using the information in file 1, I would like to extract from file2 all rows which matchs in column 3. file 1 1233 1230 1231 1232 file2 65733.00 19775.00 1220 65733.00 19793.00 1220 65733.00 19801.00 1220 65733.00 19809.00 1231 65733.00 19817.00 ... (2 Replies)
Discussion started by: jiam912
2 Replies

3. Shell Programming and Scripting

Extract lines that match a pattern

Hi all, I got a file that contains the following content, Actually it is a part of the file content, Installing XYZ XYZA Image, API 18, revision 2 Unzipping XYZ XYZA Image, API 18, revision 2 (1%) Unzipping XYZ XYZA Image, API 18, revision 2 (96%) Unzipping XYZ XYZA Image, API 18,... (7 Replies)
Discussion started by: Kashyap
7 Replies

4. Shell Programming and Scripting

Pattern Match & Extract from a string

Hi, I have long string in 2nd field, as shown below: REF1 | CLESCLJSCSHSCSMSCSNSCSRSCUDSCUFSCU7SCV1SCWPSCXGPDBACAPA0DHDPDMESED6 REF2 | SBR4PCBFPCDRSCSCG3SCHEBSCKNSCKPSCLLSCMCZXTNPCVFPCV6P4KL0DMDSDSASEWG I have a group of fixed patterns which can occur in these long strings & only... (11 Replies)
Discussion started by: karumudi7
11 Replies

5. Shell Programming and Scripting

Appending string to match pattern (data processing)

Hello i have go the following result from performing 2 testing using the same file. I have used unix script to extract the result because the files are many as shown below. 01_gravity.f.tcov 7 3 42.86 02_gravity.f.tcov 9 4 80.86... (4 Replies)
Discussion started by: ganiel24
4 Replies

6. Shell Programming and Scripting

Match first pattern first then extract second pattern match

My input file: <accession>Q91G55</accession> <name>043L_IIV6</name> <protein> <recommendedName> <location> <position position="294"/> </location> <fullName>Uncharacterized protein 043L</fullName> <accession>P18556</accession> <name>1106L_ASFB7</name> <protein> <recommendedName>... (5 Replies)
Discussion started by: patrick87
5 Replies

7. Shell Programming and Scripting

Extract data based on match against one column data from a long list data

My input file: data_5 Ali 422 2.00E-45 102/253 140/253 24 data_3 Abu 202 60.00E-45 12/23 140/23 28 data_1 Ahmad 256 7.00E-45 120/235 140/235 22 data_4 Aman 365 8.00E-45 15/65 140/65 20 data_10 Jones 869 9.00E-45 65/253 140/253 18... (12 Replies)
Discussion started by: patrick87
12 Replies

8. Shell Programming and Scripting

AWK, print no of records after pattern match.

Hi ALL :). i have a file, cat 3 + dog 5 + rat 6 - i want to print no of record having pattern "+". thanks in advance :confused:. (2 Replies)
Discussion started by: admax
2 Replies

9. Shell Programming and Scripting

Extract data from large file 80+ million records

Hello, I have got one file with more than 120+ million records(35 GB in size). I have to extract some relevant data from file based on some parameter and generate other output file. What will be the besat and fastest way to extract the ne file. sample file format :--... (2 Replies)
Discussion started by: learner16s
2 Replies

10. Shell Programming and Scripting

Need to Extract Data From 94000 records

i have a input file which does not have a delimiter All i Need to do is to identify a line and extract the data from it and run the loop again and need to ensure that it was not extracted earlier Input file ------------ abcd 12345 egfhijk ip 192.168.0.1 CNN.com abcd 12345 egfhijk ip... (12 Replies)
Discussion started by: vasimm
12 Replies
Login or Register to Ask a Question