👤
Home Man
Search
Today's Posts
Register

If you're not sure where to post a Unix or Linux question, post it here. All unix and Linux beginners welcome in this forum!

find pattern matches in consecutive lines in certain fields-awk

Tags
awk, patterns in consecutive lines, solved

👤 Login to reply

 
Thread Tools Search this Thread
# 1  
Old 1 Week Ago
find pattern matches in consecutive lines in certain fields-awk

I have a text file with many thousands of lines, a small sample of which looks like this:

InputFile:
Code:
PS002,003 D                  -1   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  509     0
PS002,003 PSQ                 0   1  7 18  1  0 -1    1  1  3 -1    -1   1   1  -1      -1      -1      -1    0  501     0
PS002,003 XNQ                 0   2 -1 -1 -1  5 -1   -1 -1  3  2     1   2   0  -1      -1      -1      -1   -1   -1    -1
PS002,003 HWN=                0   7 -1 -1 -1 -1 -1   -1  3  3  2    -1   7   2   2      -1      -1      -1    0  503     0
           * 0 -1 512 1 411 0 0 .q 4 LineNr 5 ClauseNr 1: 1: 3: 131: 0 0 SentenceNr 3 TxtType: ?Q      Pargr: 12 ClType:xYq0
           * 0 -2 111 1 411 0 0 .. 3 LineNr 10 ClauseNr 1: 1: 4: 131: 0 0 SentenceNr 6 TxtType: ?       Pargr: 1 ClType:xYq0
PS002,005 W                   0   6 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  509     0
PS002,005 B                   0   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   5   0  -1      -1      -1      -1   -1   -1    -1
PS002,005 XM>                 0   2 -1 -1 -1 11 -1   -1 -1  1  1     1   2   0  -1      -1      -1      -1   -1   -1    -1
PS002,005 H                   0   7 -1 -1 -1 -1 -1   -1  3  1  2    -1   7   5   2      -1      -1      -1    0  505     0
PS002,005 DLX                 0   1  5 18  1  0 -1    1  3  1  2    -1   1   1  -1      -1      -1      -1    0  501     0
PS002,005 >NWN                0   7 -1 -1 -1 -1 -1   -1  3  3  2    -1   7   7   2      -1      -1      -1    0  503     0
PS012,004 >BD                 0   1  5 15  1  0 -1    1  3  1  2    -1   1   1  -1      -1      -1      -1    0  501     0
PS012,004 MRJ>                0   3 -1 -1 -1  1 -1   -1 -1  0  0     2   3   3   2      -1      -1      -1    0  502     0
PS012,004 KL                  0   2 -1 -1 -1  1 -1   -1 -1  0  0     1   2   0  -1      -1      -1      -1   -1   -1    -1
PS012,004 HJN                 0   7 -1 -1 -1 -1 -1   -1  3  3  1    -1   7   2   2      -1      -1      -1    0  503     0
PS012,004 SP>                 0   2 -1 -1 -1 12 -1   -1 -1  3  1     3   2   0  -1      -1      -1      -1   -1   -1    -1
PS012,004 PLG                 0   1  6 18  1 12 -1   62 -1  3  1     3  13  -2   2      -1      -1      -1  -11  500     0

What I would like to do is that if a given line meets the conditions $16=="0" && $22=="-1" and the immediately following line has $22=="503" && $4=="7" && $16=="2" then print every set of these two consecutive lines.

Desired Output:
Code:
PS002,003 XNQ                 0   2 -1 -1 -1  5 -1   -1 -1  3  2     1   2   0  -1      -1      -1      -1   -1   -1    -1
PS002,003 HWN=                0   7 -1 -1 -1 -1 -1   -1  3  3  2    -1   7   2   2      -1      -1      -1    0  503     0
PS012,004 KL                  0   2 -1 -1 -1  1 -1   -1 -1  0  0     1   2   0  -1      -1      -1      -1   -1   -1    -1
PS012,004 HJN                 0   7 -1 -1 -1 -1 -1   -1  3  3  1    -1   7   2   2      -1      -1      -1    0  503     0

Thus far I have tried various revisions of the following awk code which has gotten me fairly close:

Code:
awk '$16=="0" && $22=="-1"{f=$0; f++; next} $22=="503" && $4=="7"{n=$0} {print f"\n"n}' InputFile

Nevertheless, I continue to not be able to figure out how to get this to work. I would very much appreciate any help to get this one-liner to work as desired. Thanks!

Last edited by jvoot; 1 Week Ago at 07:23 PM.. Reason: Left off one condition to arrive at desired output.
# 2  
Old 1 Week Ago
I'm not getting exactly your desired output, but something to start with:
Code:
awk '$16==0 && $22==-1 {l=$0;next} l && $22==503 && $4==7 {print l ORS $0;l=""}' myFile

produces:
Code:
PS002,003 XNQ                 0   2 -1 -1 -1  5 -1   -1 -1  3  2     1   2   0  -1      -1      -1      -1   -1   -1    -1
PS002,003 HWN=                0   7 -1 -1 -1 -1 -1   -1  3  3  2    -1   7   2   2      -1      -1      -1    0  503     0
PS002,005 XM>                 0   2 -1 -1 -1 11 -1   -1 -1  1  1     1   2   0  -1      -1      -1      -1   -1   -1    -1
PS002,005 >NWN                0   7 -1 -1 -1 -1 -1   -1  3  3  2    -1   7   7   2      -1      -1      -1    0  503     0
PS012,004 KL                  0   2 -1 -1 -1  1 -1   -1 -1  0  0     1   2   0  -1      -1      -1      -1   -1   -1    -1
PS012,004 HJN                 0   7 -1 -1 -1 -1 -1   -1  3  3  1    -1   7   2   2      -1      -1      -1    0  503     0


Last edited by vgersh99; 1 Week Ago at 07:42 PM..
The Following User Says Thank You to vgersh99 For This Useful Post:
jvoot (1 Week Ago)
# 3  
Old 1 Week Ago
Thank you so much vgersh99. Indeed, in the verbal parameters of my desired output I left out one condition (viz., $16=="2"). I have since fixed my original post. Thanks for pointing that out to me and the help with the code.

A slight adjustment to the code you offered reached the desired output as indicated in the original post.

Code:
awk '$16==0 && $22==-1 {l=$0;next} l && $22==503 && $4==7 && $16=="2"{print l ORS $0;l=""}' InputFile


Last edited by jvoot; 1 Week Ago at 07:26 PM..
The Following User Says Thank You to jvoot For This Useful Post:
vgersh99 (1 Week Ago)
# 4  
Old 1 Week Ago
I always prefer to have a state variable and a store variable.
Code:
awk '
met==1 && $22=="503" && $4=="7" && $16=="2" {print save; print }
{ met=0 }
$16=="0" && $22=="-1" { save=$0; met=1 }
'

The { met=0 } clears the state, in order to only continue the search in the immediately following line.
The order 2. condition then 1. condition saves a next.
The Following User Says Thank You to MadeInGermany For This Useful Post:
jvoot (1 Week Ago)
# 5  
Old 1 Week Ago
What behavior do you want with the following input file?
Code:
PS012,004 SP>                 0   2 -1 -1 -1 12 -1   -1 -1  3  1     3   2   0  -1      -1      -1      -1   -1   -1    -1
PS012,004 PLG                 0   1  6 18  1 12 -1   62 -1  3  1     3  13  -2   2      -1      -1      -1  -11  500     0
PS012,004 HJN                 0   7 -1 -1 -1 -1 -1   -1  3  3  1    -1   7   2   2      -1      -1      -1    0  503     0

The Following User Says Thank You to MadeInGermany For This Useful Post:
jvoot (1 Week Ago)
# 6  
Old 1 Week Ago
In this case @MadeInGermany, I would not want any output to be generated given your proposed input file.

Maybe it would be helpful if I gave some more sample data from my input.

InputSample
Code:
        * 0 -4 110 1 511 0 0 .. 4 LineNr 11 ClauseNr 1: 1: 3: 111: 0 0 SentenceNr 5 TxtType: Q       Pargr: 1 ClType:ZYqX
PS016,004 D                  -1   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   6   6  -1      -1      -1      -1    0  509     0
PS016,004 L>                  0  11 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1  11  11  -1      -1      -1      -1    0  510     0
PS016,004 NQJ                 0   1  4 18  1  0 -1    1  1  1 -1    -1   1   1  -1      -1      -1      -1    0  501     0
PS016,004 NWQJ                0   2 -1 -1 -1  5 -1   -1 -1  3  2     1   2   0  -1      -1      -1      -1   -1   -1    -1
PS016,004 HWN=                0   7 -1 -1 -1 -1 -1   -1  3  3  2    -1   7   2   2      -1      -1      -1    0  503     0
PS016,004 MN                  0   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   5   0  -1      -1      -1      -1   -1   -1    -1
PS016,004 DM                  0   2 -1 -1 -1  1 -1   -1 -1  0  2     3   2   5   2      -1      -1      -1    0  505     0           
        * 0 -1 620 0 0 .. 10 LineNr 14 ClauseNr 1: 1: 4: 132: 0 0 SentenceNr 12 TxtType: Q       Pargr: 1 ClType:xQt0
PS017,005 SMK                 0   1  0 18 11  0 -1    2  2  1  2    -1   1   1  -1      -1      -1      -1    0  501     0
PS017,005 HLK>                0   2 -1 -1 -1 12 -1   -1 -1  3  1     1   2   0  -1      -1      -1      -1   -1   -1    -1
PS017,005 J                   0   7 -1 -1 -1 -1 -1   -1  1  1 -1    -1   7   2   2      -1      -1      -1    0  503     0
PS017,005 B                   0   5 -1 -1 -1 -1 -1   -1 -1 -1 -1    -1   5   0  -1      -1      -1      -1   -1   -1    -1
PS017,005 CBJL                0   2 -1 -1 -1  5 -1   -1 -1  3  2     1   2   0  -1      -1      -1      -1   -1   -1    -1
PS017,005 K                   0   7 -1 -1 -1 -1 -1   -1  2  1  2    -1   7   5   2      -1      -1      -1    0  504     0
        * 0 -3 122 1 11 0 0 .. 8 LineNr 15 ClauseNr 1: 1: 3: 102: 0 0 SentenceNr 13 TxtType: Q       Pargr: 1 ClType:ZQt0

Desired Output
Code:
PS016,004 NWQJ                0   2 -1 -1 -1  5 -1   -1 -1  3  2     1   2   0  -1      -1      -1      -1   -1   -1    -1
PS016,004 HWN=                0   7 -1 -1 -1 -1 -1   -1  3  3  2    -1   7   2   2      -1      -1      -1    0  503     0
PS017,005 HLK>                0   2 -1 -1 -1 12 -1   -1 -1  3  1     1   2   0  -1      -1      -1      -1   -1   -1    -1
PS017,005 J                   0   7 -1 -1 -1 -1 -1   -1  1  1 -1    -1   7   2   2      -1      -1      -1    0  503     0

Thus, when there is a line that meets the conditions:
Code:
$16=="0" && $22=="-1"

Check the immediately following line to see if it has the conditions:
Code:
$4=="7" && $16=="2" && $22=="503"

If both of these are met, then print both lines to output; else do nothing. Ideally, I would like to be able to use the code help I receive here as a kind of template to vary the conditions on the various fields to extract a wide range of data patterns. I attempted the code that you offered and while I have not checked its accuracy in detail due to the size of the output, it seems that at a first glance it worked perfectly.
👤 Login to reply

« Previous Thread | Next Thread »
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Using awk or sed to find a pattern that has lines before and after it ran_bon_78 Shell Programming and Scripting 6 05-26-2016 02:37 PM
Eliminate consecutive lines with the same pattern sarn_nat UNIX for Dummies Questions & Answers 2 09-16-2014 12:26 PM
[awk] find pattern, change next two lines origamisven Shell Programming and Scripting 3 04-15-2014 05:12 PM
Finding the same pattern in three consecutive lines in several files in a directory shoaibjameel123 UNIX for Dummies Questions & Answers 1 12-18-2013 05:44 AM
How to insert line with between two consecutive lines that match special pattern? jjnight Shell Programming and Scripting 6 05-21-2012 02:16 AM
awk to find pattern and add lines jpsingh Shell Programming and Scripting 4 01-20-2011 04:36 PM
NAWK to remove lines that matches a specific pattern sbhuvana20 Shell Programming and Scripting 8 12-23-2010 12:29 PM
merging of 2 consecutive lines in a file for a specific pattern novice_man Shell Programming and Scripting 1 12-15-2010 12:47 AM
Merge lines if pattern matches in ksh mnjx Shell Programming and Scripting 1 08-25-2010 11:55 AM
Merge lines from one file if pattern matches giannicello Shell Programming and Scripting 6 03-12-2010 12:54 PM


All times are GMT -4. The time now is 01:22 AM.

Unix & Linux Forums Content Copyrightę1993-2018. All Rights Reserved.
×
UNIX.COM Login
Username:
Password:  
Show Password