awk - Print lines if only matching key is found


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers awk - Print lines if only matching key is found
# 1  
Old 02-01-2015
awk - Print lines if only matching key is found

I am looking to move matching lines (01 - 07) from File1 and 77 tab the matching string from File2, to File3.txt. I am almost done but

- Currently, script is not printing lines to File3.txt in order.

Thanks a lot.
Any help is appreciated.

Script I am using:
Code:
awk 'FNR == NR && ! /^[[:space:]]*$/ { key = substr($0,1,8); a[key] = $0; next }
$1 == "01" { if (key != 0)
             {
                 if (key in a)
                 {
                     printf("77\t%s\n", a[key])
                     delete a[key]
                 }
             }
             key = $4 $3 $2
           }
{ if (key in a) print $0 } ' \
     File2.txt File1.txt > File3.txt


File1.txt

Code:
 01  89  68  5000
    02  83  11
    04  83  9   02
    03  83  00
    06  83  00
    07  83  11  RT0429
    07  83  88  FS0547
    01  44  73  8800
    02  44  73
    04  44  73   02
    03  44  73
    06  44  73
    07  44  11  RT  0789

File2.txt

Code:
    50006889RT0429 NARD /3010  /E     /C A87545457          /  //                ///11        ///
    51002387 NARD /3000  /E     /S N054896334IV          /  //                ///11        ///

Current Output (File3.txt)

Code:
    07  83  11  RT0429
    77	50006889RT0429 NARD /3010  /E     /C A87545457          /  //                ///11        ///
    01  44  73  8800
    02  44  73
    04  44  73   02
    03  44  73
    06  44  73

Desired Output (File3.txt)

I need complete set from line 01 to 07 including any addition 07 lines in case the key is matched from File1, followed by 77 tab and matched string from File2.

Code:
 01  89  68  5000
    02  83  11
    04  83  9   02
    03  83  00
    06  83  00
    07  83  11  RT0429
    07  83  88  FS0547
    77	50006889RT0429 NARD /3010  /E     /C A87545457          /  //                ///11        ///


Last edited by High-T; 02-03-2015 at 04:22 PM..
# 2  
Old 02-02-2015
Using the term URGENT, esp. in bold red, is deprecated and sort of risky in these fora. Try to avoid it in the fututre.

If I remember correctly, you have been supplied with solutions to very similar problems in various threads in the recent past which you tried to condense into your script. So I think you're not too far away. Why don't you sit back and stare a while at your code snippet? The solution will jump to your eyes. (Little hint: Look at your if (key in a) command and what you are printing there...) And, you might want to add an else branch to print the non-matching records.
# 3  
Old 02-02-2015
RudiC,
Note that High-T did get an infraction for using URGENT and double-posting in another thread. I decided that one infraction was required, but didn't think two infractions were necessary at this time.

I also believe that High-T needs to look more closely at the working code he has been given and the new requirements presented in this thread and take a little time to THINK about what different needs to be done to satisfy those new requirements.

High-T,
In the 5 days since you joined this forum, you have submitted 6 threads all based on a very similar awk script. After fixing your code four times to do your tasks, you have submitted two more threads for this problem and say that we volunteers need to URGENTLY fix your code even though it looks like you are not willing to apply suggestions we have suggested in the four threads we solved for you. We are volunteers; not your paid programming staff. You have no right to expect volunteers here who have helped you with other problems (or new volunteers who may be willing to help you) to put aside other things they might have planned to do this weekend to help you meet your deadlines.

Please show us what in your current code is intended to move any lines out of File1.txt.

Please tell us what the code:
Code:
    $1=="01" &&
                FNR>1       {keyfound?"77\t" keyfound:"99"}

is intended to do. It is a nice expression, but it produces no output and sets no variables.

Given that you never use the variable lines except in these two statements:
Code:
        lines = ""
           and
    lines = lines $0 "\n"

please tell us what this variable is intended to do. Are you trying to use this to save up lines to be printed when you get the end of a set if a match was found? That would make sense; but that isn't what you're doing. Are you trying to save up lines to be reprocessed on a later run if no match is found? That would make sense, but that isn't what you're doing.

Given that File2.txt has leading spaces and that the length of the 1st field in that file is variable length (not a constant 14 characters), it would seem that:
Code:
    		a[substr($0,1,14)]=$0

might need to be something more like:
Code:
    		a[$1]=$0

The key used in your other threads could be set when you found a record starting with 01. In this thread you can't set the key until you have found both a 01 record and a 07 record. And, you can have more than one key for a single set of records. Therefore, you need to accumulate an entire set of records that may need to be printed until you have found the entire set of keys for that set and have determined whether there are any matching keys. If you look closely at your output, you will note that you haven't printed lines out of order; instead, you have printed lines starting from when you finished a matching key when you found a record starting with 07 and continued printing lines until you find another 07 record that doesn't match. BUT, I can't see how you would get the output you showed us with the code you've shown us?????

Hopefully, these comments will help you produce a working script.
This User Gave Thanks to Don Cragun For This Post:
# 4  
Old 02-03-2015
Quote:
Originally Posted by RudiC
Using the term URGENT, esp. in bold red, is deprecated and sort of risky in these fora. Try to avoid it in the fututre.

If I remember correctly, you have been supplied with solutions to very similar problems in various threads in the recent past which you tried to condense into your script. So I think you're not too far away. Why don't you sit back and stare a while at your code snippet? The solution will jump to your eyes. (Little hint: Look at your if (key in a) command and what you are printing there...) And, you might want to add an else branch to print the non-matching records.

Thanks RudiC. I have realized that I should not be using this term as I am getting help for free. I have reviewed the policy and I would be careful next time. Thanks for all your help.

---------- Post updated at 11:23 AM ---------- Previous update was at 11:21 AM ----------

Quote:
Originally Posted by Don Cragun
RudiC,
Note that High-T did get an infraction for using URGENT and double-posting in another thread. I decided that one infraction was required, but didn't think two infractions were necessary at this time.

I also believe that High-T needs to look more closely at the working code he has been given and the new requirements presented in this thread and take a little time to THINK about what different needs to be done to satisfy those new requirements.

High-T,
In the 5 days since you joined this forum, you have submitted 6 threads all based on a very similar awk script. After fixing your code four times to do your tasks, you have submitted two more threads for this problem and say that we volunteers need to URGENTLY fix your code even though it looks like you are not willing to apply suggestions we have suggested in the four threads we solved for you. We are volunteers; not your paid programming staff. You have no right to expect volunteers here who have helped you with other problems (or new volunteers who may be willing to help you) to put aside other things they might have planned to do this weekend to help you meet your deadlines.

Please show us what in your current code is intended to move any lines out of File1.txt.

Please tell us what the code:
Code:
    $1=="01" &&
                FNR>1       {keyfound?"77\t" keyfound:"99"}

is intended to do. It is a nice expression, but it produces no output and sets no variables.

Given that you never use the variable lines except in these two statements:
Code:
        lines = ""
           and
    lines = lines $0 "\n"

please tell us what this variable is intended to do. Are you trying to use this to save up lines to be printed when you get the end of a set if a match was found? That would make sense; but that isn't what you're doing. Are you trying to save up lines to be reprocessed on a later run if no match is found? That would make sense, but that isn't what you're doing.

Given that File2.txt has leading spaces and that the length of the 1st field in that file is variable length (not a constant 14 characters), it would seem that:
Code:
    		a[substr($0,1,14)]=$0

might need to be something more like:
Code:
    		a[$1]=$0

The key used in your other threads could be set when you found a record starting with 01. In this thread you can't set the key until you have found both a 01 record and a 07 record. And, you can have more than one key for a single set of records. Therefore, you need to accumulate an entire set of records that may need to be printed until you have found the entire set of keys for that set and have determined whether there are any matching keys. If you look closely at your output, you will note that you haven't printed lines out of order; instead, you have printed lines starting from when you finished a matching key when you found a record starting with 07 and continued printing lines until you find another 07 record that doesn't match. BUT, I can't see how you would get the output you showed us with the code you've shown us?????

Hopefully, these comments will help you produce a working script.
Thanks Don. You have been really helpful. I apologize for duplicate posting. I posted a new question and then I updated an old one but they were the same I admit. I promise I would respect the forum policy in future. Again thanks for your time and efforts.

---------- Post updated 02-03-15 at 10:20 AM ---------- Previous update was 02-02-15 at 11:23 AM ----------

Tried everything but the output is not coming in order Smilie

Last edited by Scrutinizer; 02-02-2015 at 12:46 PM.. Reason: Removed accidental double post..
# 5  
Old 02-03-2015
Quote:
Originally Posted by High-T
---------- Post updated 02-03-15 at 10:20 AM ---------- Previous update was 02-02-15 at 11:23 AM ----------

Tried everything but the output is not coming in order Smilie
Show us what you have tried.
# 6  
Old 02-03-2015
And, you have received several hints on what to modify in your script to make it work.
# 7  
Old 02-03-2015
I have updated the question with the script.

Code:
awk 'FNR == NR && ! /^[[:space:]]*$/ { key = substr($0,1,8); a[key] = $0; next }
$1 == "01" { if (key != 0)
             {
                 if (key in a)
                 {
                     printf("77\t%s\n", a[key])
                     delete a[key]
                 }
             }
             key = $4 $3 $2
           }
{ if (key in a) print $0 } ' \
     File2.txt File1.txt > File3.txt

 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to print matching lines in files that meet critera

In the tab delimited files below I am trying to match $2 in file1 to $2 of file2. If a match is found the awk checks $3 of file2 and if it is greater than 40% and $4 of file2 is greater than 49, the line in file1 is printed. In the desired output line3 of file1 is not printed because $3 off file2... (9 Replies)
Discussion started by: cmccabe
9 Replies

2. Shell Programming and Scripting

Find key pattern and print selected lines for each record

Hi, I need help on a complicated file that I am working on. I wanted to extract important info from a very huge file. It is space delimited file. I have hundred thousands of records in this file. An example content of the inputfile as below:- ## ID Ser402 Old; 23... (2 Replies)
Discussion started by: redse171
2 Replies

3. UNIX for Dummies Questions & Answers

awk - (URGENT!) Print lines sort and move lines if match found

URGENT HELP IS NEEDED!! I am looking to move matching lines (01 - 07) from File1 and 77 tab the matching string from File2, to File3.txt. I am almost done but - Currently, script is not printing lines to File3.txt in order. - Also the matching lines are not moving out of File1.txt ... (1 Reply)
Discussion started by: High-T
1 Replies

4. Shell Programming and Scripting

How to print few lines before and after matching word is found suing grep?

Hi, here are few lines present in the logs. I want to grep on Error and print few lines before and after Error word is found line1 Line2 Line3 Error Line4 Line5 Line6 Line7 I want the output to be Line2 Line3 Error Line5 (1 Reply)
Discussion started by: arghadeep adity
1 Replies

5. Shell Programming and Scripting

Compare file1 for matching line in file2 and print the difference in matching lines

Hello, I have two files file 1 and file 2 each having result of a query on certain database tables and need to compare for Col1 in file1 with Col3 in file2, compare Col2 with Col4 and output the value of Col1 from File1 which is a) not present in Col3 of File2 b) value of Col2 is different from... (2 Replies)
Discussion started by: RasB15
2 Replies

6. Shell Programming and Scripting

awk to print all lines after a pattern is found

Is there a way with aw to print all lines after a string is found There is a file like this ....... ........ 2012/19/11 :11.58 PM some data lne no date 2012/19/11 :11.59 PM some other data 2012/20/11 :12.00 AM some other data some line without dates some more lines without dates... (8 Replies)
Discussion started by: swayam123
8 Replies

7. Shell Programming and Scripting

awk print non matching lines based on column

My item was not answered on previous thread as code given did not work I wanted to print records from file2 where comparing column 1 and 16 for both files find rows where column 16 in file 1 does not match column 16 in file 2 Here was CODE give to issue ~/unix.com$ cat f1... (0 Replies)
Discussion started by: sigh2010
0 Replies

8. Shell Programming and Scripting

print lines from a file containing key word

i have a file containing over 1 million records,and i want to print about 300,000 line containing a some specific words. file has content. eg 1,rrt,234 3,fgt,678 4,crf,456 5,cde,drt 6,cfg,123 and i want to print the line with the word fgt,crf this is just an example,my file is so... (2 Replies)
Discussion started by: tomjones
2 Replies

9. Shell Programming and Scripting

Print lines matching value(s) in other file using awk

Hi, I have two comma separated files. I would like to see field 1 value of File1 exact match in field 2 of File2. If the value matches, then it should print matched lines from File2. I have achieved the results using cut, paste and egrep -f but I would like to use awk as it is efficient way and... (7 Replies)
Discussion started by: SBC
7 Replies

10. Shell Programming and Scripting

AIX equivalent to GNU grep's -B and -A [print lines after or before matching lines]

Hi folks I am not allowed to install GNU grep on AIX. Here my code excerpt: grep_fatal () { /usr/sfw/bin/gegrep -B4 -A2 "FATAL|QUEUE|SIGHUP" } Howto the same on AIX based machine? from manual GNU grep ‘--after-context=num’ Print num lines of trailing context after... (4 Replies)
Discussion started by: slashdotweenie
4 Replies
Login or Register to Ask a Question