Awk: Performing "for" loop within text block with two files
I am hoping to pull multiple strings from one file and use them to search within a block of text within another file.
File 1
File 2
My hope was that when $1 of File 1 matches $1 in File 2, $0 in File 2 contains the string "<Co>", and $2 of File 1 matches a string *exactly* in File 2 on a line beginning with the word "Lexeme," then print.
Thus, my desired output would look like this:
With the following code I am able to am able to do two of the three criteria listed above, namely, I am able to match $1 of File1 with $1 of File2 and also when $0 of File 1 has the string "<Co>". However, I am having difficulty with the last criteria, viz., match $2 of File 1 with the exact string in File 2 when the lines begins with "Lexeme."
I have also tried:
I think there might be something wrong with the way that I'm defining the "B" array with $2 of File 1 or defining the "for" loop in the script. Thank you so much in advance for your help.
Last edited by jvoot; 01-22-2019 at 10:45 AM..
Reason: Highlighted problem areas in script example.
Since you have multiple lines in File 1 with the same $1 values and different $2 values, there are a lot of similarities between the awk code needed to solve your problem and the problem that cmccabe presented to us three days ago: awk to add text to each line of matching id
Have you looked at the awk code that was provided to help cmccabe in that thread to see if it might help you solve your problem?
This User Gave Thanks to Don Cragun For This Post:
I read through that thread Don and while I must admit that I do not exactly follow all of it, it seems to be a bit different than what I am asking here. My problem is not so much dealing with repeated values, but rather getting that "for" loop to work in my awk script.
To isolate the problem further, I am having trouble getting $2 in File 1 to precisely match a string in File 2 on lines that begin with "Lexeme" (~/ ^L/) giving me fits. I have highlighted the two lines of code in both of the example *.awk scripts with red font. In principle something similar to awk 'FNR==NR{B[$2]; next}{for (b in B) if ($0 ~b) print} ' File[12] should do the trick, but relative to this particular issue: (A) I'm having difficulty getting something like that to work in my script; and (B) this does not produce exact matches, but rather treats the values of $2 in File 1 as strings (rather than say $2 ~/^pattern$/). Thus for the line in File 1 that reads PS005,007 ML I'll get matches such as MLL> MLT> TQML, etc.
Hopefully that hones in on my particular impasse that I'm experiencing.
You mean loop through the fields and each lookup in the array, like this?
Also, do you want to search <Co> in all the records? Or only in the records that matched with $1? then it should be
or (again) loop through the fields and compare each field:
This User Gave Thanks to MadeInGermany For This Post:
The suggestions MadeInGermany provided you will work for your stated requirements.
But, when I look at you examples it seems that your requirements might be more stringent that what you have stated. If we look at your sample File 1:
and the output you say you are trying to produce:
I note that each of the selected output line groups does not just have a line 1 $1 value that is in your A[] array and a word in a line that starts with Lexeme that is in your B[] array; it has a matched pair where the word matched in the Lexeme line had to be from a line in File 1 that had a $1 value that matched $1 in that 1st line.
The requirements you stated do not require that both of those values found in a group of lines in File 2 come from a single line in File 1. But, in each of your sample output groups of lines, both values came from the same line in File 1.
Am I reading too much into your example? Or are your requirements more stringent than what you stated.
If you do want the more stringent requirements (and I am correct in assuming that the $1 value from File 1 is supposed to appear only as $1 in the first line of a group of lines in File 2 and the $2 from that same line in File 1 is required to appear in a line of a group of lines in File 2 starting with Lexeme, then you might want something more like:
These 2 Users Gave Thanks to Don Cragun For This Post:
... ... ...
Don, while your code seems to address well the thrust of what I'm going for, but must admit that I have to study it much more to "get it." I have attempted to add a counter to the opening array with input from File 1, but to no avail. I would appreciate any help one could offer to address this latest wrinkle.
Hi jvoot,
The code in the other thread I pointed to in post #2 in this thread had to keep a counter of the number of lines in a file that had the same keys. As long as you treat both fields of a line from File 1 as a single key (the two subscripts I used in the Keys[] array), it should correctly deal with the problem you were having in your code. Your problem seems to be caused by the fact that you're using two arrays (A[], and B[]) instead of just one array to keep track of the fact that you need an entry from $1 in File 1 and an entry from $2 in File 1 that come from the same line.
Did you try running the code I suggested in post #5? Didn't it give you the output you want? If you tried the different portion of File 2 you showed us in post #6, didn't the code I suggested in post #5 still give you the output you wanted?
Since you have changed the code you originally showed us (based on MadeInGermany's suggestions) without showing us your modified code, I can't comment on what else might need to be done to make it work with your updated requirements.
Note that since your original sample File 2 contents included the "record":
which contains two lines starting with Cl and two lines matching $1=="Lexeme", I chose to make my code trigger off of the empty line between records instead of off of the ClauseType keyword that appears on multiple lines in some of your records. That may have made my code more difficult to understand and it led me to create a function to print selected output "record"s since I assume that there is no empty line after the last "record" in File 2. Calling that function when NF==0 happens on the empty line between records and calling that function in the END clause handles the case of the last record in an input file not being followed by an empty line.
If my code does what you want with your new sample input file, I'll be happy to try to explain how it works. If it doesn't work for you, then I still don't understand what you're trying to do and I won't be able to help you until I can understand requirements (i.e. specifications) for the software you're trying to write.
We could also look at the code Scrutinizer suggested, but we'd have to make it a little more complex to be sure that it looks for "<Co>" on the first line (and only on the first line) of each multi-line record and to be sure that it only looks for a match for field 2 from File 1 on lines that have "Lexeme" as the first word on a line (his code is currently looking for a match on any word in the entire multi-line record; not just on the "Lexeme" line).
Please understand that we know you have a non-computer science background and we respect that. But to write software that does what you want it to, a computer expects extremely picky details about the data it is asked to process to be clearly specified. If you can't do that, a computer will be very happy giving you results that match what you said you wanted instead of results that match what you really want. I tried to point out a couple of cases where what you said you wanted might not be what you really want, but only you can tell us whether some fields can appear anywhere or only in certain places, whether lines containing "Lexeme" can appear more than once (as in the example from you sample input above) or can only appear as the 2nd line of a multi-line "record" in your real data, etc., etc., etc.
P.S. If you'd like to play around with genes to find a cure for Type 1 diabetes, I'll be happy to write awk scripts for you to help you reach your goal.
These 3 Users Gave Thanks to Don Cragun For This Post:
Hi Team. I am trying to execute a simple for loop within an awk but its giving a different result. Below is the main code:
awk '{for(i=1;i<=6;i++) print $i}'The result should be 1 2 3 4 5 6 but its not giving this result. Can someone please help? (3 Replies)
Hi,
I'm using Linux and bash shell. I have a file (F1.txt) with contents like
Table1 Column1 123abc
Table1 Column2 xyz
Table2 Column1 543
Now, I would like to get the output as
UPDATE Table1 SET Column1='123abc';
UPDATE Table1 SET Column2='xyz';
UPDATE Table2 SET Column1='543';... (3 Replies)
logs:
"/home/abc/public_html/index.php"
"/home/abc/public_html/index.php"
"/home/xyz/public_html/index.php"
"/home/xyz/public_html/index.php"
"/home/xyz/public_html/index.php"
how to use "cut" or "awk" or "sed" to get the following result:
abc
abc
xyz
xyz
xyz (8 Replies)
Hello everyone
Sorry I have to add another sed question. I am searching a log file and need only the first 2 occurances of text which comes after (note the space) "string " and before a ",". I have tried
sed -n 's/.*string \(*\),.*/\1/p' filewith some, but limited success. This gives out all... (10 Replies)
Hi everyone,
I've been thinking and trying/changing all day long the below code, maybe some awk expert could help me to fix the for loop I've thought,
I think I'm very close to the correct output.
file1 is:
<boxes content="Grapes and Apples">
<box No.="Box MT. 53">
<quantity... (8 Replies)
Hi,
I have line in input file as below:
3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL
My expected output for line in the file must be :
"1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL"
Can someone... (7 Replies)
Hi All,
I have working (Perl) code to combine 2 input files into a single output file using the join function that works to a point, but has the following limitations:
1. I am restrained to 2 input files only.
2. Only the "matched" fields are written out to the "matched" output file and... (1 Reply)
Hi Friends,
Can any of you explain me about the below line of code?
mn_code=`env|grep "..mn"|awk -F"=" '{print $2}'`
Im not able to understand, what exactly it is doing :confused:
Any help would be useful for me.
Lokesha (4 Replies)
First off, I have been learning AWK by trial and error over the last week or so, and there are some gaps in my basic understanding of the language.
Here is my situation: I am coding and outputting results from an experiment I conducted in Psyscope, which has all been compiled into a master file.... (2 Replies)