Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk to match field4 and field5, field6 and field7, field8 and field9 and so on.

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk to match field4 and field5, field6 and field7, field8 and field9 and so on.
# 1  
Old 09-22-2015
awk to match field4 and field5, field6 and field7, field8 and field9 and so on.
Hi all,

I want to match field4 and field5, field6 and field7, field8 and field9 and so on using the following input file.

Condition:
if field4=field5, print field4
if field4=NN and field5≠NN, print field5
if field5=NN and field4≠NN, print field4
Repeat the above 3 condition for field6&7, field8&9 and so on.

Input:
Code:
loc01	1000560	G	NN	AG	NN	NN	NN	AG	NN	NN	NN	NN
loc02	1001612	C	NN	NN	NN	NN	NN	NN	NN	NN	CT	NN
loc03	1001797	C	AC	NN	NN	NN	NN	NN	NN	NN	AC	NN
loc04	1004204	G	NN	AG	NN	NN	NN	NN	AG	NN	AG	NN
loc05	1005484	C	NN	NN	CT	NN	NN	NN	NN	NN	NN	NN
loc06	1005486	C	NN	NN	CT	NN	NN	NN	NN	NN	NN	NN
loc07	1005499	G	NN	NN	AG	NN	NN	NN	NN	NN	NN	NN
loc08	100707	T	NN	NN	NN	NN	NN	NN	NN	NN	NN	NN
loc09	100937	C	NN	NN	NN	NN	NN	NN	CT	NN	NN	NN
loc10	100949	C	NN	NN	NN	NN	NN	NN	CT	NN	NN	NN
loc11	101063	T	NN	NN	NN	NN	NN	CT	NN	NN	CT	NN
loc12	1010912	G	AG	AG	AG	NN	AG	NN	AG	NN	NN	AG
loc13	1011214	A	NN	NN	NN	AG	NN	NN	NN	NN	AG	AG
loc14	1011673	T	NN	CT	CT	NN	CT	NN	CT	NN	CT	CT
loc15	1011981	A	NN	NN	NN	NN	NN	AG	NN	NN	NN	NN
loc16	1012439	C	NN	NN	NN	NN	NN	NN	NN	NN	NN	CT
loc17	1012718	T	NN	NN	CT	NN	CT	NN	CT	NN	CT	NN
loc18	1015524	A	NN	NN	NN	NN	AT	AT	AT	NN	AT	NN
loc19	1023408	C	CG	NN	NN	NN	NN	CG	NN	NN	NN	NN
loc20	102483	A	NN	NN	NN	NN	NN	NN	NN	NN	NN	NN

Output file:
Code:
loc01	1000560	G	AG	NN	AG	NN	NN
loc02	1001612	C	NN	NN	NN	NN	CT
loc03	1001797	C	AC	NN	NN	NN	AC
loc04	1004204	G	AG	NN	NN	AG	AG
loc05	1005484	C	NN	CT	NN	NN	NN
loc06	1005486	C	NN	CT	NN	NN	NN
loc07	1005499	G	NN	AG	NN	NN	NN
loc08	100707	T	NN	NN	NN	NN	NN
loc09	100937	C	NN	NN	NN	CT	NN
loc10	100949	C	NN	NN	NN	CT	NN
loc11	101063	T	NN	NN	CT	NN	CT
loc12	1010912	G	AG	AG	AG	AG	AG
loc13	1011214	A	NN	AG	NN	NN	AG
loc14	1011673	T	CT	CT	CT	CT	CT
loc15	1011981	A	NN	NN	AG	NN	NN
loc16	1012439	C	NN	NN	NN	NN	CT
loc17	1012718	T	NN	CT	CT	CT	CT
loc18	1015524	A	NN	NN	AT	AT	AT
loc19	1023408	C	CG	NN	CG	NN	NN
loc20	102483	A	NN	NN	NN	NN	NN

I have tried the following code but it does not work:
Code:
awk '{for(i=4;i<=NF;i+=2)
     if($i==$i+1) new=$i; 
else if($i=="NN" || $i+1!="NN") new=$i+1; 
else if($i!="NN" || $i=="NN") new=$i; 
      print$1,$2,$3,new;}' input.txt

What have I done wrong?
Last edited by huiyee1; 09-22-2015 at 02:38 AM..
# 2  
Old 09-22-2015
Hi, here a corrected version of your script:
Code:
awk '{
printf("%s\t%s\t%s\t",$1,$2,$3)
for(i=4;i<=NF;i+=2){
   if($i==$(i+1)) new=$i;
   else if($i=="NN" || $(i+1)!="NN") new=$(i+1);
   else if($i!="NN" || $(i+1)=="NN") new=$i;
   printf("%s\t",new)}
print ""}' input.txt

This User Gave Thanks to disedorgue For This Post:
huiyee1
# 3  
Old 09-22-2015
Slight simplification:
Code:
awk '
        {printf "%s\t%s\t%s\t", $1, $2, $3
         for (i=4; i<=NF; i+=2) {if ($i==$(i+1) || $(i+1) == "NN") printf "%s\t", $i
                                 else   if ($i == "NN") printf "%s\t", $(i+1)
                                }
         print ""
        }
' file
loc01    1000560    G    AG    NN    AG    NN    NN    
loc02    1001612    C    NN    NN    NN    NN    CT    
loc03    1001797    C    AC    NN    NN    NN    AC    
loc04    1004204    G    AG    NN    NN    AG    AG    
loc05    1005484    C    NN    CT    NN    NN    NN    
loc06    1005486    C    NN    CT    NN    NN    NN    
loc07    1005499    G    NN    AG    NN    NN    NN    
loc08    100707     T    NN    NN    NN    NN    NN    
loc09    100937     C    NN    NN    NN    CT    NN    
loc10    100949     C    NN    NN    NN    CT    NN    
loc11    101063     T    NN    NN    CT    NN    CT    
loc12    1010912    G    AG    AG    AG    AG    AG    
loc13    1011214    A    NN    AG    NN    NN    AG    
loc14    1011673    T    CT    CT    CT    CT    CT    
loc15    1011981    A    NN    NN    AG    NN    NN    
loc16    1012439    C    NN    NN    NN    NN    CT    
loc17    1012718    T    NN    CT    CT    CT    CT    
loc18    1015524    A    NN    NN    AT    AT    AT    
loc19    1023408    C    CG    NN    CG    NN    NN    
loc20    102483     A    NN    NN    NN    NN    NN

This User Gave Thanks to RudiC For This Post:
huiyee1
# 4  
Old 09-23-2015
Hi disedorgue and RudiC,

I need to add in one more condition another input file as follows,
if field4≠field5 and field4≠NN and field5≠NN, print NN

Input2:
Code:
loc01	1000560	G	AA	AG	GC	GA	AC	AG	NN	NN	NN	NN
loc02	1001612	C	NN	NN	CT	CC	NN	NN	NN	NN	CT	NN
loc03	1001797	C	AC	NN	NN	NN	NN	NN	NN	NN	AC	NN
loc04	1004204	G	NN	AG	NN	NN	NN	NN	AG	NN	AG	NN
loc05	1005484	C	NN	NN	CT	NN	NN	NN	NN	NN	NN	NN
loc06	1005486	C	CT	CG	CT	NN	NN	NN	NN	NN	NN	NN
loc07	1005499	G	NN	NN	AG	NN	NN	NN	NN	NN	NN	NN
loc08	100707	T	NN	NN	NN	NN	NN	NN	NN	NN	NN	NN
loc09	100937	C	NN	NN	NN	NN	NN	NN	CT	NN	NN	NN
loc10	100949	C	NN	NN	NN	NN	NN	NN	CT	NN	NN	NN
loc11	101063	T	NN	NN	NN	NN	NN	CT	NN	NN	CT	NN
loc12	1010912	G	AG	AG	AG	NN	AG	NN	AG	NN	NN	AG
loc13	1011214	A	NN	NN	NN	AG	NN	NN	NN	NN	AG	AG
loc14	1011673	T	NN	CT	CT	NN	CT	NN	CT	NN	CT	CT
loc15	1011981	A	NN	NN	NN	NN	NN	AG	NN	NN	NN	NN
loc16	1012439	C	NN	NN	NN	NN	NN	NN	NN	NN	NN	CT
loc17	1012718	T	NN	NN	CT	NN	CT	NN	CT	NN	CT	NN
loc18	1015524	A	NN	NN	NN	NN	AT	AT	AT	NN	AT	NN
loc19	1023408	C	CG	NN	NN	NN	NN	CG	NN	NN	NN	NN
loc20	102483	A	AA	CC	CC	CT	NN	NN	NN	NN	NN	NN

Output2:
Code:
loc01	1000560	G	NN	NN	NN	NN	NN
loc02	1001612	C	NN	NN	NN	NN	CT
loc03	1001797	C	AC	NN	NN	NN	AC
loc04	1004204	G	AG	NN	NN	AG	AG
loc05	1005484	C	NN	CT	NN	NN	NN
loc06	1005486	C	NN	CT	NN	NN	NN
loc07	1005499	G	NN	AG	NN	NN	NN
loc08	100707	T	NN	NN	NN	NN	NN
loc09	100937	C	NN	NN	NN	CT	NN
loc10	100949	C	NN	NN	NN	CT	NN
loc11	101063	T	NN	NN	CT	NN	CT
loc12	1010912	G	AG	AG	AG	AG	AG
loc13	1011214	A	NN	AG	NN	NN	AG
loc14	1011673	T	CT	CT	CT	CT	CT
loc15	1011981	A	NN	NN	AG	NN	NN
loc16	1012439	C	NN	NN	NN	NN	CT
loc17	1012718	T	NN	CT	CT	CT	CT
loc18	1015524	A	NN	NN	AT	AT	AT
loc19	1023408	C	CG	NN	CG	NN	NN
loc20	102483	A	NN	NN	NN	NN	NN

I tried to use the following script modified from disedorgue, but i failed.
Code:
 awk '{
printf("%s\t%s\t%s\t",$1,$2,$3)
for(i=4;i<=NF;i+=2){
   if($i==$(i+1)) new=$i;
   else if($i=="NN" || $(i+1)!="NN") new=$(i+1);
   else if($i!="NN" || $(i+1)=="NN") new=$i;
   else if($i!=$(i+1) || $i!="NN" || $(i+1)!="NN") new=NN;
   printf("%s\t",new)} print ""}' input2.txt

What is the mistake I have made?

---------- Post updated 09-23-15 at 12:34 AM ---------- Previous update was 09-22-15 at 08:50 PM ----------

Hi all,

I figured it out using the following script:
Code:
awk '{
printf("%s\t%s\t%s\t",$1,$2,$3)
for(i=4;i<=NF;i+=2){
   if($i==$(i+1)) new=$i;
   else if($i=="NN" && $(i+1)!="NN") new=$(i+1);
   else if($i!="NN" && $(i+1)=="NN") new=$i; 
   else if($i!=$(i+1) && $i!="NN" && $(i+1)!="NN") new="NN";
   printf("%s\t",new)} print ""}' input2.txt

# 5  
Old 09-23-2015
different approach:
Code:
awk '
        {printf "%s\t%s\t%s\t", $1, $2, $3
         for (i=4; i<=NF; i+=2) {X=$i $(i+1)
                                 if (sub ("NN", _, X)) printf "%s\t", X
                                 else   if (gsub($i, "&", X) == 2) printf "%s\t", $i
                                        else printf "NN\t"
                                }
         print ""
        }
' file

# 6  
Old 09-23-2015
For fun:
Code:
$ awk '{
printf("%s\t%s\t%s\t",$1,$2,$3)
for(i=4;i<=NF;i+=2){
   ($i==$(i+1) || ($i!="NN" && $(i+1)=="NN")) ? new=$i : ($i=="NN" && $(i+1)!="NN") ? new=$(i+1) : ($i!=$(i+1) && $i!="NN" && $(i+1)!="NN") && new="NN"
   printf("%s\t",new)} print ""}' file

Regards
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to print match or non-match and select fields/patterns for non-matches

In the awk below I am trying to output those lines that Match between file1 and file2, those Missing in file1, and those missing in file2. Using each $1,$2,$4,$5 value as a key to match on, that is if those 4 fields are found in both files the match, but if those 4 fields are not found then missing... (0 Replies)
Discussion started by: cmccabe
0 Replies

2. Shell Programming and Scripting

awk to update file based on partial match in field1 and exact match in field2

I am trying to create a cronjob that will run on startup that will look at a list.txt file to see if there is a later version of a database using database.txt as the source. The matching lines are written to output. $1 in database.txt will be in list.txt as a partial match. $2 of database.txt... (2 Replies)
Discussion started by: cmccabe
2 Replies

3. Shell Programming and Scripting

awk to match field between two files and use conditions on match

I am trying to look for $2 of file1 (skipping the header) in $2 of file2 (skipping the header) and if they match and the value in $10 is > 30 and $11 is > 49, then print the line from file1 to a output file. If no match is foung the line is not printed. Both the input and output are tab-delimited.... (3 Replies)
Discussion started by: cmccabe
3 Replies

4. Shell Programming and Scripting

awk if match

Hi, This is the file content: #160814 20:43:00 server id 2 end_log_pos 169934694 Query thread_id=8927407 exec_time=0 error_code=0 use sun_final/*!*/; SET TIMESTAMP=1471207380/*!*/; DELETE FROM `top_pack` WHERE `top_pack`.`id` = 3023 Trying like:awk... (5 Replies)
Discussion started by: ashokvpp
5 Replies

5. Shell Programming and Scripting

Using awk for match and print

I have the need to match up the lat / lon from a fileA with the lat / lon and value from fileB. fileA is a small subset of fileB I have the following awk script but it prints out all the contents from fileB. I only need the matches. awk 'FNR==NR {A=$NF; next} {A=$NF} END{for(i in A) printf... (10 Replies)
Discussion started by: ncwxpanther
10 Replies

6. Shell Programming and Scripting

awk match help

Trying to match $1 of file2.txt with $1 of file 1.txt and output the entire line of the match. Thank you :) awk 'NR==FNR{A=$2; next} A {$2=$2 " " A}1' file1.txt file2.txt > output.txt file1.txt LMNA 285.195652 MZT1P1 166.852113 HFM1 129.847940 file2.txt LMNA PTPN11... (3 Replies)
Discussion started by: cmccabe
3 Replies

7. Shell Programming and Scripting

Better way to match a list in awk

Suppose I have a list of strings in a file called stringlist... string1 string2 ... stringn Suppose also that I have another file, or stdin, or whatever, and I want to use awk to see if some field in each record matches any string in stringlist. What I've been doing is using each string... (3 Replies)
Discussion started by: treesloth
3 Replies

8. UNIX for Dummies Questions & Answers

awk display the match and 2 lines after the match is found.

Hello, can someone help me how to find a word and 2 lines after it and then send the output to another file. For example, here is myfile1.txt. I want to search for "Error" and 2 lines below it and send it to myfile2.txt I tried with grep -A but it's not supported on my system. I tried with awk,... (4 Replies)
Discussion started by: eurouno
4 Replies

9. Shell Programming and Scripting

AWK match and print

I have thousands of tables compiled in a single txt document that I'm parsing with AWK. Scattered throughout the document in random sections I would like to parse out the sections that look like this: 1 Seq. Descrição do bem Tipo do bem Valor do bem (R$) 2 1 LOCALIZADO ANA RUA PESSEGO N 96... (3 Replies)
Discussion started by: daveyabe
3 Replies

10. UNIX for Advanced & Expert Users

dynamic match thru awk

hey , my i/p text looks like this, FILE_TYPE=01|FILE_DESC=Periodic|FILE_SCHDL_TYPE=Daily|FILE_SCHDL=|FILE_SCHDL_TIME=9:00am|RESULTS=B FILE_TYPE=02|FILE_DESC=NCTO|FILE_SCHDL_TYPE=Daily|FILE_SCHDL=|FILE_SCHDL_TIME=9:00am|RESULTS=M NOTE Look carefully for the position FILE_TYPE,FILE_DESC... (23 Replies)
Discussion started by: manas_ranjan
23 Replies
Login or Register to Ask a Question