UNIX for Dummies Questions & Answers

Hi,

I have the following txt file List_With_Duplicates.txt;

Code:

a,1,1
b,3,4
c,5,2
d,6,1
e,3,3
f,3,7

When I run the command
awk -F ',' '{if($2==$3){print $1","$2","$3}}' List_With_Duplicates.txt I get the following output;

Code:

a,1,1
e,3,3

This works! as I've compared the 2nd & 3rd column for duplicates and printed the lines which match. However, say I have the following file List_No_Duplicates.txt;

Code:

b,3,4
c,5,2
d,6,1
f,3,7

This file has no duplicates. When I run the command awk -F ',' '{if($2==$3){print $1","$2","$3} else {print "No Duplicates"}}' List_No_Duplicates.txt I get the following output;

Code:

No Duplicates
No Duplicates
No Duplicates
No Duplicates

However, I just want 1 statement saying No Duplicates rather than printing the same statement for each line.

Its probably really simple explanation but I just cant get anything to work. I've tried using False & True operators and an exit statement but nothing works.

Would appreciate any help with this.
Thanks

You are telling awk to either print the line that contains duplicates or print the string No Duplicates for each input line you read from the file. Try this instead:

Code:

awk '
BEGIN {	FS = OFS = ","
}
$2 == $3 {
	print
	f++
}
END {	if(f == 0) print "No Duplicates"
}' List_No_Duplicates.txt

If someone wants to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk or nawk.

Thanks for the quick reply.

When I run this on a txt file with no duplicates it works great. But when I run it on a file with duplicates I get the following output;

Code:

 
 awk 'BEGIN { FS = OFS = "," } $2 == $3 { print f++ } END {if(f == 0) print "No Duplicates"}' List_With_Duplicates.txt

Code:

 
 0
 1

Preferred output is;

Code:

 
 a,1,1
 e,3,3

The code seems to output the iteration number of 'f' rather than the value. Is there an amendment that can be made to change this?

Thanks

Moderator's Comments:

Please use CODE tags (not ICODE tags) for multiline sample code, input, and output!

---------- Post updated at 11:41 AM ---------- Previous update was at 11:23 AM ----------

I tried;

Code:

 
 awk 'BEGIN { FS = OFS = "," } $2 == $3 { print f++ $1,$2,$3 } END {if(f == 0) print "No Duplicates"}' List_With_Duplicates.txt

And got the following output;

Code:

 
 0a,1,1
 1e,3,3

Is there a way of hiding the iteration number? When I do the same command but without the f++ the 'Non Duplicate' string is output also.

Thanks

Moderator's Comments:

Please use CODE tags (not ICODE tags) for multiline sample code, input, and output!

---------- Post updated at 11:42 AM ---------- Previous update was at 11:41 AM ----------

Sorry, was using wrong tabs

I gave you a working, easy to read and understand multi-line awk script. You converted it into a harder to read, non-working single-line awk script.

If you don't understand awk syntax well enough to convert it correctly, please at least try the code you are given before changing it and complaining that the modifications you made to the code you were given don't work.

I personally prefer multi-line scripts because they are easier to maintain. If you insist on a single line script try:

Code:

awk -F, '$2==$3{print;f++}END{if(!f)print "No Duplicates"}' OFS=',' List_With_Duplicates.txt

Wasn't complaining, just new to this and trying different things.
Appreciate your help

For your problem in post#3, drop the f++ in the print statement. (In fact, you could drop everything from it as an print on its own will print $0, which is the entire line)

I think your main impediment is you need to know how awk script work:

Every awk-script consists of three parts (although each part is optional): the first part is marked with BEGIN and is executed before the first line of the input is read. The second part is not marked with a name and is executed every time a line of input is read. The last part is named END and is executed after the last line of input is read.

For instance:

Code:

awk 'BEGIN  {
               printf( "Here we start.\n" );
            }
            {
               print;
            }
     END    {
               printf( "Here it ends.\n" );
            }' /path/to/input

This will print the input file unaltered (the single "print" command in the middle does that), surrounded by "Here we start" at the top (the effect of the BEGIN clause) and "Here it ends" at the bottom (the effect of the END clause).

Your original script printed "no duplicates" several times because the process - deciding upon printing it or not, then printing it eventually - was done for every line over and over. Dons script worked because he put that into the END-part and just manipulated a flag without printing anything in the main part.

You should be able with this to change his script into doing whatever you want it to do.

I hope this helps.

bakunin