Shell Programming and Scripting

I was reading this thread. It looks like a simpler way to say this is to only keep uniq lines based on field or column 1.

https://www.unix.com/shell-programmin...le-column.html

Can someone explain this command please? How are there no errors from using the same filename twice?

Code:

awk -F"," 'NR == FNR {  cnt[$1] ++} NR != FNR {  if (cnt[$1] == 1) print $0 }' filer.txt filer.txt

When NR==FNR , awk is reading the file for the first time and then it sets the counter. At the second pass (when NR > FNR) it prints only those records with a unique field 1. The command could be shortened to:

Code:

awk -F, 'NR==FNR{C[$1]++; next} C[$1]==1' infile infile

Why would there be errors reading the same file twice? After awk is done reading the file the first time it closes it and then reopens it.

Can you please elaborate on this? I still don't understand what is going on.

I didn't know that you could pass to 2 files to awk. What are some other purposes of passing 2 files to awk?

The purpose is to go over the same file twice, first to find the number of occurrence of field one. The second time only to print lines of which field 1 occurs exactly once..

The same can be done without arrays and with only a single pass, but than the input file needs to be sorted on field 1:

Code:

awk -F, '$1!=p{if(q)print q; q=$0; p=$1; next}{q=x} END{if(q)print q}' infile

I understand everything but the array parts.

In C[$1]++ does the $1 refer to field 1 or line 1? In C[$1]++ does it read threw all the fields or lines and then jump to C[$1]==1, or does it jump to C[$1]==1 after each increment? Does the 1 in C[$1]==1 mean true or something else?

What does an array like this mean? I've seen a few awk arrays like this.

Code:

A[$1$2$3$4]=$0

$1 refers to field 1. awk read line by line and on every line it stores the increments the counter of C[$1]. For example if we take you input file:


After awk rereads the file (then NR!=FNR, so the first part is skipped)



C[$1]==1 means "if C[$1] equals 1". In this case there is no {...} action part after this condition, so the default action is performed, which is {print $0}