Shell Programming and Scripting

Hi,

I am having the following doubts on awk. Please clarify for me.

a) What does it mean?

Code:

awk '$1=$1'

and how does it change if I append FS="" to the above code?



b) What if I use awk -vFS="\n" (i.e) setting (input) field separator to newline char, then what will be the value of $0, $1...? I guess "$0" denotes the whole record right?



c) To find the duplicate record in a file, I may use

Code:

awk 'a[$0]++ > 1 {exit 1}' file
awk '{if (found[$1] > 0) print $1 " is a dup"; ++found[$1]}' file

Both returns the same result though the validation check is different. The first one checks for "> 1" and second one checks for "> 0". This is confusing.

That's enough to list the duplicate records.

Code:

awk 'a[$0]++' file

---------- Post updated at 09:30 PM ---------- Previous update was at 09:27 PM ----------

for first question:

Code:

$ cat infile
abc jfkjff           alkjflkajfajfk
ab
abcd
abc
aB

$ awk '$1=$1' infile
abc jfkjff alkjflkajfajfk
ab
abcd
abc
aB

$ awk '$1=$1' FS="" infile
a b c   j f k j f f                       a l k j f l k a j f a j f k
a b
a b c d
a b c
a B

awk "$1=$1" is typically used to collapse extra spaces in a text file

Code:

$ echo "a    b      c      d" | awk "$1=$1"
a b c d

Thanks for this info, can you please illustrate on its working i.e. how it actually works.

Any assignment to the positional parameters causes the input line ($0) to be rebuilt with the current OFS.

The statement $1=$1 also returns a non-null value and this causes awk to perform the {} block (or in this case as there isn't a {} block to print the value of $0).

Thank you Chubler_XL, but Is it just an assignment of positional parameter or that particular field? why it is just $1 and not $0 or $2, $3... etc.,? How this assignment collapses the extra spaces?

It will not work with $0, since $0 is the entire record, not a field. $2=$2 or $7=$7 would work too, but $1=$1 is better since there isn't always a $2 and then you would be introducing empty fields.

---------- Post updated at 12:13 ---------- Previous update was at 12:09 ----------

This is correct only if $1 is not zero or "" , otherwise those lines will not get printed... So in that case it would be safer to use:

Code:

awk '{$1=$1}1'

Single quotes should be used instead of double quotes...