Shell Programming and Scripting

As we said before, if it is important to choose the 1st line in your input file for lines with the same 1st field, sort -u is not guaranteed to do that. And, sort and sort -k1,1 are not guaranteed to keep lines with the same first field in the same order in the output file as they appeared in the output file. So, sorting and then using awk to choose the first line of those with the same 1st field won't work either.

And, in your earlier statements, you said you wanted the output to be stored in your input file; but the code you now have that you say works doesn't do that. Instead, it uses an input file named by the expansion of the shell variable $result, stores the sorted results in a file with the extension .csv added to the end of the name of the input file. And, whether or not processing was successful, it removes the input file.

Assuming that your input file is specified by $result and you want the output stored in that same file if processing is successful (and the original file left unchanged if there is an error), you might try something like:

Code:

#!/bin/ksh
result="file"
awk -F, '
NR == 1 {
	print
	next
}
!A[$1]++ {
	print | "sort"
}' "$result" > "$result.$$" && cp "$result.$$" "$result"; rm -f "$result.$$"

If you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk. Although written and tested using the Korn shell, this script should work with any shell that uses basic Bourne shell syntax.

If you want awk to do the parsing & sorting try the below script...

Code:

awk -F\, '{
    if (NR == 1) print
    else !f[$1]++ && x[++i] = $0
} END {
    for (j=1; j<i; j++)
        for (k=1; k<(i-j+1); k++)
            if (x[k] > x[k+1]) {
                t = x[k]
                x[k] = x[k+1]
                x[k+1] = t
            }
    for (k=1; k<=i; k++)
        print x[k]
}' file