Shell Programming and Scripting

The first part:

Code:

{ read A; read B; printf "%s\n" "$A" "$B";  sort; } < file

reads the first line from file into the shell variable A (read A) and the second line from file into the shell variable B (read B), reprints those two lines unchanged (printf '%s\n' "$A" "$B" and sorts the remaining lines from file (sort). This sorts the data in file while keeping the headers (unsorted) at the start of the output.

All of the output from the first part is then piped (|) into the second part:

Code:

awk '!T[$1]++'

which creates an array indexed by the first field on each input line (T[$1]) and sets the value of that array to the number of times that index has been seen (after using the previous value of the variable). In awk, statements consist of a condition section and an action section. In this case the condition is !T[$1]++ and there is no action section. When there is no action section, the default action (print the line) is performed if, and only if, the condition evaluates to a non-zero numeric value or a non-empty string value. Since undefined elements of an array are treated as a zero or an empty string (depending on context), the condition !T[$1]++ is evaluated as follows the first time a line is seen with a given string in the first field:and since the condition's value is non-zero, the line is printed. On subsequent lines with the same first field the evaluation is:and since the condition's value is zero, the line is not printed.

Note that the sort in the first part grouped all lines with the same 1st field together and (if both an "add" and a "delete" appeared for that value, the line(s) with "add" for that 1st field value will come before any line(s) with "delete".

Nice, simple, straight-forward magic. The only likely problem with this approach is that if the data in your file contains a 1st field value that is identical to the 1st field in one of the header lines, those data lines will be lost. But, as long as you don't have a user named "user", that shouldn't be a problem for you.

try also:

Code:

awk 'NR==FNR {a[$1]++; next} $2 ~ /add/ {print; a[$1]=0} ++b[$1]==a[$1]' infile infile

Note that if you don't care about the order of the data lines in the output, you can also try something like:

Code:

awk '
NR < 3 {# Copy header lines unchanged.
	print
	next
}
!($1 in A) && $2 == "add" {
	# Print 1st occurrence of an "add" line for each user.
	print
	A[$1]
	next
}
!($1 in A) && !($1 in D) && $2 == "delete" {
	# If user does not have an "add" line yet and does not have a "delete"
	# line yet, capture the 1st "delete" line.
	D[$1] = $0
}
END {	# After we reach EOF on the input file, print captured "delete" lines
	# for users who did not have an "add" line.
	for(u in D)
		if(!(u in A))
			print D[u]
}' file

which doesn't need sort and only reads the input file once. (It will print all users with "add" in the 2nd field in the order in which they were first found in the input file followed by users who have no "add" but do have a "delete" in the 2nd field in random order.) With your latest sample input file, the output will be:

Code:

user role  location eid
-------------------------
AAA  add  UK  1
DDD  add  FR  3
BBB  add  IN  4
CCC  delete  AU  2

Combining MadeInGermany's modification in post #4 of my suggestion with the bit that saves the header, you could also try:

Code:

awk 'FNR<3{print; next} !($1 in A) || $2=="add" {A[$1]=$0} END{for(i in A) print A[i]}' file