UNIX for Dummies Questions & Answers

Ooops... sorry. Here is the fixed version:

Code:

$ sort input | awk -F_ '
!cnt[$1,$2,$3]{
  if(c)
    print a;
  c=0; a=$0}
cnt[$1,$2,$3]++{
  a=a "\n" $0; 
  c++
}
END{ 
  if(c) 
    print a ; 
  print "\n***********************" ;
  for(b in cnt){
    if(cnt[b]>1)
      print b " ... " cnt[b]
} }' SUBSEP="_"

Variable 'c' is a flag, whether to print or not.
This will also print the counts at the end (in random order).
Sorted before fed to awk. Not very elegant, but should work.


You could also combine other filters to achieve this:

Code:

grep  -f <( awk -F_ 'cnt[$1,$2,$3]++{print $1,$2,$3}' OFS="_" input | sort | uniq ) input

Here awk will print the first 3 fields of the duplicates only, sort and uniq will get only single unique occurence, and this is fed to grep to search for all lines containing these strings.

Then you could find the counts like:

Code:

cut -d_ -f1,2,3 input | sort | uniq -c | egrep -v "^ *1"

cut will extract only first 3 fields, 'uniq -c' will count, and 'grep -v' will filter out those that have count of 1.

Using awk only is more efficient, since you are processing your file only once.