Shell Programming and Scripting

Hi All

I have a rather large text file of approx 1m records in the format:-

20110877837-2.PDF
20100298984-3.PDF

et al...

I want to run uniq against the file to make sure there are no duplicate names.....

Code:

uniq -d /path/to/input/file.txt

However this is not producing any results even if I deliberately introduce duplicates, however if the duplicates are next to each other it does then produce output.

Does uniq only check consecutive records?

1 million records isn't that large unless they're really big records.

uniq only checks consecutive records, yes. Try sort first.

Code:

DUP=`sort < /path/to/file | uniq -d | wc -l`

[ "$DUP" -gt 0 ] && echo "$DUP duplicates"