Shell Programming and Scripting

Hi Gurus,

Do any kind souls encounter have the same script as mentioned here.

Find and compare filenames in different mount point and remove duplicates.


Thanks a million!!!
wanna13e

Give some examples, like between which you want to find duplicate, and what you wanted to remove ?

As I understand your question, if you have any two (or more) instances of files with the same name, you want to remove all but one, irrespective of whether they have the same sizes, contents, etc. Please feel free to say no to this, if this is not what you intended.

As a matter of interest, when you are removing these files, how do you intend to determine which occurence is preserved and which are deleted?

Did you intend to keep any records?

mount/volumeA/folderA/1234_A_54321.pdf
mount/volumeB/folderB/123_A_54321.pdf
mount/volumeC/folderC/123_A_54321.pdf

The script will remove the duplicate in volumeB and volumeC but keep file in volumeA.

Thank a million!
wanna13e.

---------- Post updated at 08:22 PM ---------- Previous update was at 08:20 PM ----------

Yes, i intend to keep the records as shown in the example above in my post.


Thank a million!!!
wanna13e.

lot of open source tools are available in sourceforge, as
fdupes & finddup and so on.

I think fdupes use hashing (md5/sha) to determine if you have duplicate files. Way better than checking by filename.

im not sure but i think diff command can do compare even if file is binary.
alternatively, you can create script and use diff to compare or follow what suggested above.