There is a tool that can determine the identity of files using the md5 sum.
Look at the listing
Interactive mode with a choice to remove
The following case is suitable for use in the script
All duplicates of the file will be deleted exclude only the first file (in order of sorting name files and then name dirs!) will be saved.
A simple way to change the directory with the saved file try to use -i option. It does not change the save directory, but in the reorganized sort order, the upper file may be in the folder you need
try fdupes -i dir1/ dir2/
and then use fdupes -Nd dir1/ dir2/
Well, before you delete something, be sure to read the man pages on the command and make training tests on its use.
I want to find duplicates in file on 2nd field i wrote this code:
nawk '{a++} END{for i in a {if (a>1) print}}' temp
Could not find whats wrong with this.
Appreciate help (5 Replies)
I have a folder which in turn has numerous sub folders all containing pdf files with same file named in different ways.
So I need a script if it can be written to find and print the duplicate files (That is files with same size) along with the respective paths.
So I assume here that same file... (5 Replies)
Hi All,
I am Oracle Apps Tech guy, I have a requirement to find 777 permission is there or not for all Folders and Sub-folders
Under APPL_TOP (Folder/directory) with below conditions
i) the directory names should start with xx..... (like xxau,xxcfi,xxcca...etc)
and exclude the directory... (11 Replies)
Hello,
My text file has input of the form
abc dft45.xml
ert rt653.xml
abc ert57.xml
I need to write a perl script/shell script to find duplicates in the first column and write it into a text file of the form...
abc dft45.xml
abc ert57.xml
Can some one help me plz? (5 Replies)
example data
5666700842511TAfmoham03151008075205999900000001000001000++
5666700843130MAfmoham03151008142606056667008390315100005001
6666666663130MAfmoham03151008142606056667008390315100005001
I'd like to sort on position 10-14 where the characters are eq "130MA".
Then based on positions... (0 Replies)
Hi all,
Using grep command, i want to find the pattern of text in all directories and sub-directories.
e.g: if i want to search for a pattern named "parmeter", i used the command
grep -i "param" ../*
is this correct? (1 Reply)
these are numeric ids..
222932017099186177
222932014385467392
222932017371820032
222932017409556480
I have text file having 300 millions of line as shown above. I want to find duplicates from this file. Please suggest the quicker way..
sort | uniq -d will... (3 Replies)
with below given format,
I have been trying to find out all IDs for those entries with duplicate names in 2nd and 3rd columns and their count like how many time duplication happened for any name if any,
0.237788 Aaban Aahva
0.291066 Aabheer Aahlaad
0.845814 Aabid Aahan
0.152208 Aadam... (6 Replies)
Hello All,
This is a noob question. I tried searching for the answer but the answer found did not help me .
I have a file that can have duplicates.
100
200
300
400
100
150
the number 100 is duplicated twice. I want to find the duplicate along with the line number.
expected... (4 Replies)
Discussion started by: vatigers
4 Replies
LEARN ABOUT DEBIAN
file::dircompare
DirCompare(3pm) User Contributed Perl Documentation DirCompare(3pm)NAME
File::DirCompare - Perl module to compare two directories using callbacks.
SYNOPSIS
use File::DirCompare;
# Simple diff -r --brief replacement
use File::Basename;
File::DirCompare->compare($dir1, $dir2, sub {
my ($a, $b) = @_;
if (! $b) {
printf "Only in %s: %s
", dirname($a), basename($a);
} elsif (! $a) {
printf "Only in %s: %s
", dirname($b), basename($b);
} else {
print "Files $a and $b differ
";
}
});
# Version-control like Deleted/Added/Modified listing
my (@listing, @modified); # use closure to collect results
File::DirCompare->compare('old_tree', 'new_tree', sub {
my ($a, $b) = @_;
if (! $b) {
push @listing, "D $a";
} elsif (! $a) {
push @listing, "A $b";
} else {
if (-f $a && -f $b) {
push @listing, "M $b";
push @modified, $b;
} else {
# One file, one directory - treat as delete + add
push @listing, "D $a";
push @listing, "A $b";
}
}
});
DESCRIPTION
File::DirCompare is a perl module to compare two directories using a callback, invoked for all files that are 'different' between the two
directories, and for any files that exist only in one or other directory ('unique' files).
File::DirCompare has a single public compare() method, with the following signature:
File::DirCompare->compare($dir1, $dir2, $sub, $opts);
The first three arguments are required - $dir1 and $dir2 are paths to the two directories to be compared, and $sub is the subroutine
reference called for all unique or different files. $opts is an optional hashref of options - see OPTIONS below.
The provided subroutine is called for all unique files, and for every pair of 'different' files encountered, with the following signature:
$sub->($file1, $file2)
where $file1 and $file2 are the paths to the two files. For 'unique' files i.e. where a file exists in only one directory, the subroutine
is called with the other argument 'undef' i.e. for:
$sub->($file1, undef)
$sub->(undef, $file2)
the first indicates $file1 exists only in the first directory given ($dir1), and the second indicates $file2 exists only in the second
directory given ($dir2).
OPTIONS
The following optional arguments are supported, passed in using a hash reference after the three required arguments to compare() e.g.
File::DirCompare->compare($dir1, $dir2, $sub, {
cmp => $cmp_sub,
ignore_unique => 1,
});
cmp By default, two files are regarded as different if their contents do not match (tested with File::Compare::compare). That default
behaviour can be overridden by providing a 'cmp' subroutine to do the file comparison, returning zero if the two files are equal, and
non-zero if not.
E.g. to compare using modification times instead of file contents:
File::DirCompare->compare($dir1, $dir2, $sub, {
cmp => sub { -M $_[0] <=> -M $_[1] },
});
ignore_cmp
If you want to see all corresponding files, not just 'different' ones, set the 'ignore_cmp' flag to tell File::DirCompare to skip its
file comparison checks i.e.
File::DirCompare->compare($dir1, $dir2, $sub,
{ ignore_cmp => 1 });
ignore_unique
If you want to ignore files that only exist in one of the two directories, set the 'ignore_unique' flag i.e.
File::DirCompare->compare($dir1, $dir2, $sub,
{ ignore_unique => 1 });
SEE ALSO
File::Dircmp, which provides similar functionality (and whose directory walking code I've adapted for this module), but a simpler
reporting-only interface, something like the first example in the SYNOPSIS above.
AUTHOR AND CREDITS
Gavin Carr <gavin@openfusion.com.au>
Thanks to Robin Barker for a bug report and fix for glob problems with whitespace.
COPYRIGHT AND LICENSE
Copyright 2006-2007 by Gavin Carr.
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
perl v5.10.1 2010-03-02 DirCompare(3pm)