07-02-2010
Find duplicate files
What utility do you recommend for simply finding all duplicate files among all files?
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
hello all
I like to make search on files , and the result need to be the files that are duplicated? (8 Replies)
Discussion started by: umen
8 Replies
2. Shell Programming and Scripting
I need a perl script which will create an output file after comparing two diff file in a directory path:
/export/home/abc/file1
/export/home/abc/file2
File Format: <IP>TAB<DeviceName><TAB>DESCRIPTIONS
file1:
10.1.2.1.3<tab>abc123def<tab>xyz.mm1.ppp.... (2 Replies)
Discussion started by: ricky007
2 Replies
3. Shell Programming and Scripting
I have a directory with images:
-rw-r--r-- 1 root root 26216 Mar 19 21:00 020109.210001.jpg
-rw-r--r-- 1 root root 21760 Mar 19 21:15 020109.211502.jpg
-rw-r--r-- 1 root root 23144 Mar 19 21:30 020109.213002.jpg
-rw-r--r-- 1 root root 31350 Mar 20 00:45 020109.004501.jpg
-rw-r--r-- 1 root... (2 Replies)
Discussion started by: Ikon
2 Replies
4. Shell Programming and Scripting
Hi!
I want to find duplicate files (criteria: file size) in my download folder.
I try it like this:
find /Users/frodo/Downloads \! -type d -exec du {} \; | sort > /Users/frodo/Desktop/duplicates_1.txt;
cut -f 1 /Users/frodo/Desktop/duplicates_1.txt | uniq -d | grep -hif -... (9 Replies)
Discussion started by: Dirk Einecke
9 Replies
5. Shell Programming and Scripting
I have more than 100 files like this:
SVEAVLTGPYGYT 2
SVEGNFEETQY 10
SVELGQGYEQY 28
SVERTGTGYT 6
SVGLADYNEQF 21
SVGQGYEQY 32
SVKTVLGYEQF 2
SVNNEQF 12
SVRDGLTNSPLH 3
SVRRDREGLEQF 11
SVRTSGSYEQY 17
SVSVSGSPLQETQY 78
SVVHSTSPEAF 59
SVVPGNGYT 75 (4 Replies)
Discussion started by: xshang
4 Replies
6. Shell Programming and Scripting
Hi !
I wonder if anyone can help on this : I have a directory: /xyz that has the following files:
chsLog.107.20130603.gz
chsLog.115.20130603
chsLog.111.20130603.gz
chsLog.107.20130603
chsLog.115.20130603.gz
As you ca see there are two files that are the same but only with a minor... (10 Replies)
Discussion started by: fretagi
10 Replies
7. Shell Programming and Scripting
Hi champs,
I have one of the requirement, where I need to compare two files line by line and ignore duplicates. Note, I hav files in sorted order.
I have tried using the comm command, but its not working for my scenario.
Input file1
srv1..development..employee..empname,empid,empdesg... (1 Reply)
Discussion started by: Selva_2507
1 Replies
8. Shell Programming and Scripting
I am so frustrated!!!
I want a nice command that clears away duplicate files:
find . -type f -regex '.*{1,3}\..*' | xargs -I## rm -v '##'
should work in my opinion. But it finds nothing even though I have files that have the file name:
Scooby-Doo-1.txt
Himalaya-2.jpg
Camping... (8 Replies)
Discussion started by: Mr.Glaurung
8 Replies
9. Shell Programming and Scripting
I have tried the following code and with that i couldnt achieve what i want.
#!/usr/bin/bash
find ./ -type f \( -iname "*.xml" \) | sort -n > fileList
sed -i '/\.\/fileList/d' fileList
NAMEOFTHISFILE=$(echo $0|sed -e 's/\/()$*.^|/\\&/g')
sed -i "/$NAMEOFTHISFILE/d"... (2 Replies)
Discussion started by: gold2k8
2 Replies
10. UNIX for Advanced & Expert Users
I would like find and delete old backup files in aix. How would I go about doing this? For example:
server1_1-20-2020
server1_1-21-2020
server1_1-22-2020
server1_1-23-2020
server2_1-20-2020
server2_1-21-2020
server2_1-22-2020
server2_1-23-2020
How would I go about finding and... (3 Replies)
Discussion started by: cokedude
3 Replies
LEARN ABOUT CENTOS
file::find::rule::perl
File::Find::Rule::Perl(3) User Contributed Perl Documentation File::Find::Rule::Perl(3)
NAME
File::Find::Rule::Perl - Common rules for searching for Perl things
SYNOPSIS
use File::Find::Rule ();
use File::Find::Rule::Perl ();
# Find all Perl files smaller than 10k
my @files = File::Find::Rule->perl_file
->size('<10Ki')
->in('dir');
# Locate all the modules that PAUSE will index
my @mod = File::Find::Rule->no_index
->perl_module
->in('My-Distribution');
DESCRIPTION
I write a lot of things that muck with Perl files. And it always annoyed me that finding "perl files" requires a moderately complex
File::Find::Rule pattern.
File::Find::Rule::Perl provides methods for finding various types Perl-related files, or replicating search queries run on a distribution
in various parts of the CPAN ecosystem.
METHODS
perl_module
The "perl_module" rule locates perl modules. That is, files that are named "*.pm".
This rule is equivalent to "->"file->name( '*.pm' )> and is included primarily for completeness.
perl_test
The "perl_test" rule locates perl test scripts. That is, files that are named "*.t".
This rule is equivalent to "->"file->name( '*.t' )> and is included primarily for completeness.
perl_installer
The "perl_installer" rule locates perl distribution installers. That is, it locates "Makefile.PL" and "Build.PL" files.
perl_script
The "perl_script" rule locates perl scripts.
This is any file that ends in .pl, or any files without extensions that have a perl "hash-bang" line.
perl_file
The "perl_file" rule locates all files containing Perl code.
This includes all the files matching the above "perl_module", "perl_test", "perl_installer" and "perl_script" rules.
no_index
# Provide the rules directly
$rule->no_index(
directory => [ 'inc', 't', 'examples' ],
file => [ 'Foo.pm', 'lib/Foo.pm' ],
);
# Provide a META.yml to use
$rule->no_index( 'META.yml' );
# Provide a dist root directory to look for a META.yml in
$rule->no_index( 'My-Distribution' );
# Automatically pick up a META.yml from the target directory
$rule->no_index->in( 'My-Distribution' );
The "no_index" method applies a set of rules as per the no_index section in a "META.yml" file.
SUPPORT
Bugs should always be submitted via the CPAN bug tracker
<http://rt.cpan.org/NoAuth/ReportBug.html?Queue=File-Find-Rule-Perl>
For other issues, contact the maintainer
AUTHOR
Adam Kennedy <adamk@cpan.org>
SEE ALSO
<http://ali.as/>, File::Find::Rule, File::Find::Rule::PPI
COPYRIGHT
Copyright 2006 - 2012 Adam Kennedy.
This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
The full text of the license can be found in the LICENSE file included with this module.
perl v5.16.3 2012-10-31 File::Find::Rule::Perl(3)