Sponsored Content
Top Forums Shell Programming and Scripting Find Duplicate files, not by name Post 302299627 by Ikon on Friday 20th of March 2009 02:38:31 PM
Old 03-20-2009
Nice.. Thats ALOT faster, the directory has hundreds of files.

Thanks a bunch...
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

how to find duplicate files with find ?

hello all I like to make search on files , and the result need to be the files that are duplicated? (8 Replies)
Discussion started by: umen
8 Replies

2. Shell Programming and Scripting

Find duplicate value comparing 2 files and create an output

I need a perl script which will create an output file after comparing two diff file in a directory path: /export/home/abc/file1 /export/home/abc/file2 File Format: <IP>TAB<DeviceName><TAB>DESCRIPTIONS file1: 10.1.2.1.3<tab>abc123def<tab>xyz.mm1.ppp.... (2 Replies)
Discussion started by: ricky007
2 Replies

3. Shell Programming and Scripting

Find duplicate files

What utility do you recommend for simply finding all duplicate files among all files? (4 Replies)
Discussion started by: kiasas
4 Replies

4. Shell Programming and Scripting

Find duplicate files by file size

Hi! I want to find duplicate files (criteria: file size) in my download folder. I try it like this: find /Users/frodo/Downloads \! -type d -exec du {} \; | sort > /Users/frodo/Desktop/duplicates_1.txt; cut -f 1 /Users/frodo/Desktop/duplicates_1.txt | uniq -d | grep -hif -... (9 Replies)
Discussion started by: Dirk Einecke
9 Replies

5. Shell Programming and Scripting

find duplicate string in many different files

I have more than 100 files like this: SVEAVLTGPYGYT 2 SVEGNFEETQY 10 SVELGQGYEQY 28 SVERTGTGYT 6 SVGLADYNEQF 21 SVGQGYEQY 32 SVKTVLGYEQF 2 SVNNEQF 12 SVRDGLTNSPLH 3 SVRRDREGLEQF 11 SVRTSGSYEQY 17 SVSVSGSPLQETQY 78 SVVHSTSPEAF 59 SVVPGNGYT 75 (4 Replies)
Discussion started by: xshang
4 Replies

6. Shell Programming and Scripting

Find duplicate files but with different extensions

Hi ! I wonder if anyone can help on this : I have a directory: /xyz that has the following files: chsLog.107.20130603.gz chsLog.115.20130603 chsLog.111.20130603.gz chsLog.107.20130603 chsLog.115.20130603.gz As you ca see there are two files that are the same but only with a minor... (10 Replies)
Discussion started by: fretagi
10 Replies

7. Shell Programming and Scripting

Find duplicate rows between files

Hi champs, I have one of the requirement, where I need to compare two files line by line and ignore duplicates. Note, I hav files in sorted order. I have tried using the comm command, but its not working for my scenario. Input file1 srv1..development..employee..empname,empid,empdesg... (1 Reply)
Discussion started by: Selva_2507
1 Replies

8. Shell Programming and Scripting

Find help in shell - that clears away duplicate files

I am so frustrated!!! I want a nice command that clears away duplicate files: find . -type f -regex '.*{1,3}\..*' | xargs -I## rm -v '##' should work in my opinion. But it finds nothing even though I have files that have the file name: Scooby-Doo-1.txt Himalaya-2.jpg Camping... (8 Replies)
Discussion started by: Mr.Glaurung
8 Replies

9. Shell Programming and Scripting

To Find Duplicate files using latest in Linux

I have tried the following code and with that i couldnt achieve what i want. #!/usr/bin/bash find ./ -type f \( -iname "*.xml" \) | sort -n > fileList sed -i '/\.\/fileList/d' fileList NAMEOFTHISFILE=$(echo $0|sed -e 's/\/()$*.^|/\\&/g') sed -i "/$NAMEOFTHISFILE/d"... (2 Replies)
Discussion started by: gold2k8
2 Replies

10. UNIX for Advanced & Expert Users

AIX find duplicate backup files

I would like find and delete old backup files in aix. How would I go about doing this? For example: server1_1-20-2020 server1_1-21-2020 server1_1-22-2020 server1_1-23-2020 server2_1-20-2020 server2_1-21-2020 server2_1-22-2020 server2_1-23-2020 How would I go about finding and... (3 Replies)
Discussion started by: cokedude
3 Replies
NCOPY(1)							       ncopy								  NCOPY(1)

NAME
ncopy - NetWare file copy SYNOPSIS
ncopy -V ncopy [ -vmMnpptu ] [ -s amount ] file destinationfile|directory ncopy [ -vmMnpptu ] [ -s amount ] file1 [ file2 ... ] directory ncopy -r [ -vmMnpptu ] [ -s amount ] srcdir dstdir DESCRIPTION
With ncopy you can copy files to different locations on a single NetWare file server without generating excess network traffic. The pro- gram uses a NetWare function to do the copy rather than transferring the file across the network for both the read and write. If the last argument is a directory, ncopy will copy the source file(s) into the directory. If only two files are given and the last argu- ment is not a directory, it will copy the source file to the destination file. If the source and destination files are not on the same NetWare server (or are not on NetWare servers at all), ncopy will do a normal file copy. OPTIONS
-V Show version number and exit -v Verbose copy. Will show current file and percentage completion. -m Copy MAC resource fork. Copies MAC resource fork together with data fork. -M Copy MAC resource fork to/from non-MAC filesystem. It expects/creates resource forks in subdirectory .rsrc of each directory copied. If you want to copy files from MAC volume to .rsrc scheme, you must specify both options, -mM. It is not possible to create .rsrc direc- tory on MAC-aware volume in one step, you must first copy data to non-MAC media using ncopy -mM and then copy them back using ncopy -M. If you want to copy files from .rsrc scheme on MAC volume to real MAC multiple-forks file, you must first copy data to non-MAC filesys- tem using ncopy -M and then copy them back using ncopy -mM. -n Nice NetWare copy. Will sleep for a second between copying blocks on the NetWare server. Gives other people a chance to do some work on the NetWare server when you are copying large files. This has no effect if you are not copying on a NetWare server. -s amount Nice time slice factor. Used in conjunction with the -n option, this specifies the number of 100K blocks to copy before sleeping. Default is 10. (1 Megabyte) -p Preserve file attributes and date/time during copy. -pp Preserve file attributes, date/time and owner during copy. Name of owner is preserved, not owner ID. -t Preserve trustees during copy. Trustee name is preserved, not ID. -r Perform recursive copy. -u Perform copy only if mtime or size differs. BUGS
ncopy does not preserve long (MAC, NFS, FTAM, OS2) names during copy. SEE ALSO
ncpmount(8), ncpumount(8) CREDITS
ncopy was written by Brian G. Reid (breid@tim.com) and Tom C. Henderson (thenderson@tim.com). Many thanks to Volker Lendecke (lendecke@math.uni-goettingen.de) for the ncpfs and ncplib which made ncopy possible. Some further work was done by Petr Vandrovec (van- drove@vc.cvut.cz). ncopy 17/03/1996 NCOPY(1)
All times are GMT -4. The time now is 02:52 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy