Find duplicate files but with different extensions


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Find duplicate files but with different extensions
# 1  
Old 06-24-2013
Find duplicate files but with different extensions

Hi !

I wonder if anyone can help on this : I have a directory:
Code:
/xyz

that has the following files:
Code:
chsLog.107.20130603.gz
chsLog.115.20130603
chsLog.111.20130603.gz
chsLog.107.20130603
chsLog.115.20130603.gz

As you ca see there are two files that are the same but only with a minor difference which is
Code:
gz

extension.
How can I find those files? I have tryied the
Code:
find

command, but is not very helpfull
# 2  
Old 06-24-2013
find . -name *.gz
# 3  
Old 06-24-2013
Find duplicate files but with different extensions

That will only find
Code:
gz

extension filenames
# 4  
Old 06-24-2013
Whats your expected output?
# 5  
Old 06-24-2013
Find duplicate files but with different extensions

I am expecting to see:

Code:
chsLog.107.20130603.gz
chsLog.107.20130603

# 6  
Old 06-24-2013
Try this:
Code:
find|awk '/gz$/{sub(/.gz$/,"")gz[$1]++;next}{a[$1]++}END{for(i in a)if(gz[i])print i}'

# 7  
Old 06-24-2013
Find duplicate files but with different extensions

error message:

Code:
 find|awk '/gz$/{sub(/.gz$/,"")gz[$1]++;next}{a[$1]++}END{for(i in a)if(gz[i])print i}'
find: insufficient number of arguments
find: [-H | -L] path-list predicate-list
awk: syntax error near line 1
awk: illegal statement near line 1

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

AIX find duplicate backup files

I would like find and delete old backup files in aix. How would I go about doing this? For example: server1_1-20-2020 server1_1-21-2020 server1_1-22-2020 server1_1-23-2020 server2_1-20-2020 server2_1-21-2020 server2_1-22-2020 server2_1-23-2020 How would I go about finding and... (3 Replies)
Discussion started by: cokedude
3 Replies

2. UNIX for Advanced & Expert Users

Find wild card directory and its files of some extensions

I want to use Find command to find directories that have certain name and them find files in that directory having only some extensions. So far, I have come up with this command to list directories with wild card name and list ALL the files in that directory. find . -type d -name prog\* -print... (11 Replies)
Discussion started by: sssccc
11 Replies

3. AIX

AIX : Find files ignoring certain file extensions

Hi All, I am scripting a program to find and archive files. There are certain file types that I do not want to archive. Below is the scenario. I have created a lookup file which has details on folders days and file extensions that needs to be ignored I have separated the individual into... (4 Replies)
Discussion started by: kavinmjr
4 Replies

4. Shell Programming and Scripting

Find duplicate rows between files

Hi champs, I have one of the requirement, where I need to compare two files line by line and ignore duplicates. Note, I hav files in sorted order. I have tried using the comm command, but its not working for my scenario. Input file1 srv1..development..employee..empname,empid,empdesg... (1 Reply)
Discussion started by: Selva_2507
1 Replies

5. Shell Programming and Scripting

find duplicate string in many different files

I have more than 100 files like this: SVEAVLTGPYGYT 2 SVEGNFEETQY 10 SVELGQGYEQY 28 SVERTGTGYT 6 SVGLADYNEQF 21 SVGQGYEQY 32 SVKTVLGYEQF 2 SVNNEQF 12 SVRDGLTNSPLH 3 SVRRDREGLEQF 11 SVRTSGSYEQY 17 SVSVSGSPLQETQY 78 SVVHSTSPEAF 59 SVVPGNGYT 75 (4 Replies)
Discussion started by: xshang
4 Replies

6. Shell Programming and Scripting

Find duplicate files by file size

Hi! I want to find duplicate files (criteria: file size) in my download folder. I try it like this: find /Users/frodo/Downloads \! -type d -exec du {} \; | sort > /Users/frodo/Desktop/duplicates_1.txt; cut -f 1 /Users/frodo/Desktop/duplicates_1.txt | uniq -d | grep -hif -... (9 Replies)
Discussion started by: Dirk Einecke
9 Replies

7. Shell Programming and Scripting

Find duplicate files

What utility do you recommend for simply finding all duplicate files among all files? (4 Replies)
Discussion started by: kiasas
4 Replies

8. Shell Programming and Scripting

Find Duplicate files, not by name

I have a directory with images: -rw-r--r-- 1 root root 26216 Mar 19 21:00 020109.210001.jpg -rw-r--r-- 1 root root 21760 Mar 19 21:15 020109.211502.jpg -rw-r--r-- 1 root root 23144 Mar 19 21:30 020109.213002.jpg -rw-r--r-- 1 root root 31350 Mar 20 00:45 020109.004501.jpg -rw-r--r-- 1 root... (2 Replies)
Discussion started by: Ikon
2 Replies

9. Shell Programming and Scripting

Find files with 3 different extensions

Hi all, From one directory I need to fetch only files of type *.xls,*.csv,*.txt. I tried the find . -name '*.txt,*.csv,*.xls' -print. But it throws me error. Please do help me on this. Thanks Mahalakshmi.A (11 Replies)
Discussion started by: mahalakshmi
11 Replies

10. Shell Programming and Scripting

how to find duplicate files with find ?

hello all I like to make search on files , and the result need to be the files that are duplicated? (8 Replies)
Discussion started by: umen
8 Replies
Login or Register to Ask a Question

Featured Tech Videos