Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Find duplicate files but with different extensions

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Find duplicate files but with different extensions
# 1  
Old 06-24-2013
Find duplicate files but with different extensions
Hi !

I wonder if anyone can help on this : I have a directory:
Code:
/xyz

that has the following files:
Code:
chsLog.107.20130603.gz
chsLog.115.20130603
chsLog.111.20130603.gz
chsLog.107.20130603
chsLog.115.20130603.gz

As you ca see there are two files that are the same but only with a minor difference which is
Code:
gz

extension.
How can I find those files? I have tryied the
Code:
find

command, but is not very helpfull
# 2  
Old 06-24-2013
find . -name *.gz
# 3  
Old 06-24-2013
Find duplicate files but with different extensions
That will only find
Code:
gz

extension filenames
# 4  
Old 06-24-2013
Whats your expected output?
# 5  
Old 06-24-2013
Find duplicate files but with different extensions
I am expecting to see:

Code:
chsLog.107.20130603.gz
chsLog.107.20130603

# 6  
Old 06-24-2013
Try this:
Code:
find|awk '/gz$/{sub(/.gz$/,"")gz[$1]++;next}{a[$1]++}END{for(i in a)if(gz[i])print i}'

# 7  
Old 06-24-2013
Find duplicate files but with different extensions
error message:

Code:
 find|awk '/gz$/{sub(/.gz$/,"")gz[$1]++;next}{a[$1]++}END{for(i in a)if(gz[i])print i}'
find: insufficient number of arguments
find: [-H | -L] path-list predicate-list
awk: syntax error near line 1
awk: illegal statement near line 1
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

AIX find duplicate backup files

I would like find and delete old backup files in aix. How would I go about doing this? For example: server1_1-20-2020 server1_1-21-2020 server1_1-22-2020 server1_1-23-2020 server2_1-20-2020 server2_1-21-2020 server2_1-22-2020 server2_1-23-2020 How would I go about finding and... (3 Replies)
Discussion started by: cokedude
3 Replies

2. UNIX for Advanced & Expert Users

Find wild card directory and its files of some extensions

I want to use Find command to find directories that have certain name and them find files in that directory having only some extensions. So far, I have come up with this command to list directories with wild card name and list ALL the files in that directory. find . -type d -name prog\* -print... (11 Replies)
Discussion started by: sssccc
11 Replies

3. AIX

AIX : Find files ignoring certain file extensions

Hi All, I am scripting a program to find and archive files. There are certain file types that I do not want to archive. Below is the scenario. I have created a lookup file which has details on folders days and file extensions that needs to be ignored I have separated the individual into... (4 Replies)
Discussion started by: kavinmjr
4 Replies

4. Shell Programming and Scripting

Find duplicate rows between files

Hi champs, I have one of the requirement, where I need to compare two files line by line and ignore duplicates. Note, I hav files in sorted order. I have tried using the comm command, but its not working for my scenario. Input file1 srv1..development..employee..empname,empid,empdesg... (1 Reply)
Discussion started by: Selva_2507
1 Replies

5. Shell Programming and Scripting

find duplicate string in many different files

I have more than 100 files like this: SVEAVLTGPYGYT 2 SVEGNFEETQY 10 SVELGQGYEQY 28 SVERTGTGYT 6 SVGLADYNEQF 21 SVGQGYEQY 32 SVKTVLGYEQF 2 SVNNEQF 12 SVRDGLTNSPLH 3 SVRRDREGLEQF 11 SVRTSGSYEQY 17 SVSVSGSPLQETQY 78 SVVHSTSPEAF 59 SVVPGNGYT 75 (4 Replies)
Discussion started by: xshang
4 Replies

6. Shell Programming and Scripting

Find duplicate files by file size

Hi! I want to find duplicate files (criteria: file size) in my download folder. I try it like this: find /Users/frodo/Downloads \! -type d -exec du {} \; | sort > /Users/frodo/Desktop/duplicates_1.txt; cut -f 1 /Users/frodo/Desktop/duplicates_1.txt | uniq -d | grep -hif -... (9 Replies)
Discussion started by: Dirk Einecke
9 Replies

7. Shell Programming and Scripting

Find duplicate files

What utility do you recommend for simply finding all duplicate files among all files? (4 Replies)
Discussion started by: kiasas
4 Replies

8. Shell Programming and Scripting

Find Duplicate files, not by name

I have a directory with images: -rw-r--r-- 1 root root 26216 Mar 19 21:00 020109.210001.jpg -rw-r--r-- 1 root root 21760 Mar 19 21:15 020109.211502.jpg -rw-r--r-- 1 root root 23144 Mar 19 21:30 020109.213002.jpg -rw-r--r-- 1 root root 31350 Mar 20 00:45 020109.004501.jpg -rw-r--r-- 1 root... (2 Replies)
Discussion started by: Ikon
2 Replies

9. Shell Programming and Scripting

Find files with 3 different extensions

Hi all, From one directory I need to fetch only files of type *.xls,*.csv,*.txt. I tried the find . -name '*.txt,*.csv,*.xls' -print. But it throws me error. Please do help me on this. Thanks Mahalakshmi.A (11 Replies)
Discussion started by: mahalakshmi
11 Replies

10. Shell Programming and Scripting

how to find duplicate files with find ?

hello all I like to make search on files , and the result need to be the files that are duplicated? (8 Replies)
Discussion started by: umen
8 Replies
Login or Register to Ask a Question