Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Delete duplicate directories?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Delete duplicate directories?
# 1  
Old 08-27-2016
Delete duplicate directories?
Is there a way via some bash script or just cmd to find duplicate directories?


i have main folders:

TEST1

TEST2


In folder TEST1 is some amount of same folders as in folder TEST2
can be this done? i tried fdupe but it only search for dupe files not whle dirs


thx!
# 2  
Old 08-27-2016
That's not too specific nor precise, is it? What exactly are you after? Identical directory names/branches? Identical file names? Identical files?
This User Gave Thanks to RudiC For This Post:
ZerO13
# 3  
Old 08-27-2016
I am after Identical directory names Smilie
# 4  
Old 08-27-2016
The fdupes command that you mention does a very thorough comparison of files before it does any action. You just want directory names regardless of contents?
This User Gave Thanks to RudiC For This Post:
ZerO13
# 5  
Old 08-27-2016
Try
Code:
find TEST1 TEST2 -type d | awk -F/ '{D[$NF]++} END {for (d in D) if (D[d]>1) print D[d] ": " d}'

This User Gave Thanks to RudiC For This Post:
ZerO13
# 6  
Old 08-27-2016
Thats it! thx Smilie

now how can i make it that it will detele from directory TEST1?

//i know i need to add rm -rf or something like that

//also only delete dirs with match "2: DIRblah" from your command

you are GOD! Smilie

found the way neeeded D[d]==2

Code:
find /TEST1/ /TEST2/ -type d | awk -F/ '{D[$NF]++} END {for (d in D) if (D[d]==2) print D[d] ": " d}'

Last edited by ZerO13; 08-27-2016 at 07:48 AM..
# 7  
Old 08-27-2016
You ALWAYS need to apply utmost care when removing things (files, directories) (semi-) automatically. Too soon you'll be faced with inadvertent if not fatal results!
For your request, try

Code:
find TEST1 TEST2 -type d | awk -F/ '{D[$NF]++} END {for (d in D) if (D[d]>1) print "echo rm TEST1/" d}' | sh

The echo is there for safety reasons; remove when happy.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Find duplicate values in specific column and delete all the duplicate values

Dear folks I have a map file of around 54K lines and some of the values in the second column have the same value and I want to find them and delete all of the same values. I looked over duplicate commands but my case is not to keep one of the duplicate values. I want to remove all of the same... (4 Replies)
Discussion started by: sajmar
4 Replies

2. Shell Programming and Scripting

Delete duplicate row

Hi all, how can delete duplicate files in file form, e.g. $cat file1 aaa 123 234 345 456 bbb 345 345 657 568 ccc 345 768 897 456 aaa 123 234 345 456 ddd 786 784 234 263 ccc 345 768 897 456 aaa 123 234 345 456 ccc 345 768 897 456 then i need ouput file1 some, (4 Replies)
Discussion started by: aav1307
4 Replies

3. Programming

Finding duplicate files in two base directories

Hello All, I have got some assignment to complete till this Monday and problem statement is as follow :- Problem :- Find duplicate files (especially .c and .cpp) from two project base directories with following requirement :- 1.Should be extendable to search in multiple base... (4 Replies)
Discussion started by: anand.shah
4 Replies

4. Shell Programming and Scripting

Delete duplicate rows

Hi, This is a followup to my earlier post him mno klm 20 76 . + . klm_mango unix_00000001; alp fdc klm 123 456 . + . klm_mango unix_0000103; her tkr klm 415 439 . + . klm_mango unix_00001043; abc tvr klm 20 76 . + . klm_mango unix_00000001; abc def klm 83 84 . + . klm_mango... (5 Replies)
Discussion started by: jacobs.smith
5 Replies

5. Shell Programming and Scripting

Create duplicate directories with same permissions

Hi all, I need to create duplicate directories and sub directories (only the directories not the files or file contents) with the same permissions. Can some one guide me in doing this. I could able to create but here the permissions should be the same how can i do this in linux. Thanks in... (5 Replies)
Discussion started by: Olivia
5 Replies

6. UNIX for Dummies Questions & Answers

Delete duplicate second line

Hi ALL I need a help I need to retain only the first line of 035 if I have two line before =040 , if only one then need to take that Eg: Input =035 (ABC)12324141241 =035 (XYZPQR)704124 =040 AB$QS$WEWR =035 (ABC)08080880809 =035 (XYZPQR)9809314 =040 ... (4 Replies)
Discussion started by: umapearl
4 Replies

7. UNIX for Dummies Questions & Answers

Duplicate directories

I have noticed that the same folder (and contents) lives in /u/public and /usr/public Question was this put here intentionally or by accident? Its 31Gb in size and on a 72Gb HDD that leaves little room for apps. It is a nework shared drive for all to access e.g. p: points to... (0 Replies)
Discussion started by: moondogi
0 Replies

8. Shell Programming and Scripting

Remove Duplicate Filenames in 2 very large directories

Hello Gurus, O/S RHEL4 I have a requirement to compare two linux based directories for duplicate filenames and remove them. These directories are close to 2 TB each. I have tried running a: Prompt>diff -r data1/ data2/ I have tried this as well: jason@jason-desktop:~$ cat script.sh ... (7 Replies)
Discussion started by: jaysunn
7 Replies

9. Shell Programming and Scripting

duplicate directories

Hi, I have file which users like filename ->"readfile", following entries peter john alaska abcd xyz and i have directory /var/ i want to do first cat of "readfile" line by line and first read peter in variable and also cross check with /var/ how many directories are avaialble... (8 Replies)
Discussion started by: learnbash
8 Replies

10. UNIX for Advanced & Expert Users

Delete Duplicate Cronjobs

I set up same cronjobs in two different users to generate messages at 5:30 AM Not Its generating duplicate messages. I want to delete the cron entries set up in the first user, but I am unable to view the entries in that user. I tried to find the process Id, but its not showing any id Could... (2 Replies)
Discussion started by: nskworld
2 Replies
Login or Register to Ask a Question