Delete duplicate directories?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Delete duplicate directories?
# 8  
Old 08-27-2016
RudiC is right. Clobbering directories wholesale like this can cause an awful mess. As an example: Are you sure you do not have symbolic links maybe from TEST2 to TEST1 or from TEST1 to TEST2? This would mean you do not have duplicates the way you think.

Code:
find TEST1 TEST2 -type l

would be a good thing to try before doing anything else.
# 9  
Old 08-27-2016
As a further example of what jim mcnamara and RudiC are talking about, consider the following directories in your two file hierarchies:
Code:
/TEST1
/TEST1/dir1
/TEST1/dir1/dir1
/TEST1/dir3
/TEST1/dir3/dir3
/TEST1/subdir

/TEST2/dir1
/TEST2/dir1/dir1
/TEST2/dir2
/TEST2/dir2/dir2
/TEST2/dir2/dir2/subdir

your script will print:
Code:
2: dir2
2: dir3
2: subdir

(although the order of these lines in the output is not specified). I assume that you are then going to attempt to run the commands:
Code:
rm -rf /TEST1/dir2
rm -rf /TEST1/dir3
rm -rf /TEST1/subdir

Assuming that you have appropriate permissions to remove all of the files under /TEST1:
  • the first of those commands will silently fail (since there is no directory named /TEST1/dir2 (nor any directory named dir2 anywhere under the file hierarchy rooted in /TEST1),
  • the second of those commands will succeed (even though there is no directory named dir3 anywhere in the file hierarchy rooted in /TEST2,
  • the third of those commands will succeed (even though the matching directory name (subdir) is at different levels in the /TEST1 and /TEST2 file hierarchies), and
  • the directories /TEST1/dir1 and /TEST1/dir1/dir1 will remain untouched (because your script will have assigned the value 4 to D["dir1"]) even though the directories /TEST1/dir1 and /TEST1/dir1/dir1 would seem to be the only directories that are in common with the file hierarchy rooted in /TEST2.
This makes perfect sense to me if, and only if, there are no directories in any of the directories located in the directories /TEST1 and /TEST2 (unlike the file hierarchies in my example above). Is this really what you want to do?
These 2 Users Gave Thanks to Don Cragun For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Find duplicate values in specific column and delete all the duplicate values

Dear folks I have a map file of around 54K lines and some of the values in the second column have the same value and I want to find them and delete all of the same values. I looked over duplicate commands but my case is not to keep one of the duplicate values. I want to remove all of the same... (4 Replies)
Discussion started by: sajmar
4 Replies

2. Shell Programming and Scripting

Delete duplicate row

Hi all, how can delete duplicate files in file form, e.g. $cat file1 aaa 123 234 345 456 bbb 345 345 657 568 ccc 345 768 897 456 aaa 123 234 345 456 ddd 786 784 234 263 ccc 345 768 897 456 aaa 123 234 345 456 ccc 345 768 897 456 then i need ouput file1 some, (4 Replies)
Discussion started by: aav1307
4 Replies

3. Programming

Finding duplicate files in two base directories

Hello All, I have got some assignment to complete till this Monday and problem statement is as follow :- Problem :- Find duplicate files (especially .c and .cpp) from two project base directories with following requirement :- 1.Should be extendable to search in multiple base... (4 Replies)
Discussion started by: anand.shah
4 Replies

4. Shell Programming and Scripting

Delete duplicate rows

Hi, This is a followup to my earlier post him mno klm 20 76 . + . klm_mango unix_00000001; alp fdc klm 123 456 . + . klm_mango unix_0000103; her tkr klm 415 439 . + . klm_mango unix_00001043; abc tvr klm 20 76 . + . klm_mango unix_00000001; abc def klm 83 84 . + . klm_mango... (5 Replies)
Discussion started by: jacobs.smith
5 Replies

5. Shell Programming and Scripting

Create duplicate directories with same permissions

Hi all, I need to create duplicate directories and sub directories (only the directories not the files or file contents) with the same permissions. Can some one guide me in doing this. I could able to create but here the permissions should be the same how can i do this in linux. Thanks in... (5 Replies)
Discussion started by: Olivia
5 Replies

6. UNIX for Dummies Questions & Answers

Delete duplicate second line

Hi ALL I need a help I need to retain only the first line of 035 if I have two line before =040 , if only one then need to take that Eg: Input =035 (ABC)12324141241 =035 (XYZPQR)704124 =040 AB$QS$WEWR =035 (ABC)08080880809 =035 (XYZPQR)9809314 =040 ... (4 Replies)
Discussion started by: umapearl
4 Replies

7. UNIX for Dummies Questions & Answers

Duplicate directories

I have noticed that the same folder (and contents) lives in /u/public and /usr/public Question was this put here intentionally or by accident? Its 31Gb in size and on a 72Gb HDD that leaves little room for apps. It is a nework shared drive for all to access e.g. p: points to... (0 Replies)
Discussion started by: moondogi
0 Replies

8. Shell Programming and Scripting

Remove Duplicate Filenames in 2 very large directories

Hello Gurus, O/S RHEL4 I have a requirement to compare two linux based directories for duplicate filenames and remove them. These directories are close to 2 TB each. I have tried running a: Prompt>diff -r data1/ data2/ I have tried this as well: jason@jason-desktop:~$ cat script.sh ... (7 Replies)
Discussion started by: jaysunn
7 Replies

9. Shell Programming and Scripting

duplicate directories

Hi, I have file which users like filename ->"readfile", following entries peter john alaska abcd xyz and i have directory /var/ i want to do first cat of "readfile" line by line and first read peter in variable and also cross check with /var/ how many directories are avaialble... (8 Replies)
Discussion started by: learnbash
8 Replies

10. UNIX for Advanced & Expert Users

Delete Duplicate Cronjobs

I set up same cronjobs in two different users to generate messages at 5:30 AM Not Its generating duplicate messages. I want to delete the cron entries set up in the first user, but I am unable to view the entries in that user. I tried to find the process Id, but its not showing any id Could... (2 Replies)
Discussion started by: nskworld
2 Replies
Login or Register to Ask a Question