FINDING DUPLICATE PROJECT ( directory project )


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting FINDING DUPLICATE PROJECT ( directory project )
# 1  
Old 09-21-2015
FINDING DUPLICATE PROJECT ( directory project )

I have a project tree like that.

Quote:
/dir_a
/dir_a/dir_1
/dir_a/dir_1/dir_1-1 (empty)
/dir_a/dir_1/dir_1-2/dir_1-2-1/dir_1-2-1-1/some_project/some_project.txt
/dir_a/dir_1/dir_1-2/dir_1-2-1/dir_1-2-1-1/some_project/some_project.tiff
/dir_a/dir_1/dir_1-2/dir_1-2-2 (empty)
/dir_a/dir_1/dir_1-3/dir_1-3-1 (empty)
/dir_a/dir_1/dir_1-3/dir_1-3-2/dir_1-3-2-1/dir_1-3-2-1-1/file_1-3-2-1-1.sh
/dir_a/dir_1/dir_1-3/dir_1-3-3 (empty)
/dir_a/dir_1/dir_1-4/a_project/a_project.doc
/dir_a/dir_1/dir_1-4/some_project/some_project.tiff
/dir_a/dir_1/dir_1-4/some_project/some_project.pdf
/dir_a/dir_2/dir_2-1 (empty)
/dir_a/dir_3 (empty)
/dir_a/dir_4/dir_4-1/dir_4-1-1/dir_4-1-1-1/some_project/some_project.txt
/dir_a/dir_5/dir_5-1/dir_5-1-1 (empty)
/dir_a/dir_6/dir_6-1/another_project/another_project.doc
/dir_a/dir_6/dir_6-1/another_project/another_project.tiff
/dir_a/dir_6/dir_6-1/another_project/another_project.pdf
/dir_a/dir_7/dir_7-1/dir_7-1-1/dir_7-1-1-1/dir_7-1-1-1-1/dir_1-3-2-1-1/file_1-3-2-1-1.kmz
/dir/a/dir_8/another_project/another_project.tiff
after running find command with the -no -empty option, i am able to have a list of non empty directory

Code:
    DO_MY_SEARCH="find .    -type d -not -empty -print0"
MY_EXCLUDE_DIR1="  -e NOT_IN_USE -e RTMAP -e NOT_USEFULL   "
echo " " > $MY_TEMP_RESULT_1
while IFS= read -r -d '' file; do
        CURRENT_DATA=$file
        echo $CURRENT_DATA | grep -v ${MY_EXCLUDE_DIR1} >> $MY_TEMP_RESULT_1
    done < <($DO_MY_SEARCH)

Quote:
/dir_a/dir_1
/dir_a/dir_1/dir_1-2
/dir_a/dir_1/dir_1-2/dir_1-2-1
/dir_a/dir_1/dir_1-2/dir_1-2-1/dir_1-2-1-1
/dir_a/dir_1/dir_1-2/dir_1-2-1/dir_1-2-1-1/some_project
/dir_a/dir_1/dir_1-3
/dir_a/dir_1/dir_1-3/dir_1-3-2
/dir_a/dir_1/dir_1-3/dir_1-3-2/dir_1-3-2-1
/dir_a/dir_1/dir_1-3/dir_1-3-2/dir_1-3-2-1/dir_1-3-2-1-1
/dir_a/dir_1/dir_1-4
/dir_a/dir_1/dir_1-4/a_project
/dir_a/dir_1/dir_1-4/some_project
/dir_a/dir_4
/dir_a/dir_4/dir_4-1
/dir_a/dir_4/dir_4-1/dir_4-1-1
/dir_a/dir_4/dir_4-1/dir_4-1-1/dir_4-1-1-1
/dir_a/dir_4/dir_4-1/dir_4-1-1/dir_4-1-1-1/some_project
/dir_a/dir_6
/dir_a/dir_6/dir_6-1
/dir_a/dir_6/dir_6-1/another_project
/dir_a/dir_7
/dir_a/dir_7/dir_7-1
/dir_a/dir_7/dir_7-1/dir_7-1-1
/dir_a/dir_7/dir_7-1/dir_7-1-1/dir_7-1-1-1
/dir_a/dir_7/dir_7-1/dir_7-1-1/dir_7-1-1-1/dir_7-1-1-1-1
/dir_a/dir_7/dir_7-1/dir_7-1-1/dir_7-1-1-1/dir_7-1-1-1-1/dir_1-3-2-1-1
/dir/a/dir_8
/dir/a/dir_8/another_project
I have some duplicate project :

1 : project some_project
/dir_a/dir_1/dir_1-2/dir_1-2-1/dir_1-2-1-1/some_project
and
/dir_a/dir_1/dir_1-4/some_project
and
/dir_a/dir_4/dir_4-1/dir_4-1-1/dir_4-1-1-1/some_project

2 : project dir_1-3-2-1-1
/dir_a/dir_1/dir_1-3/dir_1-3-2/dir_1-3-2-1/dir_1-3-2-1-1
and
/dir_a/dir_7/dir_7-1/dir_7-1-1/dir_7-1-1-1/dir_7-1-1-1-1/dir_1-3-2-1-1

3 : project another_project
/dir_a/dir_6/dir_6-1/another_project
and
/dir/a/dir_8/another_project

How to get the duplicate leaf name :
some_project
dir_1-3-2-1-1
another_project

Any help is welcome.
# 2  
Old 09-21-2015
How about
Code:
while read PROJ; do PROJ=${PROJ##*/}; echo $PROJ; done < $MY_TEMP_RESULT_1 | sort | uniq -d
another_project
dir_1-3-2-1-1
some_project

This User Gave Thanks to RudiC For This Post:
# 3  
Old 09-21-2015
Quote:
Originally Posted by RudiC
How about
Code:
while read PROJ; do PROJ=${PROJ##*/}; echo $PROJ; done < $MY_TEMP_RESULT_1 | sort | uniq -d
another_project
dir_1-3-2-1-1
some_project

seems to work on a real example.
As the real problem is more complicated, this give me the way to finish by hand.

Thank you very much
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. News, Links, Events and Announcements

A new project was posted on The UNIX and Linux Forums project board.

A new project was posted on your project board. Project title: Bash Shell Tutoring Estimated Budget: $50/hr Start date: Immediately Required skills: Linux, Bash, Shell, UNIX I work as a datawarehouse designer and developer. Although I usually stick to the role of an analyst,... (0 Replies)
Discussion started by: Neo
0 Replies

2. UNIX and Linux Applications

Need ideas for graduation project based on unix or linux Need ideas for graduation project based on

Dear all, i am in last year of electronics department in engineering faculty i need suggestions for a graduation project based on unix or free bsd or linux and electronics "embedded linux " i think about embedded unix for example or device drivers please i need helps (1 Reply)
Discussion started by: MOHA-1
1 Replies

3. Solaris

what is the use of /etc/project file and project administration commands?

i have two doubts.. 1. what is the use /etc/project file. i renamed this file and when i tried to switch user or login with some user account the login was happening slowly. but when i renamed it to original name it was working fine... why so? 2. unix already has useradd and grouadd for... (4 Replies)
Discussion started by: chidori
4 Replies

4. Linux

Help me in finding ideas for Linux Project

Hi guys.. I m newbie to this forum. Basically, i need help in my final year B.E project. I will need some ideas or hints to decide my Project Topic on/for Linux. If you ask me why i choose Linux.. then i wud say I want to contribute more to open source community and eagerly want to learn... (2 Replies)
Discussion started by: SRJSRJ
2 Replies

5. Solaris

SSH doesn't pick up user's project from /etc/project

We have a system running ssh. When a user logs in, they do not get the project they are assigned to (they run under "system"). I verify the project using the command "ps -e -o user,pid,ppid,args,project". If you do a "su - username", the user does get the project they are assigned to (and all... (2 Replies)
Discussion started by: kurgan
2 Replies

6. What is on Your Mind?

r-project

Hi folks, Any folk has experience on r-Project; The R Project for Statistical Computing Please shed me some light on its main application with examples. The package is availabl on Ubuntu repo; $ apt-cache policy r-base-html r-base-html: Installed: (none) Candidate: 2.4.1-1 ... (0 Replies)
Discussion started by: satimis
0 Replies

7. Shell Programming and Scripting

finding duplicate files by size and finding pattern matching and its count

Hi, I have a challenging task,in which i have to find the duplicate files by its name and size,then i need to take anyone of the file.Then i need to open the file and find for more than one pattern and count of that pattern. Note:These are the samples of two files,but i can have more... (2 Replies)
Discussion started by: jerome Sukumar
2 Replies

8. Linux

project

hi, iam doing my MCA finial year project in linux-c language.my project name 'stream control transmission protocol'.it is about message passing to server from client through packet,please help in this i want to know the coding for this or tell me the any link for this coding. ... (0 Replies)
Discussion started by: anurakrish
0 Replies

9. UNIX for Dummies Questions & Answers

project

i want to do hospital project in c++ or java in unix/linux platforms what are the required softwares i should have and how to install oracle,java in linux (1 Reply)
Discussion started by: nrusimha
1 Replies
Login or Register to Ask a Question