Excluding directories from a find


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Excluding directories from a find
# 1  
Old 08-10-2012
Excluding directories from a find

I've looked at a few similar threads, but I can't bridge from those examples to what I'm working on, so I'm hoping someone can help.

I want to extend the following statement
Code:
find $PathToCheck -type f \( -not -iwholename "$ScriptDir/*" \) -exec md5sum "{}" \;>$NewSigs

to exclude several directories that are below the directory I am searching.

When I used -name like in this post

w w w . unix.com / unix-dummies-questions-answers/16921-question-non-recursive-find-syntax.html
(had to put in spaces... forum wouldn't let me post otherwise)
Moderator's Comments:
Mod Comment The URL is:
https://www.unix.com/unix-dummies-que...nd-syntax.html

After a few honest posts this site will allow you to post links (it's an anti-spammer measure).


I got
Code:
find: warning: Unix filenames usually don't contain slashes (though pathnames do). (etc.)

and the find didn't work.

The path that I am passing in the variable ($PathToCheck) looks like:
Code:
/home/username/

and the paths I want to exclude (also want to pass in variables) will look like
Code:
/home/username/public_html/somedirectory
/home/username/public_html/otherdirectory/etc/ignorethis

(There are many other directories under /home/username/public_html/ that I want to search - just in case that information makes a difference.)

I'm trying to write my script so I can easily configure multiple exclude directories in (a) shell variable(s), but as long as I know how to write the general form of the statement with 2 or 3 exclusions, I should be able to work the rest out.

Any help would be most appreciated.

Last edited by methyl; 08-12-2012 at 02:20 AM.. Reason: Please use code tags for data and code samples
# 2  
Old 08-10-2012
Hi

Try this:


Code:
$ X="-name public_html/somedirectory -o -name public_html/otherdirectory/etc/ignorethis"
$ find . -type d  \( $X \) -prune -o  -print  -exec md5sum '{}' \;

Guru.
# 3  
Old 08-10-2012
Thanks for the quick reply guruprasadpr

I tried
Code:
ScriptDir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"  # Evaluates to /home/username/wd
X="-name $ScriptDir/ -o -name /home/username/mail"
find $PathToCheck -type d  \( $X \) -prune -o  -print  -exec md5sum '{}' >TestFind.txt

and got the following:
Code:
find: warning: Unix filenames usually don't contain slashes (though pathnames do).

That means that -name '/home/username/wd/'' will probably evaluate to false all the time on this system. You might find the -wholename test more useful, or perhaps -samefile. Alternatively, if you are using GNU grep, you could use find ... -print0 | grep -FzZ '/home/username/wd/'.
find: warning: Unix filenames usually don't contain slashes (though pathnames do). That means that -name '/home/username/mail' will probably evaluate to false all the time on this system. You might find the -wholename test more useful, or perhaps -samefile. Alternatively, if you are using GNU grep, you could use
Code:
find ... -print0 | grep -FzZ '/home/username/mail'
find: missing argument to '-exec'

Also tried this
Code:
ScriptDir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"  # Evaluates to /home/username/wd
X="-name $ScriptDir/ -o -name /home/username/mail"
find $PathToCheck -type d  \( $X \) -prune -o  -print  -exec md5sum "{}" \;>TestFind.txt

and got a ton of error messages md5sum: /../.../../: Is a directory. And all the directories under /home/username/mail were still included. -print isn't needed because I only need the output from the md5sum, and "" must be used for the substituion from find to work. The find must only give files, not directories so I also tried:

Code:
ScriptDir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"  # Evaluates to /home/username/wd
X="-name $ScriptDir/ -o -name /home/username/mail"
find $PathToCheck -type f  \( $X \) -prune -o  -exec md5sum "{}" \;>TestFind.txt

which still gave a ton of errors about Is a directory and both the directories that should have been excluded were processed.

Any suggestions?

Last edited by methyl; 08-12-2012 at 01:15 AM.. Reason: Please use code tags and avoid embedding code in text. If you get an error message, please post the exact error message.
# 4  
Old 08-10-2012
Well, think this'll work - if you need more complicated directory matching you could just get the entire list of directories and then use grep -v instead of the find -type d:

Code:
#!/bin/sh
STARTDIR="/home/XXX/.cache"
EXCLDIRS="Thunar sessions"
exclude=""

for dir in ${EXCLDIRS}; do 
    exclude="${exclude} -name ${dir} -o"
done

# Lil sloppy, but just kill the -o at the end.
exclude="`echo $exclude | sed 's/-o$//'`"

# Run the find - this will produce all the directories we -want-
find ${STARTDIR} -type d -a \! \( ${exclude} \) -print | while read line; do
    # and this will run whatever we want against the files in them...
    find ${line} -type f -exec md5sum {} \;
done

# 5  
Old 08-10-2012
Quote:
Originally Posted by nixie
I want to extend the following statement
Code:
find $PathToCheck -type f \( -not -iwholename "$ScriptDir/*" \) -exec md5sum "{}" \;>$NewSigs

to exclude several directories that are below the directory I am searching.
Code:
find "$path" -type d \( -iwholename "$ScriptDir" -o -iwholename "$anotherDir" \) -prune \
          -o -type f -exec md5sum {} +

This approach should also run faster because md5sum is no longer called once per file.

Regards,
Alister

---------- Post updated at 11:33 AM ---------- Previous update was at 11:22 AM ----------

Quote:
Originally Posted by Vryali
Code:
# Run the find - this will produce all the directories we -want-
find ${STARTDIR} -type d -a \! \( ${exclude} \) -print ...

The original solution excludes all files below the excluded directory. Your approach will include files in subdirectories below any excluded directories.

On an unrelated note, we do not know for certain if there are directories with whitespace in their names (increasingly common these days). If there are, your solution won't work.

Regards,
Alister
This User Gave Thanks to alister For This Post:
# 6  
Old 08-11-2012
Thanks for the replies Vryali & Alister

A special thanks to Alister this:
Code:
find "$path" -type d \( -iwholename "$ScriptDir" -o -iwholename "$anotherDir" \) -prune \
          -o -type f -exec md5sum {} +

is exactly what I am looking for - works great & really fast as you said.

A couple of follow ups:
  1. What does the + sign do?
  2. Thanks for the comment about the whitespace-I hadn't thought about that. (I don't believe that whitespace is legal on an apache server, but the MD5 does work on files in the tree that contain white space (I tested it)). If I did need to use a directory that incudes whitespace, could I just include escape sequences in the exclude variables?
# 7  
Old 08-12-2012
@nixie
1) Post #1 was the worst formatted post I have seen in months.
2) Please use code tags when posting code or data.
3) Please avoid punctuating code with English punctuation - particularly quotes. It makes the code read like nonsense.
4) Please do not a use Microsoft character set when posting unix code. Copy/paste via Windows Notepad to get rid of weird characters which have no meaning in unix scripts.
5) Please mention what Operating System and version you have and what Shell you are using. There is much variation on the find command and you and @alister refer to much obscure syntax which is definitely not from a unix find.


@guruprasadpr
Please avoid posting untested and erroneous code. Where does $X come from? How does you code work for multiple directories?

Last edited by methyl; 08-12-2012 at 02:22 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Find command excluding directories and some files

hello. I try to print a list of files but excluding some directories and some files. I would like to write a command for : find "from_dir" "ignore dir1, dir2, ..." "ignore file1, file2,...." "where file are older than 2017-02-03T06:00:00" Note that "DO_IT" is a local function in the script... (5 Replies)
Discussion started by: jcdole
5 Replies

2. UNIX for Advanced & Expert Users

Find all files in the current directory excluding hidden files and directories

Find all files in the current directory only excluding hidden directories and files. For the below command, though it's not deleting hidden files.. it is traversing through the hidden directories and listing normal which should be avoided. `find . \( ! -name ".*" -prune \) -mtime +${n_days}... (7 Replies)
Discussion started by: ksailesh1
7 Replies

3. Solaris

Flarcreate and excluding directories with software installed

Got a solaris 10 server that I want to take a flar of in order to use to build a new server. Works fine. Only trouble is there is software on the original server that is not needed on the new server - it will be uninstalled. I was thinking of creating the flar from the original server and... (0 Replies)
Discussion started by: psychocandy
0 Replies

4. Shell Programming and Scripting

Copy files/directories excluding multiple paterns

my directory structure is like below: basedir\ p.txt q.htm r.java b\ abc.htm xyz.java c\ p.htm q.java rst.txt my requirement is i want to copy all the files and directories... (0 Replies)
Discussion started by: ajayyadavmca
0 Replies

5. UNIX for Dummies Questions & Answers

List directories and sub directories recursively excluding files

Hi, Please help me, how to get all the direcotries, its sub directories and its sub directories recursively, need to exclude all the files in the process. I wanted to disply using a unix command all the directories recursively excluding files. I tried 'ls -FR' but that display files as... (3 Replies)
Discussion started by: pointers
3 Replies

6. UNIX for Dummies Questions & Answers

Copy Directories excluding files

Hi guys, I want to copy folder and sub folders only. I don't want the files. If i use cp -r command it will copy entirely with files. Could any one suggest me. Thanks in advance (1 Reply)
Discussion started by: karthik82
1 Replies

7. UNIX for Dummies Questions & Answers

Excluding directories with find

How do I exclude directories with the find command on Solaris? I want to skip the directories /proc and /shared. find / -nouser -print This shows me all files and directories that don't have an owner but I need to skip /shared and /proc. I've been able to get it to work on Linux... (3 Replies)
Discussion started by: x96riley3
3 Replies

8. Shell Programming and Scripting

cp -r excluding certain directories?

I want to recursively copy /home/me/someProject/* to a /home/you/ but I want to exclude directories called "classes". I can't find any option for excluding certain directories. Does such a thing exist, or any workaround, or am I missing something obvious> (2 Replies)
Discussion started by: sarnobat
2 Replies

9. Shell Programming and Scripting

excluding directories in tar

In a bash script I am writing I am having a problem excluding selected directories from tar. From the machine $SERVER I issue the command #start netcat on storage server gnetcat -l -vv -p 2011 >$FILEPATH/$SHORT_NAME.$today.tar & The the following command is then sent to the $CLIENT. #start... (2 Replies)
Discussion started by: thumper
2 Replies

10. UNIX for Dummies Questions & Answers

excluding directories while using tar

How do I exclude some directories while creating a tar file with a number of directories? thanks. (2 Replies)
Discussion started by: uchachra
2 Replies
Login or Register to Ask a Question