UNIX for Dummies Questions & Answers

Ok, so I'm new to my company and I don't have a lot of permissions. I'm using (ksh) Unix and I have two questions.

1.) When I grep for a string and it lists this, the directories are : cat,dog,mouse

grep: can't open dog
grep: can't open cat

I know that this means that I don't have permission to these files, but does it also mean that grep searched mouse and didn't find what I was looking for?

2.) How can I grep in a directory but ignore one specific directory? I tried

grep -il "string" * | grep -v "file.out"

Thanks, Brick

Grep works on files, not directories. When you give it a directory it'll complain.

If mouse is a file and nothing was printed, then nothing in mouse matched. Whether grep was looking for what you thought it was, I have no idea, since you didn't say what you actually typed. That doesn't go inside directories anyway, so will be be complaining about then ignoring any directories * finds.

What do you want grep to look in and what do you want it to ignore?

In portable grep, it works against files or stdin. So... grep doesn't really process directories. When you supply a globbing wildcard like "grep some-string *", the asterisk is expanded by the shell itself to match all patterns in the directory location your in.

If you want to perform a grep on files inside of a whole directory tree, look at using "find" instead. Despite the name, "find" is not grep... all find does is find files (files can be directories.... but not important for right now). There is an "-exec" option to "find" that will allow you to execute a command for matched file patterns, but likely you may want to pipe the output of "find" to "xargs" to do the grep command....

There is also a "-prune" option to find that can be used to exclude "find" matches and it can be used to match a directory name and it's contents will not be searched through. However, the "-prune" option works from a valid match, which may mean that the pattern specified could match in multiple places within a tree... making the "-prune" perhaps not so desirable. I know this is getting a bit confusing.

Consider:

Code:

find . -type f -print

That returns all files in the current directory and all files within the whole tree.

Let's say one of the directories underneath the current directory is called "test3"... and let's say that it contains a file called "test.txt". Then:

Code:

find . -type d -name test3 -prune -o -type f -print

This lists out all of the files but skipping any and ALL directories called test3 (you can see how that might not be exactly what you want since a directory called test1 underneath the current directory could also contain a test3 directory and the files there would not be output either).

Ok.. so far, all we're doing is generating a list of files to be searched on... here's where "xargs" comes in.

Code:

find . -type d -name test3 -prune -o -type f -print | xargs grep somestring

That should grep for somestring inside of all files that are not nested below a directory called "test3". However, there are STILL some problems.

First problem is that we probably like it (??) when grep tells us the name of the file where the match resides. The "xargs" command will process the file list coming into it and invoke whatever command (i.e. grep in our case) however many times needed to process the most files that will fit in the shell buffer (imagine find returning a large set of files... you can't invoke grep with a large glob matching wildcard either... blows up the max size of the shell line buffer... so xargs executes grep how ever many times as needed to process the entire list of files coming into it). The problem is when grep is invoked on just a single file... by default, you won't get the filename printed for the match. One solution is to pass a dummy extra file coming into the xargs at the very end. This ensures that a final one file grep is performed on our dummy file.

Code:

(find . -type d -name test3 -prune -o -type f -print; echo .) | xargs grep somestring

Of course.. all of that only works if none of the files returned contain certain special characters, for example, spaces. Spaces mess everything up. In the case with spaces, you either have to use GNU find and the -print0 option (and the corresponding GNU xargs -0 option)... or you can simulate the same sort of thing by filtering the output and escaping the unwelcome filename characters with an intermediate "sed" (for example) in between the "find" and "xargs" pipelines... but I've probably already said more than you needed to know. Probably confused more than helped...