UNIX for Dummies Questions & Answers

Hello.

I consider myself a journeyman Unix user. I can handle most day to day tasks and have created some fairly complicated command queries, but this problem has defeated me so far. Any help would be greatly appreciated!

We keep an archive in the file system with a directory structure in the form of

Code:

/[EntityName]/[Year]/[1-12]

When we need to find data in the archive we do a recursive search with grep. This worked fine for awhile, but now that we have several years worth of data it is taking too long. I would like to narrow the search by year, but I can't figure out a good way to do so since the year is two directory levels down. I've found answers on Google, but most use "--include" which my version of grep does not allow (AIX).

I have found one command that works, but it is still very slow:

Code:

find ./*/2013/* -type d -exec grep -rl "string to search" {} \;

I also don't understand why I have to include the -r flag, but

Code:

find ./*/2013/* -type d -exec grep -l "191684520" {} \;

does not work

Is there anything I can do to make this go faster? I've experimented with using xargs instead of the -exec flag, but don't really understand it.

You need -r because you are telling find to find directories, and only directories, via -type d.

As for speeding it up it is quite possibly running at maximum speed already -- there's no "extra-fast" way to read a directory short of speeding up the disk. But you could try improving it with xargs so it doesn't run grep more times than it needs to. This may mess up if your file or directory names contain spaces.

Code:

find ./*/2013/* -type f | xargs grep -l "191684520"

I guess I don't understand the -exec flag as well as I thought I did. I thought that I was gathering a list of directories to feed the grep command. Obviously I have a lot to learn. Thank you very much for your help!

If there is any chance that any filenames in your file hierarchy contain whitespace characters (space, tab, or newline), xargs won't work. Even in cases where it will work, using it is usually inefficient. It will probably also go faster if you let find walk the directories instead of having the shell give a longer list of directories to find to process (and may also avoid arg list too long issues). Instead of:

Code:

find ./*/2013/* -type f | xargs grep -l "191684520"

try:

Code:

find ./*/2013/ -type f -exec grep -l "191684520" {} +

Sometimes even a simple

Code:

grep -l "191684520" */2013/*/*

works.

I could have sworn that I had tried that and it did not work. I must have made a typo somewhere. Thanks for the simple solution.