Shell Programming and Scripting

I need a Shell script which take two inputs which are
1) main directory where it has to search and
2) pattern to search within main directory all files (.c and .h files)
It has to print number of pattern found in main directory & each sub directory.
main dir --> Total pattern found = 5
|
sub dir --> 3
|
sub dir --> 2

This is not a very elegant solution, but as a starting point try

Code:

find  /main_dir -type d -exec ./countit {} pattern \;

where countit is a shell script:

Code:

cnt=$(ls $1/$2 2>/dev/null|wc -w)  
if [ "$cnt" -gt 0 ]; then echo $1": "$cnt; fi

and main_dir and pattern need to be supplied by you.

---------- Post updated 08-08-12 at 09:49 AM ---------- Previous update was 07-08-12 at 11:23 AM ----------

Look at this thread to find a less clumsy solution than mine above. Still not too performant...

---------- Post updated at 09:57 AM ---------- Previous update was at 09:49 AM ----------

This is really performant, provided you have dirname on your system:

Code:

find main_dir -name pattern -exec dirname {} \;|uniq -c|awk '{print $2 " -> "$1}'

Hello, RudiC:

Don't take any of what follows personally. It is intended solely as a helpful critique.

None of the solutions quoted below is very good.


Filenames with n occurrences of embedded whitespace will be counted n+1 times. wc -l would be a better choice.

If the pattern matches a directory name, that subdirectory's contents will be counted even if they do not match the pattern. ls -d will prevent this, but will not prevent the matching directory from being counted if the intent is to count only files.

If pattern were the script's first argument, the script would be compatible with the much more efficient -exec ... {} + syntax. The body of the script could then be put within a for-loop iterating over "$@".

Is there even any point in using ls for this? A for-loop which expands the pattern could easily sidestep all of these issues. Within the loop, test can avoid counting anything that isn't a regular file. Also, using the pattern to generate arguments for ls may face a stricter system length limit than the shell for-loop's list expansion.

In my opinion, it's not worth trying to fix this approach's bugs. Better to abandon it.



From that post ...

Launching an entire shell once per filename is not an efficient approach.

If the pathname has whitespace or begins with a dash, there will be problems.

Why does that code make ls work harder for no reason? It is generating the long format and forcing a reverse time sort when the only thing done with the output is a line count?



This suggestion is nearly a very good one. Unfortunately, it won't yield the desired result.

find will very likely not generate all of a directory's contents in one contiguous chunk. It will begin outputting file names from dir A (for example), then descend into A/B, then A/B/C, then back up to A/B, and finally resume where it left off in A. Even if your find does not behave that way, it is allowed to do so. When this happens, the result is multiple, non-consecutive counts for the same directory.

The output of find needs to be sorted before uniq sees it. Also, I think that instead of executing dirname once per filename, it would be better to use one instance of sed to filter find output.

Code:

find main_dir -name pattern | sed 's#/[^/]*$##' | sort | uniq -c

For a massive amount of files, that sort could require a lot of memory. If necessary, one can trade memory for cpu by executing find once per directory (still much better than a full shell once per file):

Code:

find main_dir -type d -exec find {} -maxdepth 1 -name pattern \; | sed 's#/[^/]*$##' | uniq -c

If maxdepth is not available, recursion can still be avoided with a slightly cumbersome use of -prune.

Something to keep in mind: In some of the approaches the pattern is expanded by the shell and in others it's passed to find. The shell will not match a hidden file against a leading wildcard (?, *); find will.

Regards,
Alister

Addition Absolutely not! You may have inferred from the various edits that I have taken a multi step approximation to the problem and its solution. I wasn't happy with the first ones either, running scripts or multiple shells when descending directory trees. I'll carefully analyse your proposal, as I'm "always learning" (agama's motto) and I really appreciate every single of your posts.

On the other hand, it is quite intimidating to know that my posts are being scrutinized that carefully!