Swings and roundabouts. For data amounting to a million lines my suggestion is slightly slower for say 20,000 small files but faster for a small number of large files because "cat" is more efficient than "wc" at reading from disc. My method gives much less of a cpu hit. In the real world I use both constructs.

find . -type f | xargs cat |wc -l[COLOR="#738fbf"]

replacing . with the directory name ;-)

(not sure what good timing wc is. But interesting. I'm guessing loading two programs (wc and cat) into memory is slower that loading just one? I'd love it if Microsoft would bear that in mind! (oops, probably violated a rule there... sorry)

Hmm. "scottn" idea breaks if any filenames contain space characters.


I ran my test a few times first to eliminate o/s first-time buffering before posting those figures. Results were still interesting.

The "pludi" solution is very good and exhibits lateral thought, but on this occasion the UUOC argument is arguable because on my system "wc" is less efficent at reading files from disc than "cat".

BTW. I can produce the required output by using only "find" and shell commands but it proved to be horrendously slow to read large volumes of data with a shell read.



There is life in the old cat yet!

Sorry, I got a bit confused as to what thread I was writing in when I said that. I meant no disrespect. It is interesting.

You're right, but Unix filenames don't "normally" have spaces.

Time: 8 - 10 seconds (on some FS full of rubbish on my system):
Time: 2 - 3 seconds (but doesn't handle some files):
Ergo: awk rocks (and cat is cool)

Did you run each example more than once, or are the examples you posted exactly what you did test?

After the first run, depending on your filesystem and OS, there's a good chance that a lot of the data is going to be cached and not read from disk.

you only had one run of each. do more runs and find out.