How do I use the grep command to take mutiple text files in a folder and make one huge text file out of them. I'm using Mac OS X and can not find a text tool that does it so I figured I'd resort to the BSD Unix CLI for a solution... there are 5,300 files that I want to write to one huge file so that I can import it into a database in one shot... any help would be geratly appreciated. Thanx.

Why do you have to use the grep command?

cd your_folder
for i in `ls *`
do
pg $i >> all
done

With 5,300 files, the `ls *` will probably exceed the max line length and pg is a slow way to cat a file.

will work. If the xargs program is available,

(ls * | xargs cat ) > all
will be a very fast solution.

I also wonder about the mention of grep. If you need to extract certain lines switch "cat" with "grep string" in both solutions.

Wow - I tried
(ls * | xargs cat) > all

It works the same and it's cool that it's so small.. but I keep getting the error:

cat: input/output files 'all' identical

The large file is created like it should be, but is there something additional to get rid of / suppress this error?

that error is just letting you know that it also tried to run on the "all" file and it failed.

nothing to worry about.

the error message doesn't meen anything however if there are a lot of files in the folder it can be annoying to get all of those messages

you can get rid of those error messages by sending them to another file.

the command

ls * | xargs cat > all 2> /dev/null

works on my system (Sun,solaris)
it may work on yours also

Ouch! First, that line has the same problem that your original solution did, we are asking the shell to replace * with a list of all files in the current directory. And that won't fly with 5,300 files. What I meant to type is:

(ls | xargs cat) > all

I also didn't think about the output file popping into existence before the ls ran. The best solution for that is something like:

(ls | xargs cat) > /some/dir/all

cat is smart enough to catch this, but grep isn't. And I'm still thinking that the op wanted to use grep to extract certain lines.