UNIX for Dummies Questions & Answers

I need to concatenate about a thousand files (using a loop) on a UNIX server. Also, each file has a header row which is to be removed. Which of the following would give better performance?

tail+2 <filename> or sed '1,1d' <filename>

Or is there another, faster way?

Thanks,
Kaus

here is my cut,

have a parent and child process

parent process - remove the header information from the file and pass it on the filename to the child process
till child completes wait on child's pid

child process - receive the filename from the parent and start appending the file to one common file

so atlast, there would be one file which contains the contents of all the 'n' files after header information is removed

A faster way would be to write a small C program to read each file, find the second line in the file, and append the remainder of the file out to the new (concat'ed) file.

one more,

Code:

ls file_list | while read file
do
awk ' NR != 1 { print }' $file 
done > one_big_output_file

If the filenames are not too long (list too long error):

Code:

awk FNR\>1 *>final

Adjust * for your needs.

Use nawk or /usr/xpg4/bin/awk on Solaris.


If you have GNU tail:

Code:

tail -qn2 *>final