Hi!

The simplest way is to read the whole file in array, split it to four parts, process then and write the result in the output file. Because it's really simple and quick then perhaps you should do it in that way. There are a lot of things in the world else you can do or improve or learn.

But... There is always but, you know. :-) It is definitely not "unix way". Why?

Well. From the famous "The UNIX Time-Sharing System": "... there have always been fairly severe size constraints on the system and its software. Given the partially antagonistic desires for reasonable efficiency and expressive power, the size constraint has encouraged not only economy, but also a certain elegance of design."

You don't believe if I say what recourses did have the first Unix hosts. So I wouldn't - but the word "severe" says for itself. At those time the famous "unix philosophy" was born.

Doug McIlroy summarized it in this way: "This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface."

You can read more here - in the free and good book The Art of Unix Programming.

And what relation does all this stuff have to your question? Just see:

1. You need the header:

Code:

sed -n '/^START-OF-FILE/,/^START-OF-DATA/p' INPUTFILE >/tmp/header.$$

2. The footer:

Code:

sed -n '/^END-OF-DATA/,$p' INPUTFILE >/tmp/footer.$$

3. You can process your file with your perl script but print the name of your good file in the end of the script:

Code:

goodfile=$(perl process.pl)

Or you can print both names - good and bad one and then split them. Or you can give this name as the argument to the script. You just need to know this name.

4. What is the number of records(lines) in the goodfile?

Code:

goodrecs=$(wc -l "$goodfile")

5. The new footer:

Code:

sed 's/^DATARECORDS=.*$/DATARECORDS='"$goodrecs"'/' /tmp/footer.$$ >/tmp/newfooter.$$

6. Now

Code:

cat /tmp/header.$$ "$goodfile" /tmp/newfooter.$$ >OUTPUTFILE

7. And don't forget to clean after you:

Code:

rm /tmp/header.$$ /tmp/*footer.$$ # maybe the goodfile too

The beauty of the shell programming that you can do it incremental, in small pieces. You can test and debug your steps separately. And then, when you get the result, you just append your steps in a small, elegant, and really unix program - a shell script.

Regards,
Andrey (yazu)

===

Well. Sorry for my English. This post was really my English exercise. :-)