UNIX for Advanced & Expert Users

I've a list file, which has some file names.
ex: list file "list_file" will have

data_file1.txt
data_file2.txt
data_file3.txt
:
:
data_filen.txt


Above files will have the below layout:

Header1
Header2
*TM*
Data record 1
Datarecord 2
Datarecord n
*TM*
Trailer1
Trailer2
*TM*
*TM*
EOF


I've to read all the data file name one by one and extract only data records to single file "complete.txt".

I've done the shell script for the above work. But my Manager is suggested me to simplify the code.
Are there any simple logic using awk to accomplish the same?

Please help me...

With Regards / Lokesha

Code:

for file in `ls data_file*.txt`
do
   awk '/^Data/' $file
done > complete.txt

Thanks Jim mcnamara,

But you are not reading "list_file" which will have the only file names that are need to select.

Also we can't select the data as '/^Data/', because it is not always staring with the same contents. So we have to extract the data with in two
*TM*.

Any idea???

If the filenames contain no spaces:

Code:

awk 'f[FILENAME]==1&&!/^\*TM\*/;/^\*TM\*/{f[FILENAME]++}' $(<list_file)

Use nawk or /usr/xpg4/bin/awk on Solaris.

Otherwse:

Code:

(IFS=$'\n';awk 'f[FILENAME]==1&&!/^\*TM\*/;/^\*TM\*/{f[FILENAME]++}' $(<list_file))

P.S. If your shell doesn't expand $'\n' to a newline, use IFS='
'

ok:

Code:

while read file
do
   awk '/^Data/' $file
done <  list_file > complete.txt

Try the below code

With Awk only:

Code:

awk '{ f[NR] = $0 
} END { 
  for (k = 1; k <= NR; k++) {
    while ((getline < f[k]) > 0) { 
      if (p[f[k]] == 1 && $0 !~ /^\*TM\*/)
        print > complete
      if ($0 ~ /^\*TM\*/)
        p[f[k]]++
    }
  }
}' complete="complete.txt" list_file

Use nawk or /usr/xpg4/bin/awk on Solaris.