Shell Programming and Scripting

Thanks to everyone for your replies. I guess I owe a better explanation on what I'm trying to do.

I have a file that has a header, trailer and some detail records.
e.g. file1.txt

HEADER|value1|value2|value3
12345|value1|value2|value3
12345|value1|value2|value3
6789|value1|value2|value3
TRAILER|5|value2

where the value 5 in the TRAILER record is the record count of the file

My objective is to move the header,trailer and detail records beginning with 12345 to a new file and replace the record count on the new file. Given below is the code I have and it managed to do what I wanted (There might be some syntax issues since I'm on my windows PC and don't have access to the actual code I had written on my unix machine)

cat file1.txt | egrep -i "^[header|trailer|12345]" > tmp.txt # to move the desired records from file1.txt to tmp.txt

newcount=`cat tmp.txt | wc -l` # get the record count of tmp.txt

cat tmp.txt | egrep -i "^trailer" | cut -d "|" -f2 | ## get the second field from the trailer record in tmp.txt
while read count
do
sed "s/$count/$newcount/" > tmp1.txt
mv tmp1.txt file2.txt
done

My expected output on file2.txt is

HEADER|value1|value2|value3
12345|value1|value2|value3
12345|value1|value2|value3
TRAILER|4|value2

Instead of the WHILE loop, I wanted to know if we can directly use sed as I had intended in my original question to achieve the same result.

So, will something like

cat tmp.txt | sed "/^TRAILER/s/`cut -d "|" -f2`/$newcount/" > file2.txt

or
cat tmp.txt | egrep -i "^trailer" | sed "/s/`cut -d "|" -f2`/$newcount/" > file2.txt

work?



-Thanks

You can certainly get the results you want using cat, cut, egrep, sed, etc. but that method is compute intensive becauses of the number of forks, execs, etc. You can easily do what you want to do entirely within any modern shell.

For example the following works for both bash, and ksh93

Code:

count=1
IFS="|"
while read prefix value line
do
   if (( count == 1 )) || [[ $prefix = "12345" ]]
   then
      echo "$prefix|$value|$line"
      (( count++ ))
   fi
   if [[ $prefix = "TRAILER" ]]
   then
      echo "$prefix|$count|$line"
   fi
done < file1.txt

With awk:

Code:

awk -F"|" '
NR==1 || $1=="12345"{print;c++}
/TRAILER/{$2=++c;print}
' OFS="|" file1.txt

This is the output:

Code:

$ cat file1.txt
HEADER|value1|value2|value3
12345|value1|value2|value3
12345|value1|value2|value3
6789|value1|value2|value3
TRAILER|5|value2
$
$
$ awk -F"|" '
NR==1 || $1=="12345"{print;c++}
/TRAILER/{$2=++c;print}
' OFS="|" file1.txt
HEADER|value1|value2|value3
12345|value1|value2|value3
12345|value1|value2|value3
TRAILER|4|value2
$
$

Regards

But your script will not work in "any modern shell". You use a number of non-standard constructs all of which can be replaced by standard syntax with no loss of efficiency.

Translated to work in any standard UNIX shell:

Code:

count=1
while IFS="|" read prefix value line
do
   if [ $count = 1 ] || [ $prefix = "12345" ]
   then
      echo "$prefix|$value|$line"
      count=$(( $count + 1 ))
   fi
   if [ "$prefix" = "TRAILER" ]
   then
      echo "$prefix|$count|$line"
   fi
done < file1.txt