Shell Programming and Scripting

I am readinga file lin by line and based craeting a arry of unique elemenst from the second column of the line. However when i coem out of the while loop my array becomes empty , can eny one tell me what I would be doing wrong
#!/bin/bash

logfile="./mylog.dat"
begin=100
end="$(( $begin + 1000 ))!d"
index=0
Isthere=0
ENGINES=""

sed "$begin, $end" $logfile | while read line
do
i=`echo $line | awk -F "," '{print $2}'`
Isthere=0
for item in "${ENGINES[@]}" ; do
if [ "$i" = "$item" ] ; then
echo "Aleady there "
Isthere=1
break
fi
done
if [ "$Isthere" -eq 0 ] ; then
echo "adding $Isthere"
ENGINES[$index]=`expr $i`
index=$(( $index + 1))
fi
# echo $line
done
echo $index ${#ENGINES[@]} # howe ever at this point array is empty
for i in "${ENGINES[@]}" ; do
echo $i
done

Put a few lines of data in a file called test.file. Try this loop style:
cat test.file | while read line
and the array will be empty after the loop ends. Then try:
exec < test.file
while read line
and the array will have data after the loop ends. The while loop is being placed in a subshell if it is in a pipeline. ksh will not do that but other shells do.

You mean I ahve to change the below line
sed "$begin, $end" $logfile | while read line

to
exec < test.file
while read line

But I do not want to read file fully , but only from begin to end.

Change your technique to get rid of the pipeline, change your shell to ksh, or live with the empty array. These are your options. Sorry, but you can't keep the pipeline, bash, and the array contents. One has to go. Here is a simple script to illustrate the problem...

Code:

$ cat script2
#! /usr/local/bin/bash

echo "cat
dog
mouse
rabbit
lion
wolf
dog
bat
lion
hamster
rabbit
elephant
elephant
whale
cricket" > list.txt

index=0
cat list.txt | while read item ; do
                array1[index]=$item
                ((index=index+1))
done
echo array1: ${array1[@]}

exec < list.txt
index=0
while read item ; do
        array2[index]=$item
        ((index=index+1))
done
echo array2: ${array2[@]}
exit 0
$ ./script2
array1:
array2: cat dog mouse rabbit lion wolf dog bat lion hamster rabbit elephant elephant whale cricket
$

ksh is the only shell I know that will populate both arrays. This is one of the reasons that I strongly prefer ksh to other shells. (The other is ksh co-processes.) So my suggestion: switch to ksh.

Fantastic , that was new to be , I thought of getting rid of the the "disk write" happening while out putting it in to a file.

sed -n "$begin, $end" $logfile >engines
exec < engines
while read line
do
# array insert
done

Thankyou very much , I am so used t bash such that , ksh is not easy for me to use now. It does not ahve the "auto completion when we type something and the the <tab> key stroke

You can use bash as your interactive shell and still write scripts in ksh.