Shell Programming and Scripting

dear there,

this kept me awake last night, the variable ${TO} in the following script doesn't seem to hold its value.

I have a file ./filelist, which lists all files of interests. I group them by keywords in the filename, and would like to count total number of lines in each group.
---------------------------------------
#!/bin/bash -xv

TO=1

for i in key1 key2;do
TO=0
grep $i ./filelist |\
while read FILE;do
let TO=$((${TO}+`cat $FILE|wc -l`))
echo "total number of lines in file whose name containing $i increased to ${TO}"
done
echo "total number of lines in file whose name containing $i finally
equal to ${TO}"
done
------------------------------------
the content of ./filelist:
key1_week1.csv
key2_week2.csv
key1_week3.csv
key2_week4.csv
--------------------------------
note: the number of lines in each file listed above is 3, 2, 5, 9
--------------------------------

the output is wierd since the final total is wrong while the running total are right:
number of lines in file whose name containing key1 increased to 3
number of lines in file whose name containing key1 increased to 8
total number of lines in file whose name containing key1 is 0
number of lines in file whose name containing key2 increased to 2
number of lines in file whose name containing key2 increased to 11
total number of lines in file whose name containing key2 is 0

So why ${TO} holds its value after the while loop? isn't it a global variable? This puzzled me!

Thanks for your patience!

Sincerely Yours,
Michael

If you run the for loop in a subshell, you can make that code work:

Code:

#! /bin/bash
TEST=1
(
 echo $TEST
 for i in 2 3 4 ; do
   TEST=$i
   echo $TEST
 done
)
echo $TEST

More info on variable scope here:
Subshells

dear jimbalaya, it seems doing the opposite thing. What I want is to let ${TO} to hold values after the while loop. It seems that the subshell prevents that. Since I didn;t use subshell, I expect the outer variable ${TO} will be changed inside the while loop. however it didn't, that's where the problem it.

Dear jimbalaya, I just added "echo $$" to see which shell I am in, it turned out that I am in the same shell all the time. however the value of $TO is not updated inside the while loop!
---------------------
#!/bin/bash

TO=1

for i in key1 key2;do
TO=0
grep $i ./filelist |\
while read FILE;do
let TO=$((${TO}+`cat $FILE|wc -l`))
echo "To value increased to ${TO}"
echo "we are in the shell $$"
done
echo "we are in the shell $$"
echo "TO value is ${TO}"
done
--------------------------------
output
-------------------------------
To value increased to 3
we are in the shell 12851
To value increased to 8
we are in the shell 12851
we are in the shell 12851
TO value is 0
To value increased to 2
we are in the shell 12851
To value increased to 11
we are in the shell 12851
we are in the shell 12851
TO value is 0

Sorry, looking at the code i thought that's what you were trying to do.

Can you give an example of what the expected output should be?

Commands after a pipe also run in a subshell.
Use a for loop:

Code:

for FILE in "`grep $i ./filelist`"; do

instead of:

Code:

grep $i ./filelist |\
while read FILE;do

Regards

Ok, appears that the while loop using a pipe was also a subshell, so that was the problem.

Code:

#! /bin/bash
TO=1

for i in key1 key2 ; do
  TO=0

  for FILE in $(grep $i ./filelist); do
    let TO=$(( ${TO} + `cat $FILE|wc -l` ))
    echo "total number of lines in file whose name containing $i increased to ${TO}"
  done


  echo "total number of lines in file whose name containing $i finally equal to ${TO}"
done

this prints:

Code:

total number of lines in file whose name containing key1 increased to 3
total number of lines in file whose name containing key1 increased to 8
total number of lines in file whose name containing key1 finally equal to 8
total number of lines in file whose name containing key2 increased to 2
total number of lines in file whose name containing key2 increased to 11
total number of lines in file whose name containing key2 finally equal to 11

Edit: looks like someone found the fix before me.