Bash Variable scope - while loop while reading from a file


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Bash Variable scope - while loop while reading from a file
# 1  
Old 07-30-2019
Bash Variable scope - while loop while reading from a file

Cope sample1: test.sh
Code:
i=0
echo " Outside loop i = $i "
while [ $i -le 5 ]
do
i=$(( $i + 1))
echo "Inside loop i = $i "
done
echo " Out of loop i is : $i "

When run output :
Code:
Outside loop i = 0
Inside loop i = 1
Inside loop i = 2
Inside loop i = 3
Inside loop i = 4
Inside loop i = 5
Inside loop i = 6
 Out of loop i is : 6


Code sample 2: test_1.sh

Code:
cat file1
AAAAAAAAAAA
BBBBBBBBBBB
CCCCCCCCCCC
DDDDDDDDDDD
EEEEEEEEEEE

Code:
i=0
echo " Outside loop i = $i "
cat file1 |while read line
do
i=$(( $i + 1))
echo "Inside loop i = $i "
done
echo " Out of loop i is : $i "

When run ouputput:

test_1.sh
Code:
 Outside loop i = 0
Inside loop i = 1
Inside loop i = 2
Inside loop i = 3
Inside loop i = 4
Inside loop i = 5
 Out of loop i is : 0


I'm puzzled.

In the second script (test_1.sh), I expect the output of the variable i to be 5 not 0.

Can someone help me understand why it is back to 0 ( why not 5?)
# 2  
Old 07-30-2019
Hello Adarshreddy01,

IMHO that's because you are passing output of cat file1 command to while loop which is considered as a new shell(or in other language creates a sub shell to execute while loop). So once while loop is done with execution its new value of i gets lost and when you try to print its value; it prints OLD value for i.

Now lets do this with another way(Without using cat file | while.....
Code:
cat file.ksh
i=10
echo " Outside loop i = $i "
while read line
do
i=$(( $i + 1))
echo "Inside loop i = $i "
done < "file1"
echo " Out of loop i is : $i "

When you run script file.ksh we will see following.

Code:
 Outside loop i = 10
Inside loop i = 11
Inside loop i = 12
Inside loop i = 13
Inside loop i = 14
Inside loop i = 15
 Out of loop i is : 15

I believe that is proved why in your attempt i's value is getting lost.

Thanks,
R. Singh
These 2 Users Gave Thanks to RavinderSingh13 For This Post:
# 3  
Old 07-30-2019
It's good to mention that there is a major difference between bash and ksh running the same code.
Which is expected due to ksh / bash difference in implementation.

Code:
# ./in_ksh.sh
 Outside loop i = 0
Inside loop i = 1
Inside loop i = 2
Inside loop i = 3
Inside loop i = 4
Inside loop i = 5
 Out of loop i is : 5

./in_bash.sh
 Outside loop i = 0
Inside loop i = 1
Inside loop i = 2
Inside loop i = 3
Inside loop i = 4
Inside loop i = 5
 Out of loop i is : 0

Regards
Peasant.
These 2 Users Gave Thanks to Peasant For This Post:
# 4  
Old 07-30-2019
Quote:
Originally Posted by Adarshreddy01
Can someone help me understand why it is back to 0 ( why not 5?)
As Ravinder has already said it is because of the pipeline. Basically in classical Bourne Shell (and its descendants, bash - Bourne Again Shell - among them) code like this:

Code:
process1 | process2

is executed by running the process1 in the main shell and process2 in its own subshell. Therefore the while-loop you use runs in its own subshell, with its own variables and inside this shell your variables work - which is why your counter is increased - but when the subshell is left (at the end of the loop) all these variables are deleted. This is why your counter variable is zero at the end.

Notice that the other descendant of the Bourne shell - the Korn Shell, ksh - does things differently. In ksh your code would work as you obviously expect because it executes the last element of a pipeline in the main shell. You can force bash (since, IIRC version 4) to do the same but you must set this option explicitly:

Code:
shopt -s lastpipe

will set the behavior of bash to exactly the same as ksh always had. So your options are: switch to ksh (most of the code is the same but there are subtle differences between bash and ksh) or set the lastpipe option.

By the way: notice that pipelines and redirections word differently in bash, regardless of the lastpipe-setting! This simple line-counter:

Code:
(( lines = 0 ))
cat /path/to/file | while read line ; do
     (( lines++ ))
done
echo $lines

will not work (for the same reasons as your code) but:

Code:
(( lines = 0 ))
while read line ; do
     (( lines++ ))
done < /path/to/file
echo $lines

will.

I hope this helps.

bakunin

/PS: only now saw that peasant has also addressed the ksh<->bash difference. He is absolutely right.

Last edited by bakunin; 07-30-2019 at 10:45 AM.. Reason: corrected typo
These 4 Users Gave Thanks to bakunin For This Post:
# 5  
Old 07-30-2019
As is often the case with a little bit of jiggery pokery there is a pseudo-back-door.
The storage device can be your friend although it will slow things down with the disk thrashing and huge numbers.
NOTE: This uses 'dash' which means it is fully POSIX compliant.

Code:
#!/usr/local/bin/dash

i=0
echo "Outside loop i = ${i}."
cat self.txt | while read -r line
do
    # Read the line, there are 5 lines...
    i=$(( i + 1 ))
    echo "Line ${i}."
    echo "${i}" > i
done
i=$( cat i )
echo "Out of loop i = ${i}."

Results OSX 10.14.3, default terminal.
Code:
Last login: Tue Jul 30 14:28:01 on ttys000
AMIGA:amiga~> cd Desktop/Code/Shell
AMIGA:amiga~/Desktop/Code/Shell> chmod 755 subshell.sh
AMIGA:amiga~/Desktop/Code/Shell> ./subshell.sh
Outside loop i = 0.
Line 1.
Line 2.
Line 3.
Line 4.
Line 5.
Out of loop i = 5.
AMIGA:amiga~/Desktop/Code/Shell> _

This User Gave Thanks to wisecracker For This Post:
# 6  
Old 07-31-2019
Quote:
Originally Posted by wisecracker
As is often the case with a little bit of jiggery pokery there is a pseudo-back-door.
This is especially for you:

Code:
#! /bin/bash

pInitPermStor ()
{
if [ ! -d "$fStorage" ] ; then
     return $(mkdir -p "$fStorage")            # create storage dir if not there
fi

return $(rm -f "$fStorage/*")                  # empty it if it already exists
}




pRemovePermStor ()
{
if [ ! -d "$fStorage" ] ; then
     return 1                                  # shouldn't happen
fi

return $(rm -f "$fStorage")                    # clean up
}




pGetVar ()
{
local chVar="$1"

if [ -f "${fStorage}/${chVar}" ] ; then
     printf "%s=%s\n" "$chVar" $( < "${fStorage}/${chVar}" )
else
     printf "$chVar"=\"\"
fi

return 0
}


pPutVar ()
{
local chVar="$1"
local chValue="$2"

if ! printf "%s\n" "$2" > "${fStorage}/${chVar}" ; then
     return 1
fi

return 0
}


# ------------------ main()
declare fStorage="/path/to/dir/$$"

if ! pInitPermStor ; then
     printf "ERROR: cannot initialise permanent variable storage\n" >&2
     exit 1
fi

pPutVar "x" "abcd"             # set value of variable x to "abcd"
eval $( pGetVar "x" )          # retrieve and set variable x
echo var x is: $x

if ! pRemovePermStor ; then
     printf "ERROR: permanent variable storage not found\n" >&2
     exit 1
fi
exit 0

ROFL, i doubt this helps anyone but it was fun to write.

bakunin

/edited following MadeInGermanys suggestion

Last edited by bakunin; 07-31-2019 at 10:19 AM..
# 7  
Old 07-31-2019
bakunin, I suggest to once and globally define the variable
Code:
fStorage="/path/to/dir/$$"

.
because some shells run a sub shell in a separate process, and $$ changes.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

BASH: variable and function scope and subscripts

Hi, I'm a Delphi developer new to linux, new to this forums and new to BASH programming and got a new task in my work: maintaining an existing set of BASH scripts. First thing I want to do is making the code more reliable as in my opinion it's really bad written. So here's the quest: I'm... (6 Replies)
Discussion started by: rse
6 Replies

2. Shell Programming and Scripting

Detail on For loop for multiple file input and bash variable usage

Dear mentors, I just need little explanation regarding for loop to give input to awk script for file in `ls *.txt |sort -t"_" -k2n,2`; do awk script $file done which sorts file in order, and will input one after another file in order to awk script suppose if I have to input 2 or... (4 Replies)
Discussion started by: Akshay Hegde
4 Replies

3. Shell Programming and Scripting

Variable scope in bash

Hello! Before you "bash" me with - Not another post of this kind Please read on and you will understand my problem... I am using the below to extract a sum of the diskIO on a Solaris server. #!/bin/sh PATH=/usr/bin:/usr/sbin:/sbin; export PATH TEMP1="/tmp/raw-sar-output.txt$$"... (3 Replies)
Discussion started by: haaru
3 Replies

4. Shell Programming and Scripting

(BASH) Using a loop variable to grep something in a file?

Hi, I have a loop running until a variable L that is read previously in the full script. I'd like to grep some information in an input file at a line that contains the value of the loop parameter $i. I've tried to use grep, but the problem is nothing is written in the FILE files. It seems grep... (5 Replies)
Discussion started by: DMini
5 Replies

5. UNIX for Dummies Questions & Answers

Bash loops and variable scope

Hi All, I've been researching this problem and I am pretty sure that the issue is related to the while loop and the piping. There are plenty of other threads about this issue that recommend removing the pipe and using redirection. However, I haven't been able to get it working using the ssh and... (1 Reply)
Discussion started by: 1skydive
1 Replies

6. Shell Programming and Scripting

Bourne Shell - Problem with while loop variable scope.

Hello I am having issues with a script I'm working on developing on a Solaris machine. The script is intended to find out how many times a particular user (by given userid) has logged into the local system for more than one hour today. Here is my while loop: last $user | grep -v 'sshd'... (7 Replies)
Discussion started by: DaveRich
7 Replies

7. Shell Programming and Scripting

How to get the modified value of variable outside the while loop reading from a file

Hi Friends , Sorry if this is a repeated question , The input file contains 5 lines , so the the values of the variables i and count should b i=5; count=15 but the variables are not updating , the value of variables showing i=0 and count =0 only.:mad: can any1 help me please. (11 Replies)
Discussion started by: babusek
11 Replies

8. Shell Programming and Scripting

Not access variable outside loop when a reading a file

I am writing a shell script using the korn shell. It seems that I am only able to use local variables within a while loop that is reading a file. (I can't access a variable outside a previously used while loop.) It's been a while since I wrote shell scripts. Here is a sample cat file.txt... (4 Replies)
Discussion started by: ricardo.ludwig
4 Replies

9. UNIX for Dummies Questions & Answers

reading more than one variable into a for loop

Hi, I have a file (details.txt) with 3 rows of variables ie... name postcode age john D fr25dd 25 mark W ab122aa 22 phil C cd343bb 33 What I want to do is read down the list with a loop and add each field into a one line piece of text... So I have a file (test1) which reads;... (3 Replies)
Discussion started by: starsky
3 Replies

10. Shell Programming and Scripting

bash and ksh: variable lost in loop in bash?

Hi, I use AIX (ksh) and Linux (bash) servers. I'm trying to do scripts to will run in both ksh and bash, and most of the time it works. But this time I don't get it in bash (I'm more familar in ksh). The goal of my script if to read a "config file" (like "ini" file), and make various report.... (2 Replies)
Discussion started by: estienne
2 Replies
Login or Register to Ask a Question