Variable unexpectedly blank after piped loop


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Variable unexpectedly blank after piped loop
# 1  
Old 09-29-2018
Variable unexpectedly blank after piped loop

Hi,


I was trying out a new way to parse multiple lines from a variable. I liked the "while read" method, so I used echo to pipe the variable into the while loop, the basic structure looking like:


Code:
echo $var | while read nextline
 do
    a=blah
done


 echo $a # it is empty!


I am not sure why this is happening. It seems to be the behaviour I'd expect running a subshell, but I didn't think that the pipe could start a subshell.


Any references on why this is happening would be appreciated. I have attached a functional script to this post which demonstrates this issue if you need to see it in action.


Thanks for any assistance.
# 2  
Old 09-29-2018
OK you read nextline, where do you use it? (now nextline is a new variable...) Where you trying to get something out of $nextline ?
# 3  
Old 09-29-2018
In bash and most shells the while loop the part after the pipe is executed in the background (except ksh and zsh). In those shells, once the part behind the pipe finishes, any variable content will be lost. One can avoid this like this with a code block:

Code:
echo "$var" | 
{
  while read nextline
  do
     a=blah
  done
  echo "$a"
}

Or by avoiding the pipe altogether (bash/ksh93/zsh only)
Code:
while read nextline
do
   a=blah
done <<< "$var"
echo "$a"


Last edited by Scrutinizer; 10-19-2018 at 12:54 PM.. Reason: $ sign was missing
This User Gave Thanks to Scrutinizer For This Post:
# 4  
Old 10-19-2018
Quote:
Originally Posted by Scrutinizer
...
Code:
...
done <<< "$var"


Thank you very much!

I have chosen to use the above symbol to avoid the pipe, and I've also noted the code block method for use in cases where I need compatibility.


I was slightly confused by the missing dollar sign in your snippet because I thought maybe this triple-less-than is only for variables so the dollar sign is unneeded. I still need to read up on it...
# 5  
Old 10-19-2018
You're welcome. Indeed the $-sign went missing! I have added it to my post...
# 6  
Old 10-19-2018
Note that as Scrutinizer said, ksh (both ksh88 and ksh93) and zsh run the last segment of a pipeline in the current shell execution environment. So, if you were using ksh or zsh instead of bash (or whatever shell you are using), the script you showed us in post #1 in this thread would have produced the output:
Code:
blah

from the echo $a command at the end of your script.
# 7  
Old 10-19-2018
Hmmm, may I object? Actually, bash also CAN "run the last segment of a pipeline in the current shell execution environment". man bash:
Quote:
shopt [-pqsu] [-o] [optname ...]
lastpipe
If set, and job control is not active, the shell runs the last command of a pipeline not executed in the background in the current shell environment.
Code:
shopt -s lastpipe                     # set option
set +m                                # disable job control
unset a
 echo $var | while read nextline;     # from post#1

   do     a=blah;
   done


echo $a                               # it is empty!
blah                                  # it's NOT!

This User Gave Thanks to RudiC For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Use cut output as variable piped awk command

Hi, I would like use the output of my cut command as a variable in my following awk command. Here's what I've written. cut -f1 info.txt | awk -v i=xargs -F'' '{if($6 == $i) print $20}' summary.txt Where obviously the 'xargs' doesn't do what I want. How can I pass my cut result to my awk... (3 Replies)
Discussion started by: heyooo
3 Replies

2. Shell Programming and Scripting

[Solved] How to increment and add variable length numbers to a variable in a loop?

Hi All, I have a file which has hundred of records with fixed number of fields. In each record there is set of 8 characters which represent the duration of that activity. I want to sum up the duration present in all the records for a report. The problem is the duration changes per record so I... (5 Replies)
Discussion started by: danish0909
5 Replies

3. Shell Programming and Scripting

Array Variable being Assigned Values in Loop, But Gone when Loop Completes???

Hello All, Maybe I'm Missing something here but I have NOOO idea what the heck is going on with this....? I have a Variable that contains a PATTERN of what I'm considering "Illegal Characters". So what I'm doing is looping through a string containing some of these "Illegal Characters". Now... (5 Replies)
Discussion started by: mrm5102
5 Replies

4. Shell Programming and Scripting

append blank spaces at the end of a variable string

Hello, could you please help with this one. I have an input file like this: 123,4567,89000 123456789,9876543,12 and for the output I need strings to be with the fixed length, let's say 15, and if the string is -lt 15 to be populated with blanks at the end until it reach 15, like this: 123 ,4567... (1 Reply)
Discussion started by: apenkov
1 Replies

5. Shell Programming and Scripting

ignore blank lines in while loop

Hi, i am having one text file it contains some blank lines and i want to ignore that blank lines . #! /bin/bash clear rdCount=0; while read myline do echo $myline let rdCount=$rdCount+1 done < ps.txt echo "Total line=$rdCount" and ps .txt contains the data- (17 Replies)
Discussion started by: aish11
17 Replies

6. UNIX for Dummies Questions & Answers

While loop seems to exit when blank line is met

Hello everyone! I am having an issue with a script I am trying to create. I have an input file (named sort_$1.txt) like: aaaa bbbb cccc dddd eeee and I process it with the following code: while read -r EachLine2 do (11 Replies)
Discussion started by: haaru
11 Replies

7. Shell Programming and Scripting

KSH variable containing piped commands

Hi, I am trying to execute a bunch of piped command which are stored in a variable, but only one command executed at a time, and rest of the pipes, commands and their arguments are considered as argument to the very first command. Can you please help me in this? bash-2.05$ cat test.sh... (1 Reply)
Discussion started by: prashant.ladha
1 Replies

8. Shell Programming and Scripting

Unix mail with blank line on variable

I am testing a ksh script for email. In the subject/content of the mail there is some dynamic variables like date and charges. However these variables occupied the entire line erase other in that particular line For e.g. there is a mail message: This mail is intent... Your total... (2 Replies)
Discussion started by: balaji.rengaraj
2 Replies

9. Shell Programming and Scripting

how to check the variable values is blank

HI , I have to check the values of variable is blank or not. exm : ###test test1 var=`cate filename | head -1 | cut -c1-3` I need to check the first three character of 1st line . if it is blank .then exit or we need to process . Thanks in advance . (2 Replies)
Discussion started by: julirani
2 Replies

10. Shell Programming and Scripting

[bash] Check if variable is set or blank

Hello guys. In my script, i have the following code: echo "The tarfile contains these directorys" tar -tf file.tar > tarlist.txt cat tarlist | awk -F/ '{print $1 "/" $2}' | nl echo "Enter path of the directory you want to extract or just press enter to extract everything: " read path... (1 Reply)
Discussion started by: noratx
1 Replies
Login or Register to Ask a Question