While loop...

 
Thread Tools Search this Thread
Homework and Emergencies Homework & Coursework Questions While loop...
# 8  
Old 09-12-2011
First off, the "cut" command does extract ONE field (or several fields, but this is advanced stuff you want to postpone until you grasped the basics) from a line consisting of several such fields. You really should read the man page of "cut". (Tip: if you want to extract several fields one at a time why don't you assign several variables one at a time?)

Secondly, you might want to think over the way you construct a while loop. Given, the way you do it works and it doesn't really make a difference in a 10-line-script, but you write this script to learn something so your concern should not only be to make it work but to learn the most possible from the exercise.

Consider this: when you feed data to a while-loop with a redirection at the end the script is becoming hard to read as the loop becomes longer. The following two loops do the same:

Code:
cat /some/file | while read line ; do
     ...some processing...
done

while read line ; do
     ...some processing...
done < /some/file

As long as the loop is short there is no problem, but if "...some processing..." consists of 100 lines: then you will start skipping backwards an forwards as you try to read the source code. Writing the input at top of the loop is somewhat easier to understand and therefore to maintain.


A third point: "read" is able to read several values at once - the values are delimited by the "IFS", the "internal field separator". This is the value the shell uses to separate input fields - even command arguments! Per default the IFS is a space character, which is why a line like "command arg1 arg2" will be interpreted as feeding two distinct arguments "arg1" and "arg2" to "command".

Lets do an exmple: Here is a input file, save this to "infile":

Code:
a b c
one two three
1 2 3

And here is some code to read this file, save it to a script "read_infile.sh" and execute it:

Code:
#! /bin/bash

value1=""
value2=""
value3=""

cat infile | while read value1 value2 value3 ; do
     echo "value1 is: $value1"
     echo "value2 is: $value2"
     echo "value3 is: $value3"
done

That leaves just the problem of changing the IFS to a colon instead of a space. Hint: you can assign it like any variable and example how to do this can be found in this forum, just use the search function.


There is even a fourth point: it pays to analyze what consumes (system) time on running a script. The most time consuming thing is forking new processes. Every time you invoke a command in a script a new process is started and this consumes time. You can read a very enlightening thread where i learned this lesson thanks to cfajohnson and perderabo here.

What does that mean for your script? Well, the line (i have corrected the syntactical error)

Code:
log_name="$(echo $inputline | cut -f1 -d:)"

does something which can be done by "parameter expansion" too. Parameter expansion happens completely inside the shell and doesn't need its own process. Have a look at the man page of bash and search for "parameter expansion" to see what i mean.

I hope this helps.

bakunin
# 9  
Old 09-12-2011
Quote:
Originally Posted by bakunin
Secondly, you might want to think over the way you construct a while loop. Given, the way you do it works and it doesn't really make a difference in a 10-line-script, but you write this script to learn something so your concern should not only be to make it work but to learn the most possible from the exercise.

Consider this: when you feed data to a while-loop with a redirection at the end the script is becoming hard to read as the loop becomes longer. The following two loops do the same:

Code:
cat /some/file | while read line ; do
     ...some processing...
done

while read line ; do
     ...some processing...
done < /some/file

As long as the loop is short there is no problem, but if "...some processing..." consists of 100 lines: then you will start skipping backwards an forwards as you try to read the source code. Writing the input at top of the loop is somewhat easier to understand and therefore to maintain.
I must respectfully disagree. This is a useless use of cat and creates unnecessary overhead of a process and a pipeline. See https://secure.wikimedia.org/wikiped...ess_use_of_cat
See also: http://partmaps.org/era/unix/award.html

Using redirection as in the second example is more efficient and thus preferred. As a rule, whenever you catch yourself cat'ing only one file that is a red flag that there should be a more efficient way to do what you are trying to do.

Also
Code:
log_name="$(echo $inputline | cut -f1 -d:)"

contains a syntax error. The quotes need to be removed otherwise you are just assigning a literal to your variable. This was in the original code.
# 10  
Old 09-12-2011
Quote:
Originally Posted by itkamaraj
just noticed.. what is this ?

Are you trying to store the first fields in the variable ?

Code:
echo log_name = $log_name

I believe the intention is to echo the variable's name, the equals sign, and the variable's value. If so, that statement is fine except for the lack of double quotes around $log_name (to protect the results of the parameter expansion from field splitting and pathname expansion).

Regards,
Alister

---------- Post updated at 01:00 PM ---------- Previous update was at 12:28 PM ----------

Quote:
Originally Posted by gary_w
Quote:
Originally Posted by bakunin
Secondly, you might want to think over the way you construct a while loop. Given, the way you do it works and it doesn't really make a difference in a 10-line-script, but you write this script to learn something so your concern should not only be to make it work but to learn the most possible from the exercise.

Consider this: when you feed data to a while-loop with a redirection at the end the script is becoming hard to read as the loop becomes longer. The following two loops do the same:

Code:
cat /some/file | while read line ; do
     ...some processing...
done

while read line ; do
     ...some processing...
done < /some/file

As long as the loop is short there is no problem, but if "...some processing..." consists of 100 lines: then you will start skipping backwards an forwards as you try to read the source code. Writing the input at top of the loop is somewhat easier to understand and therefore to maintain.

I must respectfully disagree. This is a useless use of cat and creates unnecessary overhead of a process and a pipeline...
The UUofC is of no consequence compared to the fact that in many shells the use of a pipe relegates the reading while-loop to a subshell.

bakunin's helpful and informative post neglected to mention that if you do use a pipe, and if the while-loop runs in a subshell, all changes to the execution environment (such as variable assignments and redirections) will never be accessible to the parent shell. This alone could render the approach inconvenient if not infeasible.

If one truly, madly, deeply cares to avoid a redirection at the end of the while-loop, instead of piping cat into a while-loop, it's a better idea to use exec-redirection before the while-loop. Not only is it portable, the redirection just before the while-loop makes it clear from whence the input comes.

Code:
exec 4<&0 <inputfile
while read line; do
    ....
    ....
done
exec <&4 4<&-


Quote:
Originally Posted by gary_w
Code:
log_name="$(echo $inputline | cut -f1 -d:)"

contains a syntax error. The quotes need to be removed otherwise you are just assigning a literal to your variable. This was in the original code.
You are mistaken. Double quoting a command substitution has no effect.

Double quoting only prevents parsing steps which are capable of generating additional fields -- field splitting and pathname expansion/globbing, for example. However, those steps are already bypassed during parameter assignment. So, those double quotes are harmless.

The behavior you describe is what would happen if the command substitution where single-quoted.


Regards,
Alister
# 11  
Old 09-12-2011
Quote:
Originally Posted by alister
You are mistaken. Double quoting a command substitution has no effect.

Double quoting only prevents parsing steps which are capable of generating additional fields -- field splitting and pathname expansion/globbing, for example. However, those steps are already bypassed during parameter assignment. So, those double quotes are harmless.

The behavior you describe is what would happen if the command substitution where single-quoted.
Interesting:
Using the original poster's shebang of /bin/sh on our solaris box using the original code and a one line roster file:
Code:
#!/bin/sh
while read inputline
do
        log_name="$(echo $inputline | cut -f1 -d:)"
        echo $log_name
done < roster
exit 0

Output shows the quotes do matter:
Code:
$ zzz3
$(echo efs:123:123:123 | cut -f1 -d:)
$

Removing them works as expected (although I need to use backquotes instead of the $( ) syntax for the subshell as in our version of /bin/sh the $( ) is not supported):
Code:
#!/bin/sh
while read inputline
do
        log_name=`echo $inputline | cut -f1 -d:`
        echo $log_name
done < roster
exit 0

Output:
Code:
$ zzz3
efs
$

Using the ksh with what you describe works though:
Code:
#!/bin/ksh
while read inputline
do
        log_name="$(echo $inputline | cut -f1 -d:)"
        echo $log_name
done < roster
exit 0

So your mileage may vary also depending on your shell. Interesting discussion!
# 12  
Old 09-12-2011
Sweet, thanks to all who helped. I did not use the back quote. Yes I have to say the conversation was very entertaining and informative as well.
# 13  
Old 09-12-2011
Thanks for the info, gary_w. I should have qualified my earlier post to make it clear that my statements reflected the POSIX standard. As far as POSIX sh is concerned, double-quoting a command substitution (using either $(...) or `...` syntax), whose result is assigned to a variable, has no effect.

/bin/sh on your solaris box is probably an ancient shell that doesn't support the now ubiquitous (and POSIX-standardized) $(...) command substitution syntax. To your shell, that double-quoted string does not contain a command substitution and so it simply assigns the string as is to the variable. If the OP is using solaris, then he/she may be attempting to use unsupported syntax.

If solaris /bin/sh is POSIX-compliant with regard to a double-quoted backtick command substitution, it should give identical results for var=`cmd` and var="`cmd`".

Regards,
Alister

Last edited by alister; 09-12-2011 at 05:18 PM..
# 14  
Old 09-12-2011
Thanks to all who helped. The conversation has been interesting and informative.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

For loop or while loop from a text file

Hi all, i developed a script to measure the uptime of a process in a Solaris 10/11 environments. All is well, but i came across a situation where there are multiple processes of the same name. Basically i have the following result file: beVWARS 13357 19592122 beVWARS 14329 19591910... (4 Replies)
Discussion started by: nms
4 Replies

2. Shell Programming and Scripting

awk loop using array:wish to store array values from loop for use outside loop

Here's my code: awk -F '' 'NR==FNR { if (/time/ && $5>10) A=$2" "$3":"$4":"($5-01) else if (/time/ && $5<01) A=$2" "$3":"$4-01":"(59-$5) else if (/time/ && $5<=10) A=$2" "$3":"$4":0"($5-01) else if (/close/) { B=0 n1=n2; ... (2 Replies)
Discussion started by: klane
2 Replies

3. Shell Programming and Scripting

Reset while loop to loop same file multiple times

Hi, I want to read file multiple times. Right now i am using while loop but that is not working. ex. While read line do while read line2 do echo stmt1 #processing some data based on data., done < file2.txt done < file1.txt # This will have 10... (4 Replies)
Discussion started by: tmalik79
4 Replies

4. Shell Programming and Scripting

Array Variable being Assigned Values in Loop, But Gone when Loop Completes???

Hello All, Maybe I'm Missing something here but I have NOOO idea what the heck is going on with this....? I have a Variable that contains a PATTERN of what I'm considering "Illegal Characters". So what I'm doing is looping through a string containing some of these "Illegal Characters". Now... (5 Replies)
Discussion started by: mrm5102
5 Replies

5. Shell Programming and Scripting

My for loop decides to become an infinite loop?

Hi, I was debating if I should put this in the dummies or scripts section, I apologize in advance if I chose poorly. Fairly new to Unix and BASH scripting but I thought I made it fairly well given my limited understanding. However, the output indicates that it's looping and I'm ending up with a... (5 Replies)
Discussion started by: gotreef
5 Replies

6. Shell Programming and Scripting

S# in a for loop - concatenate $(loop counter)

Hi, hope I am posting in the right section. My problem is that I have 2 or more arguments passed and I want to check if the arguments passed exists or not. The first argument should not exist and the remaining others should exist. example: ./shells.sh argument1 argument2 argument3 ... (5 Replies)
Discussion started by: fight4love
5 Replies

7. Shell Programming and Scripting

BASH loop inside a loop question

Hi all Sorry for the basic question, but i am writing a shell script to get around a slightly flaky binary that ships with one of our servers. This particular utility randomly generates the correct information and could work first time or may work on the 12th or 100th attempt etc !.... (4 Replies)
Discussion started by: rethink
4 Replies

8. Shell Programming and Scripting

Null Handling in Until loop. . .loop won't stop

Hi Im running this script, which is supposed to find the max value build some tables and then stop running once all the tables are built. Thing is , it keeps assigning a null value to $h and then $g is null so it keep building tables i.e. testupdateNUL. How can I stop this? Here is what I have: ... (4 Replies)
Discussion started by: brandono66
4 Replies

9. Shell Programming and Scripting

Using variables created sequentially in a loop while still inside of the loop [bash]

I'm trying to understand if it's possible to create a set of variables that are numbered based on another variable (using eval) in a loop, and then call on it before the loop ends. As an example I've written a script called question (The fist command is to show what is the contents of the... (2 Replies)
Discussion started by: DeCoTwc
2 Replies

10. Shell Programming and Scripting

Is there a better way I could have run this loop. (For loop with two variables)

Sorry for such a dreadful title, but I'm not sure how to be more descriptive. I'm hoping some of the more gurutastic out there can take a look at a solution I came up with to a problem, and advice if there are better ways to have gone about it. To make a long story short around 20K pieces of... (2 Replies)
Discussion started by: DeCoTwc
2 Replies
Login or Register to Ask a Question