Variable scope in bash


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Variable scope in bash
# 1  
Old 01-11-2012
Variable scope in bash

Hello! Before you "bash" me with
- Not another post of this kind
Please read on and you will understand my problem...

I am using the below to extract a sum of the diskIO on a Solaris server.

Code:
#!/bin/sh
PATH=/usr/bin:/usr/sbin:/sbin; export PATH
TEMP1="/tmp/raw-sar-output.txt$$"
TEMP2="/tmp/sorted-sar-output.txt$$"
CONT=0
sar -d 30 4 > $TEMP1
AVG_START=`awk '/^Average/ {print NR}' $TEMP1`
TOT_LINES=`cat $TEMP1 |wc -l`
TAIL_LINES=`expr $TOT_LINES - $AVG_START`
tail -$TAIL_LINES $TEMP1 | awk '{ print $5,$2 }' | sort -k 1n,1 > $TEMP2
if [ -s "$TEMP2" ]
then
   while read file;
   do
      IO=`echo $file | awk '{ print $1 }'`
      TMP_VAL=`echo $file | awk '{ print $1 }'`
      CONT=`expr $CONT + $TMP_VAL`
   echo CONT = $CONT
   done < $TEMP2
   #Echo Result
   echo $CONT
else
   rm -f $TEMP1 $TEMP2 2>/dev/null
   exit 1
fi
rm -f $TEMP1 $TEMP2 2>/dev/null
exit 0

Even though I am not piping to while, the final
Code:
echo $CONT

returns zero.

The same code with Korn shell works perfectly but for some reason the execution is too slow and causes problems to the use I need it for. So I definately need to use Bash or something equally fast...

Please help!
This User Gave Thanks to haaru For This Post:
# 2  
Old 01-11-2012
The execution is slow because you're running dozens of externals in backticks all the time. You should use externals to process big batches of data -- not individual lines. Externals are fast enough on bulk data, but overkill for anything less, like makng 10 phone calls to say 10 words. Most of your script could probably be done with one execution of awk, if we knew the details.

If you show the input you get from sar, and the output you want from the script, we can find a much more efficient way to do so, probably in nearly pure awk. It can do much more than '{ print $1 }'

I don't see anything obviously wrong with your script, though there's many places in it where things are unquoted and quoting may be important.
# 3  
Old 01-11-2012
The script is not slow for its content.
The "sar -d 30 4" statement will take 2 minutes (sample interval of thirty seconds times the four iterations). On my test the script took 2 mins 3 secs.

The process to fish out the Average lines can be improved such that we only read $TEMP1 once.

Code:
#!/bin/sh
PATH=/usr/bin:/usr/sbin:/sbin; export PATH
TEMP1="/tmp/raw-sar-output.txt$$"
TEMP2="/tmp/sorted-sar-output.txt$$"
CONT=0
sar -d 30 4 > $TEMP1
grep \^"Average" $TEMP1 | awk '{ print $5,$2 }' | sort -k 1n,1 > $TEMP2
if [ -s "$TEMP2" ]
then
   while read file;
   do
      IO=`echo $file | awk '{ print $1 }'`
      TMP_VAL=`echo $file | awk '{ print $1 }'`
      CONT=`expr $CONT + $TMP_VAL`
   echo CONT = $CONT
   done < $TEMP2
   #Echo Result
   echo $CONT
else
   rm -f $TEMP1 $TEMP2 2>/dev/null
   exit 1
fi
rm -f $TEMP1 $TEMP2 2>/dev/null
exit 0


If performance was really important, there is no need for the workfile $TEMP1 and (as Corona688 notes) the maths can be done in awk.

The other issue with your script is possibly the design. Field 5 in "sar" is the Number of Read+Write data transfers PER SECOND. Over the sample period of just 2 minutes you need some decent level of disc activity to get this figure above zero.

Finally the question I should have asked first:
Please explain the statement "execution is too slow and causes problems to the use I need it for".


(You are aware that unix systems can be configured to accumulate "sar" statistics automatically all day every day)?
# 4  
Old 01-11-2012
Actually I altered the script to just use one AWK statement as "Corona688" suggested. It now looks like this (also corrected a bug):

Code:
#!/bin/ksh
PATH=/usr/bin:/usr/sbin:/sbin; export PATH
TEMP1="/tmp/raw-sar-output.txt$$"
sar -d 5 4 > $TEMP1
AVG_START=`awk '/^Average/ {print NR}' $TEMP1`
TOT_LINES=`cat $TEMP1 |wc -l`
TAIL_LINES=`expr $TOT_LINES - $AVG_START + 1`
tail -$TAIL_LINES $TEMP1 | awk '{ if ($1 == "Average") sum=sum + $5; else sum=sum + $4 } ; END{print sum}'
rm -f "$TEMP1" 2>/dev/null
exit 0

I noticed afterwards that the sar command was causing most of the delay but then again I am under the impression that bash was a bit faster.

To answer "methyl"'s question, I will use this script along with net-SNMP extend (or exec if you will) functionality. The idea is to graph the accumulation of read/writes on servers.

So... Can someone propose a logical interval? The servers that will be queried I assure are very very busy Smilie

Currently I have it set as
Code:
 
sar -d 5 4

this also helps so that my snmpwalks do not time out.

For the sake of learning.. Can someone answer how come the variable scope of the variable "CONT" is not global...?

Last edited by haaru; 01-11-2012 at 02:32 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Bash Variable scope - while loop while reading from a file

Cope sample1: test.sh i=0 echo " Outside loop i = $i " while do i=$(( $i + 1)) echo "Inside loop i = $i " done echo " Out of loop i is : $i " When run output : Outside loop i = 0 Inside loop i = 1 Inside loop i = 2 Inside loop i = 3 Inside loop i = 4 Inside loop i = 5 Inside... (8 Replies)
Discussion started by: Adarshreddy01
8 Replies

2. Programming

Variable Scope in Perl

I have to admit that i have not used Perl at all and this is a singular occasion where i have to patch an existing Perl script. I dearly hope i do not have to do it again for the next 15 years and therefore try to avoid having to learn the programming language in earnest. The OS is AIX 7.1, the... (2 Replies)
Discussion started by: bakunin
2 Replies

3. Shell Programming and Scripting

BASH: variable and function scope and subscripts

Hi, I'm a Delphi developer new to linux, new to this forums and new to BASH programming and got a new task in my work: maintaining an existing set of BASH scripts. First thing I want to do is making the code more reliable as in my opinion it's really bad written. So here's the quest: I'm... (6 Replies)
Discussion started by: rse
6 Replies

4. Shell Programming and Scripting

Maintain Scope of the variable in UNIX

Hi All Is there is any way to maintain the scope of the variable in unix Example x=1 j=1 while do .. .... .... while do .. .. x=x+1 done #inner most while loop ends here done #outer loop ends here (8 Replies)
Discussion started by: parthmittal2007
8 Replies

5. Shell Programming and Scripting

Help with retaining variable scope

Hi, I use Korn Shell. Searched Forum and modified the way the file is input to the while loop, but still the variable does not seem to be retaining the final count. while read name do Tmp=`echo $name | awk '{print $9 }'` Count=`cat $Tmp | wc -l`... (6 Replies)
Discussion started by: justchill
6 Replies

6. UNIX for Dummies Questions & Answers

Bash loops and variable scope

Hi All, I've been researching this problem and I am pretty sure that the issue is related to the while loop and the piping. There are plenty of other threads about this issue that recommend removing the pipe and using redirection. However, I haven't been able to get it working using the ssh and... (1 Reply)
Discussion started by: 1skydive
1 Replies

7. Shell Programming and Scripting

variable scope

Hi, I want to know about the variable scope in shell script. How can we use the script argument inside the function? fn () { echo $1 ## I want this argument should be the main script argument and not the funtion argument. } also are there any local,global types in shell script? if... (3 Replies)
Discussion started by: shellwell
3 Replies

8. Shell Programming and Scripting

scope of the variable - Naga

Hi All, I am new to unix shell scripting, in the below script "num" is an input file which contains a series of numbers example : 2 3 5 8 I want to add the above all numbers and want the result finally outside the while loop. it prints the value zero instead of the actual expected... (13 Replies)
Discussion started by: nagnatar
13 Replies

9. Shell Programming and Scripting

problem with shell variable's scope

Hi, I am stuck while developing a shell sub-routine which checks the log file for "success" or "failure". The subroutine reads the log file and checks for key word "success", if found it set the variable (found=1). It returns success or failure based on this variable. My problem is, I can... (2 Replies)
Discussion started by: cjjoy
2 Replies

10. Programming

C++ variable scope and mutexes

I've been wondering if I can make mutexes much easier to use in C++ with creative use of a locking class and variable scope, but I'm not sure if things happen in the order I want. Here's pseudocode for something that could use the class: int someclass::getvalue() { int retval; ... (0 Replies)
Discussion started by: Corona688
0 Replies
Login or Register to Ask a Question