Unix/Linux Go Back    


Shell Programming and Scripting BSD, Linux, and UNIX shell scripting — Post awk, bash, csh, ksh, perl, php, python, sed, sh, shell scripts, and other shell scripting languages questions here.

any way to speed up calculations in bash script

Shell Programming and Scripting


Closed    
 
Thread Tools Search this Thread Display Modes
    #1  
Old Unix and Linux 01-14-2009
npatwardhan npatwardhan is offline
Registered User
 
Join Date: Nov 2008
Last Activity: 23 February 2017, 5:55 AM EST
Posts: 186
Thanks: 2
Thanked 0 Times in 0 Posts
any way to speed up calculations in bash script

hi i have a script that is taking the difference of multiple columns in a file from a value from a single row..so far i have a loop to do that.. all the data is floating point..fin[i] has the difference between array1 and array2..array1 has 700 x 300= 210000 values and array2 has 700 values..


Code:
while [ $j -lt $outer ];do
b=${ARRAY2[j]}
i=0
while [ $i -lt $count1 ];do
a=${ARRAY1[k]}
fin[i]=`echo $a - $b | bc`
printf "%.1f," ${fin[i]} >> "$count".diff
((i++))
((k++))
done 
echo >> "$count".diff
((j++))
done

i am using bc to calculate the floating point subtraction. problem is i have about 600 columns and 300 rows..the script takes a long time to execute.. is there any way i can speed it up?
Sponsored Links
    #2  
Old Unix and Linux 01-15-2009
cfajohnson's Unix or Linux Image
cfajohnson cfajohnson is offline Forum Advisor  
Shell programmer, author
 
Join Date: Mar 2007
Last Activity: 11 July 2016, 2:55 PM EDT
Location: Toronto, Canada
Posts: 2,898
Thanks: 0
Thanked 134 Times in 118 Posts
Quote:
Originally Posted by npatwardhan View Post
is there any way i can speed it up?

Don't call the external command 21,000 times. Pipe your calculations to a single instance of it.

And I would use awk rather than bc.


Code:
while [ $j -lt $outer ];do
  i=0
  while [ $i -lt $count1 ];do
    printf "%f\n" "${ARRAY1[k]} - ${ARRAY2[j]}"
    i=$(( $i + 1 ))
    k=$(( $k + 1 ))
  done 
  echo
  j=$(( $j + 1 ))
done | awk '/^$/ { print; next }
{ printf "%.1f," $1 - $2 }
'  > "$count.diff"

If you are processing the entire arrays:


Code:
for a in "${ARRAY1[@]}"
do
  printf "%f $a\n" ${ARRAY2[@]}
  echo
done | awk '/^$/ { print; next }
{ printf "%.1f,", $1 - $2 }
'

Sponsored Links
    #3  
Old Unix and Linux 01-15-2009
npatwardhan npatwardhan is offline
Registered User
 
Join Date: Nov 2008
Last Activity: 23 February 2017, 5:55 AM EST
Posts: 186
Thanks: 2
Thanked 0 Times in 0 Posts
thanks. i am using bc because the numbers are floating point.. i also tried:


Code:
while [ $j -lt $outer ];do
  i=0
  while [ $i -lt $count1 ];do
    printf "%f\n" "${ARRAY1[k]} - ${ARRAY2[j]}"
    i=$(( $i + 1 ))
    k=$(( $k + 1 ))
  done 
  echo
  j=$(( $j + 1 ))
done | awk '/^$/ { print; next }
{ printf "%.1f," $1 - $2 }
'  > "$count.diff"

and i got an error:

Code:
./lines1: line 77: printf: -0.5422 - : invalid number
./lines1: line 77: printf: -0.5422 - : invalid number
./lines1: line 77: printf: -0.5422 - : invalid number
./lines1: line 77: printf: -0.5422 - : invalid number
./lines1: line 77: printf: -0.5422 - : invalid number
./lines1: line 77: printf: -0.5422 - : invalid number
./lines1: line 77: printf: -0.5422 - : invalid number
./lines1: line 77: printf: -0.5422 - : invalid number
./lines1: line 77: printf: -0.5422 - : invalid number
./lines1: line 77: printf: -0.5422 - : invalid number

seemed like it was taking the difference but errored out..do i still need bc?
    #4  
Old Unix and Linux 01-15-2009
Corona688 Corona688 is offline Forum Staff  
Mead Rotor
 
Join Date: Aug 2005
Last Activity: 21 September 2017, 3:18 PM EDT
Location: Saskatchewan
Posts: 22,407
Thanks: 1,126
Thanked 4,233 Times in 3,913 Posts
Quote:
Originally Posted by npatwardhan View Post
seemed like it was taking the difference but errored out..do i still need bc?
printf cannot subtract numbers for you, that's what bc is for.
Sponsored Links
    #5  
Old Unix and Linux 01-15-2009
cfajohnson's Unix or Linux Image
cfajohnson cfajohnson is offline Forum Advisor  
Shell programmer, author
 
Join Date: Mar 2007
Last Activity: 11 July 2016, 2:55 PM EDT
Location: Toronto, Canada
Posts: 2,898
Thanks: 0
Thanked 134 Times in 118 Posts
Quote:
Originally Posted by npatwardhan View Post
thanks. i am using bc because the numbers are floating point.. i also tried:


Code:
while [ $j -lt $outer ];do
  i=0
  while [ $i -lt $count1 ];do
    printf "%f\n" "${ARRAY1[k]} - ${ARRAY2[j]}"


Code:
    printf "%f %f\n" "${ARRAY1[k]}" "${ARRAY2[j]}"

Sponsored Links
    #6  
Old Unix and Linux 01-15-2009
npatwardhan npatwardhan is offline
Registered User
 
Join Date: Nov 2008
Last Activity: 23 February 2017, 5:55 AM EST
Posts: 186
Thanks: 2
Thanked 0 Times in 0 Posts
that is just going to print the values.. i still need to take the difference.. is there any way i can round the data values in ARRAY1 and ARRAY2? this way i can get rid of the extra precision and speed up my calculations..
Sponsored Links
    #7  
Old Unix and Linux 01-15-2009
cfajohnson's Unix or Linux Image
cfajohnson cfajohnson is offline Forum Advisor  
Shell programmer, author
 
Join Date: Mar 2007
Last Activity: 11 July 2016, 2:55 PM EDT
Location: Toronto, Canada
Posts: 2,898
Thanks: 0
Thanked 134 Times in 118 Posts
Quote:
Originally Posted by npatwardhan View Post
that is just going to print the values..

It prints the values which are piped to awk where the subtraction is performed.
Quote:
i still need to take the difference..

That's done in awk.
Quote:
is there any way i can round the data values in ARRAY1 and ARRAY2? this way i can get rid of the extra precision and speed up my calculations..

Changing the precision will not affect the speed.
Sponsored Links
Closed

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
how to make your bash script run on a machine with csh and bash npatwardhan Shell Programming and Scripting 3 11-19-2008 03:17 AM
passing variable from bash to perl from bash script arsidh Shell Programming and Scripting 10 06-04-2008 12:25 PM
Why generate "ash and bash" different output for same bash script? s. murat Shell Programming and Scripting 0 05-26-2008 07:19 AM
Non-integer calculations in bash jeriryan87 Shell Programming and Scripting 2 06-27-2007 04:17 PM
dmidecode, RAM speed = "Current Speed: Unknown" Santi Filesystems, Disks and Memory 0 02-16-2006 05:16 AM



All times are GMT -4. The time now is 04:02 AM.