Can anyone make this script run faster? Post: 302219767

Sponsored Content

Top Forums Shell Programming and Scripting Can anyone make this script run faster? Post 302219767 by chihung on Wednesday 30th of July 2008 03:03:40 AM

07-30-2008

Registered User

'Annihilannic' is right, all you need to do is to avoid forking out commands. BTW, the "du" part cannot be tuned 'cos it depends on how much stuff you have

Assuming we follow the same logic (no major change of code), how much we can tune in that ksh script. Here is my attempt on an old Sun box.
The original script run time is 8.927s and the below modified script is 2.845s, that's more than 3 times better in performance. Basically I avoid a lot print and cut in the loop

Code:

#!/usr/bin/ksh

function ConvertBytes
  {
  if (( Convert_QTY > 1073741824 ))
    then
      FormattedConvert_QTY=`echo "$Convert_QTY / 1073741824" | bc -l`
      FormattedConvert_UNIT="G"
    else
      if (( Convert_QTY > 1048576 ))
        then
          FormattedConvert_QTY=`echo "$Convert_QTY / 1048576" | bc -l`
          FormattedConvert_UNIT="M"
        else
          if (( Convert_QTY > 1024 ))
            then
              FormattedConvert_QTY=`echo "$Convert_QTY / 1024" | bc -l`
              FormattedConvert_UNIT="K"
            else
              FormattedConvert_QTY=$Convert_QTY
              FormattedConvert_UNIT="B"
          fi
      fi
  fi
}

# ****************************************************************
# *
# *  Write the directory information to a temporary file
# *
# **********************************************+*****************

TempFile_NME=/tmp/llh.ksh.$$.tmp
trap "rm -f $TempFile_NME" 0 1 2 9 15

ls -lF $ls_SW $FileSpecification | awk '{ print $1 "+" $2 "+" $3 "+" $4 "+" $5 "+" $6 "+" $7 "+" $8 "+" $9 }'>$TempFile_NME

# ****************************************************************
# *
# *  Loop through the temporary file and display formatted information
# *
# **********************************************+*****************

echo "\nPermission Lnk Owner  Group   Size       Date                   File Name"
echo   "---------- --- ------ ----- -------- ------------ ----------------------------------------"

while read InputRecord
  do
    # $1:permission, $2:lnk, ...
    set -- `echo "$InputRecord" | sed -e 's/\+/ /g'`

    if [ "$1" != "total" ]
      then

        Convert_QTY="$5"
        ConvertBytes
        Create_DTE="$6 $7 $8"

        # 'set --' cannot extract '*' for executable
        # shell will expand '*'
        File_NME=$(echo "$InputRecord" | cut -d"+" -f9)

        printf "%s %3.0f %-8s %-5s %4.0f %s %-12s %-40s\n" "$1" "$2" "$3" "$4" "$FormattedConvert_QTY" "$FormattedConvert_UNIT" "$Create_DTE" "$File_NME"
    fi
  done<$TempFile_NME

# ****************************************************************
# *
# *  Calculate total space consumed by the specified directory
# *
# **********************************************+*****************

echo   "                            --------"

TotalSize_STR=`du -ks $FileSpecification | awk '{ print $1 }'`
TotalSize_QTY=$(expr "$TotalSize_STR" ) # Convert to integer

if (( TotalSize_QTY > 1073741824 ))
  then
    TotalSize_QTY=`echo "$TotalSize_QTY / 1073741824" | bc -l`
    FormattedTotalSize_QTY=$(printf "%4.1f" "$TotalSize_QTY")
    FormattedTotalSize_QTY="${FormattedTotalSize_QTY} T"
  else
    if (( TotalSize_QTY > 1048576 ))
      then
        TotalSize_QTY=`echo "$TotalSize_QTY / 1048576" | bc`
        FormattedTotalSize_QTY=$(printf "%4d" "$TotalSize_QTY")
        FormattedTotalSize_QTY="${FormattedTotalSize_QTY} G"
      else
        if (( TotalSize_QTY > 1024 ))
          then
            TotalSize_QTY=`echo "$TotalSize_QTY / 1024" | bc`
            FormattedTotalSize_QTY=$(printf "%4d" "$TotalSize_QTY")
            FormattedTotalSize_QTY="${FormattedTotalSize_QTY} M"
          else
            FormattedTotalSize_QTY=$(printf "%4d" "$TotalSize_QTY")
            FormattedTotalSize_QTY="${FormattedTotalSize_QTY} K"
        fi
    fi
fi

echo "                              $FormattedTotalSize_QTY (for $FileSpecification and its subdirectories)"

The best bet is to write a awk script to do the formatting, run time on this is 0.104s, that's 85 times faster than the original script

Code:

#! /bin/sh

ls -lF | awk '
NR>1 {
  fac=1
  unit="B"
  if($5>1073741824) {
    fac=1073741824
    unit="G"
  } else {
    if ($5>1048576) {
      fac=1048576
      unit="M"
    } else {
      if ($5>1024) {
        fac=1024
        unit="K"
      }
    }
  }
  printf("%s %3s %-8s %-5s %4.0f %s %s %2s %-5s %-12s\n",$1,$2,$3,$4,$5/fac,unit,$6,$7,$8,$9)
}'

chihung

View Public Profile for chihung

Find all posts by chihung

10 More Discussions You Might Find Interesting

1. Solaris

looking for different debugger for Solaris or to make sunstudio faster

im using the sunstudio but it is very slow , is there ant other GUI debugger for sun Solaris or at list some ways to make it faster ? im using to debug throw telnet connection connected to remote server thanks

2. Shell Programming and Scripting

awk help to make my work faster

hii everyone , i have a file in which i have line numbers.. file name is file1.txt aa bb cc "12" qw xx yy zz "23" we bb qw we "123249" jh here 12,23,123249. is the line number now according to this line numbers we have to print lines from other file named...

3. Red Hat

Re:How to make the linux pc faster

Hi, Can any one help me out in solving the problem i have a linux database server it is tooo slow that i am unable to open even the terminial is there any solution to get rid of this problem.How to make this server faster. Thanks & Regards Venky

4. Shell Programming and Scripting

How to make copy work faster

I am trying to copy a folder which contains a list of C executables. It takes 2 mins for completion,where as the entire script takes only 3 more minutes for other process. Is there a way to copy the folder faster so that the performance of the script will improve?

5. Shell Programming and Scripting

Make script faster

Hi all, In bash scripting, I use to read files: cat $file | while read line; do ... doneHowever, it's a very slow way to read file line by line. E.g. In a file that has 3 columns, and less than 400 rows, like this: I run next script: cat $line | while read line; do ## Reads each...

6. Shell Programming and Scripting

Making script run faster

Can someone help me edit the below script to make it run faster? Shell: bash OS: Linux Red Hat The point of the script is to grab entire chunks of information that concerns the service "MEMORY_CHECK". For each chunk, the beginning starts with "service {", and ends with "}". I should...

7. Shell Programming and Scripting

awk changes to make it faster

I have script like below, who is picking number from one file and and searching in another file, and printing output. Bu is is very slow to be run on huge file.can we modify it with awk #! /bin/ksh while read line1 do echo "$line1" a=`echo $line1` if then echo "$num" cat file1|nawk...

8. Shell Programming and Scripting

Optimize shell script to run faster

data.file: contact { contact_name=royce-rolls modified_attributes=0 modified_host_attributes=0 modified_service_attributes=0 host_notification_period=24x7 service_notification_period=24x7 last_host_notification=0 last_service_notification=0 host_notifications_enabled=1...

9. Shell Programming and Scripting

How to make awk command faster?

I have the below command which is referring a large file and it is taking 3 hours to run. Can something be done to make this command faster. awk -F ',' '{OFS=","}{ if ($13 == "9999") print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12 }' ${NLAP_TEMP}/hist1.out|sort -T ${NLAP_TEMP} |uniq>...

10. Shell Programming and Scripting

How to make faster loop in multiple directories?

Hello, I am under Ubuntu 18.04 Bionic. I have one shell script run.sh (which is out of my topic) to run files under multiple directories and one file to control all processes running under those directories (control.sh). I set a cronjob task to check each of them with two minutes of intervals....