The UNIX and Linux Forums  
Hello and Welcome from United States to the UNIX and Linux Forums! Thank You for Visiting and Joining Our Global Community.

Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
.
google unix.com



Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
To Write a Shell script that takes two arguments. bobby36 Shell Programming and Scripting 3 04-05-2007 08:44 PM
why shell scripting takes more time to read a file brkavi_in Shell Programming and Scripting 1 06-23-2006 08:20 AM
Killing a process that takes too long WeezelDs Shell Programming and Scripting 3 04-10-2006 03:33 PM
fwrite takes extremely long time inna High Level Programming 5 07-06-2005 06:19 AM
Finding out how long a command takes to run cfoxwell UNIX for Dummies Questions & Answers 3 11-22-2001 01:50 AM

Closed Thread
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish Powered by Powered by Google
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 02-05-2007
ozzman ozzman is offline
Registered User
  
 

Join Date: Feb 2007
Location: Aus Syd
Posts: 10
Unhappy shell script takes long time to complete

Hi all,

I wrote this shell script to validate filed numbers for input file. But it take forever to complete validation on a file. The average speed is like 9mins/MB.

Can anyone tell me how to improve the performance of a shell script?

Thanks
  #2 (permalink)  
Old 02-05-2007
matrixmadhan matrixmadhan is offline Forum Advisor  
Technorati Master
  
 

Join Date: Mar 2005
Location: leaf node in B+ tree
Posts: 2,944
It would be great if you could post sample input / output data and if possible your script
  #3 (permalink)  
Old 02-05-2007
ozzman ozzman is offline
Registered User
  
 

Join Date: Feb 2007
Location: Aus Syd
Posts: 10
Question Don't have the script right here with me

matrixmadhan,

I don't have the script right here with me, but I can brief you how my script looks like.

Code:
    #starts with couple of constants for the file
    function1 ...
    function2 ...
    function3
    {
       function4
    }
    
    function4...
    
    while time < 00:00:00
    do 
       function1
       if [ $? -eq 0 ]
       then
           for loop 
           do 
             function2...
             function4...
             ./call_another_script
           done
       fi
     done
Will this help to determine the cause?
  #4 (permalink)  
Old 02-05-2007
vino's Avatar
vino vino is offline Forum Staff  
Supporter (in vino veritas)
  
 

Join Date: Feb 2005
Location: Bangalore, India
Posts: 2,796
There could be something in those functions that is taking too much of a time. For example, some cut or some grep or an invocation of some external tools. Also the script, call_another_script could be the culprit.

Unless you can show what those functions are, it is hard to pinpoint the exact cause.
  #5 (permalink)  
Old 02-05-2007
sb008 sb008 is offline Forum Advisor  
Registered User
  
 

Join Date: Jan 2007
Posts: 384
Put some "date" commands in your script and you might be able to find out where the delay is and focus on that part.
  #6 (permalink)  
Old 02-05-2007
reborg's Avatar
reborg reborg is offline Forum Staff  
Administrator
  
 

Join Date: Mar 2005
Location: Ireland
Posts: 4,206
I would start by setting the debug flag in the shell. It might be obvious just from that what operation is taking the time, without knowing exactly what you are doing in the functions it is not really possible for anyone to answer.
  #7 (permalink)  
Old 02-05-2007
ozzman ozzman is offline
Registered User
  
 

Join Date: Feb 2007
Location: Aus Syd
Posts: 10
Thanks for the reply

the part that take the most of the time is the following code.

Code:
  
function line_count
   {
       COUNT=`echo $1 | awk -F\| '{print NF}'`
       if [ "$COUNT" != "$2" ]
       then
          error_log "File $FN: Validation failed at line   $LINENUM. Expected $2, getting $COUNT"
          return 5
       fi
   }

   function validate_line
   {
        if [ "$1" = "$FIRST_LEVEL_HEAD" ]
        then
              line_count "$2" $FIRST_LEVEL_COUNT
              return $?
        elif [ "$1" = "$SECOND_LEVEL_HEAD" ]
        then
              line_count "$2" $SECOND_LEVEL_COUNT
              return $?
        else
              error_log "File $FN: Line $LINENUM head is not regconised"
              return 5
        fi
   }


   function validate_file
   {
      trace_log "Start to validate $FN..."
      LINENUM=0
      ERROR=0
       while read LINE
       do
            LINENUM=`expr $LINENUM + 1`
            LINE_HEAD=`echo $LINE | awk -F\| '{print $1}'`
            validate_line $LINE_HEAD "$LINE"
            if [ ! $? -eq 0 ]
            then
               ERROR=1
            fi
       done < $1
       if [ ! $ERROR -eq 0 ]
       then
           return 7
       fi
    }


    validate_file $FILE

Any suggestion ?
Sponsored Links
Closed Thread

Bookmarks

Tags
grep or

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On




All times are GMT -4. The time now is 09:48 AM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0