The UNIX and Linux Forums  

Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
Google UNIX.COM


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
help on most efficient search prvnrk Shell Programming and Scripting 16 04-08-2008 01:24 PM
Can you suggest a more efficient way for this? mikie Shell Programming and Scripting 1 11-20-2006 07:49 AM
Efficient way of Awk braindrain Shell Programming and Scripting 4 07-11-2006 11:01 AM
Efficient I/O S.P.Prasad High Level Programming 1 12-09-2003 10:38 AM
Efficient Dispatching uchachra UNIX for Advanced & Expert Users 1 01-10-2003 02:33 PM

Reply
 
Submit Tools LinkBack Thread Tools Search this Thread Display Modes
  #1  
Old 12-15-2005
Registered User
 

Join Date: Dec 2005
Posts: 5
Is there a more efficient way?

I'm using korn shell to connect to oracle, retrieve certain values, put them in a list, and iterate through them. While this method works, I can't help but think there is an easier method.

If you know of one, please suggest a shorter, more efficient method.


PHP Code:
###############  FUNCTIONS  ####################################################################
check_for_sqlplus_error() 
{
    
# $1 = $? (results of last command)
    # $2 = user defined error code
  
if [ $-ne 0 ]; then
   write_log 
" ERROR - sqlplus execution failure at: "$2
   
exit 1
  fi
  
return 0
}    

run_query()
{
    
# $1 - db login
    # $2 - user defined error code
    # $3 - SQL to execute
    # $4 - logfile to re-direct to
$SQLPLUS -$1<<EOF |  sed '/^$/d' > $4
  set HEADING   OFF  
  set FEED OFF
  set TRIMOUT   ON 
  set TRIMSPOOL ON 
  
$3;
EOF

check_for_sqlplus_error 
$? "code "$2
return 0
}

run_proc()
{
    
# $1 - db login
    # $2 - user defined error code
    # $3 - SQL to execute
    # $4 - logfile to re-direct to
$SQLPLUS -$1<<EOF |  sed '/^$/d' > $4
  set pagesize 0
  set AUTOT ON
  set serveroutput on SIZE 50000
  
$3;
EOF

check_for_sqlplus_error 
$? "code "$2
return 0
}

write_log ()
{
  echo `
date +"%m/%d/%Y %H:%M:%S"`" "$1
}

#################  MAIN  ########################################################################
# SQL*PLUS code:200 - Get the list of batches for this load 
write_log " Get the Batch List..."
run_query $ODS 200 "select distinct batch_num from cr_batch where load_desc = '$SOURCE_SYSTEM_DESC'"  $LOGPATH$SOURCE_SYSTEM_DESC.batch.lst

if [ -s $LOGPATH$SOURCE_SYSTEM_DESC.batch.lst ]; then
  write_log 
" Begin Processing Batches..."
  
while read BATCH_NUM
  
do
    if [ 
$BATCH_NUM ]; then
       
# SQL*PLUS code:300 - Run the batch load stored procedure for this batch 
       
write_log " Calling p_stage_load with $SOURCE_SYSTEM_DESC , $LOAD_RUN_NUM, $BATCH_NUM "
       
run_proc $STAGING 300 "exec p_stage_load('$SOURCE_SYSTEM_DESC', $LOAD_RUN_NUM, $BATCH_NUM )"  $LOGPATH$SOURCE_SYSTEM_DESC.$BATCH_NUM.log &
    else 
      
write_log " No batches found for '$SOURCE_SYSTEM_DESC'"
    
fi
  done 
$LOGPATH$SOURCE_SYSTEM_DESC.batch.lst
else 
    
# if there are no batches, lets just exit
    
write_log " No batches for "$SOURCE_SYSTEM_DESC" were found!...Exiting"
    
exit
fi 
Reply With Quote
Forum Sponsor
  #2  
Old 12-15-2005
Registered User
 

Join Date: Jan 2005
Posts: 682
First, FYI: sqlplus will return exit code 0 when the database isn't up; you can try it yourself. Add "WHENEVER SQLERROR" to force a non zero exit code.

Quote:
Originally Posted by SelectSplat
Code:
run_query()
{
# $1 - db login
# $2 - user defined error code
# $3 - SQL to execute
# $4 - logfile to re-direct to
$SQLPLUS -s $1<<EOF |  sed '/^$/d' > $4
  WHENEVER SQLERROR EXIT 1
  set HEADING   OFF  
  set FEED OFF
  set TRIMOUT   ON 
  set TRIMSPOOL ON 
  $3;
EOF

check_for_sqlplus_error $? "code "$2
return 0
}
Second: You certainly could combine your sqlplus calls into one.
Code:
run_query() 
{ 
    # $1 - db login 
    # $2 - user defined error code 
    # $3 - SQL to execute 
    # $4 - logfile to re-direct to 
$SQLPLUS -s $1<<EOF | sed '/^$/d' > $4 
  $3; 
EOF 

check_for_sqlplus_error $? "code "$2 
return 0 
}

run_query $ODS 200 "
  set HEADING OFF FEEDBACK OFF TRIMOUT ON TRIMSPOOL ON
  select distinct
	batch_num 
  from	cr_batch
  where	load_desc = '$SOURCE_SYSTEM_DESC';
"  $LOGPATH$SOURCE_SYSTEM_DESC.batch.lst 

...
run_query $STAGING 300 "
  set pagesize 0 AUTOT ON serveroutput on SIZE 50000
     exec p_stage_load('$SOURCE_SYSTEM_DESC', $LOAD_RUN_NUM, $BATCH_NUM );
"  $LOGPATH$SOURCE_SYSTEM_DESC.$BATCH_NUM.log & 
...
Personally, I prefer to use this kind of construct:
Code:
# Read whole lines
IFS='
'
set -A RESULTS $({
    sqlplus -s /nolog "
        connect un/pw
        select or exec ...;
"
    print RC=$?
} 2>&1)

if [[ "RC=0" != ${RESULST[(( ${#RESULTS[@]} - 1} ))] ]]
then
    write_log "sqlplus error"
    exit 1
fi

for i in ${RESULTS[@]}
do
    case $i in
        ORA*|SP2*|PLS*)
            write_log "sql error"
            exit 1
        ;;
        ... whatever else you want to look for ...
        ;;
    esac
done
If you results exceed 4098, you won't want to use an array if you Korn shell is KSH88. I believe that the newer Korn shells will allow more the 4098 elements.

Last edited by tmarikle; 12-15-2005 at 08:48 PM.
Reply With Quote
  #3  
Old 12-16-2005
Registered User
 

Join Date: Dec 2005
Posts: 5
The first two tips are GREAT thanks. SQL*Plus is new to me, as nearly all my experience has been with ISQL.

Your last suggestion is extremly interesting to me, as I can barely follow what's going on. In fact, I don't understand at all.

Is 'RESULTS' an array of strings being assigned the result set from sqlplus?
What is IFS?
What is /nolog ?
I'm completly lost with the 'if'.
If I use this method, will I still have each 'word' in each row in it's own variable?
Can you psudo code document it for me?

Also, I'm not exactly sure what version of korn we're using, but I did notice that VAR=$(command) doesn't work as I'd expect. It looks like I have the same funcationality with VAR=`command`, but I'm not 100% sure.

Sorry for all the questions. Thanks for you reply, and in advance for your further elaboration.


Quote:
Originally Posted by tmarikle

Personally, I prefer to use this kind of construct:
Code:
# Read whole lines
IFS='
'
set -A RESULTS $({
    sqlplus -s /nolog "
        connect un/pw
        select or exec ...;
"
    print RC=$?
} 2>&1)

if [[ "RC=0" != ${RESULST[(( ${#RESULTS[@]} - 1} ))] ]]
then
    write_log "sqlplus error"
    exit 1
fi

for i in ${RESULTS[@]}
do
    case $i in
        ORA*|SP2*|PLS*)
            write_log "sql error"
            exit 1
        ;;
        ... whatever else you want to look for ...
        ;;
    esac
done
If you results exceed 4098, you won't want to use an array if you Korn shell is KSH88. I believe that the newer Korn shells will allow more the 4098 elements.
Reply With Quote
  #4  
Old 12-16-2005
Registered User
 

Join Date: Jan 2005
Posts: 682
[code]
# Read whole lines
IFS defines what field seperators consist of. Normally they are space, tab, newlines, etc. I am redefining the field
seperators to newlines only so for and while loops process lines instead of words.

IFS='
'
This is defining an array from sqlplus' output plus my print command. "/nolog" just keeps sqlplus from
attempting to log in from any point except from a "connect" command from the SQL.

Code:
set -A RESULTS $({
    sqlplus -s /nolog "
        WHENEVER SQLERROR EXIT 1    <== Forgot to include my earlier recommendation
        connect un/pw
        select or exec ...;
"
    print RC=$?
} 2>&1)


My results array will consist of elements (whole lines of text) starting from 0.  Remember that I manufactured
my own message following the sqlplus to show sqlplus' exit code (RC=$?).  This sits in the array's last element.
Since we don't typically know how many rows will return I have to compute the size of the array before knowing
which element is the last.

${#RESULTS[@]} gives me the array's size in terms of the number of elements and (( ${#RESULTS[@]} - 1} )) 
points me to the array's last element (remember that the array starts with at element 0).  Therefore, the "if"
statement compares a known "good" exit code message of "RC=0" with the actual message returned following sqlplus'
execution.  If the message is not "RC=0" it must be bad and sqlplus exited with an error so we will exit too.
if [[ "RC=0" != ${RESULST[(( ${#RESULTS[@]} - 1} ))] ]]
then
    write_log "sqlplus error"
    exit 1
fi

This loop simply processes each element in the array and tests each array element for whatever we want.  "ORA*|SP2*|PLS*)"
are known Oracle error messages so we typically want to handle them.
for i in ${RESULTS[@]}
do
    case $i in
        ORA*|SP2*|PLS*)
            write_log "sql error"
            exit 1
        ;;
        ... whatever else you want to look for ...
        ;;
    esac
done
Reply With Quote
  #5  
Old 12-16-2005
Registered User
 

Join Date: Dec 2005
Posts: 5
That's outstanding.

Will this still work in my version of ksh, even when it appears that $(command) isn't working? Or do I need to do the obvious, and replace the $(command) with `command` ?
Reply With Quote
  #6  
Old 12-16-2005
Registered User
 

Join Date: Dec 2005
Posts: 5
Also, 2 other important questions regarding this technique.

Using the 'while read', I'm able to capture each 'word' of a line in a seperate variable, while processing all of the lines in the file. It appears that your technique is putting the whole line in an element of the array. If I do that, I'd need to use awk, or something similar, to address a particular 'word' in the line, correct?

Also, in this loop, I'm spawning batches off in the background. Each spawned batch load a list of tables sequentially. If one of the tables in the batch fails to load, the desired effect is to continue on with the next table. So, from your explaination, I gather that this statement...

if [[ "RC=0" != ${RESULST[(( ${#RESULTS[@]} - 1} ))] ]]

Would not be relevent. Is that correct?
Reply With Quote
  #7  
Old 12-16-2005
Registered User
 

Join Date: Jan 2005
Posts: 682
Regarding KSH: I'm using KSH88, which is about as old as they get I think. I do not know why $() doesn't work but back ticks should work fine. Bourne and Bash use back ticks; are use certain that you aren't using one of these shells?

Regarding words vs lines: this is what happened with we changed IFS. You can reset IFS to use spaces at any time letting you process words in a read loop.
Code:
IFS=' '
Regarding the if statement: mine is only testing whether sqlplus exited with a non-zero result so I don't know whether that would be relavent in your case; perhaps.
Reply With Quote
Google The UNIX and Linux Forums
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes




All times are GMT -7. The time now is 05:41 AM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited.
The UNIX and Linux Forums Content Copyright ©1993-2008. All Rights Reserved.Ad Management by RedTyger Visit The Complex Event Processing Blog

Content Relevant URLs by vBSEO 3.2.0