SQL/Plus in a coprocess example. Also saves query results into shell variables


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting SQL/Plus in a coprocess example. Also saves query results into shell variables
# 1  
Old 07-20-2011
SQL/Plus in a coprocess example. Also saves query results into shell variables

While assisting a forum member, I recommended running SQL/Plus in a coprocess (to make database connections and run a test script) for the duration of his script rather than starting/stopping it once for every row in a file he was processing.

I recalled I made a coprocess example for folks at work that illustrates doing this plus shows several ways of returning query results into shell variables. I thought I would stick my neck out and post it here as I think the technique is useful. I'm sure folks out there have done this in different ways and will let us know if they have suggestions to improve it. At any rate I hope someone out there finds this info useful.

This works on Solaris using the Korn shell against an Oracle 10g database:
Code:
#!/bin/ksh
##
##  Coprocess example.  Start SQL/Plus in the background, and communicate
##  with it via a pipe as long as you need it.  This saves overhead of
##  starting it multiple times.  It stays running while we make queries to it.
##
##  This example also shows how to save query results into shell variables
##  using 3 different methods.
##
##  print -p to send to the coprocess,
##  read -p to read from it a line at a time.
##

output=""                  ## Output from SQL/Plus goes here.
set -f output              ## Don't do filename expansion on this variable.  For when an error
                           ## from SQL/Plus contains a splat (*).
integer rc                 ## Holds a return code.
typeset -r ERRFILE=$0.err  ## Define an error file.
typeset -r EOF="DONE"      ## Text used to indicate the end of SQL/Plus output.

## Create the error file or zero it out if it already exists.
> $ERRFILE

## Start sqlplus in a coprocess.
sqlplus -S / |&

##  Exit SQL/Plus if any of the following signals are received:
##  0=normal exit, 2=interrupt, 3=quit, 9=kill, 15=termination
trap 'print -p "exit"' 0 2 3 9 15

## Send commands to SQL/Plus.
print -p "set heading off;"
print -p "set feedback off;"
print -p "set pagesize 0;"
print -p "set linesize 500;"

##
## Send a query to SQL/Plus.  It is formatted so we can set a shell variable.
##
print -p "select 'COUNT1='||count(*) as count from dual;"
print -p "prompt $EOF"  ## This is an indicator that we reached the end
                        ## of selected data.  When we read a DONE from
                        ## the coprocess we know we have no more data.

##  Read the output from the coprocess a line at a time. When DONE
##  is read, that indicates the end of output.
while read -p output
do
   if [[ "$output" == "$EOF" ]]; then
     break
   else
     ## eval forces the shell to evaluate the line twice.  First, replacing
     ## "$output" with "COUNT1=99999", then again which creates and sets
     ## a variable.
     eval $output
   fi
done

##
##  Send another query to the same running sql/plus coprocess.
##
print -p "select 'COUNT1_DATE='|| sysdate as count_date from dual;"
print -p "prompt $EOF"
while read -p output
do
   if [[ "$output" == "$EOF" ]]; then
     break
   else
     eval $output
   fi
done

##
##  Send yet another query to the same running sql/plus coprocess. This one
##  returns several values at once which eval handles as long as they are separated
##  by ';'. This way you could read multiple values from the database in one query
##  and set shell variables for them all.
##
print -p "select 'COUNT2='||count(*)||';COUNT2_DATE='||sysdate from dual;"
print -p "prompt $EOF"
while read -p output
do
   if [[ "$output" == "$EOF" ]]; then
     break
   else
     eval $output
   fi
done

##
##  You could also read a row into an array for cases where you may need
##  to process multiple rows returned.
##
##  First create a pipe-separated list of values. Don't use the deault IFS of a space as
##  data returned could contain spaces.  Use a character not likely in the data.
##
print -p "select count(*)||'|'||sysdate from dual;"
print -p "prompt $EOF"

while read -p output
do
   if [[ "$output" == "$EOF" ]]; then
     break
   else
     IFS="|"                      ## Set the Input Field Separator.
     set -A output_array $output  ## Create an array. It parses
                                  ## automatically on the IFS character.
   fi
done

##
##  Deliberately cause an error to show error handling. Call a procedure
##  that does not exist.
##
print -p "exec efs"
print -p "prompt $EOF"
while read -p output
do
   if [[ "$output" == "$EOF" ]]; then
     break
   fi
   ##
   ##  If unexpected output, log it to an error file and exit.  In this case, the
   ##  error from SQL/Plus will start with "BEGIN" followed by any number of
   ##  characters.
   ##
   print "$output" | grep '^BEGIN.*' >/dev/null        # Does output start with BEGIN?
   rc=$?
   if (( $rc == 0 )) then                              # It does.
     print -p "prompt $EOF"                            # The prompt DONE after the exec does not
                                                       # seem to print when an error happens so
                                                       # we have to issue it again here.
     while [[ "$output" != "$EOF" ]]
     do
       print "$output" >> $ERRFILE
       read -p output
     done
     #print -p "exit"
     #exit 1                                           # This should really exit but for demo
                                                       # purposes we will look the other way.
   else                                                # It doesn't so use it.
     eval $output
   fi
done

print "COUNT1 count is $COUNT1"
print "COUNT1 date is  $COUNT1_DATE\n"
print "COUNT2 count is $COUNT2"
print "COUNT2 date is  $COUNT2_DATE\n"
print "Array count3:   ${output_array[0]}"
print "Array date3:    ${output_array[1]}"
print "\n\nContents of error file:"
cat $ERRFILE

# Close the coprocess. Technically this is handled by the trap.
print -p "exit"

exit 0

Output:
Code:
$ gcw3.sh
COUNT1 count is 1
COUNT1 date is  03-AUG-2011

COUNT2 count is 1
COUNT2 date is  03-AUG-2011

Array count3:   1
Array date3:    03-AUG-2011


Contents of error file:
BEGIN efs; END;

      *
ERROR at line 1:
ORA-06550: line 1, column 7:
PLS-00201: identifier 'EFS' must be declared
ORA-06550: line 1, column 7:
PL/SQL: Statement ignored

$


Last edited by gary_w; 08-17-2011 at 11:30 AM.. Reason: Added EOF variable; indicator of end of SQL/Plus output. Also typeset it to be read-only since it's a constant.
These 3 Users Gave Thanks to gary_w For This Post:
# 2  
Old 08-03-2011
Great! This really helped me to convert one of my scripts.

The problem I have is that when the sql statement returns an error (e.g I try to use "exec proc_name" and the name of the procedure is wrong) I get this:


Code:
sql>exec delete_logsx(12345);
BEGIN delete_logsx(12345); END;

                 *
ERROR at line 1:
ORA-06550: line 1, column 18:
PLS-00302: component 'DELETE_LOGSX' must be declared
ORA-06550: line 1, column 7:
PL/SQL: Statement ignored

Beacuse there is a "*", I see this (when using set -xv):

+ read -p OUT
+ [[ * = DONE ]]
And then a LONG list of all the files in my directory.

Any suggestions?

Last edited by radoulov; 02-27-2013 at 11:37 AM..
# 3  
Old 08-03-2011
Glad you found this useful. Yes this simple example of selecting counts and setting variables does not allow for proper error handling.

You would have to add code in the test to handle output from SQL/Plus that you do not expect before doing the eval.. Given the output from calling a procedure that does not exist:
Code:
SQL> exec efs
BEGIN efs; END;

      *
ERROR at line 1:
ORA-06550: line 1, column 7:
PLS-00201: identifier 'EFS' must be declared
ORA-06550: line 1, column 7:
PL/SQL: Statement ignored

SQL>

you could test for the first word of the output being = "BEGIN" and if so, write the remaining lines until "DONE" to an error file and exit. Or test if the output does not start with what you are expecting and log the remaining lines and exit. At any rate put the common error handling code in a function so it can be reused.

I have updated the code and added an example of catching an error. Note the addition of set -f output which tells the shell not to do filename expansion on it (that is why you got the list of filenames).

Gary

Last edited by gary_w; 08-03-2011 at 12:19 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Storing the SQL results in array variables

Requirement 1) I need to execute 15 SQL queries in oracle through linux script. All these query results needs to be stored in array variables. Requirement 2) And these 15 queries needs to be executed in parallel. Requirement 3) Once all the queries executed then the shell script should... (3 Replies)
Discussion started by: Niranjancse
3 Replies

2. UNIX for Dummies Questions & Answers

Getting values of 2 columns from sql query in UNIX variables

Hi, I have connected to oracle database with sqlplus -s / <<EOF select ename, age from emp where empid=1234; EOF I want to save the values of ename and age in unix shell variables. Any pointers would be welcome.. Thanks in advance!!1 Cheers :):):):) (1 Reply)
Discussion started by: gonchusirsa
1 Replies

3. Shell Programming and Scripting

How to Assign SQL Query Results to Variables in Linux?

Hi, I am new to linux... How to Assign SQL Query Results to Variables in Linux,i want ti generate it in param files, Can anyone please explain me. Ex: SQL> Select * from EMP; O/P: Emp_No Emp_Name 1 AAA 2 BBB 3 CCC and I want expected... (5 Replies)
Discussion started by: Sravana Kumar
5 Replies

4. Shell Programming and Scripting

How to store results of multiple sql queries in shell variables in ksh?

Hi, I have a script where I make a sqlplus connection. In the script I have multiple sql queries within that sqlplus connection. I want the result of the queries to be stored in shell variables declared earlier. I dont want to use procedures. Is there anyway else. Thanks in advance.. Cheers (6 Replies)
Discussion started by: gonchusirsa
6 Replies

5. Shell Programming and Scripting

Multiple Query Results to Variables

Hello, I am very new to shell scripting and I am not sure of how best to handle the following scenario. I need to query a list of values from a table. I need to store those results and use them to selectively delete values in yet another table in a separate database. I do know how to store the... (3 Replies)
Discussion started by: flowervz
3 Replies

6. Shell Programming and Scripting

Oracle Query results to be stored in variables using unix

I want to store the sql query output into a variable #!/bin/ksh ORACLE_SID=DB01; export ORACLE_SID; export FILE_PATH=/home/asg/Tmp # Order Checking echo " removing old files " rm $FILE_PATH/Malformed_Order.txt echo " Enter the Malformed Order ....!" read orders echo "Regrade... (5 Replies)
Discussion started by: Nareshp
5 Replies

7. Shell Programming and Scripting

How to use sql data file in unix csv file as input to an sql query from shell

Hi , I used the below script to get the sql data into csv file using unix scripting. I m getting the output into an output file but the output file is not displayed in a separe columns . #!/bin/ksh export FILE_PATH=/maav/home/xyz/abc/ rm $FILE_PATH/sample.csv sqlplus -s... (2 Replies)
Discussion started by: Nareshp
2 Replies

8. Shell Programming and Scripting

Oracle Query results to be stored in variables

Hi I would like to know if there is a way to just have one ORACLE connection established, using which we can execute different queries and store the results under different variables. For e.g the following uses to two silent ORACLE connection to store the result under two different... (4 Replies)
Discussion started by: ashokjaysiv
4 Replies

9. UNIX for Advanced & Expert Users

Set shell variables from SQLPLUS query results

Hi All, I needed to get the result of two sqlplus queris into shell variables. After days of looking for the ultimate solution to this problem.. i found this... sqlplus -s USER/PASS@DB <<EOF | awk '{if(NR==1) printf("%s ", $1); if(NR==2) printf("%s ", $1);}' | read VAR1 VAR2 set head off... (2 Replies)
Discussion started by: pranavagarwal
2 Replies

10. UNIX for Dummies Questions & Answers

sql query results in unix shell script

Hi I want to get the a field from a SQL query into unix shell script variable. the whole situation is like this. 1. Opened a cursor to a table in DB2 databse. 2. Fetching individual rows with the help of cursor. 3. Each row has 4 fields. I want each of the field in individual shell... (1 Reply)
Discussion started by: skyineyes
1 Replies
Login or Register to Ask a Question