storing large data in unix array variable


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting storing large data in unix array variable
# 1  
Old 07-02-2012
storing large data in unix array variable

Hi,

I have table in sql ..from this table im storing the first coloumn values in shell array variable ...

after this passing this variable as an arugument in SQL procedure.

But the proc. is running fine only for 1024 values in array ...

How to store more than 1024 values in the array so that all the values would be affected after calling the proc.

Plz reply soon.
# 2  
Old 07-02-2012
Use loops .. Post your code for better understanding ..
# 3  
Old 07-02-2012
Hi,

I'm not sure that using shell arrays is the right answer for the problem you have.

In general, dump of a database table in memory is not a good idea.

For my pov, you should use temporary files.
# 4  
Old 07-02-2012
It depends on how you are using the variable in the proc.

If you are using the variable on a IN statement then it would have the maximum input limit. Oracle has some limit which yield you the error ORA-00939

Worth verifying the proc first before the shell.
# 5  
Old 07-03-2012
My code snippet

Code:
set -A my_array `sqlplus -s username/pwd@DB <<EOF

set feedback off
SET NEWPAGE 1
SET ECHO OFF
SET FEEDBACK OFF
SET TERM OFF
SET PAGESIZE 50000
Set LINESIZE 300
SET TRIMS ON
set verify on

select * from Icunbarred_$var vv  ;

exit;
EOF`

echo "there are ${#my_array[*]} elements in the array"
element=2

while [ $element -lt ${#my_array[*]} ]
    do
        echo "----->>" ${my_array[element]}
        sqlplus -s username/pwd@DB <<EOF
        exec nextbilldtm_change(${my_array[element]});
        exit;
EOF
        let element=element+1;
    done

------------
My table contains around 10000 records. but after updating 1024 records it halts ...

What changes need to be done in this code. Kindly reply soon.

Last edited by methyl; 07-03-2012 at 06:56 AM.. Reason: please use code tags
# 6  
Old 07-03-2012
Forget the array. I would use a sqlplus spool command to create the output file, then read it back in the subsequent process.

However, I can't see the point of the intermediate file/array unless perhaps there are two independent databases or two different Oracle acconts. Any reason to not just invoke nextbilldtm_change in the first program?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Transpose large data in UNIX

Hi I have the following sample of data: my full data dimention is 900,000* 1119 rs987435 C G 1 1 1 0 2 rs345783 C G 0 0 1 0 0 rs955894 G T 1 1 2 2 1 rs6088791 ... (7 Replies)
Discussion started by: marwah
7 Replies

2. Shell Programming and Scripting

Bash arrays: rebin/interpolate smaller array to large array

hello, i need a bit of help on how to do this effectively in bash without a lot of extra looping or massive switch/case i have a long array of M elements and a short array of N elements, so M > N always. M is not a multiple of N. for case 1, I want to stretch N to fit M arrayHuge H = (... (2 Replies)
Discussion started by: f77hack
2 Replies

3. Shell Programming and Scripting

Perl : Large amount of data put into an array

This basic code works. I have a very long list, almost 10000 lines that I am building into the array. Each line has either 2 or 3 fields as shown in the code snippit. The array elements are static (for a few reasons that out of scope of this question) the list has to be "built in". It... (5 Replies)
Discussion started by: sumguy
5 Replies

4. Shell Programming and Scripting

storing multiple values in a array variable

Am using a find command in my script .The output may be one or more. I need to store those values in a array and need to access those. Am unable to find the solution . Any help on this will be helpful. if < code> else a=<find command output which gives the file name either 1 or more> if 1... (1 Reply)
Discussion started by: rogerben
1 Replies

5. Shell Programming and Scripting

Store Large data in Variable in csh

Hi All, This is my first post!! :) How to store huge data in a single variable in csh. Right now i can store upto ~700kb of data. I still want to store more data and i dont want to use perl. Is there any method to store like that...Please let me know.. Thanks (3 Replies)
Discussion started by: vickra
3 Replies

6. Shell Programming and Scripting

Storing data in perl 2D array

Respected All, Kindly help me out. I have got file listings in a directory like this: -rw-r--r-- 1 root root 115149 2011-11-17 07:15 file1.stat.log -rw-r--r-- 1 root root 115149 2011-11-18 08:15 file2.stat.log -rw-r--r-- 1 root root 115149 2011-11-19 09:15 file3.stat.log -rw-r--r-- 1... (2 Replies)
Discussion started by: teknokid1
2 Replies

7. Shell Programming and Scripting

Filtering some data and storing the remaing to a variable.

Hi, i have a problem writing a script. Actually i have many files which contains some data, now the last line has some data as in the format : 2556||04-04-10 now i want to do tail -1 filename.txt and store the result into a variable which will be used later for some other calculations. My... (7 Replies)
Discussion started by: fidelis
7 Replies

8. Shell Programming and Scripting

Storing count(*) into unix variable

Hi, I've just started on unix shell scripting a few days ago. I tried look for the solution to my problem over the net but to no avail. The objective of my script is to count the number of records in a particular table from a Oracle database. Ok, here's my problem, when I run this script: ... (1 Reply)
Discussion started by: mervinboyz
1 Replies

9. UNIX for Advanced & Expert Users

how to read the data from an excel sheet and use those data as variable in the unix c

I have 3 columns in an excel sheet. c1 c2 c3 EIP_ACCOUNT SMALL_TS_01 select A.* from acc; All the above 3 col shoud be passed a variable in the unix code. 1.How to read an excel file 2.How to pass these data as variable to the unic script (1 Reply)
Discussion started by: Anne Grace
1 Replies

10. HP-UX

Need to split a large data file using a Unix script

Greetings all: I am still new to Unix environment and I need help with the following requirement. I have a large sequential file sorted on a field (say store#) that is being split into several smaller files, one for each store. That means if there are 500 stores, there will be 500 files. This... (1 Reply)
Discussion started by: SAIK
1 Replies
Login or Register to Ask a Question