Visit Our UNIX and Linux User Community


Parallel sqlldr to load data.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Parallel sqlldr to load data.
# 1  
Old 05-03-2014
Parallel sqlldr to load data.

I am using SQLLDR to load data in DB.For parallel loading I'm using nohup command.
The requirement is:
I have different files within a directories.
Ex: 1) Dir/folder_A/AE.txt
2) Dir/folder_A/DM.txt
3) Dir/folder_B/CM.txt
I need to loop through directories and load the data parallelly in Database table.

The problem is: if the file is being loaded I need to check if another file from same directory is still loading then this new file should wait for previous to complete load else this new file will start loading data parallely.

ex : if AE.txt is already loading and DM.txt comes then it should check if DM.txt is also from folder_A,if yes then check if AE.txt is still loading the data ,if yes then wait for it to finish else load DM.txt,when other file CM.txt comes in since it is in folder_B it can start loading data no matter DM.txt is still l loading or not.

Please help!
# 2  
Old 05-05-2014
How many directories you have? Are those fixed?

You can do something like..

Code:
for mydir in <dirs>
do 
  load_my_data $mydir &
done
wait


Inside load_my_data,

Code:
function load_my_data {
 local dir=$1
 for myfile in $dir
 do
  sqlldr ... ...
 done

}

This User Gave Thanks to clx For This Post:

Previous Thread | Next Thread
Test Your Knowledge in Computers #721
Difficulty: Easy
In 2011, Alan Minsky was inducted into IEEE Intelligent Systems AI Hall of Fame for the "significant contributions to the field of AI and intelligent systems&".
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

SQLLDR :Data not loaded completely

Hi , I am using below control file LOAD DATA APPEND INTO TABLE LSHADMIN.EG TRAILING NULLCOLS ( STUDY CHAR ) and the text file to load data is CACZ885M2301 When I run below command: sqlldr userid=apps/apps control=/home/appsuser/dataload/ctl_file.ctl... (3 Replies)
Discussion started by: Pratiksha Mehra
3 Replies

2. UNIX for Dummies Questions & Answers

Load UNIX data into excel sheet

Hi, i have some data in a temporary file in Unix (the data is taken from the result of an SQL query). Now i want to dump that data into an excel sheet. How to do that. Someone please advise. Thanks Regards, Vinit (3 Replies)
Discussion started by: vinit raj
3 Replies

3. Shell Programming and Scripting

Capture the data in Linux .While doing load test.

Hi All, I am trying to capture the data in linux .While doing load test. is there's any sample script please help me. Linux test4 2.6.18-308.8.1.el5 #1 SMP Fri May 4 16:43:02 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux Thanks, (5 Replies)
Discussion started by: sam1226
5 Replies

4. Shell Programming and Scripting

Perl script to load data by calling sqlldr

Hello all, I know this is Unix forum, but i also know that there are some experts here who can help me out with this situation; I am loading a data file into oracle table using Perl script by calling sqlldr script. It does not do anything, and no data is getting loaded. Any help,... (2 Replies)
Discussion started by: msrahman
2 Replies

5. Shell Programming and Scripting

sh and MySQL LOAD DATA

I have a csv file in a folder that is roughly 500,000 rows long. Rather than using PHP, I would like to use SH to run a MYSQL LOAD DATA command to load the data in, as I think it would be much faster and would not cause any memory problems associated with PHP. But the problem is, I am not... (18 Replies)
Discussion started by: worchyld
18 Replies

6. Shell Programming and Scripting

Load data to flat file from table.

Hi all, I need to know how to copy data from a table say ABC to a flat file say XYZ.dat in unix, Please leave ur comments and the fastest way to do so, I need to load the table records into flat file. Regards Ann (4 Replies)
Discussion started by: Haque123
4 Replies

7. Shell Programming and Scripting

To load data from variable to oracle table ....???

Hi all, How to load variable value into Oracle table? I have created a file validation shell script. After the validation completes, i need to update a Oracle table with the variable value... Can someone help me how to do it? I have searched, but not able to get answer. i have 4 variables... (2 Replies)
Discussion started by: Amit.Sagpariya
2 Replies

8. Shell Programming and Scripting

To load data from variable to oracle table

Hi all, How to load variable value into Oracle table? I have created a file validation shell script. After the validation completes, i need to update a Oracle table with the variable value... Can someone help me how to do it? I have searched, but not able to get answer. i have 4... (1 Reply)
Discussion started by: Amit.Sagpariya
1 Replies

9. Shell Programming and Scripting

Automatically Load data from all files in directory

I'm new in Unix shell scripting and i need someone to help me to make Script that run all time watching my directory that files uploaded to it via FTP (/mydir/incoming_files), if any files exists in it then (if many files exists, then sort files and load them ascending) it‘ll checks the size of the... (1 Reply)
Discussion started by: m_fighter
1 Replies

10. Shell Programming and Scripting

Need help in wrting Load Script for a Load-Resume type of load.

hi all need your help. I am wrting a script that will load data into the table. then on another load will append the data into the existing table. Regards Ankit (1 Reply)
Discussion started by: ankitgupta
1 Replies

Featured Tech Videos