Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: split process files in parallel and rejoin
Top Forums Shell Programming and Scripting split process files in parallel and rejoin Post 95377 by mahendramahendr on Tuesday 10th of January 2006 08:02:37 PM
Old 01-10-2006
Script :
======

#!/usr/bin/ksh

split -l 1000 hugefile.txt -a hugefile

for fname in hugefile*
do
sqlldr user/password control=xyz.ctl log=${fname}.log data=$fname &
done

echo "Waiting for background process to complete...."

wait

echo "All files loaded..!!"


For finding the status, you can simply grep all the log files or count the lines in bad file.... not sure whether there is a way to trap directly from sqlldr command..

The above split will not actually split the original file, hence no need to join the splitted files back... simply delete the splitted files.

I have not tested the above stuff, correct if you have any syntax errors..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

killing parallel oracle process

Hi guys: I have a an oracle job which uses 10 parallel hints and would like to killit when it hangs. I want to kill all the processes that have been spawned. what I do right now is get the pid of the scheduler process which initiated theis job and the do a ps -ef| grep 'pid' and trace through... (1 Reply)
Discussion started by: oracle8
1 Replies

2. UNIX and Linux Applications

How can i see if a unix Process Aplication i.e oracle is running in parallel

There is a unix process process in oracle running and i see running by typing ps -fea|grep GE_CLIENTES. The question is How can i see if this process is running in paralel. I dont know with a Unix command or specifically its a comand from Oracle. I kow a Parallel process ia a process that... (1 Reply)
Discussion started by: alexcol
1 Replies

3. Shell Programming and Scripting

Parallel process in Perl

HI All, I have scenerio where I need to call sub modules through for loop for (i=0; i<8000 ;i++) { .. BLOCKA } BLOCKA { .. .. subroutine 1; subroutine 2; } I want this to be run in parallel process1 BLOCKA { (6 Replies)
Discussion started by: gvk25
6 Replies

4. Programming

Parallel process in java

Hello; Please Are both threads execute in parallel? Thank you (4 Replies)
Discussion started by: chercheur857
4 Replies

5. UNIX for Dummies Questions & Answers

Running parallel process

i am having 4 process,have to run parallel and not after one by one. sample1.sh sample2.sh sample3.sh sample4.sh Thanks in advance. i (11 Replies)
Discussion started by: sagar_1986
11 Replies

6. Shell Programming and Scripting

Make process parallel

Hi, I have a file which has a list of 200 tables e.g: table.txt I need to do a count for each table and store it in a file. So I did something like this: for TABLE in `cat table.txt` do T_CNT=$(sqlplus -s -l / as sysdba <<EOF set echo off heading off feadback off SELECT count(*) FROM... (1 Reply)
Discussion started by: wahi80
1 Replies

7. Shell Programming and Scripting

Process split files with same name by group

File examples f17_mar_01_02_03_04_fsw1.xml f17_mar_01_02_03_04_fsw2.xml f17_mar_01_02_03_04_fsw3.xml f17_feb_13_20_49_06_fsw1.xml f17_feb_13_20_49_06_fsw2.xml f17_feb_13_20_49_06_fsw3.xml I have many xml files that are grouped with same file name, but are numbered from 1-to-many... (2 Replies)
Discussion started by: aachave1
2 Replies

8. Shell Programming and Scripting

Monitoring processes in parallel and process log file after process exits

I am writing a script to kick off a process to gather logs on multiple nodes in parallel using "&". These processes create individual log files. Which I would like to filter and convert in CSV format after they are complete. I am facing following issues: 1. Monitor all Processes parallelly.... (5 Replies)
Discussion started by: shunya
5 Replies

9. UNIX for Beginners Questions & Answers

Split and Rename Split Files

Hello, I need to split a file by number of records and rename each split file with actual filename pre-pended with 3 digit split number. What I have tried is the below command with 2 digit numeric value split -l 3 -d abc.txt F (# Will Produce split Files as F00 F01 F02) How to produce... (19 Replies)
Discussion started by: techedipro
19 Replies

10. UNIX for Beginners Questions & Answers

Automate splitting of files , scp files as each split completes and combine files on target server

i use the split command to split a one terabyte backup file into 10 chunks of 100 GB each. The files are split one after the other. While the files is being split, I will like to scp the files one after the other as soon as the previous one completes, from server A to Server B. Then on server B ,... (2 Replies)
Discussion started by: malaika
2 Replies

LEARN ABOUT DEBIAN

flow-split

flow-split(1)						      General Commands Manual						     flow-split(1)

NAME

       flow-split -- Split flow files into smaller files.

SYNOPSIS

       flow-split [-gGhn]  [-b big|little]  [-C comment]  [-d debug_level]  [-N nflows]  [-o outfile_basename]	[-T nseconds]  [-z z_level]

DESCRIPTION

       The flow-split utility will split a flow file into smaller files based on the the number of flows or the ammount of time that has passed.

OPTIONS

       -b big|little
		 Byte order of output.

       -C Comment
		 Add a comment.

       -d debug_level
		 Enable debugging.

       -g	 Split on source tag.

       -G	 Split on destination tag.

       -h	 Display help.

       -n	 Use symbols for tag field in filename.

       -N nflows Split after processing nflows.

       -o outfile_basename
		 The basename of the resulting files.

       -T nsecond
		 Split after processing an interval of nseconds flows.

       -z z_level
		 Configure compression level to  z_level.  0 is disabled (no compression), 9 is highest compression.

EXAMPLES

       Create 1 minute flow files from the flow archive in /flows/krc4.  Store the results in /flows/krc4.split

	 flow-cat /flows/krc4 | flow-split -T60 -o /flows/krc4.split/1min.

BUGS

       None known.

AUTHOR

       Mark Fullmer maf@splintered.net

SEE ALSO

       flow-tools(1)

																     flow-split(1)
All times are GMT -4. The time now is 04:10 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy