split process files in parallel and rejoin Post: 95445

Sponsored Content

Top Forums Shell Programming and Scripting split process files in parallel and rejoin Post 95445 by jim mcnamara on Wednesday 11th of January 2006 09:44:28 AM

01-11-2006

Registered User

the {$fname}.log files will each have a section like this at the end of the file:

Code:

Table UABBDBT:
  1 Row successfully loaded.
  0 Rows not loaded due to data errors.
  0 Rows not loaded because all WHEN clauses were failed.
  0 Rows not loaded because all fields were null.


Space allocated for bind array:                 255420 bytes(55 rows)
Read   buffer bytes: 1048576

Total logical records skipped:          0
Total logical records read:             1
Total logical records rejected:         0
Total logical records discarded:        0

Run began on Tue Jul 19 11:28:38 2005
Run ended on Tue Jul 19 11:28:39 2005

Elapsed time was:     00:00:00.45
CPU time was:         00:00:00.14

grep for "not loaded" or "discarded" or "rejected" (depending on how you set up your .ctl file.

Code:

bad_records=`grep "not loaded" whatever1.log | awk '{print $NR}'`
echo "number of bad records=$bad_records"

jim mcnamara

View Public Profile for jim mcnamara

Find all posts by jim mcnamara

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

killing parallel oracle process

Hi guys: I have a an oracle job which uses 10 parallel hints and would like to killit when it hangs. I want to kill all the processes that have been spawned. what I do right now is get the pid of the scheduler process which initiated theis job and the do a ps -ef| grep 'pid' and trace through...

2. UNIX and Linux Applications

How can i see if a unix Process Aplication i.e oracle is running in parallel

There is a unix process process in oracle running and i see running by typing ps -fea|grep GE_CLIENTES. The question is How can i see if this process is running in paralel. I dont know with a Unix command or specifically its a comand from Oracle. I kow a Parallel process ia a process that...

3. Shell Programming and Scripting

Parallel process in Perl

HI All, I have scenerio where I need to call sub modules through for loop for (i=0; i<8000 ;i++) { .. BLOCKA } BLOCKA { .. .. subroutine 1; subroutine 2; } I want this to be run in parallel process1 BLOCKA {

4. Programming

Parallel process in java

Hello; Please Are both threads execute in parallel? Thank you

5. UNIX for Dummies Questions & Answers

Running parallel process

i am having 4 process,have to run parallel and not after one by one. sample1.sh sample2.sh sample3.sh sample4.sh Thanks in advance. i

6. Shell Programming and Scripting

Make process parallel

Hi, I have a file which has a list of 200 tables e.g: table.txt I need to do a count for each table and store it in a file. So I did something like this: for TABLE in `cat table.txt` do T_CNT=$(sqlplus -s -l / as sysdba <<EOF set echo off heading off feadback off SELECT count(*) FROM...

7. Shell Programming and Scripting

Process split files with same name by group

File examples f17_mar_01_02_03_04_fsw1.xml f17_mar_01_02_03_04_fsw2.xml f17_mar_01_02_03_04_fsw3.xml f17_feb_13_20_49_06_fsw1.xml f17_feb_13_20_49_06_fsw2.xml f17_feb_13_20_49_06_fsw3.xml I have many xml files that are grouped with same file name, but are numbered from 1-to-many...

8. Shell Programming and Scripting

Monitoring processes in parallel and process log file after process exits

I am writing a script to kick off a process to gather logs on multiple nodes in parallel using "&". These processes create individual log files. Which I would like to filter and convert in CSV format after they are complete. I am facing following issues: 1. Monitor all Processes parallelly....

9. UNIX for Beginners Questions & Answers

Split and Rename Split Files

Hello, I need to split a file by number of records and rename each split file with actual filename pre-pended with 3 digit split number. What I have tried is the below command with 2 digit numeric value split -l 3 -d abc.txt F (# Will Produce split Files as F00 F01 F02) How to produce...

10. UNIX for Beginners Questions & Answers

Automate splitting of files , scp files as each split completes and combine files on target server

i use the split command to split a one terabyte backup file into 10 chunks of 100 GB each. The files are split one after the other. While the files is being split, I will like to scp the files one after the other as soon as the previous one completes, from server A to Server B. Then on server B ,...

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

killing parallel oracle process

Discussion started by: oracle8

2. UNIX and Linux Applications

How can i see if a unix Process Aplication i.e oracle is running in parallel

Discussion started by: alexcol

3. Shell Programming and Scripting

Parallel process in Perl

Discussion started by: gvk25

4. Programming

Parallel process in java

Discussion started by: chercheur857

5. UNIX for Dummies Questions & Answers

Running parallel process

Discussion started by: sagar_1986

6. Shell Programming and Scripting

Make process parallel

Discussion started by: wahi80

7. Shell Programming and Scripting

Process split files with same name by group

Discussion started by: aachave1

8. Shell Programming and Scripting

Monitoring processes in parallel and process log file after process exits

Discussion started by: shunya

9. UNIX for Beginners Questions & Answers

Split and Rename Split Files

Discussion started by: techedipro

10. UNIX for Beginners Questions & Answers

Automate splitting of files , scp files as each split completes and combine files on target server

Discussion started by: malaika