Spawn and join multiple process


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Spawn and join multiple process
# 1  
Old 04-30-2013
Spawn and join multiple process

I am doing some file manipulation and then a bcp once all the files are processed. I need to do the following

Code:
for all files in dirctory
 begin 
 -Step 1 use another shell/perl  to format  the file
 done
in the end load into db using bcp

I want to do step 1 and step in a seperate process and wait for all of it to complete before i do the bcp

Code:
FILES=./$fulldate/trades/*.trade
for f in $FILES
do
 echo "Processing $f into $f.csv"
   ./cleanfile.sh & #step1 

done

./bcpload.sh

I want to wait for all step1 to be completed before I start the bcp load
# 2  
Old 04-30-2013
Doesn't cleanfile.sh need an operand? Try:
Code:
for f in $fulldate/trades/*.trade
do
 echo "Processing $f into $f.csv"
   ./cleanfile.sh "$f" & #step1 
done
wait # This waits for all of the jobs started asynchronously above to complete.
./bcpload.sh

This User Gave Thanks to Don Cragun For This Post:
# 3  
Old 04-30-2013
Yes it does . Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Sleep 600 or fork/spawn a process or daemon?

Hi, I have a script that run every 10 minutes, from a specific timeframe of the day, for example 0500 - 1900. The script is some sort of checker script for an application log file and check for errors and email us if there is error/s reported in the log. At the moment, I schedule it... (1 Reply)
Discussion started by: newbie_01
1 Replies

2. Shell Programming and Scripting

Using expect command, spawn will not start sftp process

Hi all, I have a script that runs sftp with expect so I can login and send a file in a cronjob. I've installed this on a couple other servers and it has been fine. However, this time on this machine, it seems to be giving me an issue. It won't move past the spawn sftp command and return a... (3 Replies)
Discussion started by: ltyrrell
3 Replies

3. Shell Programming and Scripting

SPAWN Multiple Processes in Unix

Hi, I have three files in my IN directory.Each file should be copied 25 times using for loop.Each file processing should run in parallel?How to spawn multiple processes in unix?Any help would be appreciated. Thanks, Liyakath (7 Replies)
Discussion started by: liyakathali
7 Replies

4. UNIX for Dummies Questions & Answers

How to use the the join command to join multiple files by a common column

Hi, I have 20 tab delimited text files that have a common column (column 1). The files are named GSM1.txt through GSM20.txt. Each file has 3 columns (2 other columns in addition to the first common column). I want to write a script to join the files by the first common column so that in the... (5 Replies)
Discussion started by: evelibertine
5 Replies

5. Programming

spawn a process with a different user

Hello Everyone: I have the following code int main() { system("/usr/OtherUser/bin/runX"); return 0; } runX must be executed with privileges from another user, how could I do that? I know the password for such user. Thanks in advance (8 Replies)
Discussion started by: edgarvm
8 Replies

6. UNIX for Dummies Questions & Answers

Join 2 files with multiple columns: awk/grep/join?

Hello, My apologies if this has been posted elsewhere, I have had a look at several threads but I am still confused how to use these functions. I have two files, each with 5 columns: File A: (tab-delimited) PDB CHAIN Start End Fragment 1avq A 171 176 awyfan 1avq A 172 177 wyfany 1c7k A 2 7... (3 Replies)
Discussion started by: InfoSeeker
3 Replies

7. Programming

sql,multiple join,outer join issue

example sql: select a.a1,b.b1,c.c1,d.d1,e.e1 from a left outer join b on a.x=b.x left outer join c on b.y=c.y left outer join d on d.z=a.z inner join a.t=e.t I know how single outer or inner join works in sql. But I don't really understand when there are multiple of them. can... (0 Replies)
Discussion started by: robbiezr
0 Replies

8. Programming

Program to spawn multiple processes

I'm trying to make a program that will spawn multiple child processes then exit. I'm having trouble figuring out how to do this since after I fork, the child process begins running the program again (never ending). int main(void){ for(int i = 0; i < 3; i++){ fork(); }... (1 Reply)
Discussion started by: cagney58
1 Replies

9. Programming

Howto spawn multiple child processes and wait?

As far as I can tell, the bash wait command waits for a logical "AND" of all the child processes. Assuming I am coding in C: (1) What is the function I would use to create multiple bash child process running perl? (2) What is the function I would use to reinvent the bash wait command so I... (4 Replies)
Discussion started by: siegfried
4 Replies

10. Shell Programming and Scripting

How to Join with child process?

Java and perl have a join feature that allows the main thread to block until a child thread is finished. Is there a similar feature in (1) bash and (2) perl that will allow the main process to block until the child process is complete? Thanks, Siegfried (4 Replies)
Discussion started by: siegfried
4 Replies
Login or Register to Ask a Question