Sponsored Content
Top Forums Shell Programming and Scripting Need Help With Parallel Processing Post 302222023 by bakunin on Tuesday 5th of August 2008 10:51:43 PM
Old 08-05-2008
If you just want to start processes you could use the background process mechanism:

command &

will start the process "command" but will continue processing of the program issuing it after it is started. You can find out the background jobs PID with the "jobs" internal shell command and look in a loop if all started background processes have ended or not.

The following is a sketch - note, that the output of "jobs" is a little different from one system to another. This was written on AIX 5.3, maybe you need to change the regexps a bit to make them fit.

Code:
#! /bin/ksh
typeset -i iPID=0

some_command &
iPID=$(jobs -l | sed -n 's/^\[[0-9]*\]  *+  *\([^ ]*\) .*/\1/p')

while [ $(ps -fe | grep -c $iPID) -gt 1 ] ; do
     # wait for process to end - process is still running
     sleep 1
done

print - "process has finished"

exit 0

If you need to exchange messages with the subprocesses you will have to use the Korn Shells Coprocess facility: a coprocess is a background process just like above, but one that can communicate with the main process through two file descriptors: &4 and &5. There are special commands ("read -p" and "print -p") to communicate between the processes. You might want to read some documentation first, because building such a coprocess is a bit complicated at the beginning.

I hope this helps.

bakunin
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to do parallel processing??

Hi All, I am working on solaris 8 sparc machine with 2 cpu. I am trying to run my application which generates files. I run multiple instance of the application, but the results don't seem to show as if it were runing parallely. When i run the application once it takes 12 secs to generate a... (1 Reply)
Discussion started by: zing
1 Replies

2. Shell Programming and Scripting

parallel processing

Hi I want to run two shell script files parallely. These two scripts are interacting with the database. can any body help on this Pls Regards Audippa naidu.M (3 Replies)
Discussion started by: audippa
3 Replies

3. Shell Programming and Scripting

parallel processing

hi i am preparing a set of batches for a set of files sequentially There is a folder /xyz where all the files reside now all the files starting with 01 - will be appended for one below other to form a batch batch01 then all the files starting with 02 - will be appended for one below other to... (7 Replies)
Discussion started by: mad_man12
7 Replies

4. Shell Programming and Scripting

How to make parallel processing rather than serial processing ??

Hello everybody, I have a little problem with one of my program. I made a plugin for collectd (a stats collector for my servers) but I have a problem to make it run in parallel. My program gathers stats from logs, so it needs to run in background waiting for any new lines added in the log... (0 Replies)
Discussion started by: Samb95
0 Replies

5. Shell Programming and Scripting

script parallel processing

How to write script which run multiple scripts parllely, i have script called A,which has to execute B,C,D,E scripts parllely.. (2 Replies)
Discussion started by: machpee
2 Replies

6. Shell Programming and Scripting

PARALLEL PROCESSING IN PERL

HI All, I have scenerio where I need to call sub modules through for loop for (i=0; i<30 ;i++) { .. .. .. subroutine 1; subroutine 2; } I want this to be run in parallel process1 { ... ... subroutine 1; subroutine 2; (0 Replies)
Discussion started by: gvk25
0 Replies

7. Shell Programming and Scripting

Parallel processing in bash?

Hi Say I am interested in processing a big data set over shell, and each process individually takes a long time, but many such processes can be pipe-lined, is there a way to do this automatically or efficiently in shell? For example consider pinging a list addresses upto 5 times each. Of... (5 Replies)
Discussion started by: jamie_123
5 Replies

8. Shell Programming and Scripting

Implement parallel processing

Unix OS : Linux 2.6x Shell type : Korn Hi all , This is a requirement to incorporate parallel processing of a Unix code . I have two pieces of unix code , one of which will act as a parent process . This script will invoke multiple ( say four ) instances of the second script at one go... (13 Replies)
Discussion started by: kumarjt
13 Replies

9. Shell Programming and Scripting

Parallel processing - continued

Hi, I am taking up the cue from where I was left in my earlier post ( link given below ) https://www.unix.com/shell-programming-scripting/231107-implement-parallel-processing.html I actually wanted to know the significance of using the Unix "wait" , which returns the control from background to... (3 Replies)
Discussion started by: kumarjt
3 Replies

10. Shell Programming and Scripting

Parallel processing

I have 10,000 + files, each of which I need to zip using bzip2. Is ti possible to use bash to create 8 parallel streams sending a new file to be processed from the list when one of the others has finished? (1 Reply)
Discussion started by: garethsays
1 Replies
Tcl_DetachPids(3)					      Tcl Library Procedures						 Tcl_DetachPids(3)

__________________________________________________________________________________________________________________________________________________

NAME
Tcl_DetachPids, Tcl_ReapDetachedProcs, Tcl_WaitPid - manage child processes in background SYNOPSIS
#include <tcl.h> Tcl_DetachPids(numPids, pidPtr) Tcl_ReapDetachedProcs() Tcl_Pid Tcl_WaitPid(pid, statusPtr, options) ARGUMENTS
int numPids (in) Number of process ids contained in the array pointed to by pidPtr. int *pidPtr (in) Address of array containing numPids process ids. Tcl_Pid pid (in) The id of the process (pipe) to wait for. int *statusPtr (out) The result of waiting on a process (pipe). Either 0 or ECHILD. int options (in) The options controlling the wait. WNOHANG specifies not to wait when checking the process. _________________________________________________________________ DESCRIPTION
Tcl_DetachPids and Tcl_ReapDetachedProcs provide a mechanism for managing subprocesses that are running in background. These procedures are needed because the parent of a process must eventually invoke the waitpid kernel call (or one of a few other similar kernel calls) to wait for the child to exit. Until the parent waits for the child, the child's state cannot be completely reclaimed by the system. If a parent continually creates children and doesn't wait on them, the system's process table will eventually overflow, even if all the children have exited. Tcl_DetachPids may be called to ask Tcl to take responsibility for one or more processes whose process ids are contained in the pidPtr array passed as argument. The caller presumably has started these processes running in background and does not want to have to deal with them again. Tcl_ReapDetachedProcs invokes the waitpid kernel call on each of the background processes so that its state can be cleaned up if it has exited. If the process has not exited yet, Tcl_ReapDetachedProcs does not wait for it to exit; it will check again the next time it is invoked. Tcl automatically calls Tcl_ReapDetachedProcs each time the exec command is executed, so in most cases it is not necessary for any code outside of Tcl to invoke Tcl_ReapDetachedProcs. However, if you call Tcl_DetachPids in situations where the exec command may never get executed, you may wish to call Tcl_ReapDetachedProcs from time to time so that background processes can be cleaned up. Tcl_WaitPid is a thin wrapper around the facilities provided by the operating system to wait on the end of a spawned process and to check a whether spawned process is still running. It is used by Tcl_ReapDetachedProcs and the channel system to portably access the operating sys- tem. KEYWORDS
background, child, detach, process, wait Tcl Tcl_DetachPids(3)
All times are GMT -4. The time now is 03:12 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy