Sponsored Content
Top Forums Shell Programming and Scripting Need Help With Parallel Processing Post 302222023 by bakunin on Tuesday 5th of August 2008 10:51:43 PM
Old 08-05-2008
If you just want to start processes you could use the background process mechanism:

command &

will start the process "command" but will continue processing of the program issuing it after it is started. You can find out the background jobs PID with the "jobs" internal shell command and look in a loop if all started background processes have ended or not.

The following is a sketch - note, that the output of "jobs" is a little different from one system to another. This was written on AIX 5.3, maybe you need to change the regexps a bit to make them fit.

Code:
#! /bin/ksh
typeset -i iPID=0

some_command &
iPID=$(jobs -l | sed -n 's/^\[[0-9]*\]  *+  *\([^ ]*\) .*/\1/p')

while [ $(ps -fe | grep -c $iPID) -gt 1 ] ; do
     # wait for process to end - process is still running
     sleep 1
done

print - "process has finished"

exit 0

If you need to exchange messages with the subprocesses you will have to use the Korn Shells Coprocess facility: a coprocess is a background process just like above, but one that can communicate with the main process through two file descriptors: &4 and &5. There are special commands ("read -p" and "print -p") to communicate between the processes. You might want to read some documentation first, because building such a coprocess is a bit complicated at the beginning.

I hope this helps.

bakunin
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to do parallel processing??

Hi All, I am working on solaris 8 sparc machine with 2 cpu. I am trying to run my application which generates files. I run multiple instance of the application, but the results don't seem to show as if it were runing parallely. When i run the application once it takes 12 secs to generate a... (1 Reply)
Discussion started by: zing
1 Replies

2. Shell Programming and Scripting

parallel processing

Hi I want to run two shell script files parallely. These two scripts are interacting with the database. can any body help on this Pls Regards Audippa naidu.M (3 Replies)
Discussion started by: audippa
3 Replies

3. Shell Programming and Scripting

parallel processing

hi i am preparing a set of batches for a set of files sequentially There is a folder /xyz where all the files reside now all the files starting with 01 - will be appended for one below other to form a batch batch01 then all the files starting with 02 - will be appended for one below other to... (7 Replies)
Discussion started by: mad_man12
7 Replies

4. Shell Programming and Scripting

How to make parallel processing rather than serial processing ??

Hello everybody, I have a little problem with one of my program. I made a plugin for collectd (a stats collector for my servers) but I have a problem to make it run in parallel. My program gathers stats from logs, so it needs to run in background waiting for any new lines added in the log... (0 Replies)
Discussion started by: Samb95
0 Replies

5. Shell Programming and Scripting

script parallel processing

How to write script which run multiple scripts parllely, i have script called A,which has to execute B,C,D,E scripts parllely.. (2 Replies)
Discussion started by: machpee
2 Replies

6. Shell Programming and Scripting

PARALLEL PROCESSING IN PERL

HI All, I have scenerio where I need to call sub modules through for loop for (i=0; i<30 ;i++) { .. .. .. subroutine 1; subroutine 2; } I want this to be run in parallel process1 { ... ... subroutine 1; subroutine 2; (0 Replies)
Discussion started by: gvk25
0 Replies

7. Shell Programming and Scripting

Parallel processing in bash?

Hi Say I am interested in processing a big data set over shell, and each process individually takes a long time, but many such processes can be pipe-lined, is there a way to do this automatically or efficiently in shell? For example consider pinging a list addresses upto 5 times each. Of... (5 Replies)
Discussion started by: jamie_123
5 Replies

8. Shell Programming and Scripting

Implement parallel processing

Unix OS : Linux 2.6x Shell type : Korn Hi all , This is a requirement to incorporate parallel processing of a Unix code . I have two pieces of unix code , one of which will act as a parent process . This script will invoke multiple ( say four ) instances of the second script at one go... (13 Replies)
Discussion started by: kumarjt
13 Replies

9. Shell Programming and Scripting

Parallel processing - continued

Hi, I am taking up the cue from where I was left in my earlier post ( link given below ) https://www.unix.com/shell-programming-scripting/231107-implement-parallel-processing.html I actually wanted to know the significance of using the Unix "wait" , which returns the control from background to... (3 Replies)
Discussion started by: kumarjt
3 Replies

10. Shell Programming and Scripting

Parallel processing

I have 10,000 + files, each of which I need to zip using bzip2. Is ti possible to use bash to create 8 parallel streams sending a new file to be processed from the list when one of the others has finished? (1 Reply)
Discussion started by: garethsays
1 Replies
wait(1)                                                            User Commands                                                           wait(1)

NAME
wait - await process completion SYNOPSIS
/bin/sh wait [pid...] /bin/jsh /bin/ksh /usr/xpg4/bin/sh wait [pid...] wait [ % jobid...] /bin/csh wait DESCRIPTION
The shell itself executes wait, without creating a new process. If you get the error message cannot fork,too many processes, try using the wait command to clean up your background processes. If this doesn't help, the system process table is probably full or you have too many active foreground processes. There is a limit to the number of process IDs associated with your login, and to the number the system can keep track of. Not all the processes of a pipeline with three or more stages are children of the shell, and thus cannot be waited for. /bin/sh, /bin/jsh Wait for your background process whose process ID is pid and report its termination status. If pid is omitted, all your shell's currently active background processes are waited for and the return code will be 0. The wait utility accepts a job identifier, when Job Control is enabled (jsh), and the argument, jobid, is preceded by a percent sign (%). If pid is not an active process ID, the wait utility will return immediately and the return code will be 0. csh Wait for your background processes. ksh When an asynchronous list is started by the shell, the process ID of the last command in each element of the asynchronous list becomes known in the current shell execution environment. If the wait utility is invoked with no operands, it will wait until all process IDs known to the invoking shell have terminated and exit with an exit status of 0. If one or more pid or jobid operands are specified that represent known process IDs (or jobids), the wait utility will wait until all of them have terminated. If one or more pid or jobid operands are specified that represent unknown process IDs (or jobids), wait will treat them as if they were known process IDs (or jobids) that exited with exit status 127. The exit status returned by the wait utility will be the exit status of the process requested by the last pid or jobid operand. The known process IDs are applicable only for invocations of wait in the current shell execution environment. OPERANDS
The following operands are supported: One of the following: pid The unsigned decimal integer process ID of a command, for which the utility is to wait for the termination. jobid A job control job ID that identifies a background process group to be waited for. The job control job ID notation is applicable only for invocations of wait in the current shell execution environment, and only on systems supporting the job control option. USAGE
On most implementations, wait is a shell built-in. If it is called in a subshell or separate utility execution environment, such as one of the following, (wait) nohup wait ... find . -exec wait ... ; it will return immediately because there will be no known process IDs to wait for in those environments. EXAMPLES
Example 1: Using A Script To Identify The Termination Signal Although the exact value used when a process is terminated by a signal is unspecified, if it is known that a signal terminated a process, a script can still reliably figure out which signal is using kill, as shown by the following (/bin/ksh and /usr/xpg4/bin/sh): sleep 1000& pid=$! kill -kill $pid wait $pid echo $pid was terminated by a SIG$(kill -l $(($?-128))) signal. Example 2: Returning The Exit Status Of A Process If the following sequence of commands is run in less than 31 seconds (/bin/ksh and /usr/xpg4/bin/sh): sleep 257 | sleep 31 & jobs -l %% then either of the following commands will return the exit status of the second sleep in the pipeline: wait <pid of sleep 31> wait %% ENVIRONMENT VARIABLES
See environ(5) for descriptions of the following environment variables that affect the execution of wait: LANG, LC_ALL, LC_CTYPE, LC_MES- SAGES, and NLSPATH. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ |Interface Stability |Standard | +-----------------------------+-----------------------------+ SEE ALSO
csh(1), jobs(1), ksh(1), sh(1), attributes(5), environ(5), standards(5) SunOS 5.10 12 Dec 1997 wait(1)
All times are GMT -4. The time now is 07:38 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy