07-18-2013
Implement parallel processing
Unix OS : Linux 2.6x
Shell type : Korn
Hi all ,
This is a requirement to incorporate parallel processing of a Unix code .
I have two pieces of unix code , one of which will act as a parent process .
This script will invoke multiple ( say four ) instances of the second script at one go ,and each of these instances will run parallely , exclusive of the status of the other process.
How will I implement this logic ? What aspects must I consider while designing this kind of relationship ?
Please guide me on this .
Thanks
Kumarjit
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi All,
I am working on solaris 8 sparc machine with 2 cpu.
I am trying to run my application which generates files. I run multiple instance of the application, but the results don't seem to show as if it were runing parallely.
When i run the application once it takes 12 secs to generate a... (1 Reply)
Discussion started by: zing
1 Replies
2. Shell Programming and Scripting
Hi
I want to run two shell script files parallely. These two scripts are interacting with the database. can any body help on this Pls
Regards
Audippa naidu.M (3 Replies)
Discussion started by: audippa
3 Replies
3. Shell Programming and Scripting
Hi
I am looking for some kind of feature in unix that will help me write a script that can invoke multiple processes in parallel. And make sure that the multiple parallel processes complete successfully before I proceed to the next step.
Someone suggested something called timespid or... (6 Replies)
Discussion started by: imnewtothis23
6 Replies
4. Shell Programming and Scripting
hi i am preparing a set of batches for a set of files sequentially
There is a folder /xyz where all the files reside
now all the files starting with
01 - will be appended for one below other to form a batch batch01
then all the files starting with
02 - will be appended for one below other to... (7 Replies)
Discussion started by: mad_man12
7 Replies
5. Shell Programming and Scripting
Hello everybody,
I have a little problem with one of my program. I made a plugin for collectd (a stats collector for my servers) but I have a problem to make it run in parallel.
My program gathers stats from logs, so it needs to run in background waiting for any new lines added in the log... (0 Replies)
Discussion started by: Samb95
0 Replies
6. Shell Programming and Scripting
HI All,
I have scenerio where I need to call sub modules through for loop
for (i=0; i<30 ;i++)
{
..
..
..
subroutine 1;
subroutine 2;
}
I want this to be run in parallel
process1
{
...
...
subroutine 1;
subroutine 2; (0 Replies)
Discussion started by: gvk25
0 Replies
7. Shell Programming and Scripting
Hi
Say I am interested in processing a big data set over shell, and each process individually takes a long time, but many such processes can be pipe-lined, is there a way to do this automatically or efficiently in shell?
For example consider pinging a list addresses upto 5 times each. Of... (5 Replies)
Discussion started by: jamie_123
5 Replies
8. Programming
Need urgent help on thread: the goal here is the separtemask will take each image and separate different contours and for each contour in the image it will call handleobject thread. So every for loop will call the handeobject thread. However, object index variable needs to be passed in each... (0 Replies)
Discussion started by: temursalin
0 Replies
9. Shell Programming and Scripting
Hi,
I am taking up the cue from where I was left in my earlier post ( link given below )
https://www.unix.com/shell-programming-scripting/231107-implement-parallel-processing.html
I actually wanted to know the significance of using the Unix "wait" , which returns
the control from background to... (3 Replies)
Discussion started by: kumarjt
3 Replies
10. Shell Programming and Scripting
I have 10,000 + files, each of which I need to zip using bzip2.
Is ti possible to use bash to create 8 parallel streams sending a new file to be processed from the list when one of the others has finished? (1 Reply)
Discussion started by: garethsays
1 Replies
LEARN ABOUT DEBIAN
mdm-run
MDM-RUN(1) User Commands MDM-RUN(1)
NAME
mdm-run - run a command in parallel with mdm-master
SYNOPSIS
mdm-run command
mdm-sync command
DESCRIPTION
mdm-run annotates command as a candidate for parallel execution. If an mdm-master process is present (i.e., the environment variable
MDM_CMD_SOCK is defined), mdm-run submits command to the mdm-master, and it exits when the master issues the command to an mdm-slave
process (which may be connected to another terminal) for execution. mdm-sync runs command in place, but it coordinates with mdm-master to
ensure that executing command does not interfere with commands that are currently executing in parallel.
If there is no mdm-master process present (i.e., the environment variable MDM_CMD_SOCK is not defined), both mdm-run and mdm-sync simply
run command as if you entered command all by itself.
These programs are a part of the Middleman System (mdm).
OPTIONS
Neither mdm-run nor mdm-sync accepts any options.
EXIT STATUS
If there is no mdm-master process present, the exit status of mdm-run is the exit status of command. If an mdm-master process is present,
the exit status is 0 if and only if command is successfully issued to an mdm-slave process for execution.
The exit status of mdm-sync is always the exit status of command.
ENVIRONMENT
The mdm system uses the following environment variable for internal communication, and thus you should not try to set them yourself.
MDM_CMD_SOCK
mdm-run uses this environment variable to decide whether a mdm-master process is present.
EXAMPLE
See mdm.screen(1) for an example use of mdm-run.
SEE ALSO
mdm.screen(1)
Linux 2009-03-06 MDM-RUN(1)