08-05-2008
Need Help With Parallel Processing
Hi
I am looking for some kind of feature in unix that will help me write a script that can invoke multiple processes in parallel. And make sure that the multiple parallel processes complete successfully before I proceed to the next step.
Someone suggested something called timespid or timepid or something that sounds like that, and recommended that it might help me accomplish what Im looking for.
Can someone help me with this?
Thanks!
A Beginner
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi All,
I am working on solaris 8 sparc machine with 2 cpu.
I am trying to run my application which generates files. I run multiple instance of the application, but the results don't seem to show as if it were runing parallely.
When i run the application once it takes 12 secs to generate a... (1 Reply)
Discussion started by: zing
1 Replies
2. Shell Programming and Scripting
Hi
I want to run two shell script files parallely. These two scripts are interacting with the database. can any body help on this Pls
Regards
Audippa naidu.M (3 Replies)
Discussion started by: audippa
3 Replies
3. Shell Programming and Scripting
hi i am preparing a set of batches for a set of files sequentially
There is a folder /xyz where all the files reside
now all the files starting with
01 - will be appended for one below other to form a batch batch01
then all the files starting with
02 - will be appended for one below other to... (7 Replies)
Discussion started by: mad_man12
7 Replies
4. Shell Programming and Scripting
Hello everybody,
I have a little problem with one of my program. I made a plugin for collectd (a stats collector for my servers) but I have a problem to make it run in parallel.
My program gathers stats from logs, so it needs to run in background waiting for any new lines added in the log... (0 Replies)
Discussion started by: Samb95
0 Replies
5. Shell Programming and Scripting
How to write script which run multiple scripts parllely,
i have script called A,which has to execute B,C,D,E scripts parllely.. (2 Replies)
Discussion started by: machpee
2 Replies
6. Shell Programming and Scripting
HI All,
I have scenerio where I need to call sub modules through for loop
for (i=0; i<30 ;i++)
{
..
..
..
subroutine 1;
subroutine 2;
}
I want this to be run in parallel
process1
{
...
...
subroutine 1;
subroutine 2; (0 Replies)
Discussion started by: gvk25
0 Replies
7. Shell Programming and Scripting
Hi
Say I am interested in processing a big data set over shell, and each process individually takes a long time, but many such processes can be pipe-lined, is there a way to do this automatically or efficiently in shell?
For example consider pinging a list addresses upto 5 times each. Of... (5 Replies)
Discussion started by: jamie_123
5 Replies
8. Shell Programming and Scripting
Unix OS : Linux 2.6x
Shell type : Korn
Hi all ,
This is a requirement to incorporate parallel processing of a Unix code .
I have two pieces of unix code , one of which will act as a parent process .
This script will invoke multiple ( say four ) instances of the second script at one go... (13 Replies)
Discussion started by: kumarjt
13 Replies
9. Shell Programming and Scripting
Hi,
I am taking up the cue from where I was left in my earlier post ( link given below )
https://www.unix.com/shell-programming-scripting/231107-implement-parallel-processing.html
I actually wanted to know the significance of using the Unix "wait" , which returns
the control from background to... (3 Replies)
Discussion started by: kumarjt
3 Replies
10. Shell Programming and Scripting
I have 10,000 + files, each of which I need to zip using bzip2.
Is ti possible to use bash to create 8 parallel streams sending a new file to be processed from the list when one of the others has finished? (1 Reply)
Discussion started by: garethsays
1 Replies
LEARN ABOUT CENTOS
sge_shepherd
SGE_SHEPHERD(8) Sun Grid Engine Administrative Commands SGE_SHEPHERD(8)
NAME
sge_shepherd - Sun Grid Engine single job controlling agent
SYNOPSIS
sge_shepherd
DESCRIPTION
sge_shepherd provides the parent process functionality for a single Sun Grid Engine job. The parent functionality is necessary on UNIX
systems to retrieve resource usage information (see getrusage(2)) after a job has finished. In addition, the sge_shepherd forwards signals
to the job, such as the signals for suspension, enabling, termination and the Sun Grid Engine checkpointing signal (see sge_ckpt(1) for
details).
The sge_shepherd receives information about the job to be started from the sge_execd(8). During the execution of the job it actually
starts up to 5 child processes. First a prolog script is run if this feature is enabled by the prolog parameter in the cluster configura-
tion. (See sge_conf(5).) Next a parallel environment startup procedure is run if the job is a parallel job. (See sge_pe(5) for more infor-
mation.) After that, the job itself is run, followed by a parallel environment shutdown procedure for parallel jobs, and finally an epilog
script if requested by the epilog parameter in the cluster configuration. The prolog and epilog scripts as well as the parallel environment
startup and shutdown procedures are to be provided by the Sun Grid Engine administrator and are intended for site-specific actions to be
taken before and after execution of the actual user job.
After the job has finished and the epilog script is processed, sge_shepherd retrieves resource usage statistics about the job, places them
in a job specific subdirectory of the sge_execd(8) spool directory for reporting through sge_execd(8) and finishes.
sge_shepherd also places an exit status file in the spool directory. This exit status can be viewed with qacct -j JobId (see qacct(1)); it
is not the exit status of sge_shepherd itself but of one of the methods executed by sge_shepherd. This exit status can have several mean-
ings, depending on in which method an error occurred (if any). The possible methods are: prolog, parallel start, job, parallel stop, epi-
log, suspend, restart, terminate, clean, migrate, and checkpoint.
The following exit values are returned:
0 All methods: Operation was executed successfully.
99 Job script, prolog and epilog: When FORBID_RESCHEDULE is not set in the configuration (see sge_conf(5)), the job gets re-queued.
Otherwise see "Other".
100 Job script, prolog and epilog: When FORBID_APPERROR is not set in the configuration (see sge_conf(5)), the job gets re-queued. Oth-
erwise see "Other".
Other Job script: This is the exit status of the job itself. No action is taken upon this exit status because the meaning of this exit
status is not known.
Prolog, epilog and parallel start: The queue is set to error state and the job is re-queued.
Parallel stop: The queue is set to error state, but the job is not re-queued. It is assumed that the job itself ran successfully and
only the clean up script failed.
Suspend, restart, terminate, clean, and migrate: Always successful.
Checkpoint: Success, except for kernel checkpointing: checkpoint was not successful, did not happen (but migration will happen by
Sun Grid Engine).
RESTRICTIONS
sge_shepherd should not be invoked manually, but only by sge_execd(8).
FILES
sgepasswd contains a list of user names and their corresponding encrypted passwords. If available, the password file will be
used by sge_shepherd. To change the contents of this file please use the sgepasswd command. It is not advised to change that file manu-
ally.
<execd_spool>/job_dir/<job_id> job specific directory
SEE ALSO
sge_intro(1), sge_conf(5), sge_execd(8).
COPYRIGHT
See sge_intro(1) for a full statement of rights and permissions.
SGE 6.2u5 $Date$ SGE_SHEPHERD(8)