Sponsored Content
Full Discussion: Parallel Job Execution
Top Forums Shell Programming and Scripting Parallel Job Execution Post 302261247 by Prashantckc on Monday 24th of November 2008 05:51:37 AM
Old 11-24-2008
Parallel Job Execution

Hi All,
At present I am using a UNIX Script which is running a set of JOBS. These JOBS are to be repeated for 20 times, means the same set of JOBS are repeated by just passing different arguments (From 1 to 20).

Is there any way by which I can execute them parallel?

At present its all sequential process of executing the same set of JOBS one at a time.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parallel Execution

Hello, I wish to run parallel process forked from one script. Currently I doing is submit them in background. For example: --------------------------------------------------------------- #!/usr/bin/ksh process1 & process2 & process3 & ..... ..... #here I check for completion of... (4 Replies)
Discussion started by: RishiPahuja
4 Replies

2. AIX

Make : parallel execution

Hi, Is there any way to run parallel jobs using make command? I am using non-GNU make utility on AIX 4.3. I want to run 2 jobs symultaneously using make utility. Thanks. Suman (0 Replies)
Discussion started by: suman_jakkula
0 Replies

3. Shell Programming and Scripting

Job Execution in Parallel and Maintain Dependency

All, We need to run multiple jobs in parallel with the dependencies intact. I was trying to use the same with &, but in vain. Is there is any other way to resolve this issue. I need to run Set1 A.sh B.sh C.sh Set2 D.sh E.sh Set3 F.sh (10 Replies)
Discussion started by: anand_msm
10 Replies

4. High Performance Computing

Parallel Execution on Multiple System

Hi All, I am working on a project where I need to execute set of arguments (around 500) on a Simulator. If I execute this on one linux (RedHat 8.0) machine it will approximately takes 2~3 days. Since I am having more linux machines am thinking of executing these on different machines in... (7 Replies)
Discussion started by: 123an
7 Replies

5. Shell Programming and Scripting

Conditional execution and parallel jobs

how can i process jobs parallel with conditions below. Script1.ksh Script2.ksh Script3.ksh Script4.ksh Script5.ksh Script6.ksh Script7.ksh Script8.ksh Script9.ksh Script10.ksh After successful completion of Script1.ksh I need to run Script7.ksh. After successful... (4 Replies)
Discussion started by: ford2020
4 Replies

6. Shell Programming and Scripting

Parallel execution of script not syncronize

I am haveing 2 scripts, 1st script calls 2ed script for each parameter.(parameters are kept in a different txt file) 1st script for x in `cat Export_Tables_List.txt` do sh Exp_Table.sh $x & done echo -e "1) following tables are successfully exported : \n" > temp cat... (1 Reply)
Discussion started by: sbmk_design
1 Replies

7. Programming

Parallel Execution of Programs

Since there've been a few requests for a method to execute commands on multiple CPUs (logical or physical), with various levels of shell-, make-, or Perl-based solutions, ranging from well-done to well-meant, and mostly specific to a certain problem, I've started to write a C-based solution... (4 Replies)
Discussion started by: pludi
4 Replies

8. UNIX for Advanced & Expert Users

Parallel Execution of Command

Hi All, We have a table that has to store around 80-100 million records. The table is partitioned by a column called Market Code. There are 30 partitions each corresponding to one MRKT_CD. The source of this table is a join between 3-4 other tables. We are loading this table through SQLPLUS... (2 Replies)
Discussion started by: jerome_rajan
2 Replies

9. Shell Programming and Scripting

How to make parallel execution on folder ?

I have few very huge files ~ 2 Billion rows of 130 column(CDR data) in a folder, I have written shell script need to read on each file in a folder and will create a new files based on some logic. But problem is it's taking time to create a new file due to the size , So i dont want to corrupt... (6 Replies)
Discussion started by: rspwilliam
6 Replies

10. Shell Programming and Scripting

Parallel execution of Oracle PROC in UNIX

Hi, Trying to run the Oracle Proc in parallel, but not working.. below code runs each call of the procedure and its coming out of the loop before its completion and printing the last statement! not waiting for the sqlplus to complete. for name in `cat abc.txt` do (sqlplus -s ... (3 Replies)
Discussion started by: ATWC
3 Replies
SGE_PRIORITY(5) 					   Sun Grid Engine File Formats 					   SGE_PRIORITY(5)

NAME
sge_priority - Sun Grid Engine job priorities DESCRIPTION
Sun Grid Engine provide means for controlling job dispatch and run-time priorities. The dispatch priority indicates the importance of pend- ing jobs compared with each other and determines the order in which Sun Grid Engine dispatches jobs to queue instances. The run-time prior- ity determines the CPU allocation that the operating system assigns to jobs. JOBS DISPATCH PRIORITY A job's dispatch priority is affected by a number of factors: o the identity of the submitting user o the project under which the job is submitted (or alternatively, the default project of the submitting user) o any resources requested by the job o the job's submit time o the job's initiation deadline time (if specified) o the -p priority specified for the job (also known as the POSIX priority "pprio") The effect of each of these is governed by the overall policy setup, which is split into three top-level contributions. Each of these is configured through the sched_conf(5) parameters weight_priority, weight_ticket and weight_urgency. These three parameters control to what degree POSIX priority, ticket policy, and urgency policy are in effect. To facilitate interpretation, the raw priorities ("tckts"/"urg"/"ppri") are normalized ("ntckts"/"nurg"/"npprior") before they are used to calculate job priorities ("prio"). Normalization maps each raw urgency/ticket/priority value into a range between 0 and 1. npprior = normalized(ppri) nurg = normalized(urg) ntckts = normalized(tckts) prio = weight_priority * pprio + weight_urgency * nurg + weight_ticket * ntckts The higher a job's priority value, the earlier it gets dispatched. The urgency policy defines an urgency value for each job. The urgency value urg = rrcontr + wtcontr + dlcontr consists of the resource requirement contribution ("rrcontr"), the waiting time contribution ("wtcontr") and the deadline contribution ("dlcontr"). The resource requirement contribution is adding up all resource requirements of a job into a single numeric value. rrcontr = Sum over all(hrr) with an "hrr" for each hard resource request. Depending on the resource type two different methods are used to determine the value to be used for "hrr" here. For numeric type resource requests, the "hrr" represents how much of a resource a job requests (on a per-slot basis for pe jobs) and how "important" this resource is considered in comparison to other resources. This is expressed by the formula: hrr = rurg * assumed_slot_allocation * request where the resource's urgency value ("rurg") is as specified under urgency in complex(5), the job's assumed_slot_allocation represents the number of slots supposedly assigned to the job, and the per-slot request is that which was specified using the -l qsub(1) option. For string type requests the formula is simply hrr = "rurg" and directly assigns the resource urgency value as specified under urgency in complex(5). The waiting time contribution represents a weighted weighting time of the jobs wtcontr = waiting_time * weight_waiting_time with the waiting time in seconds and the weight_waiting_time value as specified in sched_conf(5). The deadline contribution has an increasing effect as jobs approach their deadline initiation time (see the -dl option in qsub(1)). It is defined as the quotient of the weight_deadline value from sched_conf(5) and the (steadily decreasing) free time in seconds until deadline initiation time dlcontr = weight_deadline / free_time or is set to 0 for non-deadline jobs. After the deadline passes, the value is static and equal to weight_deadline. The ticket policy unites functional, override and share tree policies in the ticket value ("tckts"), as is defined as the sum of the spe- cific ticket values ("ftckt"/"otckt"/"stckt") for each sub-policy (functional, override, share): tckts = ftckt + otckt + stckt The ticket policies provide a broad range of means for influencing both job dispatch and runtime priorities on a per job, per user, per project, and per department basis. See the Sun Grid Engine Installation and Administration Guide for details. JOB RUN-TIME PRIORITY The run-time priority can be dynamically adjusted in order to meet the goals set with the ticket policy. Dynamic run-time priority adjust- ment can be turned off (default) globally using reprioritize_interval in sched_conf(5) If no dynamic run-time priority adjustment is done at a host level, the priority specification in queue_conf(5) is in effect. Note that urgency and POSIX priorities do NOT affect runtime priority. SEE ALSO
sge_intro(1), complex(5), qstat(1), qsub(1), sched_conf(5), sge_conf(5) Sun Grid Engine Installation and Administration Guide COPYRIGHT
See sge_intro(1) for a full statement of rights and permissions. SGE 6.2u5 $Date$ SGE_PRIORITY(5)
All times are GMT -4. The time now is 01:02 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy