02-06-2009
Parallel Execution on Multiple System
Hi All,
I am working on a project where I need to execute set of arguments (around 500) on a Simulator. If I execute this on one linux (RedHat 8.0) machine it will approximately takes 2~3 days. Since I am having more linux machines am thinking of executing these on different machines in parallel(i.e. Paralle Execution).
I tried for sanity basis following option which is already posted in this forum;
i.e. Script1.sh & Script2.sh & Script3.sh . But I didn't find much savings in this option.
I also tried downloading a tool-JOBQUEUE from site "
Download jobqueue 0.04 for Linux - jobqueue is a program for executing jobs in parallel to complete all jobs as fast as possible. - Softpedia".
This is able to split execution on multiple system, but it won't take all arguments list set (like if arguments set list has 10, it will take 8/9 skips 1 r 2).
Could you please help me in solving the problem.
Thanks in advance
Regards,
123an
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hello,
I wish to run parallel process forked from one script.
Currently I doing is submit them in background.
For example:
---------------------------------------------------------------
#!/usr/bin/ksh
process1 &
process2 &
process3 &
.....
.....
#here I check for completion of... (4 Replies)
Discussion started by: RishiPahuja
4 Replies
2. AIX
Hi,
Is there any way to run parallel jobs using make command?
I am using non-GNU make utility on AIX 4.3.
I want to run 2 jobs symultaneously using make utility.
Thanks.
Suman (0 Replies)
Discussion started by: suman_jakkula
0 Replies
3. Shell Programming and Scripting
Hi All,
At present I am using a UNIX Script which is running a set of JOBS. These JOBS are to be repeated for 20 times, means the same set of JOBS are repeated by just passing different arguments (From 1 to 20).
Is there any way by which I can execute them parallel?
At present its all... (4 Replies)
Discussion started by: Prashantckc
4 Replies
4. Shell Programming and Scripting
how can i process jobs parallel with conditions below.
Script1.ksh
Script2.ksh
Script3.ksh
Script4.ksh
Script5.ksh
Script6.ksh
Script7.ksh
Script8.ksh
Script9.ksh
Script10.ksh
After successful completion of Script1.ksh I need to run Script7.ksh.
After successful... (4 Replies)
Discussion started by: ford2020
4 Replies
5. Shell Programming and Scripting
I am haveing 2 scripts, 1st script calls 2ed script for each parameter.(parameters are kept in a different txt file)
1st script
for x in `cat Export_Tables_List.txt`
do
sh Exp_Table.sh $x &
done
echo -e "1) following tables are successfully exported : \n" > temp
cat... (1 Reply)
Discussion started by: sbmk_design
1 Replies
6. Programming
Since there've been a few requests for a method to execute commands on multiple CPUs (logical or physical), with various levels of shell-, make-, or Perl-based solutions, ranging from well-done to well-meant, and mostly specific to a certain problem, I've started to write a C-based solution... (4 Replies)
Discussion started by: pludi
4 Replies
7. UNIX for Advanced & Expert Users
Hi All,
We have a table that has to store around 80-100 million records. The table is partitioned by a column called Market Code. There are 30 partitions each corresponding to one MRKT_CD.
The source of this table is a join between 3-4 other tables. We are loading this table through SQLPLUS... (2 Replies)
Discussion started by: jerome_rajan
2 Replies
8. Shell Programming and Scripting
I have few very huge files ~ 2 Billion rows of 130 column(CDR data) in a folder, I have written shell script need to read on each file in a folder and will create a new files based on some logic.
But problem is it's taking time to create a new file due to the size , So i dont want to corrupt... (6 Replies)
Discussion started by: rspwilliam
6 Replies
9. Homework & Coursework Questions
I have a requirement (in a shell script) to connect to several servers at once and execute a series of commands.
I am aware that ssh can be used for sequential execution. But since most of the commands that I need to execute take a long time, I have to go for the parallel option.
Is there... (2 Replies)
Discussion started by: sneha1887
2 Replies
10. UNIX for Beginners Questions & Answers
i have say x number of procedure to run, ie i have one procedure which accepts variable and i need that to run in parallel and capture the error code if in case if it fails through the unix.
sqlplus <EOF> exec test_t (abc,124); </EOF>
sqlplus <EOF> exec test_t (abc,125); </EOF>
sqlplus <EOF>... (2 Replies)
Discussion started by: ATWC
2 Replies
LEARN ABOUT XFREE86
bos_exec
BOS_EXEC(8) AFS Command Reference BOS_EXEC(8)
NAME
bos_exec - Executes a command on a remote server machine
SYNOPSIS
bos exec -server <machine name> -cmd <command to execute>
[-cell <cell name>] [-noauth] [-localauth] [-help]
bos e -s <machine name> -cm <command to execute>
[-ce <cell name>] [-n] [-l] [-h]
DESCRIPTION
The bos exec command executes the indicated command on the file server machine named by the -server argument. Its intended use is to reboot
the machine, using the /sbin/reboot command or equivalent.
OPTIONS
-server <machine name>
Indicates the server machine on which to execute the command. Identify the machine by IP address or its host name (either fully-
qualified or abbreviated unambiguously). For details, see bos(8).
-cmd <command to execute>
Specifies the complete local disk pathname of the command to execute (for example, /sbin/reboot). Surround this argument with double
quotes ("") if the command contains one or more spaces.
-cell <cell name>
Names the cell in which to run the command. Do not combine this argument with the -localauth flag. For more details, see bos(8).
-noauth
Assigns the unprivileged identity "anonymous" to the issuer. Do not combine this flag with the -localauth flag. For more details, see
bos(8).
-localauth
Constructs a server ticket using a key from the local /etc/openafs/server/KeyFile file. The bos command interpreter presents the ticket
to the BOS Server during mutual authentication. Do not combine this flag with the -cell or -noauth options. For more details, see
bos(8).
-help
Prints the online help for this command. All other valid options are ignored.
EXAMPLES
The following command reboots the machine "fs2.abc.com". The issuer has previously issued the bos shutdown command to shutdown all
processes cleanly.
% bos exec -server fs2.abc.com -cmd /sbin/shutdown -r now
PRIVILEGE REQUIRED
The issuer must be listed in the /etc/openafs/server/UserList file on the machine named by the -server argument, or must be logged onto a
server machine as the local superuser "root" if the -localauth flag is included.
The bos exec command is not available on servers running in restricted mode.
SEE ALSO
bos(8)
COPYRIGHT
IBM Corporation 2000. <http://www.ibm.com/> All Rights Reserved.
This documentation is covered by the IBM Public License Version 1.0. It was converted from HTML to POD by software written by Chas
Williams and Russ Allbery, based on work by Alf Wachsmann and Elizabeth Cassell.
OpenAFS 2012-03-26 BOS_EXEC(8)