Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Parallel Execution on Multiple System
Special Forums UNIX and Linux Applications High Performance Computing Parallel Execution on Multiple System Post 302284672 by 123an on Friday 6th of February 2009 04:53:55 AM
Old 02-06-2009
Parallel Execution on Multiple System
Hi All,

I am working on a project where I need to execute set of arguments (around 500) on a Simulator. If I execute this on one linux (RedHat 8.0) machine it will approximately takes 2~3 days. Since I am having more linux machines am thinking of executing these on different machines in parallel(i.e. Paralle Execution).

I tried for sanity basis following option which is already posted in this forum;
i.e. Script1.sh & Script2.sh & Script3.sh . But I didn't find much savings in this option.

I also tried downloading a tool-JOBQUEUE from site "Download jobqueue 0.04 for Linux - jobqueue is a program for executing jobs in parallel to complete all jobs as fast as possible. - Softpedia".
This is able to split execution on multiple system, but it won't take all arguments list set (like if arguments set list has 10, it will take 8/9 skips 1 r 2).

Could you please help me in solving the problem.

Thanks in advance

Regards,
123an
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parallel Execution

Hello, I wish to run parallel process forked from one script. Currently I doing is submit them in background. For example: --------------------------------------------------------------- #!/usr/bin/ksh process1 & process2 & process3 & ..... ..... #here I check for completion of... (4 Replies)
Discussion started by: RishiPahuja
4 Replies

2. AIX

Make : parallel execution

Hi, Is there any way to run parallel jobs using make command? I am using non-GNU make utility on AIX 4.3. I want to run 2 jobs symultaneously using make utility. Thanks. Suman (0 Replies)
Discussion started by: suman_jakkula
0 Replies

3. Shell Programming and Scripting

Parallel Job Execution

Hi All, At present I am using a UNIX Script which is running a set of JOBS. These JOBS are to be repeated for 20 times, means the same set of JOBS are repeated by just passing different arguments (From 1 to 20). Is there any way by which I can execute them parallel? At present its all... (4 Replies)
Discussion started by: Prashantckc
4 Replies

4. Shell Programming and Scripting

Conditional execution and parallel jobs

how can i process jobs parallel with conditions below. Script1.ksh Script2.ksh Script3.ksh Script4.ksh Script5.ksh Script6.ksh Script7.ksh Script8.ksh Script9.ksh Script10.ksh After successful completion of Script1.ksh I need to run Script7.ksh. After successful... (4 Replies)
Discussion started by: ford2020
4 Replies

5. Shell Programming and Scripting

Parallel execution of script not syncronize

I am haveing 2 scripts, 1st script calls 2ed script for each parameter.(parameters are kept in a different txt file) 1st script for x in `cat Export_Tables_List.txt` do sh Exp_Table.sh $x & done echo -e "1) following tables are successfully exported : \n" > temp cat... (1 Reply)
Discussion started by: sbmk_design
1 Replies

6. Programming

Parallel Execution of Programs

Since there've been a few requests for a method to execute commands on multiple CPUs (logical or physical), with various levels of shell-, make-, or Perl-based solutions, ranging from well-done to well-meant, and mostly specific to a certain problem, I've started to write a C-based solution... (4 Replies)
Discussion started by: pludi
4 Replies

7. UNIX for Advanced & Expert Users

Parallel Execution of Command

Hi All, We have a table that has to store around 80-100 million records. The table is partitioned by a column called Market Code. There are 30 partitions each corresponding to one MRKT_CD. The source of this table is a join between 3-4 other tables. We are loading this table through SQLPLUS... (2 Replies)
Discussion started by: jerome_rajan
2 Replies

8. Shell Programming and Scripting

How to make parallel execution on folder ?

I have few very huge files ~ 2 Billion rows of 130 column(CDR data) in a folder, I have written shell script need to read on each file in a folder and will create a new files based on some logic. But problem is it's taking time to create a new file due to the size , So i dont want to corrupt... (6 Replies)
Discussion started by: rspwilliam
6 Replies

9. Homework & Coursework Questions

Parallel execution on multiple servers in UNIX

I have a requirement (in a shell script) to connect to several servers at once and execute a series of commands. I am aware that ssh can be used for sequential execution. But since most of the commands that I need to execute take a long time, I have to go for the parallel option. Is there... (2 Replies)
Discussion started by: sneha1887
2 Replies

10. UNIX for Beginners Questions & Answers

Parallel execution of Oracle procedure in UNIX

i have say x number of procedure to run, ie i have one procedure which accepts variable and i need that to run in parallel and capture the error code if in case if it fails through the unix. sqlplus <EOF> exec test_t (abc,124); </EOF> sqlplus <EOF> exec test_t (abc,125); </EOF> sqlplus <EOF>... (2 Replies)
Discussion started by: ATWC
2 Replies

LEARN ABOUT X11R4

setsid

SETSID(1)                                                          User Commands                                                         SETSID(1)

NAME

       setsid - run a program in a new session

SYNOPSIS

       setsid [options] program [arguments]

DESCRIPTION

       setsid  runs  a  program in a new session. The command calls fork(2) if already a process group leader. Otherwise, it executes a program in
       the current process.

OPTIONS

       -c, --ctty
              Set the controlling terminal to the current one.

       -w, --wait
              Wait for the execution of the program to end, and return the exit value of this program as the return value of setsid.

       -V, --version
              Display version information and exit.

       -h, --help
              Display help text and exit.

SEE ALSO

       setsid(2)

AUTHOR

       Rick Sladkey <jrs@world.std.com>

AVAILABILITY

       The setsid command is part of the util-linux package and is available from https://www.kernel.org/pub/linux/utils/util-linux/.

util-linux                                                           July 2014                                                           SETSID(1)
All times are GMT -4. The time now is 07:47 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy