Sponsored Content
Full Discussion: Parallel Job Execution
Top Forums Shell Programming and Scripting Parallel Job Execution Post 302261969 by Prashantckc on Wednesday 26th of November 2008 05:38:37 AM
Old 11-26-2008
Many thanks for this help .
I have done these changes and it worked.

Only thing which I have noticed that after I call papallel execution as

script.sh arg1 &
script.sh arg2 &
script.sh arg3 &
script.sh arg4 &

The prompt is not coming in UNIX (KSH Prompt) and I have to explicitly hit an Enter key.

May be since I am executing the parallel processes and hence the issue. Is there any way by which I can return the prompt after all the parallel executions?
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parallel Execution

Hello, I wish to run parallel process forked from one script. Currently I doing is submit them in background. For example: --------------------------------------------------------------- #!/usr/bin/ksh process1 & process2 & process3 & ..... ..... #here I check for completion of... (4 Replies)
Discussion started by: RishiPahuja
4 Replies

2. AIX

Make : parallel execution

Hi, Is there any way to run parallel jobs using make command? I am using non-GNU make utility on AIX 4.3. I want to run 2 jobs symultaneously using make utility. Thanks. Suman (0 Replies)
Discussion started by: suman_jakkula
0 Replies

3. Shell Programming and Scripting

Job Execution in Parallel and Maintain Dependency

All, We need to run multiple jobs in parallel with the dependencies intact. I was trying to use the same with &, but in vain. Is there is any other way to resolve this issue. I need to run Set1 A.sh B.sh C.sh Set2 D.sh E.sh Set3 F.sh (10 Replies)
Discussion started by: anand_msm
10 Replies

4. High Performance Computing

Parallel Execution on Multiple System

Hi All, I am working on a project where I need to execute set of arguments (around 500) on a Simulator. If I execute this on one linux (RedHat 8.0) machine it will approximately takes 2~3 days. Since I am having more linux machines am thinking of executing these on different machines in... (7 Replies)
Discussion started by: 123an
7 Replies

5. Shell Programming and Scripting

Conditional execution and parallel jobs

how can i process jobs parallel with conditions below. Script1.ksh Script2.ksh Script3.ksh Script4.ksh Script5.ksh Script6.ksh Script7.ksh Script8.ksh Script9.ksh Script10.ksh After successful completion of Script1.ksh I need to run Script7.ksh. After successful... (4 Replies)
Discussion started by: ford2020
4 Replies

6. Shell Programming and Scripting

Parallel execution of script not syncronize

I am haveing 2 scripts, 1st script calls 2ed script for each parameter.(parameters are kept in a different txt file) 1st script for x in `cat Export_Tables_List.txt` do sh Exp_Table.sh $x & done echo -e "1) following tables are successfully exported : \n" > temp cat... (1 Reply)
Discussion started by: sbmk_design
1 Replies

7. Programming

Parallel Execution of Programs

Since there've been a few requests for a method to execute commands on multiple CPUs (logical or physical), with various levels of shell-, make-, or Perl-based solutions, ranging from well-done to well-meant, and mostly specific to a certain problem, I've started to write a C-based solution... (4 Replies)
Discussion started by: pludi
4 Replies

8. UNIX for Advanced & Expert Users

Parallel Execution of Command

Hi All, We have a table that has to store around 80-100 million records. The table is partitioned by a column called Market Code. There are 30 partitions each corresponding to one MRKT_CD. The source of this table is a join between 3-4 other tables. We are loading this table through SQLPLUS... (2 Replies)
Discussion started by: jerome_rajan
2 Replies

9. Shell Programming and Scripting

How to make parallel execution on folder ?

I have few very huge files ~ 2 Billion rows of 130 column(CDR data) in a folder, I have written shell script need to read on each file in a folder and will create a new files based on some logic. But problem is it's taking time to create a new file due to the size , So i dont want to corrupt... (6 Replies)
Discussion started by: rspwilliam
6 Replies

10. Shell Programming and Scripting

Parallel execution of Oracle PROC in UNIX

Hi, Trying to run the Oracle Proc in parallel, but not working.. below code runs each call of the procedure and its coming out of the loop before its completion and printing the last statement! not waiting for the sqlplus to complete. for name in `cat abc.txt` do (sqlplus -s ... (3 Replies)
Discussion started by: ATWC
3 Replies
TheSchwartz(3pm)					User Contributed Perl Documentation					  TheSchwartz(3pm)

NAME
TheSchwartz - reliable job queue SYNOPSIS
# MyApp.pm package MyApp; sub work_asynchronously { my %args = @_; my $client = TheSchwartz->new( databases => $DATABASE_INFO ); $client->insert('MyWorker', \%args); } # myworker.pl package MyWorker; use base qw( TheSchwartz::Worker ); sub work { my $class = shift; my TheSchwartz::Job $job = shift; print "Workin' hard or hardly workin'? Hyuk!! "; $job->completed(); } package main; my $client = TheSchwartz->new( databases => $DATABASE_INFO ); $client->can_do('MyWorker'); $client->work(); DESCRIPTION
TheSchwartz is a reliable job queue system. Your application can put jobs into the system, and your worker processes can pull jobs from the queue atomically to perform. Failed jobs can be left in the queue to retry later. Abilities specify what jobs a worker process can perform. Abilities are the names of "TheSchwartz::Worker" subclasses, as in the synopsis: the "MyWorker" class name is used to specify that the worker script can perform the job. When using the "TheSchwartz" client's "work" functions, the class-ability duality is used to automatically dispatch to the proper class to do the actual work. TheSchwartz clients will also prefer to do jobs for unused abilities before reusing a particular ability, to avoid exhausting the supply of one kind of job while jobs of other types stack up. Some jobs with high setup times can be performed more efficiently if a group of related jobs are performed together. TheSchwartz offers a facility to coalesce jobs into groups, which a properly constructed worker can find and perform at once. For example, if your worker were delivering email, you might store the domain name from the recipient's address as the coalescing value. The worker that grabs that job could then batch deliver all the mail for that domain once it connects to that domain's mail server. USAGE
"TheSchwartz->new( %args )" Optional members of %args are: o "databases" An arrayref of database information. TheSchwartz workers can use multiple databases, such that if any of them are unavailable, the worker will search for appropriate jobs in the other databases automatically. Each member of the "databases" value should be a hashref containing: o "dsn" The database DSN for this database. o "user" The username to use when connecting to this database. o "pass" The password to use when connecting to this database. o "verbose" A value indicating whether to log debug messages. If "verbose" is a coderef, it is called to log debug messages. If "verbose" is not a coderef but is some other true value, debug messages will be sent to "STDERR". Otherwise, debug messages will not be logged. o "prioritize" A value indicating whether to utilize the job 'priority' field when selecting jobs to be processed. If unspecified, jobs will always be executed in a randomized order. o "driver_cache_expiration" Optional value to control how long database connections are cached for in seconds. By default, connections are not cached. To re-use the same database connection for five minutes, pass driver_cache_expiration => 300 to the constructor. Improves job throughput in cases where the work to process a job is small compared to the database connection set-up and tear-down time. o "retry_seconds" The number of seconds after which to try reconnecting to apparently dead databases. If not given, TheSchwartz will retry connecting to databases after 30 seconds. "$client->list_jobs( %args )" Returns a list of "TheSchwartz::Job" objects matching the given arguments. The required members of %args are: o "funcname" the name of the function or a reference to an array of functions o "run_after" the value you want to check <= against on the run_after column o "grabbed_until" the value you want to check <= against on the grabbed_until column o "coalesce_op" defaults to '=', set it to whatever you want to compare the coalesce field too if you want to search, you can use 'LIKE' o "coalesce" coalesce value to search for, if you set op to 'LIKE' you can use '%' here, do remember that '%' searches anchored at the beginning of the string are much faster since it is can do a btree index lookup o "want_handle" if you want all your jobs to be set up using a handle. defaults to true. this option might be removed, as you should always have this on a Job object. It is important to remember that this function doesnt lock anything, it just returns as many jobs as there is up to amount of databases * FIND_JOB_BATCH_SIZE "$client->lookup_job( $handle_id )" Returns a "TheSchwartz::Job" corresponding to the given handle ID. "$client->set_verbose( $verbose )" Sets the current logging function to $verbose if it's a coderef. If not a coderef, enables debug logging to "STDERR" if $verbose is true; otherwise, disables logging. POSTING JOBS
The methods of TheSchwartz clients used by applications posting jobs to the queue are: "$client->insert( $job )" Adds the given "TheSchwartz::Job" to one of the client's job databases. "$client->insert( $funcname, $arg )" Adds a new job with funcname $funcname and arguments $arg to the queue. "$client->insert_jobs( @jobs )" Adds the given "TheSchwartz::Job" objects to one of the client's job databases. All the given jobs are recorded in one job database. "$client->set_prioritize( $prioritize )" Set the "prioritize" value as described in the constructor. WORKING
The methods of TheSchwartz clients for use in worker processes are: "$client->can_do( $ability )" Adds $ability to the list of abilities $client is capable of performing. Subsequent calls to that client's "work" methods will find jobs requiring the given ability. "$client->work_once()" Find and perform one job $client can do. "$client->work_until_done()" Find and perform jobs $client can do until no more such jobs are found in any of the client's job databases. "$client->work( [$delay] )" Find and perform any jobs $client can do, forever. When no job is available, the working process will sleep for $delay seconds (or 5, if not specified) before looking again. "$client->work_on($handle)" Given a job handle (a scalar string) $handle, runs the job, then returns. "$client->find_job_for_workers( [$abilities] )" Returns a "TheSchwartz::Job" for a random job that the client can do. If specified, the job returned matches one of the abilities in the arrayref $abilities, rather than $client's abilities. "$client->find_job_with_coalescing_value( $ability, $coval )" Returns a "TheSchwartz::Job" for a random job for a worker capable of $ability and with a coalescing value of $coval. "$client->find_job_with_coalescing_prefix( $ability, $coval )" Returns a "TheSchwartz::Job" for a random job for a worker capable of $ability and with a coalescing value beginning with $coval. Note the "TheSchwartz" implementation of this function uses a "LIKE" query to find matching jobs, with all the attendant performance implications for your job databases. "$client->get_server_time( $driver )" Given an open driver $driver to a database, gets the current server time from the database. COPYRIGHT, LICENSE &; WARRANTY This software is Copyright 2007, Six Apart Ltd, cpan@sixapart.com. All rights reserved. TheSchwartz is free software; you may redistribute it and/or modify it under the same terms as Perl itself. TheScwhartz comes with no warranty of any kind. perl v5.10.0 2008-08-19 TheSchwartz(3pm)
All times are GMT -4. The time now is 01:32 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy