I have a script (ksh) which tries to run a function in parallel for performance gains. I am also trying to limit the number of parallel child processes to avoid overloading the system by using a variable to count triggered processes and waiting for completion e.g.
The problem with this approach is that it is only as good as the worst performer in a batch, i.e. the wait command will wait for all child processed in a batch to finish before triggering the next batch. There is a more flexible way of doing this by using xargs --max-procs command, but unfortunately that seems to be built for shell commands only.
Is there a way I can use xargs to trigger user-defined functions in parallel?
Last edited by vbe; 05-17-2015 at 02:17 PM..
Reason: code tags not icode
Hi All,
I am working on solaris 8 sparc machine with 2 cpu.
I am trying to run my application which generates files. I run multiple instance of the application, but the results don't seem to show as if it were runing parallely.
When i run the application once it takes 12 secs to generate a... (1 Reply)
Hi
I want to run two shell script files parallely. These two scripts are interacting with the database. can any body help on this Pls
Regards
Audippa naidu.M (3 Replies)
Hi
I am looking for some kind of feature in unix that will help me write a script that can invoke multiple processes in parallel. And make sure that the multiple parallel processes complete successfully before I proceed to the next step.
Someone suggested something called timespid or... (6 Replies)
hi i am preparing a set of batches for a set of files sequentially
There is a folder /xyz where all the files reside
now all the files starting with
01 - will be appended for one below other to form a batch batch01
then all the files starting with
02 - will be appended for one below other to... (7 Replies)
Hi Experts,
I am trying to parse some syslog outputs into a separate file per node using the below syntax but am having issues when it comes to my Xargs statements.
The command which I was intending on using was:
cat syslogs | nawk '/From/ { print $3 }' | uniq | xargs -I {} grep {}... (5 Replies)
Hello everybody,
I have a little problem with one of my program. I made a plugin for collectd (a stats collector for my servers) but I have a problem to make it run in parallel.
My program gathers stats from logs, so it needs to run in background waiting for any new lines added in the log... (0 Replies)
Hey, I just wanted to know how many algorithms there are that cannot be accelerated by parallel processing. I know one such algorithm is Euclid's Algorithm (for GCF). does anyone know any other algorithms that cannot be accelerated by pp? if so please list the names and a general sentence of what... (2 Replies)
Hello,
I have multiple functions within a shell script. eg. function_database_backup, unix_tar_creation, etc.
I would like to run these functions in parallel, as each is independent of the other.
If these were simple commands, I could have probably run each of the commands in background.
... (1 Reply)
Hi ,
I have a script which is using a text file as I/P.
I want a code where it reads n lines from this file and pass the parameters to a function and now this script should run in such a way where a function can be called in parallel with different parameters. Please find below my script, it... (1 Reply)
I have 10,000 + files, each of which I need to zip using bzip2.
Is ti possible to use bash to create 8 parallel streams sending a new file to be processed from the list when one of the others has finished? (1 Reply)
Discussion started by: garethsays
1 Replies
LEARN ABOUT OPENSOLARIS
queuedefs
queuedefs(4) File Formats queuedefs(4)NAME
queuedefs - queue description file for at, batch, and cron
SYNOPSIS
/etc/cron.d/queuedefs
DESCRIPTION
The queuedefs file describes the characteristics of the queues managed by cron(1M). Each non-comment line in this file describes one queue.
The format of the lines are as follows:
q.[njobj][nicen][nwaitw]
The fields in this line are:
q The name of the queue. a is the default queue for jobs started by at(1); b is the default queue for jobs started by batch (see
at(1)); c is the default queue for jobs run from a crontab(1) file.
njob The maximum number of jobs that can be run simultaneously in that queue; if more than njob jobs are ready to run, only the first
njob jobs will be run, and the others will be run as jobs that are currently running terminate. The default value is 100.
nice The nice(1) value to give to all jobs in that queue that are not run with a user ID of super-user. The default value is 2.
nwait The number of seconds to wait before rescheduling a job that was deferred because more than njob jobs were running in that job's
queue, or because the system-wide limit of jobs executing has been reached. The default value is 60.
Lines beginning with # are comments, and are ignored.
EXAMPLES
Example 1 A sample file.
#
#
a.4j1n
b.2j2n90w
This file specifies that the a queue, for at jobs, can have up to 4 jobs running simultaneously; those jobs will be run with a nice value
of 1. As no nwait value was given, if a job cannot be run because too many other jobs are running cron will wait 60 seconds before trying
again to run it.
The b queue, for batch(1) jobs, can have up to 2 jobs running simultaneously; those jobs will be run with a nice(1) value of 2. If a job
cannot be run because too many other jobs are running, cron(1M) will wait 90 seconds before trying again to run it. All other queues can
have up to 100 jobs running simultaneously; they will be run with a nice value of 2, and if a job cannot be run because too many other jobs
are running cron will wait 60 seconds before trying again to run it.
FILES
/etc/cron.d/queuedefs queue description file for at, batch, and cron.
SEE ALSO at(1), crontab(1), nice(1), cron(1M)SunOS 5.11 1 Mar 1994 queuedefs(4)