Sponsored Content
Top Forums Shell Programming and Scripting Parallel processing for functions in xargs Post 302944226 by sea on Sunday 17th of May 2015 03:46:17 PM
Old 05-17-2015
Why would one to repeat oneself if its already been said?
Sometimes people (me included) are too focused on the words (or specific tools, your case) they have in their mind, rather than the situation.

There are not too many different ways ways on how to run X numbers of background jobs, and this year this is at least the 3rd thread of its kind, having the same requierments.

One could ask: did you google?
One could ask: did you search the forum?
IMO that is a very legit question with the aim to solve your own question.

I like to help others, even provide tools (with the aim) to help others.
But i dont like to repeat myself, so i redirect to whats already been said, if i'm aware of it (which applies in this case).

Sorry if that hurts your feelings.
Have a nice evening.

EDIT:
A forum is NOT "set and forget".
These 3 Users Gave Thanks to sea For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to do parallel processing??

Hi All, I am working on solaris 8 sparc machine with 2 cpu. I am trying to run my application which generates files. I run multiple instance of the application, but the results don't seem to show as if it were runing parallely. When i run the application once it takes 12 secs to generate a... (1 Reply)
Discussion started by: zing
1 Replies

2. Shell Programming and Scripting

parallel processing

Hi I want to run two shell script files parallely. These two scripts are interacting with the database. can any body help on this Pls Regards Audippa naidu.M (3 Replies)
Discussion started by: audippa
3 Replies

3. Shell Programming and Scripting

Need Help With Parallel Processing

Hi I am looking for some kind of feature in unix that will help me write a script that can invoke multiple processes in parallel. And make sure that the multiple parallel processes complete successfully before I proceed to the next step. Someone suggested something called timespid or... (6 Replies)
Discussion started by: imnewtothis23
6 Replies

4. Shell Programming and Scripting

parallel processing

hi i am preparing a set of batches for a set of files sequentially There is a folder /xyz where all the files reside now all the files starting with 01 - will be appended for one below other to form a batch batch01 then all the files starting with 02 - will be appended for one below other to... (7 Replies)
Discussion started by: mad_man12
7 Replies

5. Shell Programming and Scripting

Using xargs for multiple functions

Hi Experts, I am trying to parse some syslog outputs into a separate file per node using the below syntax but am having issues when it comes to my Xargs statements. The command which I was intending on using was: cat syslogs | nawk '/From/ { print $3 }' | uniq | xargs -I {} grep {}... (5 Replies)
Discussion started by: krypton
5 Replies

6. Shell Programming and Scripting

How to make parallel processing rather than serial processing ??

Hello everybody, I have a little problem with one of my program. I made a plugin for collectd (a stats collector for my servers) but I have a problem to make it run in parallel. My program gathers stats from logs, so it needs to run in background waiting for any new lines added in the log... (0 Replies)
Discussion started by: Samb95
0 Replies

7. Programming

Algorithms for Parallel Processing

Hey, I just wanted to know how many algorithms there are that cannot be accelerated by parallel processing. I know one such algorithm is Euclid's Algorithm (for GCF). does anyone know any other algorithms that cannot be accelerated by pp? if so please list the names and a general sentence of what... (2 Replies)
Discussion started by: azar.zorn
2 Replies

8. Shell Programming and Scripting

Calling multiple functions in parallel

Hello, I have multiple functions within a shell script. eg. function_database_backup, unix_tar_creation, etc. I would like to run these functions in parallel, as each is independent of the other. If these were simple commands, I could have probably run each of the commands in background. ... (1 Reply)
Discussion started by: neil.k
1 Replies

9. Shell Programming and Scripting

Pass parameters to a function and running functions in parallel

Hi , I have a script which is using a text file as I/P. I want a code where it reads n lines from this file and pass the parameters to a function and now this script should run in such a way where a function can be called in parallel with different parameters. Please find below my script, it... (1 Reply)
Discussion started by: Ravindra Swan
1 Replies

10. Shell Programming and Scripting

Parallel processing

I have 10,000 + files, each of which I need to zip using bzip2. Is ti possible to use bash to create 8 parallel streams sending a new file to be processed from the list when one of the others has finished? (1 Reply)
Discussion started by: garethsays
1 Replies
queuedefs(4)							   File Formats 						      queuedefs(4)

NAME
queuedefs - queue description file for at, batch, and cron SYNOPSIS
/etc/cron.d/queuedefs DESCRIPTION
The queuedefs file describes the characteristics of the queues managed by cron(1M). Each non-comment line in this file describes one queue. The format of the lines are as follows: q.[njobj][nicen][nwaitw] The fields in this line are: q The name of the queue. a is the default queue for jobs started by at(1); b is the default queue for jobs started by batch (see at(1)); c is the default queue for jobs run from a crontab(1) file. njob The maximum number of jobs that can be run simultaneously in that queue; if more than njob jobs are ready to run, only the first njob jobs will be run, and the others will be run as jobs that are currently running terminate. The default value is 100. nice The nice(1) value to give to all jobs in that queue that are not run with a user ID of super-user. The default value is 2. nwait The number of seconds to wait before rescheduling a job that was deferred because more than njob jobs were running in that job's queue, or because the system-wide limit of jobs executing has been reached. The default value is 60. Lines beginning with # are comments, and are ignored. EXAMPLES
Example 1: A sample file. # # a.4j1n b.2j2n90w This file specifies that the a queue, for at jobs, can have up to 4 jobs running simultaneously; those jobs will be run with a nice value of 1. As no nwait value was given, if a job cannot be run because too many other jobs are running cron will wait 60 seconds before trying again to run it. The b queue, for batch(1) jobs, can have up to 2 jobs running simultaneously; those jobs will be run with a nice(1) value of 2. If a job cannot be run because too many other jobs are running, cron(1M) will wait 90 seconds before trying again to run it. All other queues can have up to 100 jobs running simultaneously; they will be run with a nice value of 2, and if a job cannot be run because too many other jobs are running cron will wait 60 seconds before trying again to run it. FILES
/etc/cron.d/queuedefs queue description file for at, batch, and cron. SEE ALSO
at(1), crontab(1), nice(1), cron(1M) SunOS 5.10 1 Mar 1994 queuedefs(4)
All times are GMT -4. The time now is 07:17 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy