Sponsored Content
Top Forums Shell Programming and Scripting BASH - Handling background processes - distributed processing Post 302514641 by pbillast on Sunday 17th of April 2011 06:10:49 AM
Old 04-17-2011
In these kind of situations you'll find the environment variable $! (pid of backgound process) and the manpage for the command wait very useful.
I recently made something to launch sql statements and gzip on the output files with a maximum number of subprocesses (I chose this maximum in function of the numbers of cpu cores)

basically in the script you start a subprocess
Code:
ClientDataEncrypt <args> &
BPID=$!

and thus keep track of the backgound pids (by storing them in an array).

from there you can limit the number of subprocesses.

(P.S. on Solaris 10 pgrep is very useful in these situations, especially if you want to be able to launch ClientDataEncrypt both manually and via the watcher and want the watcher daemon to know about it)

I hope this helps

Last edited by pbillast; 04-17-2011 at 07:16 AM..
 

9 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Background processes

How do you capture the return code from a background process? I am dumping data to a fifo and then processing it in a c program. I need to know that the sql finished successfully to ensure no missing data. Thanks. ex. sqlplus user/password < get_data.sql > data_fifo.txt & bin/process_data... (2 Replies)
Discussion started by: korndog
2 Replies

2. Shell Programming and Scripting

Handling Stdout&StdErr for background jobs.

Hello Friends, sorry, i am not very familiar with Unix programming. Could you please help me on this? We have to start different components from a startup script. each components are started as below in the background in a startprocess function $nohup $file $args >>$logFile 2>&1 & ... (0 Replies)
Discussion started by: alvinbush
0 Replies

3. Solaris

Handling Stdout&StdErr for background jobs.

Hello Friends, sorry, i am not very familiar with Unix programming. Could you please help me on this? We have to start different components from a startup script. each components are started as below in the background in a startprocess function $nohup $file $args >>$logFile 2>&1 & ... (1 Reply)
Discussion started by: alvinbush
1 Replies

4. Linux

background processing in BASH

I have script 3 scripts 1 parent 2 children child1 child2 In the code below the 2 child processes fire almost Instantaneously in the background, Is that possible to know the status of pass/fail of each process "as it happens" ? In the present scenario although Child2... (5 Replies)
Discussion started by: jville
5 Replies

5. Shell Programming and Scripting

background processing in BASH

I have script 3 scripts 1 parent (p1) and 2 children child1 and child2 I have script 3 scripts 1 parent 2 children child1 child2 In the code below the 2 child processes fire almost Instantaneously in the background, Is that possible to know the status of pass/fail of each process... (12 Replies)
Discussion started by: jville
12 Replies

6. Shell Programming and Scripting

Background Processes

Ok guys so I have my first dummy shell almost done except for one tiny part: I do not know how to run a process in the background, from the code! I already know how to do that in a normal shell: $ program & However, no clue when it comes to how to program that thing. :eek: A very... (2 Replies)
Discussion started by: Across
2 Replies

7. Shell Programming and Scripting

Need help on background processes

Hi, I have a schell script parent.ksh from which I am calling three background processes a.ksh,b.ksh and c.ksh. Once these three processes completes the next step in parent.ksh should execute. How to achieve this? Please help me.... Thanks... (1 Reply)
Discussion started by: ravinunna
1 Replies

8. UNIX for Advanced & Expert Users

File Processing: Handling spaces in a line

Hi All, Iam trying to get a file processed and some lines have spaces...the below is not working Want to remove empty line Want to remove lines that start with # Avoid line with substring WHOA When trying to get the substring from the var also Iam having trouble file is like VAR=VALUE,... (13 Replies)
Discussion started by: baanprog
13 Replies

9. Shell Programming and Scripting

Help with shell script handling processes

Hello I have a file which has around 120 lines of commands. I am trying to write a shell script like which reads the 'command' file and executes line by line with some additional (common argument) with maximum 6 commands active at a time. Each of these commands when executed takes time... (5 Replies)
Discussion started by: JackyShane_36
5 Replies
ganeti-watcher(8)						   Version 2.5.2						 ganeti-watcher(8)

Name
       ganeti-watcher - Ganeti cluster watcher

Synopsis
       ganeti-watcher [--debug] [--job-age=age] [--ignore-pause]

DESCRIPTION
The ganeti-watcher is a periodically run script which is responsible for keeping the instances in the correct status. It has two separate functions, one for the master node and another one that runs on every node. If the watcher is disabled at cluster level (via the gnt-cluster watcher pause command), it will exit without doing anything. The cluster- level pause can be overridden via the --ignore-pause option, for example if during a maintenance the watcher needs to be disabled in gen- eral, but the administrator wants to run it just once. The --debug option will increase the verbosity of the watcher and also activate logging to the standard error. Master operations Its primary function is to try to keep running all instances which are marked as up in the configuration file, by trying to start them a limited number of times. Another function is to "repair" DRBD links by reactivating the block devices of instances which have secondaries on nodes that have been rebooted. The watcher will also archive old jobs (older than the age given via the --job-age option, which defaults to 6 hours), in order to keep the job queue manageable. Node operations The watcher will restart any down daemons that are appropriate for the current node. In addition, it will execute any scripts which exist under the "watcher" directory in the Ganeti hooks directory (/etc/ganeti/hooks). This should be used for lightweight actions, like starting any extra daemons. If the cluster parameter maintain_node_health is enabled, then the watcher will also shutdown instances and DRBD devices if the node is declared as offline by known master candidates. The watcher does synchronous queries but will submit jobs for executing the changes. Due to locking, it could be that the jobs execute much later than the watcher submits them. FILES
The command has a state file located at /var/lib/ganeti/watcher.data (only used on the master) and a log file at /var/log/ganeti/watcher.log. Removal of either file will not affect correct operation; the removal of the state file will just cause the restart counters for the instances to reset to zero. REPORTING BUGS
Report bugs to project website (http://code.google.com/p/ganeti/) or contact the developers using the Ganeti mailing list (ganeti@google- groups.com). SEE ALSO
Ganeti overview and specifications: ganeti(7) (general overview), ganeti-os-interface(7) (guest OS definitions). Ganeti commands: gnt-cluster(8) (cluster-wide commands), gnt-job(8) (job-related commands), gnt-node(8) (node-related commands), gnt- instance(8) (instance commands), gnt-os(8) (guest OS commands), gnt-group(8) (node group commands), gnt-backup(8) (instance import/export commands), gnt-debug(8) (debug commands). Ganeti daemons: ganeti-watcher(8) (automatic instance restarter), ganeti-cleaner(8) (job queue cleaner), ganeti-noded(8) (node daemon), ganeti-masterd(8) (master daemon), ganeti-rapi(8) (remote API daemon). Ganeti htools: htools(1) (generic binary), hbal(1) (cluster balancer), hspace(1) (capacity calculation), hail(1) (IAllocator plugin), hscan(1) (data gatherer from remote clusters). COPYRIGHT
Copyright (C) 2006, 2007, 2008, 2009, 2010, 2011 Google Inc. Permission is granted to copy, distribute and/or modify under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. On Debian systems, the complete text of the GNU General Public License can be found in /usr/share/common-licenses/GPL. Ganeti ganeti-watcher(8)
All times are GMT -4. The time now is 02:17 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy