Sponsored Content
Top Forums Shell Programming and Scripting BASH - Handling background processes - distributed processing Post 302514801 by dcarrion87 on Monday 18th of April 2011 06:00:46 AM
Old 04-18-2011
Thanks for your responses so far. I never actually really thought about managing the amount of sub processes. I am going to implement this however as it could quite easily get out of control.

I've changed the way I am dealing with the "watcher" and "worker" processes in terms of notifications. Instead of the watcher daemon doing the notifications I will have the watcher parse an email address argument, specified in the watcher config, to the worker processes.

End of the day, all these processes are doing sysloging so I know what's going on as an admin and the staff that care about a particular client system will get the notifications they need to get (if decryption failed or not for example). This way operations staff can call the worker processes manually if the worker config isn't set to auto-decrypt for that system and have the results shot to their email address...Well...the sub processes determine if they need to run or not based on the ID that's parsed by the watcher and if there's actually data in the path!

In other words, in the instance of an auto-decrypt for example:

1. ClientDataWatcher running and hits entry with auto-decrypt.
2. ClientDataWatcher checks if there is lock file matching the ID (lets assume there isn't).
3. ClientDataWatcher kicks off subtask. Touches a lock file with ID in file name.
Code:
ClientDataDecrypt "3" "person1@bla.com,person2@bla.com

4. ClientDataDecrypt sees if it has an entry for 3 in it's config.
5. ClientDataDecrypt finds entry and checks its config for folder of files it has to decrypt.
6. ClientDataDecrypt does decryption if it has to otherwise EXITS.
7. ClientDataDecrypt sends relevent notification based on above result and recipients passed.

Obviously there is a lot more stuff going on with the above but it's a summarized logic.

In terms of process sanity control, I'm going to have the watcher touch a lock file as it kicks off a process for that particular ID task based on config entry (as per above). The lock file will contain an epoch date on when the process started and it will use this to determine when it can rerun an auto-decrypt for example based on some sort of constant integer (specified somewhere).

Feedback, thoughts and abuse welcome! Smilie
 

9 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Background processes

How do you capture the return code from a background process? I am dumping data to a fifo and then processing it in a c program. I need to know that the sql finished successfully to ensure no missing data. Thanks. ex. sqlplus user/password < get_data.sql > data_fifo.txt & bin/process_data... (2 Replies)
Discussion started by: korndog
2 Replies

2. Shell Programming and Scripting

Handling Stdout&StdErr for background jobs.

Hello Friends, sorry, i am not very familiar with Unix programming. Could you please help me on this? We have to start different components from a startup script. each components are started as below in the background in a startprocess function $nohup $file $args >>$logFile 2>&1 & ... (0 Replies)
Discussion started by: alvinbush
0 Replies

3. Solaris

Handling Stdout&StdErr for background jobs.

Hello Friends, sorry, i am not very familiar with Unix programming. Could you please help me on this? We have to start different components from a startup script. each components are started as below in the background in a startprocess function $nohup $file $args >>$logFile 2>&1 & ... (1 Reply)
Discussion started by: alvinbush
1 Replies

4. Linux

background processing in BASH

I have script 3 scripts 1 parent 2 children child1 child2 In the code below the 2 child processes fire almost Instantaneously in the background, Is that possible to know the status of pass/fail of each process "as it happens" ? In the present scenario although Child2... (5 Replies)
Discussion started by: jville
5 Replies

5. Shell Programming and Scripting

background processing in BASH

I have script 3 scripts 1 parent (p1) and 2 children child1 and child2 I have script 3 scripts 1 parent 2 children child1 child2 In the code below the 2 child processes fire almost Instantaneously in the background, Is that possible to know the status of pass/fail of each process... (12 Replies)
Discussion started by: jville
12 Replies

6. Shell Programming and Scripting

Background Processes

Ok guys so I have my first dummy shell almost done except for one tiny part: I do not know how to run a process in the background, from the code! I already know how to do that in a normal shell: $ program & However, no clue when it comes to how to program that thing. :eek: A very... (2 Replies)
Discussion started by: Across
2 Replies

7. Shell Programming and Scripting

Need help on background processes

Hi, I have a schell script parent.ksh from which I am calling three background processes a.ksh,b.ksh and c.ksh. Once these three processes completes the next step in parent.ksh should execute. How to achieve this? Please help me.... Thanks... (1 Reply)
Discussion started by: ravinunna
1 Replies

8. UNIX for Advanced & Expert Users

File Processing: Handling spaces in a line

Hi All, Iam trying to get a file processed and some lines have spaces...the below is not working Want to remove empty line Want to remove lines that start with # Avoid line with substring WHOA When trying to get the substring from the var also Iam having trouble file is like VAR=VALUE,... (13 Replies)
Discussion started by: baanprog
13 Replies

9. Shell Programming and Scripting

Help with shell script handling processes

Hello I have a file which has around 120 lines of commands. I am trying to write a shell script like which reads the 'command' file and executes line by line with some additional (common argument) with maximum 6 commands active at a time. Each of these commands when executed takes time... (5 Replies)
Discussion started by: JackyShane_36
5 Replies
EVCHILD(3)								 1								EVCHILD(3)

The EvChild class

INTRODUCTION
EvChild watchers trigger when the process receives a SIGCHLD in response to some child status changes (most typically when a child dies or exits). It is permissible to install an EvChild watcher after the child has been forked(which implies it might have already exited), as long as the event loop isn't entered(or is continued from a watcher), i.e. forking and then immediately registering a watcher for the child is fine, but forking and registering a watcher a few event loop iterations later or in the next callback invocation is not. It is allowed to register EvChild watchers in the default loop only. CLASS SYNOPSIS
EvChild EvChildextends EvWatcher Properties o public$pid o public$rpid o public$rstatus Inherited properties o public$is_active o public$data o public$is_pending o public$priority Methods o public EvChild::__construct NULL (int $pid, bool $trace, callable $callback, [mixed $data], [int $priority]) o finalpublicstatic object EvChild::createStopped (int $pid, bool $trace, callable $callback, [mixed $data], [int $priority]) o public void EvChild::set (int $pid, bool $trace) Inherited methods o public int EvWatcher::clear (void ) o abstractpublic EvWatcher::__construct (void ) o public void EvWatcher::feed (int $revents) o public EvLoop EvWatcher::getLoop (void ) o public void EvWatcher::invoke (int $revents) o public bool EvWatcher::keepalive ([bool $value]) o public void EvWatcher::setCallback (callable $callback) o public void EvWatcher::start (void ) o public void EvWatcher::stop (void ) PROPERTIES
o $pid - Readonly . The process ID this watcher watches out for, or 0 , meaning any process ID. o $rpid - Readonly .The process ID that detected a status change. o $rstatus - Readonly . The process exit status caused by $rpid . PHP Documentation Group EVCHILD(3)
All times are GMT -4. The time now is 01:31 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy