Sponsored Content
Top Forums Shell Programming and Scripting Help needed on restart-from-point-of-failure in Parallel Processing Post 302995545 by saps19 on Friday 7th of April 2017 06:48:20 AM
Old 04-07-2017
Thank you rbattle1,
Quote:
You might want this:-

Code:
(/path/to/product/script/data_process "${fileLines}" || exit -1) &
make sense and corrected the script(though earlier it was running fine (Smilie), don't know how).

Currently I'm trying to get the command I'm running using below sample test script:
Code:
 $ [] cat bgPIDTest.ksh
# Some function that takes a long time to process
longprocess() {
        # Sleep up to 14 seconds
        #sleep $((RANDOM % 15))
        sleepTime=$((RANDOM % 5))
        # Randomly exit with 0 or 1
        exitCode=$((RANDOM % 2))
        echo "sleeping for: $sleepTime with exit code: $exitCode "
        sleep $sleepTime
        exit $exitCode
}
 pids=""
failCnt=0
# Run five concurrent processes
        ( longprocess ) &
        # store PID of process
        pids+=" $!"
        echo PID $pids
        ( longprocess ) &
        # store PID of process
        pids+=" $!"
        echo PID $pids
        ( longprocess ) &
        # store PID of process
        pids+=" $!"
        echo PID $pids
        ( longprocess ) &
        # store PID of process
        pids+=" $!"
        echo PID $pids
        ( longprocess ) &
        # store PID of process
        pids+=" $!"
        echo PID $pids
        ( longprocess ) &
        # store PID of process
        pids+=" $!"
        echo PID $pids
 
# Wait for all processes to finish, will take max 14s
echo "initial failCnt is $failCnt"
for p in $pids; do
        #if wait $p; then
        if ! wait $p; then
            cmdJobNM=`ps -p $p -o command=`
            failCnt=`expr $failCnt + 1`
            echo "failed command is --> $cmdJobNM, PID: $p"
        fi
done
echo "total failCnt is $failCnt"
if [ $failCnt -gt 0 ]; then
    exit -1
fi

Output:
Code:
 $ [] ksh bgPIDTest.ksh
PID 15858
sleeping for: 2 with exit code: 1
PID 15858 15859
sleeping for: 4 with exit code: 0
PID 15858 15859 15860
sleeping for: 0 with exit code: 0
PID 15858 15859 15860 15861
sleeping for: 3 with exit code: 0
PID 15858 15859 15860 15861 15862
sleeping for: 3 with exit code: 1
PID 15858 15859 15860 15861 15862 15863
initial failCnt is 0
sleeping for: 1 with exit code: 0
failed command is --> , PID: 15858
failed command is --> , PID: 15862
total failCnt is 2

Still I'm not getting the command, so that I can awk'ed the passed argument and put into a file. So the logic will be:
Once I'll restart script:
if this new file exist
take this new file
else
use old config file

once successfully done, I'll remove the error file (if any). Please suggest if this feasible.

Cheers,
Saps.
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to do parallel processing??

Hi All, I am working on solaris 8 sparc machine with 2 cpu. I am trying to run my application which generates files. I run multiple instance of the application, but the results don't seem to show as if it were runing parallely. When i run the application once it takes 12 secs to generate a... (1 Reply)
Discussion started by: zing
1 Replies

2. UNIX for Dummies Questions & Answers

Restart on power failure

How do I configure my workstation (Solaris 9) to restart and perform any check disk automatically if there is a power failure? Thanks. (1 Reply)
Discussion started by: here2learn
1 Replies

3. Shell Programming and Scripting

parallel processing

Hi I want to run two shell script files parallely. These two scripts are interacting with the database. can any body help on this Pls Regards Audippa naidu.M (3 Replies)
Discussion started by: audippa
3 Replies

4. Shell Programming and Scripting

Need Help With Parallel Processing

Hi I am looking for some kind of feature in unix that will help me write a script that can invoke multiple processes in parallel. And make sure that the multiple parallel processes complete successfully before I proceed to the next step. Someone suggested something called timespid or... (6 Replies)
Discussion started by: imnewtothis23
6 Replies

5. Shell Programming and Scripting

parallel processing

hi i am preparing a set of batches for a set of files sequentially There is a folder /xyz where all the files reside now all the files starting with 01 - will be appended for one below other to form a batch batch01 then all the files starting with 02 - will be appended for one below other to... (7 Replies)
Discussion started by: mad_man12
7 Replies

6. Shell Programming and Scripting

How to make parallel processing rather than serial processing ??

Hello everybody, I have a little problem with one of my program. I made a plugin for collectd (a stats collector for my servers) but I have a problem to make it run in parallel. My program gathers stats from logs, so it needs to run in background waiting for any new lines added in the log... (0 Replies)
Discussion started by: Samb95
0 Replies

7. Shell Programming and Scripting

script parallel processing

How to write script which run multiple scripts parllely, i have script called A,which has to execute B,C,D,E scripts parllely.. (2 Replies)
Discussion started by: machpee
2 Replies

8. Shell Programming and Scripting

Re run the script from the point of failure

Hello I have a shell script with multiple sections in it. Like, verify pre-requisites, ch co version, stop services , install product , post migration steps, start services, send status email. If the script fails at certain step, (like after product installation) it can't be re-run since the... (2 Replies)
Discussion started by: mo12
2 Replies

9. Shell Programming and Scripting

Parallel processing

I have 10,000 + files, each of which I need to zip using bzip2. Is ti possible to use bash to create 8 parallel streams sending a new file to be processed from the list when one of the others has finished? (1 Reply)
Discussion started by: garethsays
1 Replies
dpid.h(3)							     libdaemon								 dpid.h(3)

NAME
dpid.h - Contains an API for manipulating PID files. SYNOPSIS
#include <sys/types.h> Defines #define DAEMON_PID_FILE_KILL_WAIT_AVAILABLE 1 This variable is defined to 1 iff daemon_pid_file_kill_wait() is supported. Typedefs typedef const char *(* daemon_pid_file_proc_t )(void) Prototype of a function for generating the name of a PID file. Functions const char * daemon_pid_file_proc_default (void) A function for creating a pid file name from daemon_pid_file_ident. int daemon_pid_file_create (void) Creates PID pid file for the current process. int daemon_pid_file_remove (void) Removes the PID file of the current process. pid_t daemon_pid_file_is_running (void) Returns the PID file of a running daemon, if available. int daemon_pid_file_kill (int s) Kills a running daemon, if available. int daemon_pid_file_kill_wait (int s, int m) Similar to daemon_pid_file_kill() but waits until the process died. Variables const char * daemon_pid_file_ident Identification string for the PID file name, only used when daemon_pid_file_proc is set to daemon_pid_file_proc_default(). daemon_pid_file_proc_t daemon_pid_file_proc A function pointer which is used to generate the name of the PID file to manipulate. Detailed Description Contains an API for manipulating PID files. Definition in file dpid.h. Define Documentation #define DAEMON_PID_FILE_KILL_WAIT_AVAILABLE 1 This variable is defined to 1 iff daemon_pid_file_kill_wait() is supported. Since: 0.3 See also: daemon_pid_file_kill_wait() Definition at line 82 of file dpid.h. Typedef Documentation typedef const char*(* daemon_pid_file_proc_t)(void) Prototype of a function for generating the name of a PID file. Definition at line 37 of file dpid.h. Function Documentation int daemon_pid_file_create (void) Creates PID pid file for the current process. Returns: zero on success, nonzero on failure Examples: testd.c. pid_t daemon_pid_file_is_running (void) Returns the PID file of a running daemon, if available. Returns: The PID or negative on failure Examples: testd.c. int daemon_pid_file_kill (int s) Kills a running daemon, if available. Parameters: s The signal to send Returns: zero on success, nonzero on failure int daemon_pid_file_kill_wait (int s, int m) Similar to daemon_pid_file_kill() but waits until the process died. This functions is new in libdaemon 0.3. The macro DAEMON_PID_FILE_KILL_WAIT_AVAILABLE is defined iff libdaemon supports this function. Parameters: s The signal to send m Seconds to wait at maximum Returns: zero on success, nonzero on failure (timeout condition is considered a failure) Since: 0.3 See also: DAEMON_PID_FILE_KILL_WAIT_AVAILABLE Examples: testd.c. const char* daemon_pid_file_proc_default (void) A function for creating a pid file name from daemon_pid_file_ident. Returns: The PID file path int daemon_pid_file_remove (void) Removes the PID file of the current process. Returns: zero on success, nonzero on failure Examples: testd.c. Variable Documentation const char* daemon_pid_file_ident Identification string for the PID file name, only used when daemon_pid_file_proc is set to daemon_pid_file_proc_default(). Use daemon_ident_from_argv0() to generate an identification string from argv[0] Examples: testd.c. daemon_pid_file_proc_t daemon_pid_file_proc A function pointer which is used to generate the name of the PID file to manipulate. Points to daemon_pid_file_proc_default() by default. Author Generated automatically by Doxygen for libdaemon from the source code. Version 0.14 17 Jan 2010 dpid.h(3)
All times are GMT -4. The time now is 12:57 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy