Sponsored Content
Top Forums Shell Programming and Scripting Help needed on restart-from-point-of-failure in Parallel Processing Post 302995545 by saps19 on Friday 7th of April 2017 06:48:20 AM
Old 04-07-2017
Thank you rbattle1,
Quote:
You might want this:-

Code:
(/path/to/product/script/data_process "${fileLines}" || exit -1) &
make sense and corrected the script(though earlier it was running fine (Smilie), don't know how).

Currently I'm trying to get the command I'm running using below sample test script:
Code:
 $ [] cat bgPIDTest.ksh
# Some function that takes a long time to process
longprocess() {
        # Sleep up to 14 seconds
        #sleep $((RANDOM % 15))
        sleepTime=$((RANDOM % 5))
        # Randomly exit with 0 or 1
        exitCode=$((RANDOM % 2))
        echo "sleeping for: $sleepTime with exit code: $exitCode "
        sleep $sleepTime
        exit $exitCode
}
 pids=""
failCnt=0
# Run five concurrent processes
        ( longprocess ) &
        # store PID of process
        pids+=" $!"
        echo PID $pids
        ( longprocess ) &
        # store PID of process
        pids+=" $!"
        echo PID $pids
        ( longprocess ) &
        # store PID of process
        pids+=" $!"
        echo PID $pids
        ( longprocess ) &
        # store PID of process
        pids+=" $!"
        echo PID $pids
        ( longprocess ) &
        # store PID of process
        pids+=" $!"
        echo PID $pids
        ( longprocess ) &
        # store PID of process
        pids+=" $!"
        echo PID $pids
 
# Wait for all processes to finish, will take max 14s
echo "initial failCnt is $failCnt"
for p in $pids; do
        #if wait $p; then
        if ! wait $p; then
            cmdJobNM=`ps -p $p -o command=`
            failCnt=`expr $failCnt + 1`
            echo "failed command is --> $cmdJobNM, PID: $p"
        fi
done
echo "total failCnt is $failCnt"
if [ $failCnt -gt 0 ]; then
    exit -1
fi

Output:
Code:
 $ [] ksh bgPIDTest.ksh
PID 15858
sleeping for: 2 with exit code: 1
PID 15858 15859
sleeping for: 4 with exit code: 0
PID 15858 15859 15860
sleeping for: 0 with exit code: 0
PID 15858 15859 15860 15861
sleeping for: 3 with exit code: 0
PID 15858 15859 15860 15861 15862
sleeping for: 3 with exit code: 1
PID 15858 15859 15860 15861 15862 15863
initial failCnt is 0
sleeping for: 1 with exit code: 0
failed command is --> , PID: 15858
failed command is --> , PID: 15862
total failCnt is 2

Still I'm not getting the command, so that I can awk'ed the passed argument and put into a file. So the logic will be:
Once I'll restart script:
if this new file exist
take this new file
else
use old config file

once successfully done, I'll remove the error file (if any). Please suggest if this feasible.

Cheers,
Saps.
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to do parallel processing??

Hi All, I am working on solaris 8 sparc machine with 2 cpu. I am trying to run my application which generates files. I run multiple instance of the application, but the results don't seem to show as if it were runing parallely. When i run the application once it takes 12 secs to generate a... (1 Reply)
Discussion started by: zing
1 Replies

2. UNIX for Dummies Questions & Answers

Restart on power failure

How do I configure my workstation (Solaris 9) to restart and perform any check disk automatically if there is a power failure? Thanks. (1 Reply)
Discussion started by: here2learn
1 Replies

3. Shell Programming and Scripting

parallel processing

Hi I want to run two shell script files parallely. These two scripts are interacting with the database. can any body help on this Pls Regards Audippa naidu.M (3 Replies)
Discussion started by: audippa
3 Replies

4. Shell Programming and Scripting

Need Help With Parallel Processing

Hi I am looking for some kind of feature in unix that will help me write a script that can invoke multiple processes in parallel. And make sure that the multiple parallel processes complete successfully before I proceed to the next step. Someone suggested something called timespid or... (6 Replies)
Discussion started by: imnewtothis23
6 Replies

5. Shell Programming and Scripting

parallel processing

hi i am preparing a set of batches for a set of files sequentially There is a folder /xyz where all the files reside now all the files starting with 01 - will be appended for one below other to form a batch batch01 then all the files starting with 02 - will be appended for one below other to... (7 Replies)
Discussion started by: mad_man12
7 Replies

6. Shell Programming and Scripting

How to make parallel processing rather than serial processing ??

Hello everybody, I have a little problem with one of my program. I made a plugin for collectd (a stats collector for my servers) but I have a problem to make it run in parallel. My program gathers stats from logs, so it needs to run in background waiting for any new lines added in the log... (0 Replies)
Discussion started by: Samb95
0 Replies

7. Shell Programming and Scripting

script parallel processing

How to write script which run multiple scripts parllely, i have script called A,which has to execute B,C,D,E scripts parllely.. (2 Replies)
Discussion started by: machpee
2 Replies

8. Shell Programming and Scripting

Re run the script from the point of failure

Hello I have a shell script with multiple sections in it. Like, verify pre-requisites, ch co version, stop services , install product , post migration steps, start services, send status email. If the script fails at certain step, (like after product installation) it can't be re-run since the... (2 Replies)
Discussion started by: mo12
2 Replies

9. Shell Programming and Scripting

Parallel processing

I have 10,000 + files, each of which I need to zip using bzip2. Is ti possible to use bash to create 8 parallel streams sending a new file to be processed from the list when one of the others has finished? (1 Reply)
Discussion started by: garethsays
1 Replies
GRUB-FILE(1)                                                       User Commands                                                      GRUB-FILE(1)

NAME
grub-file - check file type SYNOPSIS
file OPTIONS FILE DESCRIPTION
Check if FILE is of specified type. --is-i386-xen-pae-domu Check if FILE can be booted as i386 PAE Xen unprivileged guest kernel --is-x86_64-xen-domu Check if FILE can be booted as x86_64 Xen unprivileged guest kernel --is-x86-xen-dom0 Check if FILE can be used as Xen x86 privileged guest kernel --is-x86-multiboot Check if FILE can be used as x86 multiboot kernel --is-x86-multiboot2 Check if FILE can be used as x86 multiboot2 kernel --is-arm-linux Check if FILE is ARM Linux --is-arm64-linux Check if FILE is ARM64 Linux --is-ia64-linux Check if FILE is IA64 Linux --is-mips-linux Check if FILE is MIPS Linux --is-mipsel-linux Check if FILE is MIPSEL Linux --is-sparc64-linux Check if FILE is SPARC64 Linux --is-powerpc-linux Check if FILE is POWERPC Linux --is-x86-linux Check if FILE is x86 Linux --is-x86-linux32 Check if FILE is x86 Linux supporting 32-bit protocol --is-x86-kfreebsd Check if FILE is x86 kFreeBSD --is-i386-kfreebsd Check if FILE is i386 kFreeBSD --is-x86_64-kfreebsd Check if FILE is x86_64 kFreeBSD --is-x86-knetbsd Check if FILE is x86 kNetBSD --is-i386-knetbsd Check if FILE is i386 kNetBSD --is-x86_64-knetbsd Check if FILE is x86_64 kNetBSD --is-i386-efi Check if FILE is i386 EFI file --is-x86_64-efi Check if FILE is x86_64 EFI file --is-ia64-efi Check if FILE is IA64 EFI file --is-arm64-efi Check if FILE is ARM64 EFI file --is-arm-efi Check if FILE is ARM EFI file --is-hibernated-hiberfil Check if FILE is hiberfil.sys in hibernated state --is-x86_64-xnu Check if FILE is x86_64 XNU (Mac OS X kernel) --is-i386-xnu Check if FILE is i386 XNU (Mac OS X kernel) --is-xnu-hibr Check if FILE is XNU (Mac OS X kernel) hibernated image --is-x86-bios-bootsector Check if FILE is BIOS bootsector -h, --help Display this help and exit. -u, --usage Display the usage of this command and exit. SEE ALSO
The full documentation for grub-file is maintained as a Texinfo manual. If the info and grub-file programs are properly installed at your site, the command info grub-file should give you access to the complete manual. grub-file (GRUB) 2.02-2ubuntu8.3 July 2018 GRUB-FILE(1)
All times are GMT -4. The time now is 02:21 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy