Sponsored Content
Top Forums UNIX for Advanced & Expert Users Issue with tracking successful completion of Child process running in background Post 302981964 by Don Cragun on Thursday 22nd of September 2016 12:52:21 AM
Old 09-22-2016
Running extra processes is a sure way to slow down execution.

By definition, a zombie process is a process that has terminated and still exists only because its parent has not "wait"ed for it to gather its exit status. Killing a zombie will not have any effect on that zombie. The only things that will cause a zombie to disappear are:
  1. for it parent to wait for it,
  2. for its parent to terminate, for it to be adopted by the system's zombie collector (known as init on many systems) and for the zombie collector to wait for it, or
  3. reboot the system.
Having zombies around doesn't have any appreciable effect on a system unless the system's process table is almost full (and if that is a problem for you, all of the extra processes you are creating looking for zombies and rearranging the array of running background jobs will be more of a problem).

In your code sequence:
Code:
                        if [[ $sts -eq 0 || $sts -eq 127 ]]; then
                                echo "Process completed with Process ID ${p}; exit code: $sts; at `date '+%Y/%m/%d:%H:%M:%S'`"
                                pcnt=`expr $pcnt + 1`
                        elif [[ $sts -ne 0 || $sts -ne 127 ]]; then
                                echo "Process failed for Process ID: ${p}"
                                index=`echo ${ppids[@]/$p//}|cut -d/ -f1 |wc -w |tr -d ' '`
                                unset ppids[$index]
                                pcnt=`expr $pcnt + 1`
                                fpcnt=`expr $fpcnt + 1`
                                fppids+=(${p})
                        else
                                kill -TERM ${p}
                                index=`echo ${ppids[@]/$p//}|cut -d/ -f1 |wc -w |tr -d ' '`
                                unset ppids[$index]
                                pcnt=`expr $pcnt + 1`
                        fi

There is absolutely no way that you will ever execute the else clause. If $sts expands to 0 or to 127, you will execute the first then clause. Otherwise (since we already know that $sts does not expand to 0) the first part of the or in the elif double square bracket expression must be true and the 2nd then clause will be executed.

All of the code looking for zombies and trying to reap the first zombie is a waste of CPU cycles and memory that could be better spent to running the remaining background jobs (and as noted before, killing zombies the way you are trying to kill them is just an expensive no-op).

Using nohup to run a shell script that does nothing but nohup another job echo one word and exit triples the number of processes you need to run a background job. This might or might not be part of your problem, but it certainly won't help you. And, starting up unneeded processes will slow down everything running on your system.

Showing us code with syntax errors, with undefined variables, and not telling us how many jobs you are trying to run in parallel makes it hard for us to give any firm suggestions on how to fix your code (or even to determine what might be wrong), but you might consider replacing the two snippets you showed us with a single snippet similar to the following:
Code:
#!/bin/ksh

# Note that the following four lines all fail with syntax errors...
ppids=() ---> Main array for process ids.
fppids=() ---> array to capture failed process ids.
pcnt=0 ---> success count
fpcnt=0 ---> fail count

export tot_table_cnt=0

echo
# start_time=$(date '+%Y/%m/%d:%H:%M:%S') This line ommented out: not used.
# $CONFIG_DIR is used but not set.
# $curr_date is used but not set.
# $LOG_DIR is used but not set.
# $TEMP_DIR is used but not set


while read -r table_name
do	nohup hive -S -e "do something;" > "${LOG_DIR}/${table_name}_inner_script_${curr_date}.log"&
        ppids+=($!)
        echo "Process ID:       $!."
        echo
        echo "Log File for ${table_name} is: ${LOG_DIR}/${table_name}_inner_script_${curr_date}.log"
	tot_table_cnt=$((tot_table_cnt + 1))
done < "$CONFIG_DIR/abc.txt"

echo
echo
echo 'Starting Checking the process completion of all tables:'
echo
echo "Total Number of tables: ${tot_table_cnt}."

for p in "${ppids[*]}"
do	wait $p
	sts=$?
	if [[ $sts -eq 0 ]]
	then	pcnt=$((pcnt + 1))
		echo Success
		echo "Process completed with Process ID ${p}; exit code: 0; at $(date '+%Y/%m/%d:%H:%M:%S')"
	else	echo failure
		echo "Process failed for Process ID: ${p}; exit code: $sts; at $(date '+%Y/%m/%d:%H:%M:%S')"
		fpcnt=$((fpcnt + 1))
		fppids+=(${p})
	fi
done
echo "process for all tables is complete for ${curr_date}."

if [[ $fpcnt -eq 0 ]]
then	echo
	echo 'process is successfully completed for all Tables.'
	echo 'DONE file is touched.'
	touch ${TEMP_DIR}/inner_script_completion.done
	echo
else	echo
	echo "process failed for ${fpcnt} tables."
	echo "Failed Process IDs are ${fppids[@]}."
	echo "DONE File is not touched in ${TEMP_DIR} path. Need to verify or re-run the process manually."
fi

I have no idea what hive is supposed to do. I have no idea how the variables CONFIG_DIR, curr_date, LOG_DIR, and TEMP_DIR (which are all used by your script, but not initialized) are supposed to be set. So, obviously, the above script is totally untested. But, it seems like it should get the background jobs you have started run faster than your current script (assuming that there aren't any other users consuming the cycles freed up by this simplified version of your code).
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

running process in background

I'm trying to install a solaris 9 patch cluster and when I try to use & to run in background it won't allow me to enter in my sudo password so it fails the install and sudo auth. Does Solaris not have screen like linux? If & will work what am I doing wrong? sudo ./install_cluster -q & is... (3 Replies)
Discussion started by: kingdbag
3 Replies

2. Shell Programming and Scripting

How to know the status of process running in background

I have run one shell script in background that contains a endless while loop. I am not able to know the status of that job . Please provide any command to know this. I have already used "ps -aef" , "jobs" to know it , but it didn't work. I am sure the process is running as it is generating a file... (8 Replies)
Discussion started by: sumanta
8 Replies

3. UNIX for Advanced & Expert Users

Oracle library issue in child process

Hi, I am using a daemon from which I am forking 3 processes P1,P2,P3 out of which P3 is compiled with oracle lib32/libclntsh.so and P1,P2 are non database process. The Logic of daemon is that if any one goes down simply clean the other and refork all the 3 again. P3 is getting forked first time... (1 Reply)
Discussion started by: unisuraj
1 Replies

4. UNIX for Advanced & Expert Users

how to make a parent wait on a child shells running in background?

Hi I have a shell script A which calls another 10 shell scripts which run in background. How do i make the parent script wait for the child scripts complete, or in other words, i must be able to do a grep of parent script to find out if the child scripts are still running. My Code: ... (1 Reply)
Discussion started by: albertashish
1 Replies

5. Shell Programming and Scripting

How to export a variable from a child process running in background to the parent

Hi All, I have a script which calls a child script with a parameter to be run in the background . childscript.ksh $a & Can any one suggest me how do i export a variable from the child script to parent script? Note that the child script is in background If the child script is in... (3 Replies)
Discussion started by: aixjadoo
3 Replies

6. UNIX for Advanced & Expert Users

send a new value to a variable in a running background process

Hi guys, I have a issue with a background process, I need to update the value of a variable in that process which is running at this time and it will be running for at least 2 days. Any idea? I will apreciate your help. regards. Razziel. (2 Replies)
Discussion started by: razziel
2 Replies

7. UNIX for Dummies Questions & Answers

Background Process Completion

I have my unix machine configured to run locate.updatedb on login in the background and after it completes, when I run a command such as ls-- the console returns the results of ls and + Done sudo /usr/libexec/locate.updatedbIs there... (3 Replies)
Discussion started by: Prodiga1
3 Replies

8. Shell Programming and Scripting

command to see process running at background

Hi , I want to see all the background process that are running in unix box machine...please guide me is there any specific command for that..since I am executing some scripts at background..!!:confused: (1 Reply)
Discussion started by: nks342
1 Replies

9. Shell Programming and Scripting

How to determine the completion of a background process to trigger something else?

I've been thinking about a peculiar problem, and so far haven't been able to find out a convincing solution to that. To put it simply, I have a shell script (assume it to be parent_script.sh), calling another shell script (child_script.sh) 5 times, in nohup mode to be executed in the background.... (3 Replies)
Discussion started by: Aviktheory11
3 Replies

10. UNIX for Beginners Questions & Answers

Running process in the background

Hi, I have this simple c program that creates duplicate process with fork(): #include <sys/types.h> main() { if (fork() == 0) while(1); else while(1); } I tried running it in the background gcc -o test first.c test & And I got this list of running process: (4 Replies)
Discussion started by: uniran
4 Replies
All times are GMT -4. The time now is 07:21 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy