I have a need to run any number of identical scripts simultaneously, so I've created a driver script which reads a template script, edits these appropriately and then submits them via nohup. The spawned scripts all check to see at some point how many of their number are running and once the count has dropped to 2 (I'm grepping so that means only 1 is still running) it should spawn the final script which should only run once.
The variable of the count is gotten at by
Code:
number_running=`ps -ef | grep simple | -wc l`
This number_running is always zero somehow.
Moderator's Comments:
Please use CODE tags when displaying sample input, output, and code segments.
Last edited by Don Cragun; 02-23-2018 at 08:10 PM..
Reason: Add CODE tags.
I have a need to run any number of identical scripts simultaneously, so I've created a driver script which reads a template script, edits these appropriately and then submits them via nohup. The spawned scripts all check to see at some point how many of their number are running and once the count has dropped to 2 (I'm grepping so that means only 1 is still running) it should spawn the final script which should only run once.
The variable of the count is gotten at by
Code:
number_running=`ps -ef | grep simple | -wc l`
This number_running is always zero somehow.
Moderator's Comments:
Please use CODE tags when displaying sample input, output, and code segments.
Unless you redirected diagnostic output to /dev/null, one would assume that the code marked in red above would produce a message similar to:
Code:
-ksh: -wc: not found
since most systems do not have a utility named -wc. One would guess that you intended to use:
Code:
number_running=`ps -ef | grep simple | wc -l`
Note that the output produced by wc -l may include leading <space>s (depending on what version of wc you're using). This may make a difference in the way you write the test to determine the value returned. And, although you can look for 2 instead of 1 to account for the grep command being captured by the grep, there are ways to avoid that. One way to do that is to add another grep to filter out the grep:
but that is rather inefficient. A preferred way to do that is to use a BRE that will match the lines you want to match and avoid matching the grep command too:
Code:
number_running=`ps -ef | grep '[s]imple' | wc -l`
which will just return the number of processes with that contain the string simple in the initial parts of their argument list.
Ok...I ran a test and on my mac, everything works as it should, but not when it's spawned via dbms_scheduler on exadata....
This is the logic (it works)
This is the template:test.ksh
Code:
#!/bin/ksh
set -x
export PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin
MAIN_DIR="/var/root/scripts"
cd $MAIN_DIR
DATE=$(date)
HOSTNAME=$(hostname)
FILE_CFG=$MAIN_DIR/simple_INSTANCE.cfg
FILE_LOG=$MAIN_DIR/simple_INSTANCE.log
sleep $(print $((RANDOM%100+1)))
typeset -i number_running=`ps -ef | grep runit_ | grep -v grep | wc -l`
if [[ $number_running -eq 1 ]]
then
echo $number_running > $FILE_LOG
nohup ./run_once_last.ksh &
fi
This is the cfg file (simple.cfg)
Code:
NCDP x y z
NCDT q r s
EDBP a b c
JUNO d e f
This code reads the template and creates as many ksh's and spawns as exists in the cfg (runit.ksh)
Code:
#!/bin/ksh
set -x
export PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin
MAIN_DIR="/var/root/scripts"
cd $MAIN_DIR
DATE=$(date)
HOSTNAME=$(hostname)
FILE_CFG=$MAIN_DIR/simple.cfg
FILE_LOG=$MAIN_DIR/simple.log
function doit {
sed 's/INSTANCE/$1/g' < test.ksh > runit_$1.ksh
echo ${arr_cfg[@]} > runit_$1.cfg
chmod +x runit_$1.ksh
nohup ./runit_$1.ksh &
}
cat ${FILE_CFG}|grep -v '#'|while read PARAMS
do
set -A arr_cfg $PARAMS
doit ${arr_cfg[0]} ${arr_cfg[@]}
done
Here's one genned:
Code:
#!/bin/ksh
set -x
export PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin
MAIN_DIR="/var/root/scripts"
cd $MAIN_DIR
DATE=$(date)
HOSTNAME=$(hostname)
FILE_CFG=$MAIN_DIR/simple_$1.cfg
FILE_LOG=$MAIN_DIR/simple_$1.log
sleep $(print $((RANDOM%100+1)))
typeset -i number_running=`ps -ef | grep runit_ | grep -v grep | wc -l`
if [[ $number_running -eq 1 ]]
then
echo $number_running > $FILE_LOG
nohup ./run_once_last.ksh &
fi
This is run_once_last.ksh
Code:
#!/bin/ksh
set -x
export PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin
MAIN_DIR="/var/root/scripts"
cd $MAIN_DIR
DATE=$(date)
HOSTNAME=$(hostname)
echo imdone > imdone.txt
It appears to work here, but when I schedule the job to run in oracle's dbms_scheduler, which kicks off the base job, everything works except the count check which would kick off the run_once_last.ksh
Obviously this is just an example, but it's all about attempting to run all oracle instances rman backups simultaneously but only run one tape backup, and that after the last backup that finishes completes....
BTW, I appreciate the sounding board. I'm not much of a scripter, I just hack away...
There are lots of scripts involved here. All of them include set -x to enable tracing, but you haven't shown us any of the results of those traces.
You say it doesn't work, but you don't explain the symptoms of how it fails? Is the count falling to 0 without kicking off the last job? Does the count never fall below 2?
After you start this job, what does the output from ps -ef look like at some point before your code fails? (Is there some oracle job running with an argument that contains the string you're counting?)
Why use nohup ?
Why generate scripts from scripts ?
A simple while loop reading one configuration file in one script should
Spawn the required rman backup processes in background.
After the while loop has finished (spawned the processes), using wait, wait for those to finish.
After that line, a invoke a backup to tape.
This is most rudimentary, and it will not check success of background processes.
But the code you posted only counts processes, so i guess that is not the requirement.
As i see it, you wish your program to know if the processes which backup database on some filesystem are done or do not exist anymore, then issue a tape backup on that filesystem.
Why use nohup ?
Why generate scripts from scripts ?
A simple while loop reading one configuration file in one script should
Spawn the required rman backup processes in background.
After the while loop has finished (spawned the processes), using wait, wait for those to finish.
After that line, a invoke a backup to tape.
This is most rudimentary, and it will not check success of background processes.
But the code you posted only counts processes, so i guess that is not the requirement.
As i see it, you wish your program to know if the processes which backup database on some filesystem are done or do not exist anymore, then issue a tape backup on that filesystem.
Is this correct ?
Regards
Peasant.
That's an excellent suggestion. I'm already using a wait. I'll try that monday. That would simplify things. Oh, and the scripts spawned check their own success by checking the logs they've produced, grepping for errors and emailing a success or failure message. The only failures I've run into of late is with netbackup's media manager hiccoughing and trying on a second channel. Since we use ASM, we can only push to netbackup, the people here didn't want to get the zfs backup appliance and there's no nfs mount which can handle the backups not to mention the 1gb pipe we have to back up tens of terabytes of data. So I'm having to use a single channel per node to back up to asm, compress this and then push to netbackup. If I ramp up 8 channels per node I end up creating hundreds of pieces in backup sets and then netbackup's media manager treats each piece as a separate job (no multiplexing this way) and there's the positioning/repositioning which greatly slows down the backup to tape (which is the biggest choke point). ....I added that FYI as a background story...
I have a long running ksh script that I need to run with "nohup" in the backgound which is all well and good but at the very start of the script it needes to output to the screen to query the user and accept a response before continuing.
Any thoughts on how to accomplish this other than... (11 Replies)
Dear Members,
I have a table REQUESTS in Oracle which has an attribute REQUEST_ACTION. The entries in REQUEST_ACTION are like, ME, MD, ND, NE etc.
I would like to create a script which will will call other scripts based on the request action.
Can we directly read from the REQUEST_ACTION... (2 Replies)
So this one just plain confuses me. I have a bunch of somewhat CPU intensive processes that all communicate using a shared memory region. Some of these programs are threaded and some also change the scheduling to FIFO or round robin. The good news is that everything works as long as I spawn... (3 Replies)
Hi,
I wanted to know the exact behavior of the shell scripts in the below scenario.
I have 2 scripts - a.ksh and b.ksh
Both these scripts call a 3rd script xyz.ksh.
What will happen if both a.ksh and b.ksh run at the same time?
What will happen if a.ksh starts 2-3 seconds before b.ksh?... (3 Replies)
I recently patched a RHEL5 host from 5.2 to 5.5. There are some scripts that simply do a ps and grep to check for itself before proceeding. I personally don't like the way the scripts are written and would have done it differently. Before I can proceed with patching the production servers, I... (12 Replies)
Hi
I am new to this Scripting process and would like to know How can i write a ksh script that will call other ksh scripts and write the output to a file and/or email.
For example
-------
Script ABC
-------
a.ksh
b.ksh
c.ksh
I need to call all three scripts execute them and... (2 Replies)
Im writing a script in the Ksh, as the title suggests.
OK so im sincerely tring to be lazy. Im trying to make a script that will use another file as a sort of variable library
So basically i dont need to include the variables themselves, just want to make a reference to the file, so the... (2 Replies)
Hi,
I want to write a script which should be run only on foreground. Is there any way that the script can check itself whether it was run using nohup or ksh and if the user runs the script using nohup then it should prompt the user to run it using ksh?
If (The user triggers the script using... (4 Replies)
i have scipt which is calling some
other scripts and some built-in utilities like SED, find etc..
i started this script using nohup
(e.g: nohup scriptname &)
now i want to kill this script and all the child processes
of this script.
the problem is, process id of child processes are... (4 Replies)
I'm getting rather frustrated with an interactive script I'm writing.
The script is divided up, with section for setting variable at the top, then functions (which make up most of the script) then basically a line at the end which calls the first function- the program moves between the... (5 Replies)
02-23-2018
