Shell Programming and Scripting

Without you showing us the contents of /opt/CPsuite-R77/fw1/bin/upgrade_tools/upgrade_export (if it is a shell script) or the source code that was used to produce /opt/CPsuite-R77/fw1/bin/upgrade_tools/upgrade_export (if it is a binary executable) we have absolutely no way to guess at what the argument to it means. It that parameter is used to create an output file, maybe /opt/CPsuite-R77/fw1/bin/upgrade_tools/upgrade_export did create that file for you somewhere (in a directory you haven't searched where cron ran your job). If that parameter names an input file, you absolutely either need to be sure that your script changes its current working directory to that directory before it starts /opt/CPsuite-R77/fw1/bin/upgrade_tools/upgrade_export or you need to provide the name of that file as an absolute pathname.

The common thing about the hundreds of threads started in this forum about a job that works when run on the submitter's shell command line works, but doesn't work when run from cron is that cron does not store your current shell execution environment in crontab and setup that same shell execution environment for the job when it is run. YOU have to make sure that any script you run from cron creates a shell execution environment that uses the proper shell to run your cron job, positions itself in a directory where your script has access to input and output files it needs, and has environment variables set appropriately for any variables the processes started by your script need to operate properly.

Your request is essentially saying my shell script doesn't work. Fix it for me. And, without any details about what your script does, you KNOW that we can't possibly do that other than to make the generic suggestions we have already given you.

Thx again for the response. I don't know how to show the source code for the executable. How do I do that?

I do know that the job is NOT creating the output file anywhere.

"Your request is essentially saying my shell script doesn't work. Fix it for me." - That's not what I'm saying (or not my intent). The script does work. It runs the upgrade_export executable that creates an output file -

Code:

/opt/CPsuite-R77/fw1/bin/upgrade_tools/mgt-svr-bkup-$today.tgz

So I'm not asking to fix the script, I'm wondering why it runs manually and not in crontab. If the reason is I need to set the correct shell execution environment, then at least I have something to work on. But my thought in posting this was maybe there was something obvious I was missing that was the problem. I'm new at this and trying to figure it out.

If you want a simple fix - start with environment variables.

There are two commands: set and env. set is a shell builtin and shows all variables including those visible locally (not exported). env command cannot do that, it sees only those environment variables the process got at process creation. env is a separate physical file not a builtin. So you want to use env to really see what is going on -in your case not going on.


One crude way:
create a one line script called script.shl:

Code:

/usr/bin/env

run it two ways -- (redirect the output of each run to different file names. For crontab /tmp is a handy place) manually, then in crontab

The diff command will see what crontab does to your login variables. Add 'em to your script that runs the problem executable - be sure to export the variables.

To post code use code tags -- these are examples with embedded spaces; html does not interpret them
[ c o d e ]
script text goes here
or FORTRAN, C, C++ ,etc. code goes here between a separate set of code tags for each program file.
[ / c o d e ]

Obviously, remove the spaces from the example to make it work.

Code:

script text goes here
or FORTRAN, C, C++ ,etc. code goes here between a separate set of code tags for each program file.

Look at the threads at the bottom of this page. There are five threads there with exactly the same problem you are having and the same suggestions we have been making here solved the problem for all of them.

When you run the program manually and it works, what shell are you using? Change the 1st line of your script from:

Code:

shell=/bin/bash

to:

Code:

#!pathname

where pathname is the absolute pathname of the shell you are using. Does that change make your script work when run by cron?

If not, when you run the program manually and it works, what directory is your current working directory? What is the output from the command:

Code:

pwd -P

? Add a command to your script before you invoke /opt/CPsuite-R77/fw1/bin/upgrade_tools/upgrade_export that changes your current working directory in your script to that directory:

Code:

cd directory

where directory is the string returned by the above pwd command. After that change, does your script work when run by cron?

When you run the program manually and it works, what environment variables does your program need to run correctly? Does your program work correctly if you invoke it manually with the commands:

Code:

today=$(date +"%m-%d-%y")
env - PATH="/bin:/usr/bin" /opt/CPsuite-R77/fw1/bin/upgrade_tools/upgrade_export mgt-svr-bkup-$today << EOF
y
n
EOF

? If not, what environment variables does /opt/CPsuite-R77/fw1/bin/upgrade_tools/upgrade_export need to run correctly? Find out (using the methods suggested by Jim McNamara) and set those environment variables in your script before you invoke /opt/CPsuite-R77/fw1/bin/upgrade_tools/upgrade_export.

Thanks, Jim!
I created the script /usr/bin/env and ran it manually and in crontab.

The PATH variable in the manually-run output file is long. The PATH variable in the file run in crontab is short - PATH=/usr/bin:/bin. So, thinking I needed to tell cron to use the long PATH variable, I added it into the script: (PATH=/opt/CPsuite-R77/...), but that didn't work when I ran the job from crontab.

Thanks for the response, Don.
"Look at the threads at the bottom of this page. There are five threads there with exactly the same problem you are having and the same suggestions we have been making here solved the problem for all of them."

If that were true, I'd have figured it out by now. The problems in the other threads are "similar", but not "exactly the same". I read through them before I posted my issue. Had I been able to solve my problem using suggestions posted in the other threads, I wouldn't have posted mine.

I get that my problem probably has to do with setting the environment variables, but it's not clear to me yet how to do that.

"When you run the program manually and it works, what shell are you using? Change the 1st line of your script from:

Code:

shell=/bin/bash

to:

Code:

#!pathname

where pathname is the absolute pathname of the shell you are using. Does that change make your script work when run by cron ?"

Since the first line in the script is shell=bin/bash, I'm using the bash shell, right? Changed the first line to #!/bin/bash. Still doesn't work using cron.

I'm running the job from my home directory -

Code:

/home/admin

"? Add a command to your script before you invoke

Code:

/opt/CPsuite-R77/fw1/bin/upgrade_tools/upgrade_export

that changes your current working directory in your script to that directory:

Code:

cd directory

where directory is the string returned by the above pwd command. After that change, does your script work when run by cron ?"

Still doesn't work.

"When you run the program manually and it works, what environment variables does your program need to run correctly? Does your program work correctly if you invoke it manually with the commands:"

Code:

today=$(date +"%m-%d-%y")
env - PATH="/bin:/usr/bin" /opt/CPsuite-R77/fw1/bin/upgrade_tools/upgrade_export mgt-svr-bkup-$today << EOF
y
n
EOF

Yes.

Yes, No, Maybe!
If the default shell for your system is /bin/bash, then having the line:

Code:

shell=/bin/bash

in your script does nothing and your script is being run by bash. If the default shell for your system is NOT /bin/bash, then having the line:

Code:

shell=/bin/bash

in your script does nothing and your script is NOT being run by bash; it is being run by the default shell on your system.

If you type the command:

Code:

shell=/dev/null

into your shell, does the shell you are using magically change to /dev/null? NO. All it does is set a shell variable that later lines in your script (or, when done interactively, in your login session) can access by testing the value of $shell.

By convention (when your login shell is not a csh derivative), when you login the shell variable SHELL is exported and set by the system to an absolute pathname of your login shell. My login shell is /bin/ksh and after I login, $SHELL expands to /bin/ksh. If I then invoke bash, $SHELL still expands to /bin/ksh. Note also that case matters: $SHELL and $shell are two different variables.

If the script you are asking cron to run is an executable file and the 1st two characters in that file are #!, the system will use the interpreter named by the pathname on the rest of that line as the program used to interpret the rest of that file.

Now, back to your environment problem...
If you don't know what environment variables /opt/CPsuite-R77/fw1/bin/upgrade_tools/upgrade_export needs to have set in order for it to work correctly, you need to set all of the variables in your script to the same values that they have when you run /opt/CPsuite-R77/fw1/bin/upgrade_tools/upgrade_export manually and it works.

When you login with bash as your login shell, it runs three files from your home directory if they are present: .bash_profile, .bash_login, and .profile. If those files are where the variables /opt/CPsuite-R77/fw1/bin/upgrade_tools/upgrade_export needs to run are defined, the following might work for you. Change:

Code:

shell=/bin/bash

in your script to:

Code:

#!/bin/bash
HOME="/absolute/path/to/your/home/directory"
cd "$HOME"
[ -f .bash_profile ] && . .bash_profile
[ -f .bash_login ] && . .bash_login
[ -f .profile ] && . .profile

One would expect that one or more of those three files initializes PATH (although it appears that your script doesn't depend on any utilities that are not on the default PATH that you aren't referencing with absolute pathnames) and other shell variables to the values you'll need to run /opt/CPsuite-R77/fw1/bin/upgrade_tools/upgrade_export, but we can't know that for sure since we have no idea what other commands you have run after you last logged in to your system before you successfully run /opt/CPsuite-R77/fw1/bin/upgrade_tools/upgrade_export from your interactive shell. And, as we have repeatedly said, without knowing what /opt/CPsuite-R77/fw1/bin/upgrade_tools/upgrade_export is doing, we are just guessing. If /opt/CPsuite-R77/fw1/bin/upgrade_tools/upgrade_export tries to open your controlling terminal for some reason, it might just not be possible to run /opt/CPsuite-R77/fw1/bin/upgrade_tools/upgrade_export from cron at all.