Shell Programming and Scripting

This isn't a question--its a solution! Below is a script that I wrote for my own script file development which does what the title says. Its the closest that you can get to compiling what are otherwise purely interpreted script files. I offer it here simply for the benefit of anyone else writing script files. (That, plus I would like feedback if anything is wrong or suboptimal with it.)

Personally, I do not understand why sh even has the -n option: I would have thought that it should always be done. It gives you more safety, detecting known problems before damage can be done. Was it made an option because in the stone ages when unix was written, computers were so slow that automatic syntax checking would have been a major performance impact?

Regardless, here's the script:

Code:

#!/bin/sh
set -e	# exit on first failed command
set -u	# exit if encounter never set variable


#--------------------------------------------------
# Checks the syntax of either a single shell script file,
# or every shell script file in a directory,
# or every shell script file in an entire directory tree.
#
# Usage:
#	sh checkSyntax.sh [-p path] [-R]
#
#	-p	if supplied, then requires a value that is either the path to a single .sh file
#		or the path to a directory;
#		if omitted, then the current working directory will be searched
#	
#	-R	if supplied, and if the path to be searched is a directory,
#		then also searches subdirectories for .sh files
#
# +++ KNOWN BUGS:
# --the find/while loop below screws up if any element in a path contains leading whitespace
#
# +++ In the future:
# --there are many more options from the find command (e.g. how symbolic links are handled--currently they are not followed)
# which might want to expose as command line arguments of this script.
#--------------------------------------------------


# Initialize option variables to preclude inheriting values from the environment:
opt_p="./"	# the path to search for the find command used below; default value is the current working directory
opt_R="-maxdepth 1"	# the maxdepth option for the find command used below; default value limit the search to just path itself (i.e. do not drill down into subdirectories)

# Parse command-line options:
while getopts 'p:R' option
do
	case "$option" in
	"p")
		opt_p="$OPTARG"
		;;
	"R")
		opt_R=""
		;;
	?)
		echo "Script will now quit with an exit code of 1"	# Note: if supply an invalid option, getopts should have already printed an error line by this point, so no need to duplicate it
		exit 1
		;;
	esac
done

# Find all the relevant .sh files and check their syntax:
find $opt_p $opt_R -type f -iname "*.sh" | while read shFile	# see Note below...
do
	( sh -n "$shFile" )	# sh -n will merely syntax check (never execute) shFile, and will print to stdout any errors found; encase in parentheses to execute in a subshell so that if a syntax error is found, which causes sh -n to have a non-zero exit code, then this parent shell does not see it; critical since it will stop executing (due to the set -e line at the top of the file) otherwise
done

# Note: originaly used this line instead of the first line above:
#	for shFile in `find $opt_p $opt_R -type f -iname "*.sh"`	# to understand this line, execute "man find"; was inspired by this script: Bounty: A recursive .M3U index generating shell script | debianHELP
# Dropped it because it fails on path elements which contain whitespace.
# The solution above is discussed here: https://www.unix.com/shell-programmin...e-between.html

[Note to the people running this website: I received the following email over a day ago that someone responded to my original posting. And yet, it has never appeared here! I always see just 0 replies. Whats up? Is this website buggy?

When I try clicking on the "Contact us" link on the bottom of the page, I get an "Invalid Forum specified" error message, and when I try to click on the administrator link within that error message I get the same error message again.

If someone reading this knows how to contact the technical people running this website to notify them of these bugs, I would be grateful.

Since my posts do seem to be appearing on these forums, I am going to copy the text of the email that was sent to me and paste it below:]


ok, since you have gone to the trouble to post this, I will give some feedback which would make this more usable/portable. Currently it will not work on a stricter POSIX implementation of Unix due to your use of extented option from GNU find.

1. Your set -u is doing nothing at all since you have defaults for the only variables you use.
2. Use either a variable set to the full path (preferable) for a full path to programs you run.
eg
Code:
---------
FIND=/usr/bin/find
...
$FIND $opt_p $opt_R -type f -iname "*.sh"...
---------

3. Use POSIX arguments to find, it takes more work but is portable.
4. Instead of saying that the script will exit, print a usage message, you have that in the comments, but a usage message is better.

As an improvement, instead of assuming bourne/bourne compatability, test for a magic number in the first line of the script and run the <script interpreter> -n based on that, if there is none, then by all means default to using bourne.

I do not understand your objection. Of course it does nothing for the script that I posted, because it has now been debugged and is working fine. The purpose of it is to detect any issues if some maintenance programmer comes along and mucks with it.

Personally, I think that it should always be an error to use an uninitialized variable; if I had written sh, it would not even be an option. Neither would -n. (Both would always be on.) But I have to work with whats there.


First, why would I ever want to use the full path to a command? That makes it totally unportable! The whole purpose of something like the PATH env var is to get around issues like this.

Second, you did not complete your thought: what is the or clause?


I hear you, and wish that I could obey, but after looking into the paucity of stuff actually supported by POSIX find I gave up. It does not support any of the functionality that I really want this function to have, like recursive finding in a directory tree and case insensitivity. If you know of an elegant way to do those things I am open, but I really do not want to write all of them inside this function.

When I did a "man find" just now on a linux box, it did note that many of the non-POSIX options that I am using are not unique to GNU's find but are implemented on other systems too, somaybe it is mostly portable...


Rant mode on: there is nothing more annoying than the lazy unix programmer habit of responding to an input error with the correct usage.

I usually have already read the man page or whatever before using the command, so that usage message tells me NOTHING.

What I really need to know is what--out of the million things that could possibly be wrong--is the actual wrong thing with my input, not what the correct generic form is.

If a command needs exactly 7 args and I have supplied 8, dang it, tell me that I have 1 extra argument as the error message--don't tell me
I could waste minutes staring at my input trying to determine what is wrong until I realize what the problem is that the error message should have told me in the first place.

This unix practice is done solely, as near as I can tell, because it makes the programmer's job slightly easier. But it is bad for the user.


I am slightly confused by your request. My function currently only looks for *.sh files, so why should I not assume that they are bourne shell scripts? Do people routinely write korn shell, c shell, etc script files and use the same .sh file extension for all of them, instead of using something sensible like .ksh, .csh, etc?

I do not even know if other shells support the -n syntax check option, so I would just as soon ignore all script files except those purporting to be bourne shell scripts.

Would it be a better idea to abandon searching for *.sh files, and instead look at every normal file and read its first line and only assume that it is a bourne shell script if the first line starts with "#!/bin/sh"?

OK, that explains it.

But they still need to fix their support links--those have been broken for weeks now; I have tried contacting them thru other means, but no response.

Usually replies which has a URL posted in it are subject to moderator's approvals. Perhaps, that particular reply may have had an external URL. Also like porter mentioned, we lost about a couple of hours of post.

Thanks for reporting this. We will have the admins look into this.

You are entitled to thinks so. I disagree but that is a question of opinion, by all means do that. I just said it was unnecessary in this case.

Sorry I didn't follow you here.

The purpose of PATH is to tell the shell in which directories and in which order to look for commands. A properly written script would not normally depend on login environment to know that. There are some exceptions, but I really don't see a reason for having one here. The script will currently break if the user has a different command (or different non-pompatible version of a command) with the same name as one of the ones you use in their PATH before the one you want; or has a command aliased to behave in a way you don't expect which would be worse. If you really want to use PATH, include it in the script, don't depend on the user to set it.

I did, but you didn't receive it.

What I said was, what is preferable if you don't want to use full paths directly use a variable, which I always do by the way, and it is easier to port also.

for eample:

Code:

FIND=/usr/bin/find
...
$FIND $opt_p $opt_R -type f

Also doing it in that way if the command you are using has a verbose option so for example 'tar' you could do:

Code:

TAR="/usr/bin/tar v"

and turn on verbose output for all instances of the command in the script while debugging.

Then to port it is a simple matter of updating the variable if required, or adding a clause for each OS you expect to run on for any non-shared paths.

I'm guessing you mean non-recursive, since it recurses by default:

Code:

find <dir> \( ! -name <dir> -prune \)

Case insenitivity you can do with grep -i or more simply :

Code:

-name "*.[sS][hH]"

Maybe...Solaris and HPUX for example do not.

It has little or nothing to do with laziness. It is more effort for the programmer to write the message, and saves the user a lot of time if they forget the order of the arguments, there are a lot of characters in a man page and some people use slow connections. Most users I have dealt with much prefer to run a commmand with no arguments or -h and get a usage message. Maybe the -h options solves it from either POV. I am not saying don't print the message that says what is wrong, just display the generic form as a reminder too or tell the user how to do so, this kind of thing:

Code:

# check.sh -B
Illegal option -B
    Try `check.sh -h' for more information

Yes.

bourne derived shells such as ksh and bash do.

You could if you wanted to. Something like this inserted in the find would do it:

Code:

find ..... -exec awk ' FNR==1 && /\/sh$/ { print FILENAME; quit}' \; | .....

Mechanism, not policy.

Mechanism, not policy.

So what was that you said about portability?

Normally they

(a) print usage

(b) say what option was offending

(c) do not try and read your mind.