Shell Programming and Scripting

Yes and no. Shell language has something compiled code might lack: architecture independence. A shell script (well: most shell scripts, there might be exceptions) won't care about being run in 32-bit or 64-bit environments, about being run on certain OS versions (try that with a program relying on some shared libraries), etc.

Of course you are not wrong. What was discussed about mixing external programs and shell constructs. Of course you can solve this example completely in awk, but wether you write in awk and call shell functions via the system() subfunction call or you write in shell and call awk as an external program you pay dearly for it because the exec() and fork() calls which are necessary.

As i have understood cfajohnson and perderabo this was their concern, not which tool was faster if you never leave it.

bakunin

If real time is a concern, using shell constructs, instead of external commands, can make a great deal of difference.

An example to keep things in perspective:

If you need to parse a huge file and you need the results fast, would you use awk or the shell's construct to do it?

My personal perspective:

1) I am not very good at memorizing stuff, nor do I think the human brain is optimized for that task. I'm much better at making inferences and deductions instead.
2) I know what parameter substitutions are -- in general. But I can't bring myself to remember the difference between

Code:

${parameter##pattern}
and
${parameter%%pattern}

Everytime I see one of those, I have to open a terminal and make a small test to figure out which is which. And thats only an example, there are many other similar looking constructs that do not give the slightest hint to what they actually do. My memory is weak, and when I can't use my deductive and inferential powers, I end up wasting time figuring out what they do. A classic example, if you only knew minimal bash, how much time do you need to understand what this does:

Code:

path="/home/neked/testfile"
s=${path##*/}

versus:

Code:

path="/home/neked/testfile"
s=$(basename $path)

The second code provides some semantics for you to infer what the code does. The first one relies on your memory. The brief way to sum my point is that parameter substitutions are NOT easily readable.

3) Even if you were a parameter substitution guru, and you used meaningful variable names and comments to make clearer what your parameter substitution tricks do, then future maintainers of the code might not be the same. This bit me a couple of days ago: I had to spend 20 minutes debugging a bash script riddled with those parameter substitution scripts. I estimate I would've spent closer to 5 minutes if the code was written using more obvious external tools (sed, basename, awk). This is about 15 minutes of human time wasted in order to save less than a few milliseconds of CPU time. Especially since the whole script runs only once a night, and does not exceed 0.030 seconds runtime on my modest 7 years old computer.

4) The conclusion for me is that parameter substitution should only be used when and only if the need arises. Anything else is premature optimization at the cost of more developer hours debugging and maintaining the code. If I have a script that takes 4 seconds to execute, which could be optimized into running within less than a second using parameter substitution, that still would not -- on its own -- make a convincing case to use param substitution. For a convincing case to be made, the need to reclaim the additional seconds of CPU time must be established and weighed against the loss in human seconds needed to maintain and develop the code.

I've just substituted all occurrences of parameter substitutions with equivalent sed and basename commands in the script I refer to above, the execution time went up from an average of 0.030 seconds to 0.061 seconds. That means parameter substitution use shaved an average of 0.03 seconds per run. Considering that I've had to spend roughly 15 more minutes understanding the code to debug it, this means the code has to run 15*60/0.03 times to pay off the extra time I invested to debug it. That's 30,000 times. Considering that it runs once a night, this means we'll have to wait more than 80 years!

Even after 80 years, those 0.020 seconds shaved off CPU time per night are not as worthy to me as 15 minutes of my time which I could've spent aternative activity redacted

Ironically, this post sounds more interesting after the censorship.