Shell Programming and Scripting

I have a script which outputs some timing data a line at a time. There are approx. 10 lines echoed, each line looks something like this:

Code:

0.741 http://checkip.dyndns.org 94.170.119.226

Since I needed to add all the values in the first column, I piped the output to grep, matching and printing the first numerical column, then piped it on to tr to replace new lines with '+', then piped to sed to replace the final '+' with nothing, and finally to bc to add the values. Then bc gave me a stdin syntax error - damn it, maybe I'm not quite so clever as I thought I was.

On examining the code I discovered that if I left off the final pipe to bc I got what I thought I should have - that being:

Code:

$ getip test | grep -o "^[0-9.]*" | tr "\n\r\f" "+" | sed "s/\(.*\)+/\1/" 
0.409+0.087+0.116+0.294+0.214+0.342+0.595+0.232+0.380+0.494 [Note: no new line, prompt is here as expected]

$ echo 0.409+0.087+0.116+0.294+0.214+0.342+0.595+0.232+0.380+0.494 | bc
3.163

So I copied and pasted those values into an echo statement, piped to bc, and it worked fine as shown above.

As another test I ran the command again, this time assigning the result to a variable, and then piped the variable to bc. That also worked fine, see code below (the top line has the error shown when piped to bc):

Code:

$ getip test | grep -o "^[0-9.]*" | tr "\n\r\f" "+" | sed "s/\(.*\)+/\1/" | bc
(standard_in) 1: syntax error

$ var1=$(getip test | grep -o "^[0-9.]*" | tr "\n\r\f" "+" | sed "s/\(.*\)+/\1/")

$ echo $var1
0.332+0.111+0.081+0.283+0.155+0.355+0.607+0.297+0.382+0.452+0.294

$ echo $var1 | bc
3.349

What I want to know is why I get the syntax error from bc when I run it as one line? Any ideas?

Many thanks.

PS. The somewhat simpler alternative works with no problems.

Code:

$ getip test | grep -o "^[0-9.]*" | awk '{sum+=$1} END {print sum}'
2.999

I suspect it has something to do with extra spaces being stripped out by the echo.

This'd be far simpler to do with awk.

Code:

getip | awk '{ T+=$1 } END { print T }'

FWIW - awk was made precisley for what you are doing

Code:

getip test | awk '{sum += $1} END {print sum}'

By default awk does double precision arithmetic (floating point with 15 significant digits)

Corona688 and jim mcnamara: Did neither of you see the PS in my post !! I know I failed to realize that $1 would give me the first column (I assumed that would be the whole line). It's not that I can't do it, just that I want to understand what I am doing wrong with the pipe to bc. Anyone know?

Corona688: There are no extra spaces, or any spaces at all come to think of it.

Thanks guys. Cheers.

I always endeavour to answer the question, and the actual problem. Seeing your ps actually encouraged me to suggest more awk things.

It may be the complete lack of a newline that causes bc to fail, too. echo will add one on the end, but just piping it in would end up with nothing. Many commands are picky about lines without newlines -- some versions of awk and sed, for instance.

how about:

Code:

getip test | grep -o "^[0-9.]*" | tr "\n\r\f" "+" | sed 's/+$/&0/' | bc

Explanation of the error.
Your "tr" command strips out all line terminators (\n) for some unknown reason and converts them to "+" signs. My version of "sed" will not process lines without a line terminator, but yours appears to work (to my surprise).

The "bc" program fails because the input stream is not terminated with a linefeed character (\n).

When you "echo" the variable $var1 the "echo" statement appends a linefeed character (\n).

Btw. The unix standard version of "tr" will not translate three different characters into one. The length of the translate list must be the same both sides.

Code:

echo "xyz" | tr 'xyz' '+'
+yz
echo "xyz" | tr 'xyz' '+++'
+++

In your script this would only matter if there were carriage-return or form-feed characters in the line (which there are clearly not).