Shell Programming and Scripting

I'm using while read in a script to create a file but when I paste on the screen it echos out different than the data. The file is created correctly and the script does as it should. Just trying to resolve what's being show on the screen.

Code:

    while read FILES
        do
        echo $FILES >> $UPLOADDIR/DATA 2>>/dev/null
        if [ "$FILES" = "" ];then
          break
        fi
        done

Example:

Pasted data:

Code:

dir1/dir2/dir3/dir4/file1
dir1/dir2/dir3/dir4/file2
dir1/dir2/dir3/dir4/file3
dir1/dir2/dir3/dir4/file4
dir1/dir2/dir3/dir4/file5
dir1/dir2/dir3/dir4/file6
dir1/dir2/dir3/dir4/file7

What shows up on the screen:

Code:

dir1/dir2/dir3/dir4/file1
dir1/dir2/dir3/dir4/file2
dir1/dir2/dir3/dir4/file3
dir1/dir2/dir3/dir4/file4
dir1/dir2/dir3/dir4/file5
dir1/dir2/dir3/dir4/file6
dir1/dir2/dir3/dir4/file7
dir1/dir2/dir3/dir4/file2
dir1/dir2/dir3/dir4/file3
dir1/dir2/dir3/dir4/file4
dir1/dir2/dir3/dir4/file5
dir1/dir2/dir3/dir4/file6
dir1/dir2/dir3/dir4/file7

Moderator's Comments:

edit by bakunin: Please use code tags next time for your code and data.

There seem to be some misunderstandings about how the shell works:

You haven't told us which shell and which OS you use. If it is bash, a "\n" at the end of your output is implied and your construction is just superfluous, in Korn shell you should use the "print" command instead of the "echo" and your construction would still be superfluous.

The only thing you read is a stream of filenames and you do absolutely nothing with it, just put it into another file. You could do that without using a while loop, yes? Replace:

Code:

<some-stream-of-filenames> | while read FILES
        do
        echo $FILES >> $UPLOADDIR/DATA 2>>/dev/null
        if [ "$FILES" = "" ];then
          break
        fi
done

with

Code:

<some-stream-of-filenames> >> $UPLOADDIR/DATA

Another questionable thing is:

Code:

echo $FILES >> $UPLOADDIR/DATA 2>>/dev/null

You don't need to append to "/dev/null", because it has no content that needs to preserved (in fact no content at all), therefore:

Code:

echo $FILES >> $UPLOADDIR/DATA 2>/dev/null

Further, which output of "echo" should go there? There is none, therefore the redirection is superfluous:

Code:

echo $FILES >> $UPLOADDIR/DATA

Another thing is that filenames can contain whitespace in Unix. Whitespace is the field delimiter in shell, which is why it is wise to protect strings which could contain whitespace from the interpretation by the shell. Therefore:

Code:

echo "$FILES" >> "$UPLOADDIR/DATA"

Furthermore, you use a "break" to leave the loop once input is exhausted:

Code:

        if [ "$FILES" = "" ];then
          break
        fi

This is completely unnecessary, because you use "while read ..." "read" will return "FALSE" if the input ends and therefore the loop will be left automatically.

Finally: You always append to your output file ("$UPLOADDIR/DATA"). If you run the script several times you will have the output of your stream several times in your file. If you want to have only the results from one run in your file you will have to empty it prior to the loop, like this:

Code:

cat /dev/null > "$UPLOADDIR/DATA"
<some-stream-of-filenames> | while read FILES
        do
        echo $FILES >> $UPLOADDIR/DATA 2>>/dev/null
        if [ "$FILES" = "" ];then
          break
        fi
done

But again, the loop does nothing and a

Code:

<some-stream-of-filenames> > "$UPLOADDIR/DATA"

would do the same. In case you want to process the filenames in the loop somehow and you use Korn shell you should do it this way:

Code:

exec 3> "$UPLOADDIR/DATA"       # to create a new empty file
# exec 3>> "$UPLOADDIR/DATA"    # alternatively to append to the file

<some-stream-of-filenames> | while read FILES ; do
     print -u3 "$FILES"
done

exec 3>&-

I hope this helps.

bakunin

There is nothing in the script that is shell or OS specific, so the shell and the OS are both irrelevant.

In all shells, echo adds a newline.

Neither echo nor print is a good idea.

Use printf; it is the only portable method.

Across implementations in different OSes ksh might either be ksh88 or ksh93, which exhibit differences regarding the treatment of "echo". Furthermore, if "/bin/echo" is found in the PATH, ksh treats its own built-in differently, even if it is there. With bash the situation may or may not be that complicated, but i lack experience with all bash-implementations there may be, so i can't say for sure.

Therefore my opinion is that knowing the OS and the shell thread-opener talks about may or may not matter. I can only decide once i know these, not beforehand.

In some shells "echo" is not a built-in but a call to "/bin/echo". If the implentation of each respective OS does or doesn't add a newline i don't know. I haven't used each and every Unix-like system there is.

With Korn shell "print" is a built-in and is preferred. (see here) With bash "echo" is a built-in and is probably preferred too (not so sure as i don't use bash more than i have to, but most bash scripts i have seen use "echo" for that purpose).

To ask for "portability" in general without specifying where you would want to port it is completely arbitrary: in general it is a good idea to write scripts in a way so that they work on as many systems as possible with as few changes as possible. To ask for a ksh script that it has to work in bash (or vice versa) makes about as much sense as asking for a FORTRAN program to compile on a C-compiler without error.

Again, i don't say portability should not be an issue. But "portability" as a goal makes only sense if there is a reference platform/shell/environment defined to which it should be portable. Without such a definition it is just a buzzword.

Probably you imply POSIX as this reference here and mean "portability towards the POSIX-specification", which is as good a goal as any else. If the thread-opener has POSIX-compatible systems and might face the necessity to port the script from one of these POSIX-compatible systems to another then you are right - on the other hand, if he has another set of systems he might be better off to enforce compatibility between the systems he actually has than towards a generic standard some of his systems do not adhere to.

Insofar it would also be a good idea to know more about his actual environment instead of declaring this "irrelevant" like above.

I hope this helps.

bakunin