Shell Programming and Scripting

Hello,

I do not know why the output from these two methods differs. One method retains the newlines, the other method appears to ignore or lose the newlines.

Writing a file with the redirection operator:

egrep -e 'matchstring' infile.txt > outfile.txt

The resulting outfile.txt contains separated line when viewing the file.

However, when I do this method (with the goal of having all the output lines in a variable called OUTFILE), the resulting text appears to have lost all the line breaks:

OUTFILE=$(egrep -e 'matchstring' infile.txt)

Thank you!

The reason is that several characters - newline among them - have special meaning to the shell and are therefore interpreted instead of used like "normal" characters.

The general way to protect these special characters - that is, make the shell treat them like they were normal ones - is to enclose a string into double quotes:

Code:

myvar="foo bar"      # works
myvar=foo bar        # will not work because the space after foo is interpreted
                     # as end-of-statement and that leaves "bar", which the shell
                     # cannot understand.

In your case you have to protect your variable two times: first when you fill it with content, second when you print it out. The reason is that the shell interprets the commands in a certain step-by-step process: in one step all the variables are replaced by their content and in the next step the command is interpreted. To understand what i mean make the following test:

Code:

myvar="-lisa a*"
ls ${myvar}
ls "${myvar}"

With the first command you will see the same directory listing format as if you had entered "ls -lisa a*" directly. The reason is that first the shell replaces "${myvar}" with its contents and only then executes the resulting line as the string "-lisa a*" was not protected it splits into two distinct strings, the first one being "-lisa" and the second one being "a*". This is exactly what the ls command expects.

The second command will give a strange error message. Because we protected the string it is still one string, the space character included. The ls command doesn't have any option "a*" (this can only be a parameter) and it doesn't have an option " " (space) either! This is where the error message will come from.

You see this mechanism of protecting (or not protecting) variables can be used both ways. Here is your solution:

Code:

OUTFILE="$(egrep -e 'matchstring' infile.txt)"    # notice the double quotes

echo "$OUTFILE"            # with newlines
echo $OUTFILE              # newlines replaced by spaces

I hope this helps.

bakunin

Code:

IFS=""
OUTFILE=$(egrep -e 'matchstring' infile.txt)

You peeps are awesome!
Thank you for the shell quoting refresher course!

Cheers!