Shell Programming and Scripting

Hello, I am writing a script in ksh to output a data file that is pipe delimited (not my choice). In any case, it has me scratching my head since the output is not as expected.

Code:

exec 3<acc.txt   # contains 3 digit number, 1 per line
exec 4<ssid.txt  # contains 8 digit numbers, 1 per line

set -A acc_f "Y|Y|Y|Y" "N|N|N|N" "Y|N|Y|N" "N|Y|N|Y"
set -A stype L S R N

while read -u3 acc; do
  for i in 0 1 2 3; do
      read -u4 ssid
      echo "$ssid|2012|$acc|${acc_f[$i]}|${stype[$i]}"
  done
done

The output ignores "$ssid|2012|$acc" for some reason. I can print them separately if I include some echoes before that line but that line only prints the right half starting from "|${acc_f[$i]}...". Does anyone have any ideas what is going on?

Thanks,
Ben

Moderator's Comments:

Please view this code tag video for how to use code tags when posting code and data.

First off, it would have been nice to tell the interested audience which shell you are using. I take it, from the "read -u" command, which is not there in bash, that you are using a ksh (probably ksh93), but instead of letting me hone my investigative talents it is generally more fruitful to describe your environment (this includes the OS you are on - crystal balls are in short supply nowadays).

And while i am at it: giving the thread a meaningful title would also have helped. Something like "display variables after reading from file" would probably have gotten more answers in shorter time.

You should probably use "print" instead of "echo" - "print" is a builtin and offers more possibilities. But this only as an aside.

I suppose it is because "|" is a special character to the shell. Try the following, which is a bit safer:

Code:

print - "${ssid}|2012|${acc}|${acc_f[$i]}|${stype[$i]}"

I hope this helps.

bakunin

Firstly, thanks for the response. While I did mention I was using ksh I should have mentioned that I am using both ksh93 and cygwin. I am assuming there is something not right with this combination. When I substituted your print line, the output I got inexplicably had the fields in the wrong order: |N|Y|N|Y|N|2012|901

I.e. it wrote the right half of the print first and skipped the ${ssid} completely before printing the left side. I just rewrote it in awk in my frustration and it worked fine.

In any case, I appreciate your help and scripting advice.
Thanks,
Ben

As a quick update to this problem. I ran my ksh93 script at home on CentOS and had the same puzzling results. I did a od -cx on the files after a quick look at the results, what stood out was the \r\n in the files. I did a dos2unix on the files and the script now works. I presume this is the result of working with Windows based cygwin. The results were still quite unexpected. I thought I might update this info here as it I guess pays to check what is really contained in your data files.

Thanks to the carriage returns, it only looked like it wasn't being printed. When the shell read the lines into the variables, using the \n as the delimiter, the string assigned to the variables ended in \r. So, when you were echoing them, the embedded \r in the string was causing the teriminal's cursor to return to column1 and overwrite what had been there.

Once you've been bitten by this, you tend to remember it.

Thank you for reporting back with your findings. At some point in the future, someone may find it useful.

Regards,
Alister

Thank you for posting the solution. It is a common problem which appears daily on unix.com. Mainstream unix uses line-feed as a record terminator in text files whereas MSDOS (and thence Microsoft Windows) uses the two characters carriage-return line-feed.

This is a long-standing design issue with Microsoft Operating Systems. It does mean that in early MSDOS you could output a text file to a printer port (with no proper printer driver) and it would print ok.

If you want to quickly check a file, this command will make line terminators visible :

Code:

sed -n l filename.txt

Absolutely right.

Another way is in vi to enter ":set list" in command-mode. This makes all non-printable characters (line feeds, tabs, etc.) visible. Switch off with ":set nolist".

Thanks go to the threads O/P for writing a follow-up and bothering to share his solution. Thanks also to alister for the explanation.

I hope this helps.

bakunin