UNIX for Dummies Questions & Answers

We are facing a problem with PIPE (|) as a delimiter in one of our FTP flat files.

Earlier we used comma (,) as the delimiter and never faced such an issue. Now we have changed it to PIPE and are in this situation.

Is there anything to do specifically with PIPE when transferring files between Mainframe & UNIX?

Is there any work around for this without changing the delimiter symbol from PIPE to some another?

Probably the reason for this is some ASCII-EBCDIC conversion problem: Unix machines, including AIX machines, encode files in ASCII (American Standard Code for Information Interchange) character sets. Not so the IBM-hosts, as they traditionally use a different character set called EBCDIC (Extended Binary Coded Decimal Interchange Code).

Usually there is some sort of conversion table, which tells the conversion program what character number x is to be replaced with in the other character set. Probably this conversion table is faulty at the place of the pipe character. I would bet on "M-3" not being a 3-character string like its representation here in this text but in fact some single-char non-printable character.

You can either correct this conversion process or run sed (or tr) over the file to correct the characters:

Code:

tr '<bad-char>' '|' <sourcefile> > <corrected_file>
sed 's/<bad-char>/|/g' <sourcefile> > <corrected_file>

I hope this helps.

bakunin