Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Encoding Problem while using "|" (PIPE) as delimiter from Mainframe to Unix Post 302169037 by bakunin on Wednesday 20th of February 2008 04:36:20 AM
Old 02-20-2008
Probably the reason for this is some ASCII-EBCDIC conversion problem: Unix machines, including AIX machines, encode files in ASCII (American Standard Code for Information Interchange) character sets. Not so the IBM-hosts, as they traditionally use a different character set called EBCDIC (Extended Binary Coded Decimal Interchange Code).

Usually there is some sort of conversion table, which tells the conversion program what character number x is to be replaced with in the other character set. Probably this conversion table is faulty at the place of the pipe character. I would bet on "M-3" not being a 3-character string like its representation here in this text but in fact some single-char non-printable character.

You can either correct this conversion process or run sed (or tr) over the file to correct the characters:

Code:
tr '<bad-char>' '|' <sourcefile> > <corrected_file>
sed 's/<bad-char>/|/g' <sourcefile> > <corrected_file>

I hope this helps.

bakunin
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Commands on Digital Unix equivalent to for "top" and "sar" on other Unix flavour

Hi, We have a DEC Alpha 4100 Server with OSF1 Digital Unix 4.0. Can any one tell me, if there are any commands on this Unix which are equivalent to "top" and "sar" on HP-UX or Sun Solaris ? I am particularly interested in knowing the CPU Load, what process is running on which CPU, etc. ... (1 Reply)
Discussion started by: sameerdes
1 Replies

2. UNIX for Dummies Questions & Answers

Unix "at" / "Cron" Command New Problem...Need help

Hi All, I am trying to schedule a one time job using the at command with the help of shell script for my project. The shell script should take a parameter as a command line argument from the at command itself. Is it possible to take a command line parameter for a shell script in the command... (3 Replies)
Discussion started by: Mohanraj
3 Replies

3. UNIX and Linux Applications

A question/problem about oracle "tns listener" and "enterprise manager"

hi, I have * an IBM P550 machine, * an AIX 5.3 running on it and * an oracle database, already installed on it. The problem (or question of my own) is: Oracle tns listener, "CT_LISTENER", and the enterprise manager (EM) of the instance, which is uniq instance and called... (0 Replies)
Discussion started by: talipk
0 Replies

4. UNIX for Advanced & Expert Users

A question/problem about oracle "tns listener" and "enterprise manager"

hi, I have a problem about the Oracle related components. I'm not able to find any answer yet, and waiting for your responses... Here is the configuration of my system: * an IBM P550 machine, * an AIX 5.3 running on it and * an oracle database, already installed on it. The problem (or... (1 Reply)
Discussion started by: talipk
1 Replies

5. Programming

fork&pipe "interpretting" shell - problem

hello everybode.Got some sort of "problems" with this stuff; well this is a program int main() { int Pipe; int origStdin, origStdout; int childPID; origStdin = dup(0); origStdout = dup(1); pipe(Pipe); if( (childPID = fork()) < 0 ) { perror(... (2 Replies)
Discussion started by: IdleProc
2 Replies

6. UNIX for Dummies Questions & Answers

"vi" text editor character encoding?

Hi! I've got a shell account on a FreeBSD machine. It doesn't have 'vim' installed, but only the original 'vi' text editor ("Version 1.79 (10/23/96) The CSRG, University of California, Berkeley.") So, in PuTTY I've chosen "UTF-8 translation" to have my non-english characters appear correctly.... (2 Replies)
Discussion started by: Gew
2 Replies

7. Shell Programming and Scripting

sort text having delimiter with "|" (pipe)

i am having text file below NARGU S S 12358 SALES REP |22| Acccount/s RAJU R B 64253 SALES REP |12| Acccount/s RUKMAN S 32588 SALES REP |10| Acccount/s NARGUND S S 12356... (3 Replies)
Discussion started by: suryanarayana
3 Replies

8. Shell Programming and Scripting

Substituting comma "," for dot "." in a specific column when comma"," is a delimiter

Hi, I'm dealing with an issue and losing a lot of hours figuring out how i would solve this. I have an input file which looks like this: ('BLABLA +200-GRS','Serviço ','TarifaçãoServiço','wap.bla.us.0000000121',2985,0,55,' de conversão em escada','Dia','Domingos') ('BLABLA +200-GRR','Serviço... (6 Replies)
Discussion started by: poliver
6 Replies

9. Shell Programming and Scripting

How to allign output data in UNIX that is separated with a pipe "|" symbol ?

Experts , In the given output of the log file, the 2nd field that is separated by "|" pipe is not aligned well due to the uneven data length, I would like it to align the 2nd column with 37 length (that is disturbed in the output) including the pipe . The two pepe "|" would be in a aligned way... (2 Replies)
Discussion started by: rveri
2 Replies

10. Shell Programming and Scripting

Scripting to fix the issue in UNIX file having delimiter "|"

hello All, I am new user to shell scripting, kindly advise on the below? I have a file where i have gaps & the delimiter falls in next line and new line is also created , plz see the example :employee.txt Now the issue here is , i wan to write a script , where i can use "|" to get the... (6 Replies)
Discussion started by: sunnyd1
6 Replies
MBSINIT(3)						     Linux Programmer's Manual							MBSINIT(3)

NAME
mbsinit - test for initial shift state SYNOPSIS
#include <wchar.h> int mbsinit(const mbstate_t *ps); DESCRIPTION
Character conversion between the multibyte representation and the wide character representation uses conversion state, of type mbstate_t. Conversion of a string uses a finite-state machine; when it is interrupted after the complete conversion of a number of characters, it may need to save a state for processing the remaining characters. Such a conversion state is needed for the sake of encodings such as ISO-2022 and UTF-7. The initial state is the state at the beginning of conversion of a string. There are two kinds of state: The one used by multibyte to wide character conversion functions, such as mbsrtowcs, and the one used by wide character to multibyte conversion functions, such as wcsrtombs, but they both fit in a mbstate_t, and they both have the same representation for an initial state. For 8-bit encodings, all states are equivalent to the initial state. For multibyte encodings like UTF-8, EUC-*, BIG5 or SJIS, the wide character to multibyte conversion functions never produce non-initial states, but the multibyte to wide character conversion functions like mbrtowc do produce non-initial states when interrupted in the middle of a character. One possible way to create an mbstate_t in initial state is to set it to zero: mbstate_t state; memset(&state,0,sizeof(mbstate_t)); On Linux, the following works as well, but might generate compiler warnings: mbstate_t state = { 0 }; The function mbsinit tests whether *ps corresponds to an initial state. RETURN VALUE
mbsinit returns non-zero if *ps is an initial state, or if ps is a null pointer. Otherwise it returns 0. CONFORMING TO
ISO/ANSI C, UNIX98 SEE ALSO
mbsrtowcs(3), wcsrtombs(3) NOTES
The behaviour of mbsinit depends on the LC_CTYPE category of the current locale. GNU
2000-11-20 MBSINIT(3)
All times are GMT -4. The time now is 08:20 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy