Sponsored Content
Operating Systems AIX Locales and conversion tables confusion - Some characters don't get translated Post 302503376 by Adamm on Thursday 10th of March 2011 10:37:12 AM
Old 03-10-2011
Locales and conversion tables confusion - Some characters don't get translated

Hello,

my program running on AIX receives a message (from MQ) which is in CCSID 870 (LATIN-2 EBCDIC). It needs to convert it to CCSID 912 (ISO-8859-2). For this conversion, it uses the Websphere MQ functionality, but which, according to the manual, uses AIX's system conversion tables. (Btw - it only works like this on AIX. On Linux or Windows, Websphere MQ uses its own coversion logic, ignoring system conversion tables - and it works there.)

The problem is, that some characters like š,č,ť,ž get translated to chr(26).

My checklist was:
1. The LANG environment variable is set to sk_SK
2. Conversion tables seem to be there
(root):/usr/lib/nls/loc/iconvTable# ls -l
...
-r--r--r-- 1 bin bin 288 Aug 12 2010 IBM-1250_IBM-5346
-r--r--r-- 1 bin bin 288 Aug 12 2010 IBM-1282_IBM-5346
-r--r--r-- 1 bin bin 288 Aug 12 2010 IBM-1284_IBM-5346
-r--r--r-- 1 bin bin 288 Aug 12 2010 IBM-1285_IBM-5346
-r--r--r-- 1 bin bin 288 Aug 12 2010 IBM-5346_IBM-1250
-r--r--r-- 1 bin bin 288 Aug 12 2010 IBM-5346_IBM-1282
-r--r--r-- 1 bin bin 288 Aug 12 2010 IBM-5346_IBM-1284
-r--r--r-- 1 bin bin 288 Aug 12 2010 IBM-5346_IBM-1285
-r--r--r-- 1 bin bin 288 Aug 12 2010 IBM-5346_IBM-852
-r--r--r-- 1 bin bin 288 Aug 12 2010 IBM-5346_IBM-870
-r--r--r-- 1 bin bin 288 Aug 12 2010 IBM-5346_ISO8859-2
-r--r--r-- 1 bin bin 288 Aug 12 2010 IBM-852_IBM-5346
-r--r--r-- 1 bin bin 288 Aug 12 2010 IBM-870_IBM-5346
-r--r--r-- 1 bin bin 288 Aug 12 2010 ISO8859-2_IBM-5346

Can you please help me find out what am I forgeting? Thanx.
 

8 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Locales - Solaris 8

Can anyone tell me the difference between the 2 Locales: North American Partial Locales en_US.ISO8859-1 en_US.ISO8859-15 Does it matter which one to use? What are the differences? Thanks. -I have found the below info but I don't know what it means(Also no RFC# for 8859-15): ... (2 Replies)
Discussion started by: finster
2 Replies

2. AIX

Locales in AIX

Currently we have the following locales installed on two of our servers SERVER-1 $ locale -a C POSIX en_US en_US.8859-15 en_US.ISO8859-1 EN_US EN_US.UTF-8 ZH_CN ZH_CN.UTF-8 HI_IN HI_IN.UTF-8 ko_KR ko_KR.IBM-eucKR zh_TW zh_TW.IBM-eucTW SERVER-2 (0 Replies)
Discussion started by: voorkey
0 Replies

3. Shell Programming and Scripting

Converting tables of row data into columns of tables

I am trying to transpose tables listed in the format into format. Any help would be greatly appreciated. Input: test_data_1 1 2 90% 4 3 91% 5 4 90% 6 5 90% 9 6 90% test_data_2 3 5 92% 5 4 92% 7 3 93% 9 2 92% 1 1 92% ... Output:... (7 Replies)
Discussion started by: justthisguy
7 Replies

4. AIX

SMTP and characters conversion

Hello, I'm working with an AIX 5.3 This server receive emails from an application (ServiceDesk management) and these emails are read and integrated in another application via an EAI. I work for a french company with french people that write in french : with accentued characters (é,ç,à,...). Sadly,... (2 Replies)
Discussion started by: Djizus
2 Replies

5. Shell Programming and Scripting

Setting locales

I'm having a problem with my locale setting. Essentially, when using Xmgrace or Awk, all . are replaced with , leading to wrong arithmetic. Here is an explanation of the problem: awk printing thousands with commas. see LC_ALL=en_US.UTF-8 How can I permanently set the locale... (1 Reply)
Discussion started by: chrisjorg
1 Replies

6. IP Networking

Does Translated IP Matter For Proxy Server (SQUID)

I am using Squid to create a proxy server for framework related to an application. My users sit on a private network in 191.xx.xx.xx space and my proxy sits in a different private network in 188.xx.xx.xx space. There will be a NAT in place to allow bidirectional communications but is there an... (5 Replies)
Discussion started by: BobSpero
5 Replies

7. Linux

File conversion and removing special characters from a file in Linux

I have a .CSV file when I check for the special characters in the file using the command cat -vet filename.csv, i get very lengthy lines with "^@", "^I^@" and "^@^M" characters in between each alphabet in all of the records. Using the code below file filename.csv I get the output as I have a... (2 Replies)
Discussion started by: dhruuv369
2 Replies

8. UNIX for Beginners Questions & Answers

Replacing string/special characters using a 'conversion' table

Hi, Does anyone know if there is a script or program available out there that uses a conversion table to replace special characters from a file? I am trying to remove some special characters from a file but there are several unprintable/control characters that some I need to remove but some I... (2 Replies)
Discussion started by: newbie_01
2 Replies
iconv(1)						      General Commands Manual							  iconv(1)

Name
       iconv - international codeset conversion

Syntax
       iconv [-d] -f fromcodeset -t tocodeset [file...]

Description
       The  command  converts the encoding of characters in its input from one codeset to another codeset.  The fromcodeset argument specifies the
       codeset used to encode the data in the input; that is, it specifies the input codeset.  The tocodeset argument  specifies  the  codeset	to
       which  you  want  the input data converted; that is, it specifies the output codeset.  The command performs the conversion by reading rules
       from a conversion table you create.  The command reads its input from standard input or from one or more files named on the  command  line.
       The command writes its output to standard output.

       You define conversion rules in a conversion table. The conversion rules specify how converts a particular character or group of characters,
       which are called tokens.  The conversion table is a text file that contains two lists.  In the left-hand list, you specify each	token  you
       want  to convert.  In the right-hand list, you specify the token you want to create in the output file. For example, if you issued the fol-
       lowing command:
       % iconv -fupper -tlower conversion_file

       This command uses the conversion table located in the file that specifies how to convert from an uppercase codeset to a lowercase  codeset.
       The following shows part of the conversion table:
       #
       # Converts from uppercase to lowercase
       #
       #    Input token 	Output token
       #    -------------	------------------
		 A		     a
		 B		     b
		 C		     c
		 D		     d
		 E		     e
		 F		     f
		 G		     g
		 .
		 .
		 .
		 Z		     z
       #
       # Convert tabs to spaces using octal
       #
		11		  404040
       #
       # Convert the A umlaut to lowercase
       #
		    A			    a

       Each  line  in the conversion table must contain two strings, an input token and an output token.  The tokens must be delimited with spaces
       or tabs.  The backslash character (  ) either causes the command to recognize a character it normally ignores or introduces a three  digit
       octal  constant.   All octal constants in the conversion table must contain three digits.  Lines that begin with a hash symbol (#) are com-
       ments.  The command ignores comment lines and blank lines.

       You name the conversion table file using the name of the input codeset, an underscore, and the name of the output codeset.  For example, if
       your input codeset is ISO646 and your output codeset is ISO8859, you might name the conversion table file

       The  command  searches  for  the conversion table file in the directory specified by the pathname.  If the ${ICONV} environment variable is
       undefined, the command searches the directory.

       The operation of the command is 8-bit transparent.

Options
       -d Deletes any characters that are omitted from the conversion table.  By default, the command sends characters that  are  omitted  to  the
	  output file without modifying them.

       -f Specifies the name of the input codeset.

       -t Specifies the name of the output codeset.

Restrictions
       The  conversion	table file name can contain no more than 255 characters.  You may need to truncate the name of the input codeset or output
       codeset when you name the conversion table file.

Examples
       The following shows an example of using the command:

       This command converts the data in from ISO646 encoding to ISO8859 encoding.  The command reads the conversion table from the file.  If  the
       ${ICONV} environment variable is undefined, the command uses the file. If that file does not exist, the command issues an error message and
       does not convert the data file.	The command writes the results	of any conversion it performs to the file

Files
See Also
       environ(5int)
       Guide to Developing International Software

																	  iconv(1)
All times are GMT -4. The time now is 06:35 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy