Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: ASCII to UTF-8 conversion
Top Forums Shell Programming and Scripting ASCII to UTF-8 conversion Post 302578298 by methyl on Thursday 1st of December 2011 09:06:38 AM
Old 12-01-2011
Could try a unix "diff" on the unix version against the Windows version and see if there are any significant differences.
If the file is something like XML format, the unix command "iconv" is not suitable.

It's hard to be specific without knowing what Operating System and version you have and something about your data and your Windows process.
 

10 More Discussions You Might Find Interesting

1. Programming

Howto convert Ascii -> UTF-8 & back C++

While working with russian text under FreeBSD&MySQL I need to convert a string from MySQL to the Unicode format. I've just started my way in C++ under FreeBSD , so please explain me how can I get ascii code of Char variable and also how can i get a character into variable with the specified ascii... (3 Replies)
Discussion started by: macron
3 Replies

2. Shell Programming and Scripting

ascii conversion

after converting my ebcidic file to ascii i get the following output 2097152+0 records in 1797345+1 records out Why is there a difference in number of records. Is the converson chopping off any records. All i am doing is just a conversion using the following script dd if=xaa cbs=152 ... (0 Replies)
Discussion started by: rintingtong
0 Replies

3. Shell Programming and Scripting

ascii to ebcdic conversion

Hello, I need a program for ascii to ebsdic conversion. If anybody can help, it'll be greatly appreciated. Thanks. (1 Reply)
Discussion started by: er_ashu
1 Replies

4. UNIX for Advanced & Expert Users

UTF-8 to EBCDIC conversion in UNIX

Hi all, At present a file from AS400 system is being FTPed to an AIX system. Now, a similar file needs to be sent from our Unix box (Solaris) Is there any tool available which does the conversion in Unix from UTF-8 to EBCDIC? Any suggestions/ pointers are really appreciated. Thanks,... (4 Replies)
Discussion started by: sridhar_423
4 Replies

5. Shell Programming and Scripting

need help with ascii to decimal conversion

Hi, Can anyone please help me ascci to decimal conversion in bash I have a file which contains stream of numbers like this,these are ascci values 729711810132973278105991013268971213233 I want to covert it to its actual value like upper code's decimal is "Have a Nice Day!" ... (15 Replies)
Discussion started by: sunilmenhdiratt
15 Replies

6. Shell Programming and Scripting

binary to ascii conversion

Hi, I have got a library file, created by compiling C code. The file information with "file" command, gives it a "application/x-archive" type file. I want to extract the release string of my software from this file, so that i can know which version of C files were used to create the lib. Can... (3 Replies)
Discussion started by: atulmt
3 Replies

7. Red Hat

Can't convert 7bit ASCII to UTF-8

Hello, I am trying to convert a 7bit ASCII file to UTF-8. I have used iconv before though it can't recognize it for some reason and says unknown file encoding. When I used ascii2uni package with different package, ./ascii2uni -a K -a I -a J -a X test_file > new_test_file It still... (2 Replies)
Discussion started by: rockf1bull
2 Replies

8. UNIX for Dummies Questions & Answers

Conversion from ansii to UTF 16

Hi I have a big file which is in ansii . I want to convert it to UTF-16 .Please help me on this as I am stuck at this point in unix . (8 Replies)
Discussion started by: harry00514
8 Replies

9. Shell Programming and Scripting

Convert UTF-8 file to ASCII/ISO8859-1 OR replace characters

I am trying to develop a script which will work on a source UTF-8 file and perform one or more of the following It will accept the target encoding as an argument e.g. US-ASCII or ISO-8859-1, etc 1. It should replace all occurrences of characters outside target character set by " " (space) or... (3 Replies)
Discussion started by: hemkiran.s
3 Replies

10. UNIX for Advanced & Expert Users

EBCDIC to ASCII conversion

Hi, I have a input file which is EBCIDIC and it has packed decimals. Can anyone help me to convert EBCIDIC file to ASCII(Need to convert even Packed decimal values also to normal format). Thanks swapna (12 Replies)
Discussion started by: swapna_1990
12 Replies

LEARN ABOUT SUNOS

iconv_zh

iconv_zh(5)							File Formats Manual						       iconv_zh(5)

NAME

       iconv_zh - code set conversion tables

DESCRIPTION

       The following code set conversions are supported:

		  Code Set Conversions Supported
       Code	    Symbol	       TargetCode   Symbol

       GB 2312-80   zh_CN.euc	       UTF-8	    UTF-8
       GB 2312-80   zh_CN.euc	       HZ-GB-2312   HZ-GB-2312
       GB 2312-80   zh_CN.euc	       ISO-2022-CN  zh_CN.iso2022-CN
       GBK	    zh_CN.gbk	       UTF-8	    UTF-8
       GBK	    zh_CN.gbk	       HZ-GB-2312   HZ-GB-2312
       GBK	    zh_CN.gbk	       ISO-2022-CN  zh_CN.iso2022-CN
       GB18030-2000 zh_CN.gb18030      UTF-8	    UTF-8
       GB18030-2000 zh_CN.gb18030      IS0-2022-CN  zh_CN.iso2022-CN
       HZ-GB-2312   HZ-GB-2312	       UTF-8	    UTF-8
       HZ-GB-2312   HZ-GB-2312	       GB 2312-80   zh_CN.euc
       HZ-GB-2312   HZ-GB-2312	       GBK	    zh_CN.gbk
       ISO-2022-CN  zh_CN.iso2022-CN   GB 2312-80   zh_CN.euc
       ISO-2022-CN  zh_CN.iso2022-CN   GBK	    zh_CN.gbk
       ISO-2022-CN  zh_CN.iso2022-CN   GB18030-2000 zh_CN.gb18030
       IS0-2022-CN  zh_CN.iso2022-CN   UTF-8	    UTF-8
       Code Page 935 zh_CN-cp935       UTF-8	    UTF-8
       UTF-8	    UTF-8	       GB 2312-80   zh_CN.euc
       UTF-8	    UTF-8	       GBK	    zh_CN.gbk
       UTF-8	    UTF-8	       GB18030-2000 zh_CN.gb18030
       UTF-8	    UTF-8	       HZ-GB-2312   HZ-GB-2312
       UTF-8	    UTF-8	       IS0-2022-CN  zh_CN.iso2022-CN
       UTF-8	    UTF-8	       Code Page 935 zh_CN-cp935

       GB 2312-80   zh_CN.euc	       Big-5	    zh_TW-big5
       GBK	    zh_CN.gbk	       Big-5	    zh_TW-big5
       GBK	    zh_CN.gbk	       Big-5p	    zh_TW-big5p
       GBK	    zh_CN.gbk	       BIG5HK	    zh_HK-big5hk
       GB18030-2000 zh_CN.gb18030      Big-5	    zh_TW-big5
       GB18030-2000 zh_CN.gb18030      Big-5p	    zh_TW-big5p
       GB18030-2000 zh_CN.gb18030      BIG5HK	    zh_HK-big5hk
       Big-5	    zh_TW-big5	       GB18030-2000 zh_CN.gb18030
       Big-5	    zh_TW-big5	       GBK	    zh_CN.gbk
       Big-5	    zh_TW-big5	       GB 2312-80   zh_CN.euc
       Big-5p	    zh_TW-big5p        GB18030-2000 zh_CN.gb18030
       Big-5p	    zh_TW-big5p        GBK	    zh_CN.gbk
       BIG5HK	    zh_HK-big5hk       GB18030-2000 zh_CN.gb18030
       BIG5HK	    zh_HK-big5hk       GBK	    zh_CN.gbk

       Conversion are performed as described below, For all conver- sions, if the source code set includes characters not inclu- ded in the target
       code set, conversion and output for all such characters will be done using a substitute characters.

zh_CN.euc to UTF-8 and UTF-8 to zh_CN.euc
       Conversion modules can be used to convert GB 2312-80 characters between EUC-CN and UTF-8 encodings. If input data which does not  have  the
       corresponding  character  in  the target code is encountered, it will be replaced with the substitute character (zh_CN.euc: '??'  (0x3f3f),
       UTF-8: U+FFFD(0xefbfbd)).

zh_CN.gbk to UTF-8 and UTF-8 to zh_CN.gbk
       Conversion modules can be used to convert GBK characters between GBK and UTF-8 encodings. If input data which does not have the correspond-
       ing  character  in  the	target	code  is  encountered, it will be replaced with the subsitute character (zh_CN.gbk: '??'  (0x3f3f), UTF-8:
       U+FFFD(0xefbfbd)).

zh_CN.gb18030 to UTF-8 and UTF-8 to zh_CN.gb18030
       Conversion modules can be used to convert GB18030-2000 characters between GB18030 and UTF-8 encodings. If input data which  does  not  have
       the  corresponding  character  in  the  target  code is encountered, it will be replaced with the substitute character (zh_CN.GB18030: '??'
       (0x3f3f), UTF-8: U+FFFD(0xefbfbd)).

zh_CN.euc to zh_CN.iso2022-CN and zh_CN.iso2022-CN to zh_CN.euc
       Conversion modules are provided to convert GB 2312-80 characters between IS0-2022-CN and EUC-CN encodings.

zh_CN.gbk to zh_CN.iso2022-CN and zh_CN.iso2022-CN to zh_CN.euc
       Conversion modules are provided to convert GB 2312-80, CNS 11643-1992 plane 1 and 2 characters between ISO-2022-CN and EUC-CN encoding.	If
       input  data  which  doesn't  belong  to above charset is encountered, it will be replaced with the substitute character (zh_CN.gbk: 0xa1f5,
       zh_CN.iso2022-CN: 0x2175).

zh_CN.euc to HZ-GB-2312 and HZ-GB-2312 to zh_CN.euc
       Conversion modules are provided to convert GB 2312-80 characters between EUC-CN and HZ encodings.

zh_CN.gbk to HZ-GB-2312 and HZ-GB-2312 to zh_CN.gbk
       Conversion modules are provided to convert GB 2312-80 characters between EUC-CN and HZ encodings. If input data which does  not	belong	to
       the above charset is encountered, it will be replaced with the substitute characters (HZ-GB-2312: 0x2175).

zh_CN.iso2022-CN to UTF-8 and UTF-8 to zh_CN.iso2022-CN
       Conversion  modules  are  provided to convert GB 2312-80 and CNS 11643 plane 1 and 2 characters between ISO-2022-CN and UTF-8 encodings. If
       input data  which  does	not  belong  to  the  above  charsets  is  encountered,  it  will  be  replaced  with  the  substitute	characters
       (zh_CN.iso2022-CN: '??'	(0x3f3f), UTF-8: U+FFFD).

HZ-GB-2312 to UTF-8 and UTF-8 to HZ-GB-2312
       Conversion  modules  can be used to convert GB 2312-80 characters between HZ and UTF-8 encoding. If input data which does not belong to the
       above charset is encountered, it will be replaced with the substitute characters ( HZ-GB-2312: '??' (0x3f3f), UTF-8: U+FFFD).

zh_CN-cp935 to UTF-8 and UTF-8 to zh_CN-cp935
       Conversion modules are provided to convert GB 2312-80 characters between IBM Code Page 935 and UTF-8 encodings.

zh_CN.euc to zh_TW-big5 and zh_TW-big5 to zh_CN.euc
       Conversion modules are provided to convert between simplified chinese (EUC-CN encoding) and traditional chinese (BIG5 encoding).  If  input
       data  which  does  not  have corresponding simplified/traditional chinese character is encountered, it will be replaced with the substitute
       character (zh_CN.euc: '__' (0x5f5f), zh_TW-big5: '__' (0x5f5f)).

zh_CN.gbk to zh_TW-big5 and zh_TW-big5 to zh_CN.gbk
       Conversion modules are provided to convert GBK characters between GBK and BIG5 encoding. If input data which neither has corresponding sim-
       plified/traditional  chinese  character	nor belongs to the other charset is encountered, it will be replaced with the substitute character
       (zh_CN.gbk: '__' (0x5f5f), zh_TW-big5: '__' (0x5f5f)).

zh_CN.gbk to zh_TW-big5p and zh_TW-big5p to zh_CN.gbk
       Conversion modules can be used to convert GBK or Big-5p characters between GBK and Big-5p encodings. If input data which does not belong to
       the other charset, it will be replaced with the substitute character (zh_CN.gbk: '__' (0x5f5f), zh_TW-big5p: '__' (0x5f5f)).

zh_CN.gbk to zh_HK-big5hk and zh_HK-big5hk to zh_CN.gbk
       Conversion modules can be used to convert GBK characters between GBK and BIG5HK encodings. if input data which does not belong to the other
       charset, it will be replaced with the substitute character (zh_CN.gbk: '__' (0x5f5f), zh_HK-big5hk: '__' (0x5f5f)).

SEE ALSO

       iconv(1), iconv(3), iconv(5), iconv_zh_TW(5)

								    2 Nov 2001							       iconv_zh(5)
All times are GMT -4. The time now is 11:26 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy