Unix character set problem


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Unix character set problem
# 1  
Old 04-18-2009
Unix character set problem

Hi All,

We are getting file into our unix box with multibyte characters. When we tried to view the file the record looks like this

Frédéric

Actually the data sent to us is

Frédéric

--> my locale charmap of unix is set to UTF8 only ... but still i am getting this problem.

I created the same record in windows desktop and ftp ed the file to unix server. File looks fine when ftp ed.

We thought error might be during writing the file to unix from other source. Then source sender send the data along with ascii characters of that file.

so the file looks like this...

Frédéric

70 114 233 100 233 114 105 99 <-- ascii values for above record

Ascii values are coming correctly but data looks different...

Help me out on this...
# 2  
Old 04-18-2009
It may probably be a terminal font rendering issue, or your terminal may be started in another locale. Even though you switched the locale in the shell, text may still not be rendered properly at the terminal emulator level. This is common with X-based terminals.

So which kind of terminal are you using, and are you sure a Unicode font with the needed characters is used for rendering the terminal text?
# 3  
Old 04-18-2009
Thanks for the reply.

We are using putty. With this interface we tried to change the character set ..we didnt get proper data ....

Is there any other interface like if we use other interface it is possible to view the data properly...please suggest...
# 4  
Old 04-18-2009
With Putty, you need to make sure you are selecting the proper encoding. Also check the font used. Both may be configured as preferences for specific sites.
# 5  
Old 04-20-2009
Thanks once again....

Its working fine when i change the settings in putty configuration.

But if we have to change them manually. Is there any command in unix which automatically change the settings of putty to UTF8 and font changes.

Please suggest.
# 6  
Old 04-24-2009
Hi,

We are receiving the file in unix with korean and china characters along with french characters. when we are using UTF-8 mode only french characters are loaded properly when loaded into oracle database.

Which character set should I use to capture korean characters .... Normally I heard UTf-8 will hold all the types... but here I am not able to ....Please help me on this....
# 7  
Old 04-24-2009
UTF-8 will represent those Korean, Chinese, French characters. Basically just any character in existence in the world. However, your fonts may not have the glyphs for the subset of characters you probably need.

There is practically no font in existence that covers each and every character in the Unicode character set. As a Chinese, I have some fonts on my system that is able to display Chinese. For Korean text, however, you may need to find some fonts that contain Hangul glyphs. On my Vista system, there are fonts like GulimChe that appear to support Hangul.

Check Windows update and see if you are able to download some MS language packs (including fonts) for these areas.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Russian character set issue.

Hi All, I'm facing issue while opening xls file while contains Russian/Siberian character I tried various options which I could get from google but still issue persists hence thought of taking help here, We are trying to export data from Oracle via shell script using sqlplus utility. After... (8 Replies)
Discussion started by: arvindshukla81
8 Replies

2. Shell Programming and Scripting

How to set character limit on READ?

Hello, I created the following (snippet from larger code): echo -n "A1: " read A1 VERIFY=$(echo -n $A1|wc -c) if ; then echo -e "TOO MANY CHARACTERS" fi echo -n "A2: " read A2 echo -n "A3: " read A3 echo -e "Concat: $B1/$B2/$B3" Basically what it does is it... (4 Replies)
Discussion started by: jl487
4 Replies

3. UNIX for Dummies Questions & Answers

Character set problem

Hi, I'm trying to edit a file with vi, but all special characters (áéíóú etc) don't seem to show correctly. They don't seem to be supported by the OS (SunOS 5.10). I'm using MobaXterm as the terminal emulator, which is configured to use ISO-8859-1. The same charset is used on Solaris. If I open... (4 Replies)
Discussion started by: Subbeh
4 Replies

4. Shell Programming and Scripting

Unix-problem of New line character

Hi All, "Please read the below information carefully." i have tried the below code for counting the number of lines present in text file ignoring blank lines #! /bin/bash clear rdCount=0; while read myline do if ; then echo "line is empty" else echo $myline let... (10 Replies)
Discussion started by: aish11
10 Replies

5. UNIX for Advanced & Expert Users

ASCII Character Set

I thought I would point this out. This has a lot of the non printing characters. ASCII Character Set (7 Replies)
Discussion started by: cokedude
7 Replies

6. Solaris

help me to change the character set

dears i am using solaris 10 i am facing a problem when i make setup for solaris i choose the country egypt and i select the language north america but i forget to do that the i found the date Jun written in arabic i want to change character set to written in english -rw-r--r-- 1 root ... (4 Replies)
Discussion started by: hosney00ux
4 Replies

7. Programming

character set conversion in unix C

Hi, Could anybody explain how to change the character set of a particular string in C in unix. we are using HP-UX as OS. We require to change the input string which is in cp1250 format to utf-8. A sample code would help. Thnx in advance (1 Reply)
Discussion started by: gucho
1 Replies

8. UNIX for Advanced & Expert Users

iconv -l and ANSEL character set

I am forced to use the ANSEL character set for some GEDCOM documents but must convert them to a more modern set for another app which doesn't recognize ANSEL. I am unable to locate an ISO code for ANSEL in a search of the web. Would someone plese identify the ANSEL character set from the list given... (4 Replies)
Discussion started by: Whiterock
4 Replies

9. Programming

character set solaris

hi , i am trying to work on a script that transforms some special Dutch characters and send them to a Xerox printer .. the problem is that while doing so iam unable to identify th correct character set that is used by solaris , to transfer these characcters to Xerox character set . thanks... (2 Replies)
Discussion started by: ppass
2 Replies

10. Solaris

latin 2 character-set with xterm

Hi, We have problems with the latin 2 Character-set with xterm. We have installed SunRay-Server with Solaris 8. Our Thinclients use hu- and cz-keyboards. I have set the right local-settings and xmodemaps. If I use the dtterm all is running fine. As soon as I use the xterm, it cannot display... (0 Replies)
Discussion started by: paho
0 Replies
Login or Register to Ask a Question