UTF8 encoding


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting UTF8 encoding
# 1  
Old 05-24-2010
UTF8 encoding

Hi experts,

I have a gz file from other system(solaris), which is ftped to our system(solaris).

After gunzip, the file is a xml file and we are using ORACLE built in xml transformiing tool ORAXSL to transform XML to TXT.

Now the issue is we come accross issue regarding UTF8 as below:
Error occurred while parsing RDC2010052100149253.xml: Invalid UTF8 encoding.

As informed by our SA/DBA, we have set NLS_LANG as below in shell script, still got same error. (Even we use dos2unix -ascii to transform the unzipped xml file)

export NLS_LANG=AMERICAN_AMERICA.UTF8

Is there anyway to fix this issue?

Any help to clue will be highly appreciated.
# 2  
Old 05-24-2010
First of all, does your XML contain an encoding declaration as the first line? Something like:
Code:
<?xml version="1.0" encoding="UTF-8"?>

If not, please let us what the encoding declaration is.

Next of all, not all valid UTF8 characters are valid XML characters. See Section 2.2 of Extensible Markup Language (XML) 1.0 (Second Edition) This applies to CDATA also. If this is the case, all you can do is write a filter to scrub the offending characters.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Solaris

View file encoding then change encoding.

Hi all!! I´m using command file -i myfile.xml to validate XML file encoding, but it is just saying regular file . I´m expecting / looking an output as UTF8 or ANSI / ASCII Is there command to display the files encoding? Thank you! (2 Replies)
Discussion started by: mrreds
2 Replies

2. Shell Programming and Scripting

Latin character to UTF8

Hi., Can any one please share information on how to convert latin character to utf 8 using shell script or what ever perl or python ? Thanks. (1 Reply)
Discussion started by: alnhk
1 Replies

3. Linux

How to create new empty utf8 file for appending?

Hey i try to create empty text file with utf-8 encoding without success what is the right way to do this ? tried with touch test.txt iconv -f UTF-8 -t UTF-8 test.txt or iconv -f latin1 -t UTF-8 test.txt (8 Replies)
Discussion started by: umen
8 Replies

4. UNIX for Advanced & Expert Users

Problem With UTF8 Byte Order Make

Hi Im migrating a few websites from my old webserver (CentOS-5) to a new server (CentOS6) , one of these websites is multilingual and has a lot of utf8 files(html,php) with different languages (i.e arabic, persian, russian ,etc). In old server when i do: file mailer.php I get : ... (6 Replies)
Discussion started by: mohs3n
6 Replies

5. Debian

Locales UTF8 - not working

Hello, I'm facing a strange problem in one of my Debian server, what is happening right now it that I have runned dpkg-reconfigure locales to set en_US UTF-8 so in that way I could use accentuation in my system. # locale -a C en_US.utf8 POSIX pt_BR.utf8 However, when I create a new... (12 Replies)
Discussion started by: pxb368@motorola
12 Replies

6. Shell Programming and Scripting

Convert to UTF8 File - Unix

All, I have several *.dat files which is created in windows (ANSI Endoing) Or PC File format, once I copy those files to unix. How can I convert those file to utf8 encoding ? I tired iconv, it says not supported Please help Thanks - S (5 Replies)
Discussion started by: Shanks
5 Replies

7. Shell Programming and Scripting

How to find the file encoding and updating the file encoding?

Hi, I am beginner to Unix. My requirement is to validate the encoding used in the incoming file(csv,txt).If it is encoded with UTF-8 format,then the file should remain as such otherwise i need to chnage the encoding to UTF-8. Please advice me how to proceed on this. (7 Replies)
Discussion started by: cnraja
7 Replies

8. Linux

xterm font size and utf8

Hi everyone! I want to change the default font size of xterm. I tried appending the following line in .Xresources xterm -fn -*-fixed-medium-*-*-*-14-*-*-*-*-*-*-* The size changes, but when I try to write in greek, nothing is printed on the screen. I tried appending a similar line to... (0 Replies)
Discussion started by: kerb41
0 Replies

9. Shell Programming and Scripting

utf8 strings in Perl

Hi All I need help on how to handle utf8 strings (match, split etc.) in Perl. Thanks in advance. (2 Replies)
Discussion started by: my_Perl
2 Replies

10. UNIX for Advanced & Expert Users

Utf8-utf16

Hi All, When we create a flat file using a PLSQL program , the flat file is being created in UTF8 format.This file has lot of german characters.When we use this file to load data into MS SQL Server, the german characters are coming as junk. When we create a flat file in oracle it is being ... (1 Reply)
Discussion started by: Suppandi
1 Replies
Login or Register to Ask a Question