Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Help with Converting UTF-8 data to Unicode

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help with Converting UTF-8 data to Unicode
# 1  
Old 01-12-2012
Help with Converting UTF-8 data to Unicode
How can I get an error when converting 3rd line, since it has invalid characters

Code:
abcde
a®cdée
a�cd�

Unicode for
® = ®
é = é

I used "iconv -f UTF-8 -t ISO-8859-15 in.txt > out.txt"

Last edited by methyl; 01-12-2012 at 06:31 PM.. Reason: please use code tags for code and data
# 2  
Old 01-12-2012
Just pasting the invalid characters into your web browser doesn't tell us what they are, as they will have passed through many levels of translation which have garbled them beyond recognition.

Conversion can't fix them if they're actually invalid in the first place, either. It'll find the "right" equivalent, if possible, for it to remain "broken" the same way in the different character set.

Can you show us part of a hex dump of the text file so we can see what the offending characters are? hexdump -C file
# 3  
Old 01-12-2012
Please post a Hexadecimal and Ascii dump of both in.txt and out.txt. As you have omitted to mention what Operating System and version you have, I can only suggest the unix "od" command (complete with suitable parameters for your system).
Login or Register to Ask a Question

Previous Thread | Next Thread

5 More Discussions You Might Find Interesting

1. Linux

Help to Convert file from UNIX UTF-8 to Windows UTF-16

Hi, I have tried to convert a UTF-8 file to windows UTF-16 format file as below from unix machine unix2dos < testing.txt | iconv -f UTF-8 -t UTF-16 > out.txt and i am getting some chinese characters as below which l opened the converted file on windows machine. LANG=en_US.UTF-8... (3 Replies)
Discussion started by: phanidhar6039
3 Replies

2. UNIX for Advanced & Expert Users

[ask]unicode utf-8 for arabic font

hlow all, i want to read arabic font in cli (cat, vi ,etc) in windows i can see the for why in linux i can't see that. this for the example وَمَنْ يَشْكُرْ فَإِنَّمَا يَشْكُرُ لِنَفْسِهِ what should i do ? i need your advice for read that font in cli...:confused: thx before (0 Replies)
Discussion started by: zvtral
0 Replies

3. Shell Programming and Scripting

Converting Unicode file to UTF8 format

Hi, I have a file in my desktop which is a unicode format. After this file is transferred to Unix using FTP, we are seeing some special character (like rectangle box type) at the first line. The same file is saved as UTF8 (using textpad tool, selecting encode to UTF-8 option) on my desktopand... (7 Replies)
Discussion started by: vfrg
7 Replies

4. UNIX for Dummies Questions & Answers

grep and UNICODE (utf-16) file

I'm using shell scripting in Applescript. When searching a file with the ANSEL character set (for GEDCOM files) using (grep '1 CHAR ANSEL' filepath) gives the expected result. When searching a UNICODE formatted file (utf-16), searching for text known to exist in the file using (grep '1 CHAR... (4 Replies)
Discussion started by: Whiterock
4 Replies

5. Shell Programming and Scripting

converting string to unicode

How can I can convert a string in a shell script that looks something like: ]] to unicode equivalent? thanks a lot, webtekie (1 Reply)
Discussion started by: webtekie
1 Replies
Login or Register to Ask a Question