Help with Converting UTF-8 data to Unicode Post: 302589817

Sponsored Content

Top Forums Shell Programming and Scripting Help with Converting UTF-8 data to Unicode Post 302589817 by Corona688 on Thursday 12th of January 2012 05:28:03 PM

01-12-2012

Registered User

Just pasting the invalid characters into your web browser doesn't tell us what they are, as they will have passed through many levels of translation which have garbled them beyond recognition.

Conversion can't fix them if they're actually invalid in the first place, either. It'll find the "right" equivalent, if possible, for it to remain "broken" the same way in the different character set.

Can you show us part of a hex dump of the text file so we can see what the offending characters are? hexdump -C file

Corona688

View Public Profile for Corona688

Visit Corona688's homepage!

Find all posts by Corona688

5 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

converting string to unicode

How can I can convert a string in a shell script that looks something like: ]] to unicode equivalent? thanks a lot, webtekie

2. UNIX for Dummies Questions & Answers

grep and UNICODE (utf-16) file

I'm using shell scripting in Applescript. When searching a file with the ANSEL character set (for GEDCOM files) using (grep '1 CHAR ANSEL' filepath) gives the expected result. When searching a UNICODE formatted file (utf-16), searching for text known to exist in the file using (grep '1 CHAR...

3. Shell Programming and Scripting

Converting Unicode file to UTF8 format

Hi, I have a file in my desktop which is a unicode format. After this file is transferred to Unix using FTP, we are seeing some special character (like rectangle box type) at the first line. The same file is saved as UTF8 (using textpad tool, selecting encode to UTF-8 option) on my desktopand...

4. UNIX for Advanced & Expert Users

[ask]unicode utf-8 for arabic font

hlow all, i want to read arabic font in cli (cat, vi ,etc) in windows i can see the for why in linux i can't see that. this for the example وَمَنْ يَشْكُرْ فَإِنَّمَا يَشْكُرُ لِنَفْسِهِ what should i do ? i need your advice for read that font in cli...:confused: thx before

5. Linux

Help to Convert file from UNIX UTF-8 to Windows UTF-16

Hi, I have tried to convert a UTF-8 file to windows UTF-16 format file as below from unix machine unix2dos < testing.txt | iconv -f UTF-8 -t UTF-16 > out.txt and i am getting some chinese characters as below which l opened the converted file on windows machine. LANG=en_US.UTF-8...

LEARN ABOUT DEBIAN

mdb-hexdump

MDBTools(1)															       MDBTools(1)

NAME

       mdb-hexdump - Hexdump utility from MDB Tools

SYNOPSIS

       mdb-hexdump file [pagenumber]

DESCRIPTION

       mdb-hexdump is a utility program distributed with MDB Tools.

       mdb-hexdump makes a hex dump of a binary file (such as an mdb file).

ENVIRONMENT

       MDB_JET3_CHARSET
	      Defines the charset of the input JET3 (access 97) file. Default is CP1252. See iconv(1).

       MDBICONV
	      Defines the output charset. Default is UTF-8. mdbtools must have been compiled with iconv.

       MDBOPTS
	      semi-column separated list of options:

	      o  use_index

	      o  no_memo

	      o  debug_like

	      o  debug_write

	      o  debug_usage

	      o  debug_ole

	      o  debug_row

	      o  debug_props

	      o  debug_all is a shortcut for all debug_* options

SEE ALSO

       gmdb2(1) mdb-export(1) mdb-prop(1) mdb-sql(1) mdb-ver(1) mdb-array(1) mdb-header(1) mdb-parsecsv(1) mdb-schema(1) mdb-tables(1)

AUTHORS

       The mdb-hexdump utility was written by Brian Bruns.

0.7								   13 July 2013 						       MDBTools(1)