Help to Convert file from UNIX UTF-8 to Windows UTF-16 Post: 302886930

10 More Discussions You Might Find Interesting

1. Programming

Howto convert Ascii -> UTF-8 & back C++

While working with russian text under FreeBSD&MySQL I need to convert a string from MySQL to the Unicode format. I've just started my way in C++ under FreeBSD , so please explain me how can I get ascii code of Char variable and also how can i get a character into variable with the specified ascii...

2. UNIX for Dummies Questions & Answers

grep and UNICODE (utf-16) file

I'm using shell scripting in Applescript. When searching a file with the ANSEL character set (for GEDCOM files) using (grep '1 CHAR ANSEL' filepath) gives the expected result. When searching a UNICODE formatted file (utf-16), searching for text known to exist in the file using (grep '1 CHAR...

3. UNIX for Advanced & Expert Users

Convert UTF-8 encoded hex value to a character

Hi, I have a non-ascii character (Ŵ), which can be represented in UTF-8 encoding as equivalent hex value (\xC5B4). Is there a function in unix to convert this hex value back to display the charcter ?

4. UNIX for Advanced & Expert Users

UTF-8 to EBCDIC conversion in UNIX

Hi all, At present a file from AS400 system is being FTPed to an AIX system. Now, a similar file needs to be sent from our Unix box (Solaris) Is there any tool available which does the conversion in Unix from UTF-8 to EBCDIC? Any suggestions/ pointers are really appreciated. Thanks,...

5. Red Hat

Can't convert 7bit ASCII to UTF-8

Hello, I am trying to convert a 7bit ASCII file to UTF-8. I have used iconv before though it can't recognize it for some reason and says unknown file encoding. When I used ascii2uni package with different package, ./ascii2uni -a K -a I -a J -a X test_file > new_test_file It still...

6. UNIX for Dummies Questions & Answers

Issue with UTF-8 BOM character in text file

Sometimes we recieve some excel files containing French/Japanese characters over the mail, and these files are manually transferred to the server by using SFTP (security is not a huge concern here). The data is changed to text format before transferring it using Notepad. Problem is: When saving...

7. Shell Programming and Scripting

Trying to convert utf-8 to WINDOWS-1251

Hello all i have utf-8 file that i try to convert to WINDOWS-1251 on linux without any success the file name is utf-8 when i try to do : file -bi test.txt it gives me : text/plain; charset=utf-8 when i try to convert the file i do : /usr/bin/iconv -f UTF-8 -t WINDOWS-1251 test.txt >...

8. Shell Programming and Scripting

Copying a file with UTF char on UNIX server

Hi, I need to run a SQL which check for special UTF char in DB. When I try to copy that in UNIX file it changes it to some wierd chat. How can in retain the UTF chars in my script? e.g. ο|π|ρ|σ|τ|υ|φ|χ|ψ Any help will be appriciated. Thanks,

9. Shell Programming and Scripting

Convert UTF-8 file to ASCII/ISO8859-1 OR replace characters

I am trying to develop a script which will work on a source UTF-8 file and perform one or more of the following It will accept the target encoding as an argument e.g. US-ASCII or ISO-8859-1, etc 1. It should replace all occurrences of characters outside target character set by " " (space) or...

10. UNIX for Beginners Questions & Answers

Convert files to UTF-8 on AIX 7.1

Dears, I have a shell script - working perfectly on Oracle Linux - that detects the encoding (the charset to be exact) of the files in a specified directory using the "file" command (The file command outputs the charset in Linux, but doesn't do that in AIX), then if the file isn't a UTF-8 text...

LEARN ABOUT DEBIAN

yaz-iconv

YAZ-ICONV(1)							     Commands							      YAZ-ICONV(1)

NAME

       yaz-iconv - YAZ Character set conversion utility

SYNOPSIS

       yaz-iconv [-f from] [-t to] [-v] [file...]

DESCRIPTION

       yaz-iconv converts data in file in character set specified by from to output in character set as specified by to.

       This yaz-iconv utility similar to the iconv found on many POSIX systems (Glibc, Solaris, etc).

       If no file is specified, yaz-iconv reads from standard input.

OPTIONS

       -ffrom]
	   Specify the character set from of the input file. Should be used in conjunction with option -t.

       -tto]
	   Specify the character set of of the output. Should be used in conjunction with option -f.

       -v
	   Print more information about the conversion process.

ENCODINGS

       The yaz-iconv command and the API as defined in yaz/yaz-iconv.h is a wrapper for the library system call iconv. But YAZ' iconv utility also
       implements conversions on its own. The table below lists characters sets (or encodings). that are supported by YAZ. Each character set is
       marked with either encode or decode. If an encoding is encode-enabled YAZ may convert to to the designated encoding. If an encoding is
       decode-enabled, YAZ may convert from the designated encoding.

       marc8 (encode, decode)
	   The MARC8[1] encoding as defined by the Library of Congress. Most MARC21/USMARC records usees this encoding.

       marc8s (encode, decode)
	   Like MARC8 but with conversion prefers non-combined characters in the Latin-1 plane over combined characters.

       marc8lossy (encode)
	   Lossy encoding of MARC-8.

       marc8lossless (encode)
	   Lossless encoding of MARC8.

       utf8 (encode, decode)
	   The most commonly used UNICODE encoding on the Internet.

       iso8859-1 (encode, decode)
	   ISO-8859-1, AKA Latin-1.

       iso5426 (decode)
	   ISO 5426. Some MARC records (UNIMARC) uses this encoding.

       iso5428:1984 (encode, decode)
	   ISO 5428:1984.

       advancegreek (encode, decode)
	   An encoding for Greek used by some vendors (Advance).

       danmarc (decode)

	   Danmarc (in danish)[2] is an encoding based on UNICODE which is used for DanMARC2 records.

EXAMPLES

       The following command converts from ISO-8859-1 (Latin-1) to UTF-8.

	       yaz-iconv -f ISO-8859-1 -t UTF-8 -X <input.lst >output.lst

FILES

       prefix/bin/yaz-iconv

       prefix/include/yaz/yaz-iconv.h

SEE ALSO

       yaz(7) iconv(1)

NOTES

	1. MARC8
	   http://www.loc.gov/marc/specifications/speccharmarc8.html

	2. Danmarc (in danish)
	   http://www.kat-format.dk/danMARC2/Danmarc2.4.htm#felt+Indl.+4

YAZ 4.2.30							    04/16/2012							      YAZ-ICONV(1)

10 More Discussions You Might Find Interesting

1. Programming

Howto convert Ascii -> UTF-8 & back C++

Discussion started by: macron

2. UNIX for Dummies Questions & Answers

grep and UNICODE (utf-16) file

Discussion started by: Whiterock

3. UNIX for Advanced & Expert Users

Convert UTF-8 encoded hex value to a character

Discussion started by: sumirmehta

4. UNIX for Advanced & Expert Users

UTF-8 to EBCDIC conversion in UNIX

Discussion started by: sridhar_423