02-04-2008
Actually file encoding should not be guessed or detected,
its better that is clarified from the originating source.
Since you have the control to create the file, it depends on the character set being used to create the file whether its ANSI / UTF-8/ LATIN-1/ SJIS
encoding conversion tests can be done using iconv command
10 More Discussions You Might Find Interesting
1. Solaris
Under Unix however we had many many many many problems. We had to use Ansi2utf8(), repstr() and XMLval() to prevent "Invalid token" errors. And because we didn't know what the raw XML result was, it allways was a big problem to find the cause of it. (0 Replies)
Discussion started by: devotedsinner
0 Replies
2. Shell Programming and Scripting
Hello!
The system is AIX 5.3
Give please command or script to get the file encoding
Thanks (2 Replies)
Discussion started by: vinment
2 Replies
3. AIX
Hello!
The system is AIX 5.3
Give please command or script to get the file encoding (1 Reply)
Discussion started by: vinment
1 Replies
4. Shell Programming and Scripting
Hi,
I have got a zip (binary) file transferred from MacOS (thus it has additional __MACOSX directory packed inside). On extracting this zip, there are few *.xml files available. When I opened this *.xml file in vim editor using Cygwin (on windows) the editor displayed in the bottom. I tried... (4 Replies)
Discussion started by: royalibrahim
4 Replies
5. HP-UX
how to find the character encoding of a file in hp_ux (1 Reply)
Discussion started by: alokjyotibal
1 Replies
6. Shell Programming and Scripting
Hi,
I am beginner to Unix.
My requirement is to validate the encoding used in the incoming file(csv,txt).If it is encoded with UTF-8 format,then the file should remain as such otherwise i need to chnage the encoding to UTF-8.
Please advice me how to proceed on this. (7 Replies)
Discussion started by: cnraja
7 Replies
7. UNIX for Dummies Questions & Answers
Hi, I am trying to determine the encoding for the file, because to convert to UTF-8, it seems as though I have to know the encoding of the source.
Tried this
file <filename>
give me this:
<filename>:data or International Language text
Tried to see the locale and this is the output:... (6 Replies)
Discussion started by: MIA651
6 Replies
8. Shell Programming and Scripting
I am using sed on Arabic file (utf-8 encoding) like bellow:
sed 's/./& /g' file
and all I get is:
1 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
I tried change the LANG variable to
LANG=en_US.UTF-8
but I still get the same "?" output. What is the... (1 Reply)
Discussion started by: Viernes
1 Replies
9. Solaris
Hi all!!
I´m using command file -i myfile.xml to validate XML file encoding, but it is just saying regular file . I´m expecting / looking an output as UTF8 or ANSI / ASCII
Is there command to display the files encoding?
Thank you! (2 Replies)
Discussion started by: mrreds
2 Replies
10. Shell Programming and Scripting
how can i know what format a file is
* example:
UTF-8
ANSI
UCS2
i am in a... (8 Replies)
Discussion started by: tricampeon81
8 Replies
LEARN ABOUT CENTOS
create_conversion
CREATE
CONVERSION(7) PostgreSQL 9.2.7 Documentation CREATE CONVERSION(7)
NAME
CREATE_CONVERSION - define a new encoding conversion
SYNOPSIS
CREATE [ DEFAULT ] CONVERSION name
FOR source_encoding TO dest_encoding FROM function_name
DESCRIPTION
CREATE CONVERSION defines a new conversion between character set encodings. Also, conversions that are marked DEFAULT can be used for
automatic encoding conversion between client and server. For this purpose, two conversions, from encoding A to B and from encoding B to A,
must be defined.
To be able to create a conversion, you must have EXECUTE privilege on the function and CREATE privilege on the destination schema.
PARAMETERS
DEFAULT
The DEFAULT clause indicates that this conversion is the default for this particular source to destination encoding. There should be
only one default encoding in a schema for the encoding pair.
name
The name of the conversion. The conversion name can be schema-qualified. If it is not, the conversion is defined in the current schema.
The conversion name must be unique within a schema.
source_encoding
The source encoding name.
dest_encoding
The destination encoding name.
function_name
The function used to perform the conversion. The function name can be schema-qualified. If it is not, the function will be looked up in
the path.
The function must have the following signature:
conv_proc(
integer, -- source encoding ID
integer, -- destination encoding ID
cstring, -- source string (null terminated C string)
internal, -- destination (fill with a null terminated C string)
integer -- source string length
) RETURNS void;
NOTES
Use DROP CONVERSION to remove user-defined conversions.
The privileges required to create a conversion might be changed in a future release.
EXAMPLES
To create a conversion from encoding UTF8 to LATIN1 using myfunc:
CREATE CONVERSION myconv FOR 'UTF8' TO 'LATIN1' FROM myfunc;
COMPATIBILITY
CREATE CONVERSION is a PostgreSQL extension. There is no CREATE CONVERSION statement in the SQL standard, but a CREATE TRANSLATION
statement that is very similar in purpose and syntax.
SEE ALSO
ALTER CONVERSION (ALTER_CONVERSION(7)), CREATE FUNCTION (CREATE_FUNCTION(7)), DROP CONVERSION (DROP_CONVERSION(7))
PostgreSQL 9.2.7 2014-02-17 CREATE CONVERSION(7)