Sponsored Content
Full Discussion: How to know file encoding?
Top Forums Shell Programming and Scripting How to know file encoding? Post 303038018 by rdrtx1 on Wednesday 21st of August 2019 11:30:04 AM
Old 08-21-2019
ASCII is a subset of UTF-8. But if there is a need to transliterate, try something like:
Code:
iconv -f UTF-8 -t ASCII//TRANSLIT < input_file


Last edited by rdrtx1; 02-18-2020 at 08:03 PM..
These 2 Users Gave Thanks to rdrtx1 For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

File encoding in Unix

1. I have a shell script which creates a file using cat command. How can i find what encoding the file follows (e.g. UTF8, ANSI)? 2. I want to convert that file to PC-ANSI format. How can i achieve that? I am using HP-Unix. (6 Replies)
Discussion started by: ssmallya
6 Replies

2. Shell Programming and Scripting

Delete original wav file if lame was successful encoding.

In a bash script: src=”cooltrack.wav” dst=”cooltrack.mp3” lame $src $dst I would like to add some line that would delete the source wav file like: rm $src but I would like this only if the encoding was successful. What should I include before deleting the original to check that the... (2 Replies)
Discussion started by: Aia
2 Replies

3. Shell Programming and Scripting

get the file encoding

Hello! The system is AIX 5.3 Give please command or script to get the file encoding Thanks (2 Replies)
Discussion started by: vinment
2 Replies

4. AIX

get the file encoding

Hello! The system is AIX 5.3 Give please command or script to get the file encoding (1 Reply)
Discussion started by: vinment
1 Replies

5. Shell Programming and Scripting

Cygwin vi XML file encoding problem

Hi, I have got a zip (binary) file transferred from MacOS (thus it has additional __MACOSX directory packed inside). On extracting this zip, there are few *.xml files available. When I opened this *.xml file in vim editor using Cygwin (on windows) the editor displayed in the bottom. I tried... (4 Replies)
Discussion started by: royalibrahim
4 Replies

6. HP-UX

how to find the character encoding of a file in hp_ux

how to find the character encoding of a file in hp_ux (1 Reply)
Discussion started by: alokjyotibal
1 Replies

7. Shell Programming and Scripting

How to find the file encoding and updating the file encoding?

Hi, I am beginner to Unix. My requirement is to validate the encoding used in the incoming file(csv,txt).If it is encoded with UTF-8 format,then the file should remain as such otherwise i need to chnage the encoding to UTF-8. Please advice me how to proceed on this. (7 Replies)
Discussion started by: cnraja
7 Replies

8. UNIX for Dummies Questions & Answers

Determing the encoding of a file

Hi, I am trying to determine the encoding for the file, because to convert to UTF-8, it seems as though I have to know the encoding of the source. Tried this file <filename> give me this: <filename>:data or International Language text Tried to see the locale and this is the output:... (6 Replies)
Discussion started by: MIA651
6 Replies

9. UNIX for Advanced & Expert Users

ISO 88591 file encoding charset in Linux

Hello Experts, please help to provide any insight as I am facing issue migrating java application from hpux to redhat. The java program is using InputStreamReader to read a file without specifying any charset parameter. However, in new Linux Redhat 5.6 environent, when reading a file that... (1 Reply)
Discussion started by: sonic_air
1 Replies

10. Solaris

View file encoding then change encoding.

Hi all!! I´m using command file -i myfile.xml to validate XML file encoding, but it is just saying regular file . I´m expecting / looking an output as UTF8 or ANSI / ASCII Is there command to display the files encoding? Thank you! (2 Replies)
Discussion started by: mrreds
2 Replies
ICONV(1)                                                         Linux User Manual                                                        ICONV(1)

NAME
iconv - convert text from one character encoding to another SYNOPSIS
iconv [options] [-f from-encoding] [-t to-encoding] [inputfile]... DESCRIPTION
The iconv program reads in text in one encoding and outputs the text in another encoding. If no input files are given, or if it is given as a dash (-), iconv reads from standard input. If no output file is given, iconv writes to standard output. If no from-encoding is given, the default is derived from the current locale's character encoding. If no to-encoding is given, the default is derived from the current locale's character encoding. OPTIONS
-f from-encoding, --from-code=from-encoding Use from-encoding for input characters. -t to-encoding, --to-code=to-encoding Use to-encoding for output characters. If the string //IGNORE is appended to to-encoding, characters that cannot be converted are discarded and an error is printed after conversion. If the string //TRANSLIT is appended to to-encoding, characters being converted are transliterated when needed and possible. This means that when a character cannot be represented in the target character set, it can be approximated through one or several similar looking characters. Characters that are outside of the target character set and cannot be transliterated are replaced with a ques- tion mark (?) in the output. -l, --list List all known character set encodings. -c Silently discard characters that cannot be converted instead of terminating when encountering such characters. -o outputfile, --output=outputfile Use outputfile for output. -s, --silent This option is ignored; it is provided only for compatibility. --verbose Print progress information on standard error when processing multiple files. -?, --help Print a usage summary and exit. --usage Print a short usage summary and exit. -V, --version Print the version number, license, and disclaimer of warranty for iconv. EXIT STATUS
Zero on success, nonzero on errors. ENVIRONMENT
Internally, the iconv program uses the iconv(3) function which in turn uses gconv modules (dynamically loaded shared libraries) to convert to and from a character set. Before calling iconv(3), the iconv program must first allocate a conversion descriptor using iconv_open(3). The operation of the latter function is influenced by the setting of the GCONV_PATH environment variable: * If GCONV_PATH is not set, iconv_open(3) loads the system gconv module configuration cache file created by iconvconfig(8) and then, based on the configuration, loads the gconv modules needed to perform the conversion. If the system gconv module configuration cache file is not available then the system gconv module configuration file is used. * If GCONV_PATH is defined (as a colon-separated list of pathnames), the system gconv module configuration cache is not used. Instead, iconv_open(3) first tries to load the configuration files by searching the directories in GCONV_PATH in order, followed by the system default gconv module configuration file. If a directory does not contain a gconv module configuration file, any gconv modules that it may contain are ignored. If a directory contains a gconv module configuration file and it is determined that a module needed for this conversion is available in the directory, then the needed module is loaded from that directory, the order being such that the first suitable module found in GCONV_PATH is used. This allows users to use custom modules and even replace system-provided modules by pro- viding such modules in GCONV_PATH directories. FILES
/usr/lib/gconv Usual default gconv module path. /usr/lib/gconv/gconv-modules Usual system default gconv module configuration file. /usr/lib/gconv/gconv-modules.cache Usual system gconv module configuration cache. CONFORMING TO
POSIX.1-2001. EXAMPLE
Convert text from the ISO 8859-15 character encoding to UTF-8: $ iconv -f ISO-8859-15 -t UTF-8 < input.txt > output.txt The next example converts from UTF-8 to ASCII, transliterating when possible: $ echo abc B a EUR ac | iconv -f UTF-8 -t ASCII//TRANSLIT abc ss ? EUR abc SEE ALSO
locale(1), iconv(3), nl_langinfo(3), charsets(7), iconvconfig(8) COLOPHON
This page is part of release 4.15 of the Linux man-pages project. A description of the project, information about reporting bugs, and the latest version of this page, can be found at https://www.kernel.org/doc/man-pages/. GNU 2018-02-02 ICONV(1)
All times are GMT -4. The time now is 04:32 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy