01-13-2018
Characters that don't exist in the target char set are difficult to convert. The
-c option would not necessarily help as it just silently deletes inconvertible chars.
Not sure what your OS / shell /
iconv versions are. Does the latter offer this option (
man iconv)
Quote:
-t to-encoding, --to-code=to-encoding
Use to-encoding for output characters.
. . .
If the string //TRANSLIT is appended to to-encoding, characters being converted are transliterated when needed and possible. This means that when a character cannot be represented in the target character set, it can be approximated through one or several similar looking characters. Characters that are outside of the target character set and cannot be transliterated are replaced with a question mark (?) in the output.
? Would his come close to what you need?
Last edited by RudiC; 01-14-2018 at 09:55 AM..
This User Gave Thanks to RudiC For This Post:
10 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
Hi,
One of our application is producing log files. But if we open the log file in vi or less or view mode, it shows all the special characters in it. The 'cat' shows correctly but it shows only last page. If I do 'cat' <file_name> | more, then again it shows special characters.
... (1 Reply)
Discussion started by: divakarp
1 Replies
2. Shell Programming and Scripting
Hi,
I need some advise on treating non printable chars over ascii value 126
Case 1 :
On some fields in the text , I need to retiain then 'as-is' and load to a database.I understand it also depends on database codepage.
but i just wanna know how do i ensure it do not change while loading... (1 Reply)
Discussion started by: braindrain
1 Replies
3. Shell Programming and Scripting
here is my simple script to show process and owners except me:
ps `-ef |grep xterm |grep -v aucar` | while read a1 a2 a3 a4 a5 a6 a7 a8
do
echo KILL..\($a1\).. $a2 |more
done
how can I pass values from command "ps -ef |grep xterm|grep -v aucar" to ?
because above command... (2 Replies)
Discussion started by: xramm
2 Replies
4. UNIX for Dummies Questions & Answers
I was trying to run a code to check if a fax number is empty or not.
for that, I've written the following code which is throwing an error.
#!/bin/ksh
fax= "999-999-9999"
if ; then
fax_no="000-000-0000"
else
fax_no=$fax
fi
echo $fax_no
And I get the... (7 Replies)
Discussion started by: hooaamai
7 Replies
5. Shell Programming and Scripting
Hi, I'm having trouble with awk print all characters between 2 patterns. I tried more then one solution found on this forum but with no success.
Probably my mistakes are due to the special characters "" and "]"in the search patterns.
Well, have a log file like this:
logfile.txt
... (3 Replies)
Discussion started by: ginolatino
3 Replies
6. Shell Programming and Scripting
I have a file with multiple lines. From each line I want to get all strings that starts with '+' and ends with '/'. Then I want the strings to be separated by ' + '
Example input:
+$A$/NOUN+At/NSUFF_FEM_PL+K/CASE_INDEF_ACC
Sample output:
$A$ + At + K (20 Replies)
Discussion started by: Viernes
20 Replies
7. UNIX for Dummies Questions & Answers
Hi guys,
I am trying to find the following string in a file, but I always get pattern not found error, not sure what is missing here. Can you help please?
I do a less to open the xrates.log and then do a /'="18"' in the file and tried various combinations to search the below string.
String... (8 Replies)
Discussion started by: santokal
8 Replies
8. UNIX for Advanced & Expert Users
Running SunOs 5.6. Solaris.
I've been able to remove all special characters from a fixed length file which appear in the first column but as a result all subsequent columns have shifted to the left by the amount of characters deleted.
It is a space separated file. Line 1 in input file is... (6 Replies)
Discussion started by: iffy290
6 Replies
9. Solaris
Hi all!!
I´m using command file -i myfile.xml to validate XML file encoding, but it is just saying regular file . I´m expecting / looking an output as UTF8 or ANSI / ASCII
Is there command to display the files encoding?
Thank you! (2 Replies)
Discussion started by: mrreds
2 Replies
10. UNIX for Beginners Questions & Answers
Hi Team,
I have a file a1.txt with data as follows.
dfjakjf...asdfkasj</EnableQuotedIDs><SQL><SelectStatement modified='1' type='string'><!
The delimiter string: <SelectStatement modified='1' type='string'><!
dlm="<SelectStatement modified='1' type='string'><!
The above command is... (7 Replies)
Discussion started by: kmanivan82
7 Replies
ICONV(1) Linux User Manual ICONV(1)
NAME
iconv - convert text from one character encoding to another
SYNOPSIS
iconv [options] [-f from-encoding] [-t to-encoding] [inputfile]...
DESCRIPTION
The iconv program reads in text in one encoding and outputs the text in another encoding. If no input files are given, or if it is given
as a dash (-), iconv reads from standard input. If no output file is given, iconv writes to standard output.
If no from-encoding is given, the default is derived from the current locale's character encoding. If no to-encoding is given, the default
is derived from the current locale's character encoding.
OPTIONS
-f from-encoding, --from-code=from-encoding
Use from-encoding for input characters.
-t to-encoding, --to-code=to-encoding
Use to-encoding for output characters.
If the string //IGNORE is appended to to-encoding, characters that cannot be converted are discarded and an error is printed after
conversion.
If the string //TRANSLIT is appended to to-encoding, characters being converted are transliterated when needed and possible. This
means that when a character cannot be represented in the target character set, it can be approximated through one or several similar
looking characters. Characters that are outside of the target character set and cannot be transliterated are replaced with a ques-
tion mark (?) in the output.
-l, --list
List all known character set encodings.
-c Silently discard characters that cannot be converted instead of terminating when encountering such characters.
-o outputfile, --output=outputfile
Use outputfile for output.
-s, --silent
This option is ignored; it is provided only for compatibility.
--verbose
Print progress information on standard error when processing multiple files.
-?, --help
Print a usage summary and exit.
--usage
Print a short usage summary and exit.
-V, --version
Print the version number, license, and disclaimer of warranty for iconv.
EXIT STATUS
Zero on success, nonzero on errors.
ENVIRONMENT
Internally, the iconv program uses the iconv(3) function which in turn uses gconv modules (dynamically loaded shared libraries) to convert
to and from a character set. Before calling iconv(3), the iconv program must first allocate a conversion descriptor using iconv_open(3).
The operation of the latter function is influenced by the setting of the GCONV_PATH environment variable:
* If GCONV_PATH is not set, iconv_open(3) loads the system gconv module configuration cache file created by iconvconfig(8) and then, based
on the configuration, loads the gconv modules needed to perform the conversion. If the system gconv module configuration cache file is
not available then the system gconv module configuration file is used.
* If GCONV_PATH is defined (as a colon-separated list of pathnames), the system gconv module configuration cache is not used. Instead,
iconv_open(3) first tries to load the configuration files by searching the directories in GCONV_PATH in order, followed by the system
default gconv module configuration file. If a directory does not contain a gconv module configuration file, any gconv modules that it
may contain are ignored. If a directory contains a gconv module configuration file and it is determined that a module needed for this
conversion is available in the directory, then the needed module is loaded from that directory, the order being such that the first
suitable module found in GCONV_PATH is used. This allows users to use custom modules and even replace system-provided modules by pro-
viding such modules in GCONV_PATH directories.
FILES
/usr/lib/gconv
Usual default gconv module path.
/usr/lib/gconv/gconv-modules
Usual system default gconv module configuration file.
/usr/lib/gconv/gconv-modules.cache
Usual system gconv module configuration cache.
CONFORMING TO
POSIX.1-2001.
EXAMPLE
Convert text from the ISO 8859-15 character encoding to UTF-8:
$ iconv -f ISO-8859-15 -t UTF-8 < input.txt > output.txt
The next example converts from UTF-8 to ASCII, transliterating when possible:
$ echo abc B a EUR ac | iconv -f UTF-8 -t ASCII//TRANSLIT
abc ss ? EUR abc
SEE ALSO
locale(1), iconv(3), nl_langinfo(3), charsets(7), iconvconfig(8)
COLOPHON
This page is part of release 4.15 of the Linux man-pages project. A description of the project, information about reporting bugs, and the
latest version of this page, can be found at https://www.kernel.org/doc/man-pages/.
GNU 2018-02-02 ICONV(1)