Sponsored Content
Full Discussion: Explain iconv command
Top Forums UNIX for Beginners Questions & Answers Explain iconv command Post 303032481 by RudiC on Tuesday 19th of March 2019 06:10:28 AM
Old 03-19-2019
Not sure I understand correctly - you want to iconv multi-byte UTF-8 records to ASCII but retain record length? So - if a two byte representation (like Ñ) is converted to N, a space should be added, and for three bytes, two spaces, to keep the record length, PROVIDED the target representations is a one byte char. This doesn't always come true, e.g € -> EUR in ASCII.
If above assumption is true, some conditioning upfront the iconv might help, like
Code:
LC_ALL=C sed 's/[\xC0-\xDF]./& /g; s/[\xE0-\xEF]../& /g' non-ascii.txt | iconv -futf8 -tASCII//TRANSLIT//IGNORE

adds one space for two byte repr., two for three byte repr. For longer / more exotic codes, it must be expanded equivalently.
This User Gave Thanks to RudiC For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

please explain the command

Hi all , please explain the following command : perl -e 'select(undef,undef,undef,.15)' Thanks and Regards Navatha (2 Replies)
Discussion started by: Navatha
2 Replies

2. UNIX for Dummies Questions & Answers

Can anyone explain what this command is doing?

Specifically what is the purpose of sed? What is f? Why is the 'cp f $phonefile' line needed when the script ‘goes live'? Why might that two commands following sed be commented out at the present time ( i.e., during development)? Thanks in... (2 Replies)
Discussion started by: knp808
2 Replies

3. Shell Programming and Scripting

Please Explain me this command

find . -type f -ctime +3 -exec mv {} /somedirectory/ \; in particular "-ctime v/s -mtime" and "difference between +3 and -3" (5 Replies)
Discussion started by: Rambo
5 Replies

4. Shell Programming and Scripting

Help with iconv command

Hi , I am using iconv command to convert a file in UTF-16 format to UTF-8 format. This command will work for few files but for some showing an error as bad input character. But if i copy the contents of the file for which it is showing "bad input character" to a new file and perform the... (0 Replies)
Discussion started by: Shruthi8818
0 Replies

5. UNIX for Dummies Questions & Answers

Help with iconv command

Hi , I am using iconv command to convert a file in UTF-16 format to UTF-8 format. This command will work for few files but for some showing an error as bad input character. But if i copy the contents of the file for which it is showing "bad input character" to a new file and perform the... (2 Replies)
Discussion started by: Shruthi8818
2 Replies

6. Shell Programming and Scripting

Characterset conversion problem using iconv command

Hi Friends, I am not able to conver character set from UTF-8 to IBM-284 throwing an error "cannot open convertor" . Could you please help me how to get out of this error. Below command is working fine iconv -f ISO8859-15 -t UTF-8 fromfile.txt > tofile.txt But the below command is... (2 Replies)
Discussion started by: sivakumarl
2 Replies

7. Shell Programming and Scripting

Help with command iconv

I need to convert a utf16 file to utf8. When i use the iconv command to do so it gives an error saying invalid function. When I ran the iconv -l function it did not list the utf16 and utf8 as part of its internal table. Is there anyway I can add these encodings in the library? Is there any other... (3 Replies)
Discussion started by: gaun
3 Replies

8. UNIX for Dummies Questions & Answers

Please explain this command?

Hi, I saw this. But I don't know why we need this? ls mydir > foo.txt ## I know what this will do, it will take the results and write to the file called foo.txt ls mydir > foo.txt 2>&1 ## Don't know why we need 2>&1 Thanks. (2 Replies)
Discussion started by: samnyc
2 Replies

9. Shell Programming and Scripting

Can any one explain this sqlplus command?

Hi , i am new to unix i need a small clarification regarding this sqlplus -s $USER_NAME/$PASSWD@$ORA_SID<< EOF >> SQL_CONN_LOG.log In the above command what is the meaning of <<EOF>> Thanks, krishna. (2 Replies)
Discussion started by: rams_krishna
2 Replies

10. Red Hat

Please help to explain the command

su - keibatch -c ""date ; /usr/local/kei/batch/apb/bin/JKEIKYK4140.sh -run "&$C$6&" WSUKE100201"" Not clear about : date ; /usr/local/kei/batch/apb/bin/JKEIKYK4140.sh -run "&$C$6&" WSUKE100201 Please help (2 Replies)
Discussion started by: honda_city
2 Replies
ICONV_OPEN(3)						     Linux Programmer's Manual						     ICONV_OPEN(3)

NAME
iconv_open - allocate descriptor for character set conversion SYNOPSIS
#include <iconv.h> iconv_t iconv_open(const char *tocode, const char *fromcode); DESCRIPTION
The iconv_open() function allocates a conversion descriptor suitable for converting byte sequences from character encoding fromcode to character encoding tocode. The values permitted for fromcode and tocode and the supported combinations are system-dependent. For the GNU C library, the permitted values are listed by the iconv --list command, and all combinations of the listed values are supported. Furthermore the GNU C library and the GNU libiconv library support the following two suffixes: //TRANSLIT When the string "//TRANSLIT" is appended to tocode, transliteration is activated. This means that when a character cannot be repre- sented in the target character set, it can be approximated through one or several similarly looking characters. //IGNORE When the string "//IGNORE" is appended to tocode, characters that cannot be represented in the target character set will be silently discarded. The resulting conversion descriptor can be used with iconv(3) any number of times. It remains valid until deallocated using iconv_close(3). A conversion descriptor contains a conversion state. After creation using iconv_open(), the state is in the initial state. Using iconv(3) modifies the descriptor's conversion state. To bring the state back to the initial state, use iconv(3) with NULL as inbuf argument. RETURN VALUE
The iconv_open() function returns a freshly allocated conversion descriptor. In case of error, it sets errno and returns (iconv_t) -1. ERRORS
The following error can occur, among others: EINVAL The conversion from fromcode to tocode is not supported by the implementation. VERSIONS
This function is available in glibc since version 2.1. ATTRIBUTES
For an explanation of the terms used in this section, see attributes(7). +-------------+---------------+----------------+ |Interface | Attribute | Value | +-------------+---------------+----------------+ |iconv_open() | Thread safety | MT-Safe locale | +-------------+---------------+----------------+ CONFORMING TO
POSIX.1-2001, POSIX.1-2008, SUSv2. SEE ALSO
iconv(1), iconv(3), iconv_close(3) COLOPHON
This page is part of release 4.15 of the Linux man-pages project. A description of the project, information about reporting bugs, and the latest version of this page, can be found at https://www.kernel.org/doc/man-pages/. GNU
2017-09-15 ICONV_OPEN(3)
All times are GMT -4. The time now is 11:04 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy