Sponsored Content
Full Discussion: Explain iconv command
Top Forums UNIX for Beginners Questions & Answers Explain iconv command Post 303032481 by RudiC on Tuesday 19th of March 2019 06:10:28 AM
Old 03-19-2019
Not sure I understand correctly - you want to iconv multi-byte UTF-8 records to ASCII but retain record length? So - if a two byte representation (like Ñ) is converted to N, a space should be added, and for three bytes, two spaces, to keep the record length, PROVIDED the target representations is a one byte char. This doesn't always come true, e.g € -> EUR in ASCII.
If above assumption is true, some conditioning upfront the iconv might help, like
Code:
LC_ALL=C sed 's/[\xC0-\xDF]./& /g; s/[\xE0-\xEF]../& /g' non-ascii.txt | iconv -futf8 -tASCII//TRANSLIT//IGNORE

adds one space for two byte repr., two for three byte repr. For longer / more exotic codes, it must be expanded equivalently.
This User Gave Thanks to RudiC For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

please explain the command

Hi all , please explain the following command : perl -e 'select(undef,undef,undef,.15)' Thanks and Regards Navatha (2 Replies)
Discussion started by: Navatha
2 Replies

2. UNIX for Dummies Questions & Answers

Can anyone explain what this command is doing?

Specifically what is the purpose of sed? What is f? Why is the 'cp f $phonefile' line needed when the script ‘goes live'? Why might that two commands following sed be commented out at the present time ( i.e., during development)? Thanks in... (2 Replies)
Discussion started by: knp808
2 Replies

3. Shell Programming and Scripting

Please Explain me this command

find . -type f -ctime +3 -exec mv {} /somedirectory/ \; in particular "-ctime v/s -mtime" and "difference between +3 and -3" (5 Replies)
Discussion started by: Rambo
5 Replies

4. Shell Programming and Scripting

Help with iconv command

Hi , I am using iconv command to convert a file in UTF-16 format to UTF-8 format. This command will work for few files but for some showing an error as bad input character. But if i copy the contents of the file for which it is showing "bad input character" to a new file and perform the... (0 Replies)
Discussion started by: Shruthi8818
0 Replies

5. UNIX for Dummies Questions & Answers

Help with iconv command

Hi , I am using iconv command to convert a file in UTF-16 format to UTF-8 format. This command will work for few files but for some showing an error as bad input character. But if i copy the contents of the file for which it is showing "bad input character" to a new file and perform the... (2 Replies)
Discussion started by: Shruthi8818
2 Replies

6. Shell Programming and Scripting

Characterset conversion problem using iconv command

Hi Friends, I am not able to conver character set from UTF-8 to IBM-284 throwing an error "cannot open convertor" . Could you please help me how to get out of this error. Below command is working fine iconv -f ISO8859-15 -t UTF-8 fromfile.txt > tofile.txt But the below command is... (2 Replies)
Discussion started by: sivakumarl
2 Replies

7. Shell Programming and Scripting

Help with command iconv

I need to convert a utf16 file to utf8. When i use the iconv command to do so it gives an error saying invalid function. When I ran the iconv -l function it did not list the utf16 and utf8 as part of its internal table. Is there anyway I can add these encodings in the library? Is there any other... (3 Replies)
Discussion started by: gaun
3 Replies

8. UNIX for Dummies Questions & Answers

Please explain this command?

Hi, I saw this. But I don't know why we need this? ls mydir > foo.txt ## I know what this will do, it will take the results and write to the file called foo.txt ls mydir > foo.txt 2>&1 ## Don't know why we need 2>&1 Thanks. (2 Replies)
Discussion started by: samnyc
2 Replies

9. Shell Programming and Scripting

Can any one explain this sqlplus command?

Hi , i am new to unix i need a small clarification regarding this sqlplus -s $USER_NAME/$PASSWD@$ORA_SID<< EOF >> SQL_CONN_LOG.log In the above command what is the meaning of <<EOF>> Thanks, krishna. (2 Replies)
Discussion started by: rams_krishna
2 Replies

10. Red Hat

Please help to explain the command

su - keibatch -c ""date ; /usr/local/kei/batch/apb/bin/JKEIKYK4140.sh -run "&$C$6&" WSUKE100201"" Not clear about : date ; /usr/local/kei/batch/apb/bin/JKEIKYK4140.sh -run "&$C$6&" WSUKE100201 Please help (2 Replies)
Discussion started by: honda_city
2 Replies
KICONV(3)						   BSD Library Functions Manual 						 KICONV(3)

NAME
kiconv_add_xlat16_cspair, kiconv_add_xlat16_cspairs, kiconv_add_xlat16_table -- kernel side iconv library LIBRARY
Kernel side iconv library (libkiconv, -lkiconv) SYNOPSIS
#include <sys/iconv.h> int kiconv_add_xlat16_cspair(const char *tocode, const char *fromcode, int flag); int kiconv_add_xlat16_cspairs(const char *foreigncode, const char *localcode); int kiconv_add_xlat16_table(const char *tocode, const char *fromcode, const void *data, int datalen); DESCRIPTION
The kiconv library provides multi-byte character conversion tables for kernel side iconv service. The kiconv_add_xlat16_cspair() function defines a conversion table using iconv(3) between fromcode charset and tocode charset. You can spec- ify flag to determine if tolower(3) / toupper(3) conversion is included in the table. The flag has following values. KICONV_LOWER KICONV_FROM_LOWER It generates a tolower table in addition to a character conversion table. The difference between two is tolower tocode or tolower fromcode. KICONV_UPPER KICONV_FROM_UPPER It generates a toupper table in addition to a character conversion table. The difference between two is toupper tocode or toupper fromcode. A tolower/toupper conversion is limited to single-byte characters. The kiconv_add_xlat16_cspairs() function defines two conversion tables which are from localcode to foreigncode and from foreigncode to localcode. These conversion tables also contain both tolower and toupper tables. The kiconv_add_xlat16_table() function defines a conversion table directly pointed by data whose length is datalen, not using iconv(3). SEE ALSO
iconv(3), tolower(3), toupper(3) BSD
July 17, 2003 BSD
All times are GMT -4. The time now is 04:07 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy