Sponsored Content
Full Discussion: Explain iconv command
Top Forums UNIX for Beginners Questions & Answers Explain iconv command Post 303032452 by eskay on Monday 18th of March 2019 02:48:12 PM
Old 03-18-2019
Unicode Characters in Fixed Length file

I have a requirement to remove all non-ascii characters from a fixed length file. I used the below command which is removing special characters but somehow the total record length is being truncated to one space less. If it is a multi-byte string then many characters at the end are being truncated.

Code:
 cat non-ascii.txt | iconv -f utf8 -t ascii//TRANSLIT//IGNORE > ascii.txt


Last edited by eskay; 03-18-2019 at 04:22 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

please explain the command

Hi all , please explain the following command : perl -e 'select(undef,undef,undef,.15)' Thanks and Regards Navatha (2 Replies)
Discussion started by: Navatha
2 Replies

2. UNIX for Dummies Questions & Answers

Can anyone explain what this command is doing?

Specifically what is the purpose of sed? What is f? Why is the 'cp f $phonefile' line needed when the script ‘goes live'? Why might that two commands following sed be commented out at the present time ( i.e., during development)? Thanks in... (2 Replies)
Discussion started by: knp808
2 Replies

3. Shell Programming and Scripting

Please Explain me this command

find . -type f -ctime +3 -exec mv {} /somedirectory/ \; in particular "-ctime v/s -mtime" and "difference between +3 and -3" (5 Replies)
Discussion started by: Rambo
5 Replies

4. Shell Programming and Scripting

Help with iconv command

Hi , I am using iconv command to convert a file in UTF-16 format to UTF-8 format. This command will work for few files but for some showing an error as bad input character. But if i copy the contents of the file for which it is showing "bad input character" to a new file and perform the... (0 Replies)
Discussion started by: Shruthi8818
0 Replies

5. UNIX for Dummies Questions & Answers

Help with iconv command

Hi , I am using iconv command to convert a file in UTF-16 format to UTF-8 format. This command will work for few files but for some showing an error as bad input character. But if i copy the contents of the file for which it is showing "bad input character" to a new file and perform the... (2 Replies)
Discussion started by: Shruthi8818
2 Replies

6. Shell Programming and Scripting

Characterset conversion problem using iconv command

Hi Friends, I am not able to conver character set from UTF-8 to IBM-284 throwing an error "cannot open convertor" . Could you please help me how to get out of this error. Below command is working fine iconv -f ISO8859-15 -t UTF-8 fromfile.txt > tofile.txt But the below command is... (2 Replies)
Discussion started by: sivakumarl
2 Replies

7. Shell Programming and Scripting

Help with command iconv

I need to convert a utf16 file to utf8. When i use the iconv command to do so it gives an error saying invalid function. When I ran the iconv -l function it did not list the utf16 and utf8 as part of its internal table. Is there anyway I can add these encodings in the library? Is there any other... (3 Replies)
Discussion started by: gaun
3 Replies

8. UNIX for Dummies Questions & Answers

Please explain this command?

Hi, I saw this. But I don't know why we need this? ls mydir > foo.txt ## I know what this will do, it will take the results and write to the file called foo.txt ls mydir > foo.txt 2>&1 ## Don't know why we need 2>&1 Thanks. (2 Replies)
Discussion started by: samnyc
2 Replies

9. Shell Programming and Scripting

Can any one explain this sqlplus command?

Hi , i am new to unix i need a small clarification regarding this sqlplus -s $USER_NAME/$PASSWD@$ORA_SID<< EOF >> SQL_CONN_LOG.log In the above command what is the meaning of <<EOF>> Thanks, krishna. (2 Replies)
Discussion started by: rams_krishna
2 Replies

10. Red Hat

Please help to explain the command

su - keibatch -c ""date ; /usr/local/kei/batch/apb/bin/JKEIKYK4140.sh -run "&$C$6&" WSUKE100201"" Not clear about : date ; /usr/local/kei/batch/apb/bin/JKEIKYK4140.sh -run "&$C$6&" WSUKE100201 Please help (2 Replies)
Discussion started by: honda_city
2 Replies
ICONV(1)						     Linux Programmer's Manual							  ICONV(1)

NAME
iconv - character set conversion SYNOPSIS
iconv [OPTION...] [-f encoding] [-t encoding] [inputfile ...] iconv -l DESCRIPTION
The iconv program converts text from one encoding to another encoding. More precisely, it converts from the encoding given for the -f option to the encoding given for the -t option. Either of these encodings defaults to the encoding of the current locale. All the input- files are read and converted in turn; if no inputfile is given, the standard input is used. The converted text is printed to standard out- put. The encodings permitted are system dependent. For the libiconv implementation, they are listed in the iconv_open(3) manual page. Options controlling the input and output format: -f encoding, --from-code=encoding Specifies the encoding of the input. -t encoding, --to-code=encoding Specifies the encoding of the output. Options controlling conversion problems: -c When this option is given, characters that cannot be converted are silently discarded, instead of leading to a conversion error. --unicode-subst=formatstring When this option is given, Unicode characters that cannot be represented in the target encoding are replaced with a placeholder string that is constructed from the given formatstring, applied to the Unicode code point. The formatstring must be a format string in the same format as for the printf command or the printf() function, taking either no argument or exactly one unsigned integer argument. --byte-subst=formatstring When this option is given, bytes in the input that are not valid in the source encoding are replaced with a placeholder string that is constructed from the given formatstring, applied to the byte's value. The formatstring must be a format string in the same format as for the printf command or the printf() function, taking either no argument or exactly one unsigned integer argument. --widechar-subst=formatstring When this option is given, wide characters in the input that are not valid in the source encoding are replaced with a placeholder string that is constructed from the given formatstring, applied to the byte's value. The formatstring must be a format string in the same format as for the printf command or the printf() function, taking either no argument or exactly one unsigned integer argument. Options controlling error output: -s, --silent When this option is given, error messages about invalid or unconvertible characters are omitted, but the actual converted text is unaffected. The iconv -l or iconv --list command lists the names of the supported encodings, in a system dependent format. For the libiconv implementa- tion, the names are printed in upper case, separated by whitespace, and alias names of an encoding are listed on the same line as the encoding itself. EXAMPLES
iconv -f ISO-8859-1 -t UTF-8 converts input from the old West-European encoding ISO-8859-1 to Unicode. iconv -f KOI8-R --byte-subst="<0x%x>" --unicode-subst="<U+%04X>" converts input from the old Russian encoding KOI8-R to the locale encoding, substituting an angle bracket notation with hexadecimal numbers for invalid bytes and for valid but unconvertible characters. iconv --list lists the supported encodings. SEE ALSO
iconv_open(3) GNU
January 22, 2006 ICONV(1)
All times are GMT -4. The time now is 04:41 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy