The offset of outbuf after iconv() will be advanced, so you need to do some pointer arithmetic to go back by the number of advanced output bytes, or maintain another pointer to the output buffer before iconv() and use that.
For example, this works on my system:
Seems like the return value of iconv() is not too meaningful except for checking error condition. Most programs I have seen using iconv() just throw it away.
I am forced to use the ANSEL character set for some GEDCOM documents but must convert them to a more modern set for another app which doesn't recognize ANSEL. I am unable to locate an ISO code for ANSEL in a search of the web. Would someone plese identify the ANSEL character set from the list given... (4 Replies)
Here is my question,
volume of records processed : 5M ( approx )
Its basically very simple operation that am trying to do and I had achieved the output that am interested. What am looking for really is to improve the performance, an optimized way to do that.
with respect to iconv, am... (3 Replies)
hi,
i am new to unix shell scripting and really desparate about this, so i appreciate any help on this matter. it maybe something obvious as well.
two different scripts, the first one is working fine, the second one is not:
#!/bin/bash
if
then
echo "usage: $0 <Input file> <Output... (1 Reply)
hi
I am trying iconv on my linux machine for conversion of RUSSIAN to ENGLISH, but i am not able to get exact result.
i want to know what initial setting in linux machine we need to do to get desired output
I created sample russian file using google translate in CP866 endcoding
and full... (5 Replies)
Hi ,
I am using iconv command to convert a file in UTF-16 format to UTF-8 format. This command will work for few files but for some showing an error as bad input character.
But if i copy the contents of the file for which it is showing "bad input character" to a new file and perform the... (0 Replies)
Hi ,
I am using iconv command to convert a file in UTF-16 format to UTF-8 format. This command will work for few files but for some showing an error as bad input character.
But if i copy the contents of the file for which it is showing "bad input character" to a new file and perform the... (2 Replies)
I need to convert a utf16 file to utf8. When i use the iconv command to do so it gives an error saying invalid function.
When I ran the iconv -l function it did not list the utf16 and utf8 as part of its internal table. Is there anyway I can add these encodings in the library? Is there any other... (3 Replies)
Hi
I am using iconv command to convert the encoding of files.
Below is the command used:
iconv -f UCS-2 -t UTF-8 inputfile.txt> outputfile.txt
The command is working fine for files less than 2GB.
When I try converting the files of more than 2GB size I get an error as 'Can't open file'. I... (2 Replies)
Hi All,
I am using iconv to convert huge files. the process is getting killed. I tried the option in below link
https://www.unix.com/shell-programming-and-scripting/258825-iconv-large-files.html
i.e
iconv -f UCS-2 -t UTF-8 < inputfile.txt > outputfile.txt
However, the process still gets... (4 Replies)
I have a requirement to remove all non-ascii characters from a fixed length file. I used the below command which is removing special characters but somehow the total record length is being truncated to one space less. If it is a multi-byte string then many characters at the end are being truncated.... (8 Replies)
Discussion started by: eskay
8 Replies
LEARN ABOUT X11R4
iconv
ICONV(3) Linux Programmer's Manual ICONV(3)NAME
iconv - perform character set conversion
SYNOPSIS
#include <iconv.h>
size_t iconv (iconv_t cd,
const char* * inbuf, size_t * inbytesleft,
char* * outbuf, size_t * outbytesleft);
DESCRIPTION
The argument cd must be a conversion descriptor created using the function iconv_open.
The main case is when inbuf is not NULL and *inbuf is not NULL. In this case, the iconv function converts the multibyte sequence starting
at *inbuf to a multibyte sequence starting at *outbuf. At most *inbytesleft bytes, starting at *inbuf, will be read. At most *out-
bytesleft bytes, starting at *outbuf, will be written.
The iconv function converts one multibyte character at a time, and for each character conversion it increments *inbuf and decrements
*inbytesleft by the number of converted input bytes, it increments *outbuf and decrements *outbytesleft by the number of converted output
bytes, and it updates the conversion state contained in cd. The conversion can stop for four reasons:
1. An invalid multibyte sequence is encountered in the input. In this case it sets errno to EILSEQ and returns (size_t)(-1). *inbuf is left
pointing to the beginning of the invalid multibyte sequence.
2. The input byte sequence has been entirely converted, i.e. *inbytesleft has gone down to 0. In this case iconv returns the number of non-
reversible conversions performed during this call.
3. An incomplete multibyte sequence is encountered in the input, and the input byte sequence terminates after it. In this case it sets
errno to EINVAL and returns (size_t)(-1). *inbuf is left pointing to the beginning of the incomplete multibyte sequence.
4. The output buffer has no more room for the next converted character. In this case it sets errno to E2BIG and returns (size_t)(-1).
A different case is when inbuf is NULL or *inbuf is NULL, but outbuf is not NULL and *outbuf is not NULL. In this case, the iconv function
attempts to set cd's conversion state to the initial state and store a corresponding shift sequence at *outbuf. At most *outbytesleft
bytes, starting at *outbuf, will be written. If the output buffer has no more room for this reset sequence, it sets errno to E2BIG and
returns (size_t)(-1). Otherwise it increments *outbuf and decrements *outbytesleft by the number of bytes written.
A third case is when inbuf is NULL or *inbuf is NULL, and outbuf is NULL or *outbuf is NULL. In this case, the iconv function sets cd's
conversion state to the initial state.
RETURN VALUE
The iconv function returns the number of characters converted in a non-reversible way during this call; reversible conversions are not
counted. In case of error, it sets errno and returns (size_t)(-1).
ERRORS
The following errors can occur, among others:
E2BIG There is not sufficient room at *outbuf.
EILSEQ An invalid multibyte sequence has been encountered in the input.
EINVAL An incomplete multibyte sequence has been encountered in the input.
CONFORMING TO
UNIX98
SEE ALSO iconv_open(3), iconvctl(3), iconv_close(3)GNU February 2, 2004 ICONV(3)