08-31-2016
Iconv on large files
Hi All,
I am using iconv to convert huge files. the process is getting killed. I tried the option in below link
Iconv for large files
i.e
HTML Code:
iconv -f UCS-2 -t UTF-8 < inputfile.txt > outputfile.txt
However, the process still gets killed. We do not have iconv-chunk.
Can you please suggest anyway to process without explicitly splitting the inputfile by myself in the script.
10 More Discussions You Might Find Interesting
1. Programming
I want to use iconv.h to convert some text to another charset.
The code is below:
#include <stdio.h>
#include <stdlib.h>
#include <iconv.h>
int main()
{
iconv_t cd;
char instr="汉字";
char *inbuf;
char *outbuf;
unsigned int insize=7;
... (4 Replies)
Discussion started by: yong
4 Replies
2. UNIX for Dummies Questions & Answers
How do we check 'large files' is enabled on a Unix box -- HP-UX B11.11 (2 Replies)
Discussion started by: ranj@chn
2 Replies
3. UNIX for Advanced & Expert Users
Here is my question,
volume of records processed : 5M ( approx )
Its basically very simple operation that am trying to do and I had achieved the output that am interested. What am looking for really is to improve the performance, an optimized way to do that.
with respect to iconv, am... (3 Replies)
Discussion started by: matrixmadhan
3 Replies
4. Shell Programming and Scripting
Hello everyone!
I have 2 types of files in the following format:
1) *.fa
>1234
...some text...
>2345
...some text...
>3456
...some text...
.
.
.
.
2) *.info
>1234 (7 Replies)
Discussion started by: ad23
7 Replies
5. Solaris
Hello everyone. Need some help copying a filesystem. The situation is this: I have an oracle DB mounted on /u01 and need to copy it to /u02. /u01 is 500 Gb and /u02 is 300 Gb. The size used on /u01 is 187 Gb. This is running on solaris 9 and both filesystems are UFS.
I have tried to do it using:... (14 Replies)
Discussion started by: dragonov7
14 Replies
6. Shell Programming and Scripting
Hi ,
I am using iconv command to convert a file in UTF-16 format to UTF-8 format. This command will work for few files but for some showing an error as bad input character.
But if i copy the contents of the file for which it is showing "bad input character" to a new file and perform the... (0 Replies)
Discussion started by: Shruthi8818
0 Replies
7. UNIX for Dummies Questions & Answers
Hi ,
I am using iconv command to convert a file in UTF-16 format to UTF-8 format. This command will work for few files but for some showing an error as bad input character.
But if i copy the contents of the file for which it is showing "bad input character" to a new file and perform the... (2 Replies)
Discussion started by: Shruthi8818
2 Replies
8. AIX
Hi Friends,
I need to install two converter packages UTF-8_IBM-284 and UTF-8_IBM-500 in the location /usr/lib/nls/loc/iconv/* or /usr/lib/nls/loc/iconvTable/* . Could you please let me know how to get this converter files.
Thanks in Advance,
Siva. (1 Reply)
Discussion started by: sivakumarl
1 Replies
9. Shell Programming and Scripting
I need to convert a utf16 file to utf8. When i use the iconv command to do so it gives an error saying invalid function.
When I ran the iconv -l function it did not list the utf16 and utf8 as part of its internal table. Is there anyway I can add these encodings in the library? Is there any other... (3 Replies)
Discussion started by: gaun
3 Replies
10. Shell Programming and Scripting
Hi
I am using iconv command to convert the encoding of files.
Below is the command used:
iconv -f UCS-2 -t UTF-8 inputfile.txt> outputfile.txt
The command is working fine for files less than 2GB.
When I try converting the files of more than 2GB size I get an error as 'Can't open file'. I... (2 Replies)
Discussion started by: vrcr
2 Replies
LEARN ABOUT OPENDARWIN
iconv
ICONV(1) Linux Programmer's Manual ICONV(1)
NAME
iconv - character set conversion
SYNOPSIS
iconv [-c] [-s] [-f encoding] [-t encoding] [inputfile ...]
iconv -l
DESCRIPTION
The iconv program converts text from one encoding to another encoding. More precisely, it converts from the encoding given for the -f
option to the encoding given for the -t option. Either of these encodings defaults to the encoding of the current locale. All the input-
files are read and converted in turn; if no inputfile is given, the standard input is used. The converted text is printed to standard out-
put.
When option -c is given, characters that cannot be converted are silently discarded, instead of leading to a conversion error.
When option -s is given, error messages about invalid or unconvertible characters are omitted, but the actual converted text is unaffected.
The encodings permitted are system dependent. For the libiconv implementation, they are listed in the iconv_open(3) manual page.
The iconv -l command lists the names of the supported encodings, in a system dependent format. For the libiconv implementation, the names
are printed in upper case, separated by whitespace, and alias names of an encoding are listed on the same line as the encoding itself.
SEE ALSO
iconv_open(3)
GNU
January 13, 2002 ICONV(1)