Shell Programming and Scripting

Hi

I am using iconv command to convert the encoding of files.
Below is the command used:

Code:

iconv -f UCS-2 -t UTF-8 inputfile.txt> outputfile.txt

The command is working fine for files less than 2GB.
When I try converting the files of more than 2GB size I get an error as 'Can't open file'. I now understand that iconv will not be able to convert huge files.
I am using ksh shell
Can someone provide a solution to this.

Thank you in advance.
vrcr

iconv on your system appears not to support largefiles.

There is an approach called chunking - try this:
https://github.com/mla/iconv-chunks

It is a perl that allows iconv to process 100MB chunks of gigantic files and output to stdout - you redirect to the new file.

Unless the utility in question needs to move around within the file (as opposed to just processing text from the start of the file to the end of the file), you can frequently get by much more simply by just redirecting the input file (so the shell opens the file for the utility instead of the utility opening the file itself):

Code:

iconv -f UCS-2 -t UTF-8 < inputfile.txt > outputfile.txt