Sponsored Content
Full Discussion: Iconv on large files
Top Forums UNIX for Advanced & Expert Users Iconv on large files Post 302980685 by Don Cragun on Wednesday 31st of August 2016 07:44:50 PM
Old 08-31-2016
My crystal ball is a little bit cloudy this afternoon. Exactly what happens?
  1. What diagnostics are produced?
  2. What is the size of inputfile.txt?
  3. When it is killed, what is the size of outputfile.txt?
  4. How much space is available in the filesystem where you are creating outputfile.txt?
  5. What operating system are you using (including the release number)?
 

10 More Discussions You Might Find Interesting

1. Programming

about iconv

I want to use iconv.h to convert some text to another charset. The code is below: #include <stdio.h> #include <stdlib.h> #include <iconv.h> int main() { iconv_t cd; char instr="汉字"; char *inbuf; char *outbuf; unsigned int insize=7; ... (4 Replies)
Discussion started by: yong
4 Replies

2. UNIX for Dummies Questions & Answers

large files?

How do we check 'large files' is enabled on a Unix box -- HP-UX B11.11 (2 Replies)
Discussion started by: ranj@chn
2 Replies

3. UNIX for Advanced & Expert Users

iconv and xmllint

Here is my question, volume of records processed : 5M ( approx ) Its basically very simple operation that am trying to do and I had achieved the output that am interested. What am looking for really is to improve the performance, an optimized way to do that. with respect to iconv, am... (3 Replies)
Discussion started by: matrixmadhan
3 Replies

4. Shell Programming and Scripting

Divide large data files into smaller files

Hello everyone! I have 2 types of files in the following format: 1) *.fa >1234 ...some text... >2345 ...some text... >3456 ...some text... . . . . 2) *.info >1234 (7 Replies)
Discussion started by: ad23
7 Replies

5. Solaris

How to safely copy full filesystems with large files (10Gb files)

Hello everyone. Need some help copying a filesystem. The situation is this: I have an oracle DB mounted on /u01 and need to copy it to /u02. /u01 is 500 Gb and /u02 is 300 Gb. The size used on /u01 is 187 Gb. This is running on solaris 9 and both filesystems are UFS. I have tried to do it using:... (14 Replies)
Discussion started by: dragonov7
14 Replies

6. Shell Programming and Scripting

Help with iconv command

Hi , I am using iconv command to convert a file in UTF-16 format to UTF-8 format. This command will work for few files but for some showing an error as bad input character. But if i copy the contents of the file for which it is showing "bad input character" to a new file and perform the... (0 Replies)
Discussion started by: Shruthi8818
0 Replies

7. UNIX for Dummies Questions & Answers

Help with iconv command

Hi , I am using iconv command to convert a file in UTF-16 format to UTF-8 format. This command will work for few files but for some showing an error as bad input character. But if i copy the contents of the file for which it is showing "bad input character" to a new file and perform the... (2 Replies)
Discussion started by: Shruthi8818
2 Replies

8. AIX

Installing Converter files for iconv conversion

Hi Friends, I need to install two converter packages UTF-8_IBM-284 and UTF-8_IBM-500 in the location /usr/lib/nls/loc/iconv/* or /usr/lib/nls/loc/iconvTable/* . Could you please let me know how to get this converter files. Thanks in Advance, Siva. (1 Reply)
Discussion started by: sivakumarl
1 Replies

9. Shell Programming and Scripting

Help with command iconv

I need to convert a utf16 file to utf8. When i use the iconv command to do so it gives an error saying invalid function. When I ran the iconv -l function it did not list the utf16 and utf8 as part of its internal table. Is there anyway I can add these encodings in the library? Is there any other... (3 Replies)
Discussion started by: gaun
3 Replies

10. Shell Programming and Scripting

Iconv for large files

Hi I am using iconv command to convert the encoding of files. Below is the command used: iconv -f UCS-2 -t UTF-8 inputfile.txt> outputfile.txt The command is working fine for files less than 2GB. When I try converting the files of more than 2GB size I get an error as 'Can't open file'. I... (2 Replies)
Discussion started by: vrcr
2 Replies
PICONV(1)						 Perl Programmers Reference Guide						 PICONV(1)

NAME
piconv -- iconv(1), reinvented in perl SYNOPSIS
piconv [-f from_encoding] [-t to_encoding] [-s string] [files...] piconv -l DESCRIPTION
piconv is perl version of iconv, a character encoding converter widely available for various Unixen today. This script was primarily a technology demonstrator for Perl 5.8.0, but you can use piconv in the place of iconv for virtually any case. piconv converts the character encoding of either STDIN or files specified in the argument and prints out to STDOUT. Here is the list of options. -f from_encoding Specifies the encoding you are converting from. Unlike iconv, this option can be omitted. In such cases, the current locale is used. -t to_encoding Specifies the encoding you are converting to. Unlike iconv, this option can be omitted. In such cases, the current locale is used. Therefore, when both -f and -t are omitted, piconv just acts like cat. -s string uses string instead of file for the source of text. Same as iconv. -l Lists all available encodings, one per line, in case-insensitive order. Note that only the canonical names are listed; many aliases exist. For example, the names are case-insensitive, and many standard and common aliases work, such as "latin1" for "ISO-8859-1", or "ibm850" instead of "cp850", or "winlatin1" for "cp1252". See Encode::Supported for a full discussion. -C N Check the validity of the stream if N = 1. When N = -1, something interesting happens when it encounters an invalid character. -c Same as "-C 1". -p Same as "-C -1". -h Show usage. -D Invokes debugging mode. Primarily for Encode hackers. -S scheme Selects which scheme is to be used for conversion. Available schemes are as follows: from_to Uses Encode::from_to for conversion. This is the default. decode_encode Input strings are decode()d then encode()d. A straight two-step implementation. perlio The new perlIO layer is used. NI-S' favorite. Like the -D option, this is also for Encode hackers. SEE ALSO
iconv(1) locale(3) Encode Encode::Supported Encode::Alias PerlIO perl v5.8.0 2003-02-18 PICONV(1)
All times are GMT -4. The time now is 08:14 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy