Sponsored Content
Full Discussion: Iconv on large files
Top Forums UNIX for Advanced & Expert Users Iconv on large files Post 302980687 by tostay2003 on Wednesday 31st of August 2016 08:15:31 PM
Old 08-31-2016
Please find my response below

Quote:
What diagnostics are produced?
I just get "Killed" message

Quote:
What is the size of inputfile.txt ?
Size is approx 60 GB

Quote:
When it is killed, what is the size of outputfile.txt ?
It is getting killed approx at 50GB

Quote:
How much space is available in the filesystem where you are creating outputfile.txt ?
There is around 500 GB on the filesystem

Quote:
What operating system are you using (including the release number)?
Linux 2.6.32-642.3.1.e16.x86_64
 

10 More Discussions You Might Find Interesting

1. Programming

about iconv

I want to use iconv.h to convert some text to another charset. The code is below: #include <stdio.h> #include <stdlib.h> #include <iconv.h> int main() { iconv_t cd; char instr="汉字"; char *inbuf; char *outbuf; unsigned int insize=7; ... (4 Replies)
Discussion started by: yong
4 Replies

2. UNIX for Dummies Questions & Answers

large files?

How do we check 'large files' is enabled on a Unix box -- HP-UX B11.11 (2 Replies)
Discussion started by: ranj@chn
2 Replies

3. UNIX for Advanced & Expert Users

iconv and xmllint

Here is my question, volume of records processed : 5M ( approx ) Its basically very simple operation that am trying to do and I had achieved the output that am interested. What am looking for really is to improve the performance, an optimized way to do that. with respect to iconv, am... (3 Replies)
Discussion started by: matrixmadhan
3 Replies

4. Shell Programming and Scripting

Divide large data files into smaller files

Hello everyone! I have 2 types of files in the following format: 1) *.fa >1234 ...some text... >2345 ...some text... >3456 ...some text... . . . . 2) *.info >1234 (7 Replies)
Discussion started by: ad23
7 Replies

5. Solaris

How to safely copy full filesystems with large files (10Gb files)

Hello everyone. Need some help copying a filesystem. The situation is this: I have an oracle DB mounted on /u01 and need to copy it to /u02. /u01 is 500 Gb and /u02 is 300 Gb. The size used on /u01 is 187 Gb. This is running on solaris 9 and both filesystems are UFS. I have tried to do it using:... (14 Replies)
Discussion started by: dragonov7
14 Replies

6. Shell Programming and Scripting

Help with iconv command

Hi , I am using iconv command to convert a file in UTF-16 format to UTF-8 format. This command will work for few files but for some showing an error as bad input character. But if i copy the contents of the file for which it is showing "bad input character" to a new file and perform the... (0 Replies)
Discussion started by: Shruthi8818
0 Replies

7. UNIX for Dummies Questions & Answers

Help with iconv command

Hi , I am using iconv command to convert a file in UTF-16 format to UTF-8 format. This command will work for few files but for some showing an error as bad input character. But if i copy the contents of the file for which it is showing "bad input character" to a new file and perform the... (2 Replies)
Discussion started by: Shruthi8818
2 Replies

8. AIX

Installing Converter files for iconv conversion

Hi Friends, I need to install two converter packages UTF-8_IBM-284 and UTF-8_IBM-500 in the location /usr/lib/nls/loc/iconv/* or /usr/lib/nls/loc/iconvTable/* . Could you please let me know how to get this converter files. Thanks in Advance, Siva. (1 Reply)
Discussion started by: sivakumarl
1 Replies

9. Shell Programming and Scripting

Help with command iconv

I need to convert a utf16 file to utf8. When i use the iconv command to do so it gives an error saying invalid function. When I ran the iconv -l function it did not list the utf16 and utf8 as part of its internal table. Is there anyway I can add these encodings in the library? Is there any other... (3 Replies)
Discussion started by: gaun
3 Replies

10. Shell Programming and Scripting

Iconv for large files

Hi I am using iconv command to convert the encoding of files. Below is the command used: iconv -f UCS-2 -t UTF-8 inputfile.txt> outputfile.txt The command is working fine for files less than 2GB. When I try converting the files of more than 2GB size I get an error as 'Can't open file'. I... (2 Replies)
Discussion started by: vrcr
2 Replies
App::Info::Lib::Iconv(3pm)				User Contributed Perl Documentation				App::Info::Lib::Iconv(3pm)

NAME
App::Info::Lib::Iconv - Information about libiconv SYNOPSIS
use App::Info::Lib::Iconv; my $iconv = App::Info::Lib::Iconv->new; if ($iconv->installed) { print "App name: ", $iconv->name, " "; print "Version: ", $iconv->version, " "; print "Bin dir: ", $iconv->bin_dir, " "; } else { print "libiconv is not installed. :-( "; } DESCRIPTION
App::Info::Lib::Iconv supplies information about the libiconv library installed on the local system. It implements all of the methods defined by App::Info::Lib. Methods that trigger events will trigger them only the first time they're called (See App::Info for documentation on handling events). To start over (after, say, someone has installed libiconv) construct a new App::Info::Lib::Iconv object to aggregate new meta data. Some of the methods trigger the same events. This is due to cross-calling of shared subroutines. However, any one event should be triggered no more than once. For example, although the info event "Searching for 'iconv.h'" is documented for the methods "version()", "major_version()", and "minor_version()", rest assured that it will only be triggered once, by whichever of those four methods is called first. INTERFACE
Constructor new my $iconv = App::Info::Lib::Iconv->new(@params); Returns an App::Info::Lib::Iconv object. See App::Info for a complete description of argument parameters. When called, "new()" searches the the list of directories returned by the "search_bin_dirs()" method for an executable file with a name returned by the "search_exe_names()" method. If the executable is found, libiconv will be assumed to be installed. Otherwise, most of the object methods will return "undef". Events: info Searching for iconv unknown Path to iconv executable? confirm Path to iconv executable? Class Method key_name my $key_name = App::Info::Lib::Iconv->key_name; Returns the unique key name that describes this class. The value returned is the string "libiconv". Object Methods installed print "libiconv is ", ($iconv->installed ? '' : 'not '), "installed. "; Returns true if libiconv is installed, and false if it is not. App::Info::Lib::Iconv determines whether libiconv is installed based on the presence or absence of the iconv application, as found when "new()" constructed the object. If libiconv does not appear to be installed, then most of the other object methods will return empty values. name my $name = $iconv->name; Returns the name of the application. In this case, "name()" simply returns the string "libiconv". version my $version = $iconv->version; Returns the full version number for libiconv. App::Info::Lib::Iconv attempts to parse the version number from the iconv.h file, if it exists. Events: info Searching for 'iconv.h' Searching for include directory error Cannot find include directory Cannot find 'iconv.h' Cannot parse version number from file 'iconv.h' unknown Enter a valid libiconv include directory Enter a valid libiconv version number major_version my $major_version = $iconv->major_version; Returns the libiconv major version number. App::Info::Lib::Iconv attempts to parse the version number from the iconv.h file, if it exists. For example, if "version()" returns "1.7", then this method returns "1". Events: info Searching for 'iconv.h' Searching for include directory error Cannot find include directory Cannot find 'iconv.h' Cannot parse version number from file 'iconv.h' unknown Enter a valid libiconv include directory Enter a valid libiconv version number minor_version my $minor_version = $iconv->minor_version; Returns the libiconv minor version number. App::Info::Lib::Iconv attempts to parse the version number from the iconv.h file, if it exists. For example, if "version()" returns "1.7", then this method returns "7". Events: info Searching for 'iconv.h' Searching for include directory error Cannot find include directory Cannot find 'iconv.h' Cannot parse version number from file 'iconv.h' unknown Enter a valid libiconv include directory Enter a valid libiconv version number patch_version my $patch_version = $iconv->patch_version; Since libiconv has no patch number in its version number, this method will always return false. executable my $executable = $iconv->executable; Returns the path to the Iconv executable, which will be defined by one of the names returned by "search_exe_names()". The executable is searched for in "new()", so there are no events for this method. bin_dir my $bin_dir = $iconv->bin_dir; Returns the path of the directory in which the iconv application was found when the object was constructed by "new()". Events: info Searching for bin directory error Cannot find bin directory unknown Enter a valid libiconv bin directory inc_dir my $inc_dir = $iconv->inc_dir; Returns the directory path in which the file iconv.h was found. App::Info::Lib::Iconv searches for iconv.h in the following directories: /usr/local/include /usr/include /sw/include Events: info Searching for include directory error Cannot find include directory unknown Enter a valid libiconv include directory lib_dir my $lib_dir = $iconv->lib_dir; Returns the directory path in which a libiconv library was found. The search looks for a file with a name returned by "search_lib_names()" in a directory returned by "search_lib_dirs()". Events: info Searching for library directory error Cannot find library directory unknown Enter a valid libiconv library directory so_lib_dir my $so_lib_dir = $iconv->so_lib_dir; Returns the directory path in which a libiconv shared object library was found. The search looks for a file with a name returned by "search_so_lib_names()" in a directory returned by "search_lib_dirs()". Returns the directory path in which a libiconv shared object library was found. App::Info::Lib::Iconv searches for these files: <Events:> info Searching for shared object library directory error Cannot find shared object library directory unknown Enter a valid libiconv shared object library directory home_url my $home_url = $iconv->home_url; Returns the libiconv home page URL. download_url my $download_url = $iconv->download_url; Returns the libiconv download URL. search_exe_names my @search_exe_names = $iconv->search_exe_names; Returns a list of possible names for the Iconv executable. By default, the only name returned is iconv (iconv.exe on Win32). search_bin_dirs my @search_bin_dirs = $iconv->search_bin_dirs; Returns a list of possible directories in which to search an executable. Used by the "new()" constructor to find an executable to execute and collect application info. The found directory will also be returned by the "bin_dir" method. By default, the directories returned are those in your path, followed by these: /usr/local/bin /usr/bin /bin /sw/bin /usr/local/sbin /usr/sbin /sbin /sw/sbin search_lib_names my @seach_lib_names = $self->search_lib_nams Returns a list of possible names for library files. Used by "lib_dir()" to search for library files. By default, the list is: libiconv3.a libiconv3.la libiconv3.so libiconv3.so.0 libiconv3.so.0.0.1 libiconv3.dylib libiconv3.0.dylib libiconv3.0.0.1.dylib libiconv.a libiconv.la libiconv.so libiconv.so.0 libiconv.so.0.0.1 libiconv.dylib libiconv.2.dylib libiconv.2.0.4.dylib libiconv.0.dylib libiconv.0.0.1.dylib search_so_lib_names my @seach_so_lib_names = $self->search_so_lib_nams Returns a list of possible names for shared object library files. Used by "so_lib_dir()" to search for library files. By default, the list is: libiconv3.so libiconv3.so.0 libiconv3.so.0.0.1 libiconv3.dylib libiconv3.0.dylib libiconv3.0.0.1.dylib libiconv.so libiconv.so.0 libiconv.so.0.0.1 libiconv.dylib libiconv.0.dylib libiconv.0.0.1.dylib search_lib_dirs my @search_lib_dirs = $iconv->search_lib_dirs; Returns a list of possible directories in which to search for libraries. By default, it returns all of the paths in the "libsdirs" and "loclibpth" attributes defined by the Perl Config module -- plus /sw/lib (in support of all you Fink users out there). search_inc_names my @search_inc_names = $iconv->search_inc_names; Returns a list of include file names to search for. Used by "inc_dir()" to search for an include file. By default, the only name returned is iconv.h. search_inc_dirs my @search_inc_dirs = $iconv->search_inc_dirs; Returns a list of possible directories in which to search for include files. Used by "inc_dir()" to search for an include file. By default, the directories are: /usr/local/include /usr/include /sw/include KNOWN ISSUES
This is a pretty simple class. It's possible that there are more directories that ought to be searched for libraries and includes. TO DO
Improve this class by borrowing code from Matt Seargent's AxKit Makefil.PL. BUGS
Please send bug reports to <bug-app-info@rt.cpan.org> or file them at <http://rt.cpan.org/NoAuth/Bugs.html?Dist=App-Info>. AUTHOR
David Wheeler <david@justatheory.com> based on code by Sam Tregar <sam@tregar.com>. SEE ALSO
App::Info, App::Info::Lib, Text::Iconv COPYRIGHT AND LICENSE
Copyright (c) 2002-2008, David Wheeler. Some Rights Reserved. This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.10.1 2011-03-15 App::Info::Lib::Iconv(3pm)
All times are GMT -4. The time now is 08:38 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy