Sponsored Content
Top Forums Shell Programming and Scripting Perl script error to split huge data one by one. Post 302377415 by patrick87 on Friday 4th of December 2009 04:03:47 AM
Old 12-04-2009
I got try one way to archive the same goals but it is not a good solution and no effective Smilie
First of all. I split all the content into a lot of file based on its header. After then I run each of the file with the desired program and cat each output result to a combined file at the end.
It is worked as well. Just not a good solution Smilie
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script to check the unique numbers in huge data

Friends, I have to write a shell script,the description is---- i Have to check the uniqueness of the numbers in a file. A file is containing 200thousand tickets and a ticket have 15 numbers in asecending order.And there is a strip that is having 6 tickets that means 90 numbers.I... (7 Replies)
Discussion started by: namishtiwari
7 Replies

2. Shell Programming and Scripting

Split a huge data into few different files?!

Input file data contents: >seq_1 MSNQSPPQSQRPGHSHSHSHSHAGLASSTSSHSNPSANASYNLNGPRTGGDQRYRASVDA >seq_2 AGAAGRGWGRDVTAAASPNPRNGGGRPASDLLSVGNAGGQASFASPETIDRWFEDLQHYE >seq_3 ATLEEMAAASLDANFKEELSAIEQWFRVLSEAERTAALYSLLQSSTQVQMRFFVTVLQQM ARADPITALLSPANPGQASMEAQMDAKLAAMGLKSPASPAVRQYARQSLSGDTYLSPHSA... (7 Replies)
Discussion started by: patrick87
7 Replies

3. Shell Programming and Scripting

Problem running Perl Script with huge data files

Hello Everyone, I have a perl script that reads two types of data files (txt and XML). These data files are huge and large in number. I am using something like this : foreach my $t (@text) { open TEXT, $t or die "Cannot open $t for reading: $!\n"; while(my $line=<TEXT>){ ... (4 Replies)
Discussion started by: ad23
4 Replies

4. Shell Programming and Scripting

Help- counting delimiter in a huge file and split data into 2 files

I’m new to Linux script and not sure how to filter out bad records from huge flat files (over 1.3GB each). The delimiter is a semi colon “;” Here is the sample of 5 lines in the file: Name1;phone1;address1;city1;state1;zipcode1 Name2;phone2;address2;city2;state2;zipcode2;comment... (7 Replies)
Discussion started by: lv99
7 Replies

5. AIX

Error while copying huge amount of data in aix

Hi When i copy 300GB of data from one filesystem to the other filesystem in AIX I get the error : tar: 0511-825 The file 'SAPBRD.dat' is too large. The command I used is : # tar -cf - . | (cd /sapbackup ; tar -xf - ) im copying as root The below is my ulimit -a output : ... (3 Replies)
Discussion started by: samsungsamsung
3 Replies

6. Shell Programming and Scripting

how to split a huge file by every 100 lines

into small files. i need to add a head.txt and tail.txt into small files at the begin and end, and give a name as q1.xml q2.xml q3.xml .... thank you very much. (2 Replies)
Discussion started by: dtdt
2 Replies

7. UNIX for Dummies Questions & Answers

Split a huge 7 GB File Based on Pattern into 4 files

Hi, I have a Huge 7 GB file which has around 1 million records, i want to split this file into 4 files to contain around 250k messages each. Please help me as Split command cannot work here as it might miss tags.. Format of the file is as below <!--###### ###### START-->... (6 Replies)
Discussion started by: KishM
6 Replies

8. Shell Programming and Scripting

Split a folder with huge number of files in n folders

We have a folder XYZ with large number of files (>350,000). how can i split the folder and create say 10 of them XYZ1 to XYZ10 with 35,000 files each. (doesnt matter which files go where). (12 Replies)
Discussion started by: AlokKumbhare
12 Replies

9. UNIX for Advanced & Expert Users

Need Optimization shell/awk script to aggreagte (sum) for all the columns of Huge data file

Optimization shell/awk script to aggregate (sum) for all the columns of Huge data file File delimiter "|" Need to have Sum of all columns, with column number : aggregation (summation) for each column File not having the header Like below - Column 1 "Total Column 2 : "Total ... ...... (2 Replies)
Discussion started by: kartikirans
2 Replies

10. Solaris

Split huge File System

Gents I have huge NAS File System as /sys with size 10 TB and I want to Split each 1TB in spirit File System to be mounted in the server. How to can I do that without changing anything in the source. Please your support. (1 Reply)
Discussion started by: AbuAliiiiiiiiii
1 Replies
Plack::App::CGIBin(3pm) 				User Contributed Perl Documentation				   Plack::App::CGIBin(3pm)

NAME
Plack::App::CGIBin - cgi-bin replacement for Plack servers SYNOPSIS
use Plack::App::CGIBin; use Plack::Builder; my $app = Plack::App::CGIBin->new(root => "/path/to/cgi-bin")->to_app; builder { mount "/cgi-bin" => $app; }; # Or from the command line plackup -MPlack::App::CGIBin -e 'Plack::App::CGIBin->new(root => "/path/to/cgi-bin")->to_app' DESCRIPTION
Plack::App::CGIBin allows you to load CGI scripts from a directory and convert them into a PSGI application. This would give you the extreme easiness when you have bunch of old CGI scripts that is loaded using cgi-bin of Apache web server. HOW IT WORKS
This application checks if a given file path is a perl script and if so, uses CGI::Compile to compile a CGI script into a sub (like ModPerl::Registry) and then run it as a persistent application using CGI::Emulate::PSGI. If the given file is not a perl script, it executes the script just like a normal CGI script with fork & exec. This is like a normal web server mode and no performance benefit is achieved. The default mechanism to determine if a given file is a Perl script is as follows: o Check if the filename ends with ".pl". If yes, it is a Perl script. o Open the file and see if the shebang (first line of the file) contains the word "perl" (like "#!/usr/bin/perl"). If yes, it is a Perl script. You can customize this behavior by passing "exec_cb" callback, which takes a file path to its first argument. For example, if your perl-based CGI script uses lots of global variables and such and are not ready to run on a persistent environment, you can do: my $app = Plack::App::CGIBin->new( root => "/path/to/cgi-bin", exec_cb => sub { 1 }, )->to_app; to always force the execute option for any files. AUTHOR
Tatsuhiko Miyagawa SEE ALSO
Plack::App::File CGI::Emulate::PSGI CGI::Compile Plack::App::WrapCGI See also Plack::App::WrapCGI if you compile one CGI script into a PSGI application without serving CGI scripts from a directory, to remove overhead of filesystem lookups, etc. perl v5.14.2 2011-11-02 Plack::App::CGIBin(3pm)
All times are GMT -4. The time now is 02:47 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy