Sponsored Content
Top Forums Shell Programming and Scripting Slow Perl script: how to speed up? Post 302506322 by gimley on Sunday 20th of March 2011 01:51:36 AM
Old 03-20-2011
Dear Pravin27
I had written an awk script to do the job. But it was giving false positives when there was too much data.
Here is the script:
Practically the same as yours. When I had put it up, it was suggested to use an "O" for a large array:
BEGIN {FS="="}
NR==FNR{O[$1]++;next} !($1 in O)
Unluckily the script is fast but gives false positives, hence the PERL which gives correct positives but is slow.
I tried yours, it also gives false positives on a huge fle compare.
Manyt hanks.

Gimley
 

9 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

network speed is slow

Hello, everyone: i encounter a problem these days , pls help me ,thanks in advance. my env: machine: ES40 A ES40 B os: true64 Unix 4.0f note: src.tar 8M network card speed 100M my problem: ... (3 Replies)
Discussion started by: q30
3 Replies

2. Shell Programming and Scripting

Optimize/speed-up perl extraction

Hi, Is there a way I can extract my data faster. You know my data is 1.2 GB text file with 8Million rows with 38 columns/fields. Imagine how huge this is. How I can optimized the data extraction using perl. That is why I'm creating a script to filter only those informations that I need. Is... (3 Replies)
Discussion started by: pinpe
3 Replies

3. UNIX for Advanced & Expert Users

speed test +20,000 file existance checks too slow

Need to make a very fast file existence checker. Passing in 20-50K num of files In the code below ${file} is a file with a listing of +20,000 files. test_speed is the script. I am commenting out the results of <time test_speed try>. The normal "test -f" is much much too slow when a system... (2 Replies)
Discussion started by: nullwhat
2 Replies

4. Shell Programming and Scripting

Speed up this script!

I have a script that processes a fair amount of data -- say, 25-50 megs per run. I'd like ideas on speeding it up. The code is actually just a preprocessor -- I'm using another language to do the heavy lifting. But as it happens, the preprocessing takes much more time than the final processing... (3 Replies)
Discussion started by: CRGreathouse
3 Replies

5. Filesystems, Disks and Memory

data from blktrace: read speed V.S. write speed

I analysed disk performance with blktrace and get some data: read: 8,3 4 2141 2.882115217 3342 Q R 195732187 + 32 8,3 4 2142 2.882116411 3342 G R 195732187 + 32 8,3 4 2144 2.882117647 3342 I R 195732187 + 32 8,3 4 2145 ... (1 Reply)
Discussion started by: W.C.C
1 Replies

6. Shell Programming and Scripting

Net::SSH::Perl slow to login.

I have some sample code that's supposed to ssh to another machine using Net::SSH::Perl, execute a command, and print the output of that command. It's very basic, and it works. However, I noticed that upon logging in: $ssh->login('username','password'); It takes roughly 10-13 seconds to... (2 Replies)
Discussion started by: mrwatkin
2 Replies

7. Shell Programming and Scripting

How can i speed this script up?

Hi, Im quite new to scripting and would like a bit of assistance with trying to speed up the following script. At the moment it is quite slow.... Any way to improve it? total=111120 while do total=`expr $total + 1` INCREMENT=$total firstline = "blablabla" secondline = "blablabla"... (5 Replies)
Discussion started by: brunlea
5 Replies

8. Shell Programming and Scripting

Help me with speed up this script

hey guys i have a perl script wich use to compare hashes but it tookes a long time to do that so i wich i will have the soulition to do it soo fast he is the code <redacted> (1 Reply)
Discussion started by: benga
1 Replies

9. Solaris

Rsync quite slow (using very little cpu): how to improve its speed?

I have "inherited" a OmniOS (illumos based) server. I noticed rsync is significantly slower in respect to my reference, FreeBSD 12-CURRENT, running on exactly same hardware. Using same hardware, same command with same source and target disks, OmniOS r151026 gives: test@omniosce:~# time... (11 Replies)
Discussion started by: priyadarshan
11 Replies
Plack::App::CGIBin(3pm) 				User Contributed Perl Documentation				   Plack::App::CGIBin(3pm)

NAME
Plack::App::CGIBin - cgi-bin replacement for Plack servers SYNOPSIS
use Plack::App::CGIBin; use Plack::Builder; my $app = Plack::App::CGIBin->new(root => "/path/to/cgi-bin")->to_app; builder { mount "/cgi-bin" => $app; }; # Or from the command line plackup -MPlack::App::CGIBin -e 'Plack::App::CGIBin->new(root => "/path/to/cgi-bin")->to_app' DESCRIPTION
Plack::App::CGIBin allows you to load CGI scripts from a directory and convert them into a PSGI application. This would give you the extreme easiness when you have bunch of old CGI scripts that is loaded using cgi-bin of Apache web server. HOW IT WORKS
This application checks if a given file path is a perl script and if so, uses CGI::Compile to compile a CGI script into a sub (like ModPerl::Registry) and then run it as a persistent application using CGI::Emulate::PSGI. If the given file is not a perl script, it executes the script just like a normal CGI script with fork & exec. This is like a normal web server mode and no performance benefit is achieved. The default mechanism to determine if a given file is a Perl script is as follows: o Check if the filename ends with ".pl". If yes, it is a Perl script. o Open the file and see if the shebang (first line of the file) contains the word "perl" (like "#!/usr/bin/perl"). If yes, it is a Perl script. You can customize this behavior by passing "exec_cb" callback, which takes a file path to its first argument. For example, if your perl-based CGI script uses lots of global variables and such and are not ready to run on a persistent environment, you can do: my $app = Plack::App::CGIBin->new( root => "/path/to/cgi-bin", exec_cb => sub { 1 }, )->to_app; to always force the execute option for any files. AUTHOR
Tatsuhiko Miyagawa SEE ALSO
Plack::App::File CGI::Emulate::PSGI CGI::Compile Plack::App::WrapCGI See also Plack::App::WrapCGI if you compile one CGI script into a PSGI application without serving CGI scripts from a directory, to remove overhead of filesystem lookups, etc. perl v5.14.2 2011-11-02 Plack::App::CGIBin(3pm)
All times are GMT -4. The time now is 08:10 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy