03-20-2011
Dear Pravin27
I had written an awk script to do the job. But it was giving false positives when there was too much data.
Here is the script:
Practically the same as yours. When I had put it up, it was suggested to use an "O" for a large array:
BEGIN {FS="="}
NR==FNR{O[$1]++;next} !($1 in O)
Unluckily the script is fast but gives false positives, hence the PERL which gives correct positives but is slow.
I tried yours, it also gives false positives on a huge fle compare.
Manyt hanks.
Gimley
9 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
Hello, everyone:
i encounter a problem these days , pls help me ,thanks in advance.
my env:
machine: ES40 A ES40 B
os: true64 Unix 4.0f
note: src.tar 8M network card speed 100M
my problem:
... (3 Replies)
Discussion started by: q30
3 Replies
2. Shell Programming and Scripting
Hi,
Is there a way I can extract my data faster. You know my data is 1.2 GB text file with 8Million rows with 38 columns/fields. Imagine how huge this is.
How I can optimized the data extraction using perl. That is why I'm creating a script to filter only those informations that I need. Is... (3 Replies)
Discussion started by: pinpe
3 Replies
3. UNIX for Advanced & Expert Users
Need to make a very fast file existence checker. Passing in 20-50K num of files
In the code below ${file} is a file with a listing of +20,000 files. test_speed is the script. I am commenting out the results of <time test_speed try>.
The normal "test -f" is much much too slow when a system... (2 Replies)
Discussion started by: nullwhat
2 Replies
4. Shell Programming and Scripting
I have a script that processes a fair amount of data -- say, 25-50 megs per run. I'd like ideas on speeding it up. The code is actually just a preprocessor -- I'm using another language to do the heavy lifting. But as it happens, the preprocessing takes much more time than the final processing... (3 Replies)
Discussion started by: CRGreathouse
3 Replies
5. Filesystems, Disks and Memory
I analysed disk performance with blktrace and get some data:
read:
8,3 4 2141 2.882115217 3342 Q R 195732187 + 32
8,3 4 2142 2.882116411 3342 G R 195732187 + 32
8,3 4 2144 2.882117647 3342 I R 195732187 + 32
8,3 4 2145 ... (1 Reply)
Discussion started by: W.C.C
1 Replies
6. Shell Programming and Scripting
I have some sample code that's supposed to ssh to another machine using Net::SSH::Perl, execute a command, and print
the output of that command. It's very basic, and it works. However, I noticed that upon logging in:
$ssh->login('username','password');
It takes roughly 10-13 seconds to... (2 Replies)
Discussion started by: mrwatkin
2 Replies
7. Shell Programming and Scripting
Hi,
Im quite new to scripting and would like a bit of assistance with trying to speed up the following script. At the moment it is quite slow....
Any way to improve it?
total=111120
while
do
total=`expr $total + 1`
INCREMENT=$total
firstline = "blablabla"
secondline = "blablabla"... (5 Replies)
Discussion started by: brunlea
5 Replies
8. Shell Programming and Scripting
hey guys i have a perl script wich use to compare hashes but it tookes a long time to do that so i wich i will have the soulition to do it soo fast
he is the code
<redacted> (1 Reply)
Discussion started by: benga
1 Replies
9. Solaris
I have "inherited" a OmniOS (illumos based) server.
I noticed rsync is significantly slower in respect to my reference, FreeBSD 12-CURRENT, running on exactly same hardware.
Using same hardware, same command with same source and target disks, OmniOS r151026 gives:
test@omniosce:~# time... (11 Replies)
Discussion started by: priyadarshan
11 Replies
LEARN ABOUT DEBIAN
plack::app::cgibin
Plack::App::CGIBin(3pm) User Contributed Perl Documentation Plack::App::CGIBin(3pm)
NAME
Plack::App::CGIBin - cgi-bin replacement for Plack servers
SYNOPSIS
use Plack::App::CGIBin;
use Plack::Builder;
my $app = Plack::App::CGIBin->new(root => "/path/to/cgi-bin")->to_app;
builder {
mount "/cgi-bin" => $app;
};
# Or from the command line
plackup -MPlack::App::CGIBin -e 'Plack::App::CGIBin->new(root => "/path/to/cgi-bin")->to_app'
DESCRIPTION
Plack::App::CGIBin allows you to load CGI scripts from a directory and convert them into a PSGI application.
This would give you the extreme easiness when you have bunch of old CGI scripts that is loaded using cgi-bin of Apache web server.
HOW IT WORKS
This application checks if a given file path is a perl script and if so, uses CGI::Compile to compile a CGI script into a sub (like
ModPerl::Registry) and then run it as a persistent application using CGI::Emulate::PSGI.
If the given file is not a perl script, it executes the script just like a normal CGI script with fork & exec. This is like a normal web
server mode and no performance benefit is achieved.
The default mechanism to determine if a given file is a Perl script is as follows:
o Check if the filename ends with ".pl". If yes, it is a Perl script.
o Open the file and see if the shebang (first line of the file) contains the word "perl" (like "#!/usr/bin/perl"). If yes, it is a Perl
script.
You can customize this behavior by passing "exec_cb" callback, which takes a file path to its first argument.
For example, if your perl-based CGI script uses lots of global variables and such and are not ready to run on a persistent environment, you
can do:
my $app = Plack::App::CGIBin->new(
root => "/path/to/cgi-bin",
exec_cb => sub { 1 },
)->to_app;
to always force the execute option for any files.
AUTHOR
Tatsuhiko Miyagawa
SEE ALSO
Plack::App::File CGI::Emulate::PSGI CGI::Compile Plack::App::WrapCGI
See also Plack::App::WrapCGI if you compile one CGI script into a PSGI application without serving CGI scripts from a directory, to remove
overhead of filesystem lookups, etc.
perl v5.14.2 2011-11-02 Plack::App::CGIBin(3pm)