Sponsored Content
Top Forums Shell Programming and Scripting Slow Perl script: how to speed up? Post 302506323 by pravin27 on Sunday 20th of March 2011 02:02:45 AM
Old 03-20-2011
Hi, I hope this will run faster than your script.

Code:
#!/usr/bin/perl

open (F1, $ARGV[0]);
open (F2, $ARGV[1]);
while (<F2>) {
chomp;
$hash{(split(/=/))[0]}++;
}
while (<F1>) {
    chomp;
    next if $hash{$_};
    print $_,"\n";
}
close(F1);
close(F2);

This User Gave Thanks to pravin27 For This Post:
 

9 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

network speed is slow

Hello, everyone: i encounter a problem these days , pls help me ,thanks in advance. my env: machine: ES40 A ES40 B os: true64 Unix 4.0f note: src.tar 8M network card speed 100M my problem: ... (3 Replies)
Discussion started by: q30
3 Replies

2. Shell Programming and Scripting

Optimize/speed-up perl extraction

Hi, Is there a way I can extract my data faster. You know my data is 1.2 GB text file with 8Million rows with 38 columns/fields. Imagine how huge this is. How I can optimized the data extraction using perl. That is why I'm creating a script to filter only those informations that I need. Is... (3 Replies)
Discussion started by: pinpe
3 Replies

3. UNIX for Advanced & Expert Users

speed test +20,000 file existance checks too slow

Need to make a very fast file existence checker. Passing in 20-50K num of files In the code below ${file} is a file with a listing of +20,000 files. test_speed is the script. I am commenting out the results of <time test_speed try>. The normal "test -f" is much much too slow when a system... (2 Replies)
Discussion started by: nullwhat
2 Replies

4. Shell Programming and Scripting

Speed up this script!

I have a script that processes a fair amount of data -- say, 25-50 megs per run. I'd like ideas on speeding it up. The code is actually just a preprocessor -- I'm using another language to do the heavy lifting. But as it happens, the preprocessing takes much more time than the final processing... (3 Replies)
Discussion started by: CRGreathouse
3 Replies

5. Filesystems, Disks and Memory

data from blktrace: read speed V.S. write speed

I analysed disk performance with blktrace and get some data: read: 8,3 4 2141 2.882115217 3342 Q R 195732187 + 32 8,3 4 2142 2.882116411 3342 G R 195732187 + 32 8,3 4 2144 2.882117647 3342 I R 195732187 + 32 8,3 4 2145 ... (1 Reply)
Discussion started by: W.C.C
1 Replies

6. Shell Programming and Scripting

Net::SSH::Perl slow to login.

I have some sample code that's supposed to ssh to another machine using Net::SSH::Perl, execute a command, and print the output of that command. It's very basic, and it works. However, I noticed that upon logging in: $ssh->login('username','password'); It takes roughly 10-13 seconds to... (2 Replies)
Discussion started by: mrwatkin
2 Replies

7. Shell Programming and Scripting

How can i speed this script up?

Hi, Im quite new to scripting and would like a bit of assistance with trying to speed up the following script. At the moment it is quite slow.... Any way to improve it? total=111120 while do total=`expr $total + 1` INCREMENT=$total firstline = "blablabla" secondline = "blablabla"... (5 Replies)
Discussion started by: brunlea
5 Replies

8. Shell Programming and Scripting

Help me with speed up this script

hey guys i have a perl script wich use to compare hashes but it tookes a long time to do that so i wich i will have the soulition to do it soo fast he is the code <redacted> (1 Reply)
Discussion started by: benga
1 Replies

9. Solaris

Rsync quite slow (using very little cpu): how to improve its speed?

I have "inherited" a OmniOS (illumos based) server. I noticed rsync is significantly slower in respect to my reference, FreeBSD 12-CURRENT, running on exactly same hardware. Using same hardware, same command with same source and target disks, OmniOS r151026 gives: test@omniosce:~# time... (11 Replies)
Discussion started by: priyadarshan
11 Replies
Inline::Files(3)					User Contributed Perl Documentation					  Inline::Files(3)

NAME
Inline::Files - Multiple virtual files at the end of your code VERSION
This document describes version 0.68 of Inline::Files, released July 23, 2011. SYNOPSIS
use Inline::Files; my Code $here; # etc. # etc. # etc. __FOO__ This is a virtual file at the end of the data __BAR__ This is another virtual file __FOO__ This is yet another such file WARNING
It is possible that this module may overwrite the source code in files that use it. To protect yourself against this possibility, you are strongly advised to use the "-backup" option described in "Safety first". This module is still experimental. Regardless of whether you use "-backup" or not, by using this module you agree that the authors will b<under no circumstances> be responsible for any loss of data, code, time, money, or limbs, or for any other disadvantage incurred as a result of using Inline::Files. DESCRIPTION
Inline::Files generalizes the notion of the "__DATA__" marker and the associated "<DATA>" filehandle, to an arbitrary number of markers and associated filehandles. When you add the line: use Inline::Files; to a source file you can then specify an arbitrary number of distinct virtual files at the end of the code. Each such virtual file is marked by a line of the form: __SOME_SYMBOL_NAME_IN_UPPER_CASE__ The following text -- up to the next such marker -- is treated as a file, whose (pseudo-)name is available as an element of the package array @SOME_SYMBOL_NAME_IN_UPPER_CASE. The name of the first virtual file with this marker is also available as the package scalar $SOME_SYMBOL_NAME_IN_UPPER_CASE. The filehandle of the same name is magical -- just like "ARGV" -- in that it automatically opens itself when first read. Furthermore -- just like "ARGV" -- the filehandle re-opens itself to the next appropriate virtual file (by "shift"-ing the first element of @SOME_SYMBOL_NAME_IN_UPPER_CASE into $SOME_SYMBOL_NAME_IN_UPPER_CASE) whenever it reaches EOF. So, just as with "ARGV", you can treat all the virtual files associated with a single symbol either as a single, multi-part file: use Inline::Files; while (<FILE>) { print "$FILE: $_"; } __FILE__ File 1 here __FILE__ File 2 here __OTHER_FILE__ Other file 1 __FILE__ File 3 here or as a series of individual files: use Inline::Files; foreach $filename (@FILE) { open HANDLE, $filename; print "<<$filename>> "; while (<HANDLE>) { print; } } __FILE__ File 1 here __FILE__ File 2 here __OTHER_FILE__ Other file 1 __FILE__ File 3 here Note that these two examples completely ignore the lines: __OTHER_FILE__ Other file 1 which would be accessed via the "OTHER_FILE" filehandle. Unlike "<ARGV>"/@ARGV/$ARGV, Inline::Files also makes use of the hash associated with an inline file's symbol. That is, when you create an inline file with a marker "__WHATEVER__", the hash %WHATEVER will contain information about that file. That information is: $WHATEVER{file} The name of the disk file in which the inlined "__WHATEVER__" files were defined; $WHATEVER{line} The line (starting from 1) at which the current inline "__WHATEVER__" file being accessed by "<WHATEVER>" started. $WHATEVER{offset} The byte offset (starting from 0) at which the current inline "__WHATEVER__" file being accessed by "<WHATEVER>" started. $WHATEVER{writable} Whether the the current inline file being accessed by "<WHATEVER>" is opened for output. The hash and its elements are read-only and the entry values are only meaningful when the corresponding filehandle is open. Writable virtual files If the source file that uses Inline::Files is itself writable, then the virtual files it contains may also be opened for write access. For example, here is a very simple persistence mechanism: use Inline::Files; use Data::Dumper; open CACHE or die $!; # read access (uses $CACHE to locate file) eval join "", <CACHE>; close CACHE or die $!; print "$var was '$var' "; while (<>) { chomp; $var = $_; print "$var now '$var' "; } open CACHE, ">$CACHE" or die $!; # write access print CACHE Data::Dumper->Dump([$var],['var']); close CACHE or die $!; __CACHE__ $var = 'Original value'; Unlike "ARGV", if a virtual file is part of a writable file and is automagically opened, it is opened for full read/write access. So the above example, could be even simpler: use Inline::Files; use Data::Dumper; eval join "", <CACHE>; # Automagically opened print "$var was '$var' "; while (<>) { chomp; $var = $_; print "$var now '$var' "; } seek CACHE, 0, 0; print CACHE Data::Dumper->Dump([$var],['var']); __CACHE__ $var = 'Original value'; In either case, the original file is updated only at the end of execution, on an explicit "close" of the virtual file's handle, or when "Inline::Files::Virtual::vf_save" is explicitly called. Creating new Inline files on the fly. You can also open up new Inline output files at run time. Simply use the open function with a valid new Inline file handle name and no file name. Like this: use Inline::Files; open IFILE, '>'; print IFILE "This line will be placed into a new Inline file "; print IFILE "which is marked by '__IFILE__' "; Safety first Because Inline::Files handles are often read-write, it's possible to accidentally nuke your hard-won data. But Inline::Files can save you from yourself. If Inline::Files is loaded with the "-backup" option: use Inline::Files -backup; then the source file that uses it is backed up before the inline files are extracted. The backup file is the name of the source file with the suffix ".bak" appended. You can also specify a different name for the backup file, by associating that name with the "-backup" flag: use Inline::Files -backup => '/tmp/sauve_qui_peut'; SEE ALSO
The Inline::Files::Virtual module The Filter::Util::Call module BUGS ADDED BY Alberto Simoes (ambs@cpan.org) UNWITTING PAWN OF AN AUTHOR
Damian Conway (damian@conway.org) EVIL MASTERMIND BEHIND IT ALL
Brian Ingerson (INGY@cpan.org) COPYRIGHT
Copyright (c) 2001-2009. Damian Conway. All rights reserved. This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself. See http://www.perl.com/perl/misc/Artistic.html perl v5.16.3 2011-07-23 Inline::Files(3)
All times are GMT -4. The time now is 05:26 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy