How can i speed this script up? Post: 302611289

Sponsored Content

Top Forums Shell Programming and Scripting How can i speed this script up? Post 302611289 by Corona688 on Thursday 22nd of March 2012 02:05:22 PM

03-22-2012

Registered User

Quote:

Originally Posted by brunlea

Hi,

Im quite new to scripting and would like a bit of assistance with trying to speed up the following script. At the moment it is quite slow....

Do you realize you're never clearing that XML file? It's just growing and growing every loop!

Anyway, you can put all that text in a here-document to avoid 99 echo calls writing to the same file:

Code:

cat <<EOF >file.xml
line1
lne2 $variable
line3
EOF

Instead of running the same thing 9 times to do 9 queries, you could probably run the same thing once to handle all nine queries, tying them directly in with a pipe.

Code:

total=111120
while [ $total -lt 111130 ]
do
        total=`expr $total + 1`

# Note the ending EOF below the last 'blablabla' MUST be at the very
# beginning of the line or the here-document will eat the entire rest of the
# script!
        cat <<EOF
blablabla
blablabla $total
blablabla
blablabla
blablabla
blablabla
blablabla
EOF

        # Print this to stderr so it doesn't end up in your query or what have you
        echo file done >&2
done | database_query_using_stream

What is this 'database query which uses file'? I suspect that's going to be the limiting step here and we can't tell you how to rewrite it without seeing it.

Corona688

View Public Profile for Corona688

Visit Corona688's homepage!

Find all posts by Corona688

10 More Discussions You Might Find Interesting

1. Filesystems, Disks and Memory

dmidecode, RAM speed = "Current Speed: Unknown"

Hello, I have a Supermicro server with a P4SCI mother board running Debian Sarge 3.1. This is the "dmidecode" output related to RAM info: RAM speed information is incomplete.. "Current Speed: Unknown", is there anyway/soft to get the speed of installed RAM modules? thanks!! Regards :)...

2. Shell Programming and Scripting

any way to speed up calculations in bash script

hi i have a script that is taking the difference of multiple columns in a file from a value from a single row..so far i have a loop to do that.. all the data is floating point..fin has the difference between array1 and array2..array1 has 700 x 300= 210000 values and array2 has 700 values.. ...

3. Shell Programming and Scripting

Help to improve speed of text processing script

Hey together, You should know, that I'am relatively new to shell scripting, so my solution is probably a little awkward. Here is the script: #!/bin/bash live_dir=/var/lib/pokerhands/live for limit in `find $live_dir/ -type d | sed -e s#$live_dir/##`; do cat $live_dir/$limit/*...

4. Shell Programming and Scripting

Speed up this script!

I have a script that processes a fair amount of data -- say, 25-50 megs per run. I'd like ideas on speeding it up. The code is actually just a preprocessor -- I'm using another language to do the heavy lifting. But as it happens, the preprocessing takes much more time than the final processing...

5. Filesystems, Disks and Memory

data from blktrace: read speed V.S. write speed

I analysed disk performance with blktrace and get some data: read: 8,3 4 2141 2.882115217 3342 Q R 195732187 + 32 8,3 4 2142 2.882116411 3342 G R 195732187 + 32 8,3 4 2144 2.882117647 3342 I R 195732187 + 32 8,3 4 2145 ...

6. Shell Programming and Scripting

Any trick to speed up script?

Hi Guys, I have a script that I am using to convert some text files to xls files. I create multiple temp. files in the process of conversion. Other than reducing the temp. files, are there any general tricks to help speed up the script? I am running it in the bash shell. Thanks.

7. Shell Programming and Scripting

Slow Perl script: how to speed up?

I had written a perl script to compare two files: new and master and get the output of the first file i.e. the first file: words that are not in the master file STRUCTURE OF THE TWO FILES The first file is a series of names ramesh sushil jonga sudesh lugdi whereas the second file (could be...

8. Shell Programming and Scripting

Help me with speed up this script

hey guys i have a perl script wich use to compare hashes but it tookes a long time to do that so i wich i will have the soulition to do it soo fast he is the code <redacted>

9. Shell Programming and Scripting

Speed up the loop in shell script

Hi I have written a shell script which will test 300 to 500 IPs to find which are pinging and which are not pinging. the script which give output as 10.x.x.x is pining 10.x.x.x. is not pining - - - 10.x.x.x is pining like above. But, this script is taking...

10. Shell Programming and Scripting

Need to Speed up shell script

Hello, I am basic level shell script developer. I have developed the following script. The shell script basically tracking various files containing certain strings. I am finding options to make the script run more faster. Any help/suggestion would be appreciated :) #! /bin/bash # Greps for...

LEARN ABOUT SUSE

xml::sax::pureperl

SAX::PurePerl(3)					User Contributed Perl Documentation					  SAX::PurePerl(3)

NAME

       XML::SAX::PurePerl - Pure Perl XML Parser with SAX2 interface

SYNOPSIS

	 use XML::Handler::Foo;
	 use XML::SAX::PurePerl;
	 my $handler = XML::Handler::Foo->new();
	 my $parser = XML::SAX::PurePerl->new(Handler => $handler);
	 $parser->parse_uri("myfile.xml");

DESCRIPTION

       This module implements an XML parser in pure perl. It is written around the upcoming perl 5.8's unicode support and support for multiple
       document encodings (using the PerlIO layer), however it has been ported to work with ASCII/UTF8 documents under lower perl versions.

       The SAX2 API is described in detail at http://sourceforge.net/projects/perl-xml/, in the CVS archive, under libxml-perl/docs. Hopefully
       those documents will be in a better location soon.

       Please refer to the SAX2 documentation for how to use this module - it is merely a front end to SAX2, and implements nothing that is not in
       that spec (or at least tries not to - please email me if you find errors in this implementation).

BUGS

       XML::SAX::PurePerl is slow. Very slow. I suggest you use something else in fact. However it is great as a fallback parser for XML::SAX,
       where the user might not be able to install an XS based parser or C library.

       Currently lots, probably. At the moment the weakest area is parsing DOCTYPE declarations, though the code is in place to start doing this.
       Also parsing parameter entity references is causing me much confusion, since it's not exactly what I would call trivial, or well documented
       in the XML grammar. XML documents with internal subsets are likely to fail.

       I am however trying to work towards full conformance using the Oasis test suite.

AUTHOR

       Matt Sergeant, matt@sergeant.org. Copyright 2001.

       Please report all bugs to the Perl-XML mailing list at perl-xml@listserv.activestate.com.

LICENSE

       This is free software. You may use it or redistribute it under the same terms as Perl 5.7.2 itself.

perl v5.12.1							    2008-08-05							  SAX::PurePerl(3)