Sponsored Content
Top Forums Shell Programming and Scripting Need to Speed up shell script Post 302988931 by RudiC on Thursday 5th of January 2017 08:04:00 AM
Old 01-05-2017
Every single redirection opens, reads / writes, and closes the target file. So, for every loop of your script you're handling roughly 80 file operations (times 2, for open and close). Try redirecting the entire script's output, e.g. like
Code:
./script > result.log

.
And, you're running 17 greps per loop, each causing a process creation with its resource consumption, - try running it once with either an alternation search pattern ("PAT1|PAT2|PAT3") or a pattern file (-f option), which would simplify some later treatment, and deal with your pretty print later with e.g. a single sort and / or a single awk command.
 

10 More Discussions You Might Find Interesting

1. Filesystems, Disks and Memory

dmidecode, RAM speed = "Current Speed: Unknown"

Hello, I have a Supermicro server with a P4SCI mother board running Debian Sarge 3.1. This is the "dmidecode" output related to RAM info: RAM speed information is incomplete.. "Current Speed: Unknown", is there anyway/soft to get the speed of installed RAM modules? thanks!! Regards :)... (0 Replies)
Discussion started by: Santi
0 Replies

2. Shell Programming and Scripting

any way to speed up calculations in bash script

hi i have a script that is taking the difference of multiple columns in a file from a value from a single row..so far i have a loop to do that.. all the data is floating point..fin has the difference between array1 and array2..array1 has 700 x 300= 210000 values and array2 has 700 values.. ... (11 Replies)
Discussion started by: npatwardhan
11 Replies

3. Shell Programming and Scripting

Help to improve speed of text processing script

Hey together, You should know, that I'am relatively new to shell scripting, so my solution is probably a little awkward. Here is the script: #!/bin/bash live_dir=/var/lib/pokerhands/live for limit in `find $live_dir/ -type d | sed -e s#$live_dir/##`; do cat $live_dir/$limit/*... (19 Replies)
Discussion started by: lorus
19 Replies

4. Shell Programming and Scripting

Speed up this script!

I have a script that processes a fair amount of data -- say, 25-50 megs per run. I'd like ideas on speeding it up. The code is actually just a preprocessor -- I'm using another language to do the heavy lifting. But as it happens, the preprocessing takes much more time than the final processing... (3 Replies)
Discussion started by: CRGreathouse
3 Replies

5. Filesystems, Disks and Memory

data from blktrace: read speed V.S. write speed

I analysed disk performance with blktrace and get some data: read: 8,3 4 2141 2.882115217 3342 Q R 195732187 + 32 8,3 4 2142 2.882116411 3342 G R 195732187 + 32 8,3 4 2144 2.882117647 3342 I R 195732187 + 32 8,3 4 2145 ... (1 Reply)
Discussion started by: W.C.C
1 Replies

6. Shell Programming and Scripting

Any trick to speed up script?

Hi Guys, I have a script that I am using to convert some text files to xls files. I create multiple temp. files in the process of conversion. Other than reducing the temp. files, are there any general tricks to help speed up the script? I am running it in the bash shell. Thanks. (6 Replies)
Discussion started by: npatwardhan
6 Replies

7. Shell Programming and Scripting

Slow Perl script: how to speed up?

I had written a perl script to compare two files: new and master and get the output of the first file i.e. the first file: words that are not in the master file STRUCTURE OF THE TWO FILES The first file is a series of names ramesh sushil jonga sudesh lugdi whereas the second file (could be... (4 Replies)
Discussion started by: gimley
4 Replies

8. Shell Programming and Scripting

How can i speed this script up?

Hi, Im quite new to scripting and would like a bit of assistance with trying to speed up the following script. At the moment it is quite slow.... Any way to improve it? total=111120 while do total=`expr $total + 1` INCREMENT=$total firstline = "blablabla" secondline = "blablabla"... (5 Replies)
Discussion started by: brunlea
5 Replies

9. Shell Programming and Scripting

Help me with speed up this script

hey guys i have a perl script wich use to compare hashes but it tookes a long time to do that so i wich i will have the soulition to do it soo fast he is the code <redacted> (1 Reply)
Discussion started by: benga
1 Replies

10. Shell Programming and Scripting

Speed up the loop in shell script

Hi I have written a shell script which will test 300 to 500 IPs to find which are pinging and which are not pinging. the script which give output as 10.x.x.x is pining 10.x.x.x. is not pining - - - 10.x.x.x is pining like above. But, this script is taking... (6 Replies)
Discussion started by: kumar85shiv
6 Replies
CG(1)																	     CG(1)

NAME
cg - Recursively grep for a pattern and store it. SYNOPSIS
cg [ -l ] | [ [ -i ] pattern [ files ] ] DESCRIPTION
cg does a search though text files (usually source code) recursively for a pattern, storing matches and displaying the output in a human- readable fashion. It is intended to give some of the functionaly of AT&T's cscope(1) tool, with the advantages of simplicity and not being language-specific. The script will colorize output if configured as such. It is typically run with a Perl regular expression to search for. The search can be made case insensitive by using the -i option. A list of files may also be specified with an additional argument after the pattern. Put the files pattern in quotes to make it be matched by Perl rather than by the shell. Running the script with no arguments will recall the results of the previous search. After the search, entries found can be edited using the vg(1) script. The -l option shows the last log made. SOME EXAMPLES
cg - alone recalls the previous search results. cg -i pattern - search the default list of files for all files matching the pattern (and case-insensitively). cg pattern '*.c' - search recursively for pattern in all *.c files. This automatically converts '*' to '.*' and '.' to '.' for you and does a Perl pattern match on all files in the tree. cg pattern *.c - search through the shell-expanded list of *.c files, so not done recursively (in other words, only the files your shell pass to the script as arguments). cg -l - show the last log made. COMMAND-LINE OPTIONS -i Do a case-insensitive search. -l Show the last log made. -p Toggle the default pager option. cg has a bulit-in pager function, which can be enabled or disabled by default (in .cgvgrc). If the default is enabled, this option disables the pager; if the default is disabled, this option enables it. -P Force the built-in pager to be disabled. FILES
${HOME}/.cglast Log file of the last search. ${HOME}/.cgvgrc Per-user configuration file (if the defaults are not desireable). ${HOME}/.cgvg/* Log files in $HOSTNAME.shell_pid form with the log of the last search. SEE ALSO
vg(1), perl(1), find(1), grep(1), cscope(1) AUTHOR
cg was written by Joshua Uziel <uzi@uzix.org>. 13 Mar 2002 CG(1)
All times are GMT -4. The time now is 08:19 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy