Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: egrep is very slow : How to improve performance
Top Forums Shell Programming and Scripting egrep is very slow : How to improve performance Post 302166545 by HPAVC on Tuesday 12th of February 2008 06:13:15 AM
Old 02-12-2008
Quote:
Originally Posted by hidnana
I have uploaded the $key as a screenshot as I don't have the text version right now..., it's a big string concatenated by "|".
I had a feeling it was a bloaded regular expression, a regex optimizer is what you need.

You are giving egrep (which is a grep -E dedicated) a pile of 'check for this or this or this or this'. The form you have it in is quite unwieldy. If that could be reduced to this ...

Code:
TP-CAP-P[0-9]{9}-[0-9]{9}

If your not keen on the regular expression thing you can use a program like regex buddy to load your data in (or a dozen mb or so) and then test it.

Code:
TP-CAP-P123456789-103456789
TP-CAP-P124456789-103456789
TP-CAP-P123458789-123456709
TP-CAP-P123456789-123056719
TP-CAP-P123459989-123406789

and get a sense of the regex back (this is from mkregexp from just the above).

Code:
qr/(?=[1CPT])(?:1(?:23(?:4(?:5670|0678)9|056719)|03456789)|P12(?:345(?:[68]7|99)89|4456789)|(?:T|CA)P)/

My guess is that you want to do something else, but for what your doing a 30seconds isn't that bad for huge files.
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

improve performance by using ls better than find

Hi , i'm searching for files over many Aix servers with rsh command using this request : find /dir1 -name '*.' -exec ls {} \; and then count them with "wc" but i would improve this search because it's too long and replace directly find with ls command but "ls *. " doesn't work. and... (3 Replies)
Discussion started by: Nicol
3 Replies

2. Shell Programming and Scripting

How to improve grep performance...

Hi All, I am using grep command to find string "abc" in one file . content of file is *********** abc = xyz def= lmn ************ i have given the below mentioned command to redirect the output to tmp file grep abc file | sort -u | awk '{print #3}' > out_file Then i am searching... (2 Replies)
Discussion started by: pooga17
2 Replies

3. UNIX for Dummies Questions & Answers

Improve Performance

hi someone tell me which ways i can improve disk I/O and system process performance.kindly refer some commands so i can do it on my test machine.thanks, Mazhar (2 Replies)
Discussion started by: mazhar99
2 Replies

4. Shell Programming and Scripting

Any way to improve performance of this script

I have a data file of 2 gig I need to do all these, but its taking hours, any where i can improve performance, thanks a lot #!/usr/bin/ksh echo TIMESTAMP="$(date +'_%y-%m-%d.%H-%M-%S')" function showHelp { cat << EOF >&2 syntax extreme.sh FILENAME Specify filename to parse EOF... (3 Replies)
Discussion started by: sirababu
3 Replies

5. Shell Programming and Scripting

Want to improve the performance of script

Hi All, I have written a script as follows which is taking lot of time in executing/searching only 3500 records taken as input from one file in log file of 12 GB Approximately. Working of script is read the csv file as an input having 2 arguments which are transaction_id,mobile_number and search... (6 Replies)
Discussion started by: poweroflinux
6 Replies

6. Programming

Help with improve the performance of grep

Input file: #content_1 12314345345 242467 #content_14 436677645 576577657 #content_100 3425546 56 #content_12 243254546 1232454 . . Reference file: content_100 (1 Reply)
Discussion started by: cpp_beginner
1 Replies

7. UNIX for Dummies Questions & Answers

How to improve the performance of this script?

Hi , i wrote a script to convert dates to the formate i want .it works fine but the conversion is tkaing lot of time . Can some one help me tweek this script #!/bin/bash file=$1 ofile=$2 cp $file $ofile mydates=$(grep -Po '+/+/+' $ofile) # gets 8/1/13 mydates=$(echo "$mydates" | sort |... (5 Replies)
Discussion started by: vikatakavi
5 Replies

8. Programming

Improve the performance of my C++ code

Hello, Attached is my very simple C++ code to remove any substrings (DNA sequence) of each other, i.e. any redundant sequence is removed to get unique sequences. Similar to sort | uniq command except there is reverse-complementary for DNA sequence. The program runs well with small dataset, but... (11 Replies)
Discussion started by: yifangt
11 Replies

9. Shell Programming and Scripting

Improve script - slow process with big files

Gents, Please can u help me to improve this script to be more faster, it works perfectly but for big files take a lot time to end the job.. I see the problem is in the step (while) and in this part the script takes a lot time.. Please if you can find a best way to do will be great. ... (13 Replies)
Discussion started by: jiam912
13 Replies

10. Solaris

Rsync quite slow (using very little cpu): how to improve its speed?

I have "inherited" a OmniOS (illumos based) server. I noticed rsync is significantly slower in respect to my reference, FreeBSD 12-CURRENT, running on exactly same hardware. Using same hardware, same command with same source and target disks, OmniOS r151026 gives: test@omniosce:~# time... (11 Replies)
Discussion started by: priyadarshan
11 Replies

LEARN ABOUT DEBIAN

cgi::application::plugin::dbiprofile::graph::svgtt

CGI::Application::Plugin::DBIProfile::Graph::SVGTT(3pm) User Contributed Perl DocumentationCGI::Application::Plugin::DBIProfile::Graph::SVGTT(3pm)

NAME

       CGI::Application::Plugin::DBIProfile::Graph::SVGTT - SVT::TT::Graph::Bar Graph output for CAP:DBIProfile.

SYNOPSIS

	   # in httpd.conf
	   SetVar CAP_DBIPROFILE_GRAPHMODULE CGI::Application::Plugin::DBIProfile::Graph::SVGTT
	   PerlSetVar CAP_DBIPROFILE_GRAPHMODULE CGI::Application::Plugin::DBIProfile::Graph::SVGTT

	   # in your CGI::Application subclass (needed to install callback)
	   use CGI::Application;
	   use CGI::Application::Plugin::DBIProfile::Graph::SVGTT;

DESCRIPTION

       This module provides a SVG::TT::Graph::Bars graphing option for CAP:DBIProfile.

       This also provides an example of non-inline graphs for DBIProfile.

       The following settings control the output:

       $CGI::Application::Plugin::DBIProfile::Graph::SVGTT::WIDTH
	   Width of output image.

       $CGI::Application::Plugin::DBIProfile::Graph::SVGTT::HEIGHT
	   Height of output image.

REQUIREMENTS

       SVG::TT::Graph

SEE ALSO

       SVG::TT::Graph::Bar
       CGI::Application::Plugin::DBIProfile
       CGI::Application::Plugin::DBIProfile::Graph::HTML

AUTHOR

	   Joshua I Miller, L<unrtst@cpan.org>

COPYRIGHT &; LICENSE
       Copyright 2007 Joshua Miller, all rights reserved.

       This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

perl v5.12.4							    2011-11-10		   CGI::Application::Plugin::DBIProfile::Graph::SVGTT(3pm)
All times are GMT -4. The time now is 09:03 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy