Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: How to speed up grep?
Top Forums Shell Programming and Scripting How to speed up grep? Post 302375108 by methyl on Thursday 26th of November 2009 01:36:05 PM
Old 11-26-2009
Not sure of your exact requirement - particularly whether there are dependencies. The "egrep" command can search for multiple messages in one pass.

Code:
ls -1tr logfile* | while read FILENAME
do
    egrep -i "message 1|message 2|message 3|message 4" "${FILENAME}"
done

I sometimes use "egrep" to eliminate known good messages ( egrep -iv ) and then only look at the exceptions.

Unless you only look at the logs every 10 days there is little point in reading old logs a second time unless the search criteria changes after the event.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Speed it up!

I wonder, are there any "tricks" to increase my server's access time in general? (4 Replies)
Discussion started by: pappous
4 Replies

2. UNIX for Dummies Questions & Answers

grep - speed of search

I grepped for a string from a directory of very large files. This took quite a long time (not a problem). When I grepped for a different string from the same files immediately after, the output was MUCH quicker. My question is, does anybody know why the second grep was so much quicker than the... (1 Reply)
Discussion started by: davirime
1 Replies

3. UNIX for Dummies Questions & Answers

Speed of mv vs. cp

Hi, Is mv (move) command quicker than cp (copy command)? I have large files and I want to know if mv actually copy the data to a new file then deletes the old or whether it just alters information the file system without physically moving data - Unfortuanately I don't have large files to test... (2 Replies)
Discussion started by: GMMike
2 Replies

4. AIX

cpu speed

how do i determine the speed of a cpu on AIX 4.3.3 or 5.1? (5 Replies)
Discussion started by: csaunders
5 Replies

5. Filesystems, Disks and Memory

dmidecode, RAM speed = "Current Speed: Unknown"

Hello, I have a Supermicro server with a P4SCI mother board running Debian Sarge 3.1. This is the "dmidecode" output related to RAM info: RAM speed information is incomplete.. "Current Speed: Unknown", is there anyway/soft to get the speed of installed RAM modules? thanks!! Regards :)... (0 Replies)
Discussion started by: Santi
0 Replies

6. Shell Programming and Scripting

Optimizing for a Speed-up

How would one go about optimizing this current .sh program so it works at a more minimal time. Such as is there a better way to count what I need than what I have done or better way to match patterns in the file? Thanks, #declare variables to be used. help=-1 count=0 JanCount=0 FebCount=0... (3 Replies)
Discussion started by: switch
3 Replies

7. Shell Programming and Scripting

Speed up this script!

I have a script that processes a fair amount of data -- say, 25-50 megs per run. I'd like ideas on speeding it up. The code is actually just a preprocessor -- I'm using another language to do the heavy lifting. But as it happens, the preprocessing takes much more time than the final processing... (3 Replies)
Discussion started by: CRGreathouse
3 Replies

8. Filesystems, Disks and Memory

data from blktrace: read speed V.S. write speed

I analysed disk performance with blktrace and get some data: read: 8,3 4 2141 2.882115217 3342 Q R 195732187 + 32 8,3 4 2142 2.882116411 3342 G R 195732187 + 32 8,3 4 2144 2.882117647 3342 I R 195732187 + 32 8,3 4 2145 ... (1 Reply)
Discussion started by: W.C.C
1 Replies

9. Shell Programming and Scripting

Speed Up Grep

Hi, I have to grep string from 20 - 30 files each carries 200 - 300 MB size and append to the file. How to speed the grepping time. cat catalina.out_2012_01_01 | grep "xxxxx" >> backup.txt PLZ, Suggest me, Regards, Nanthagopal A (5 Replies)
Discussion started by: nanthagopal
5 Replies

10. Shell Programming and Scripting

Can I speed up my grep command?

I've got at least 30,000 XML files that I'm using the grep command to get their filename. Can I use the head command to grab just the beginning 8 lines and compare that instead of parsing the whole document? It would speed things up! or maybe grep -m? (49 Replies)
Discussion started by: emc^24sho
49 Replies

LEARN ABOUT BSD

zgrep

ZGREP(1)						      General Commands Manual							  ZGREP(1)

NAME

       zgrep - search possibly compressed files for a regular expression

SYNOPSIS

       zgrep [ grep_options ] [ -e ] pattern filename...

DESCRIPTION

       Zgrep  invokes  grep  on  compressed  or  gzipped  files.   These  grep	options  will  cause  zgrep  to  terminate  with  an  error  code:
       (-[drRzZ]|--di*|--exc*|--inc*|--rec*|--nu*).  All other options specified are passed directly to grep.  If no file is specified,  then  the
       standard input is decompressed if necessary and fed to grep.  Otherwise the given files are uncompressed if necessary and fed to grep.

       If the GREP environment variable is set, zgrep uses it as the grep program to be invoked.

EXIT CODE

       2 - An option that is not supported was specified.

AUTHOR

       Charles Levert (charles@comm.polymtl.ca)

SEE ALSO

       grep(1), gzexe(1), gzip(1), zdiff(1), zforce(1), zmore(1), znew(1)

																	  ZGREP(1)
All times are GMT -4. The time now is 06:43 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy