Shell Programming and Scripting

I'm in need of some kind of script that will search for a string in each logfile in a directory but we don't want to use GREP. GREP seems to use up to much of our memory causing the server to use up a lot of swap space. Our log files are bigger than 500M on a daily basis. We lately started turning over our logs when it reaches 1G to help but it seems to still use up a lot of our memory causing the server to use up too much of its swap space.

Any recommendation on how we can search for strings/words in each logfile thats within a directory without having to use grep?

You might try something like this (untested):

Code:

SEARCH="this string"
awk -v S="$SEARCH" '/S/ {print $0}' *

However, i'm pretty the issue is NOT grep.
In fact, it is probably due to wrong usage (eg: too many pipes).
The more commands you use for a single search/compare the longer it takes and the more memory it uses.

hth

You can probably do it with sed or awk. What characters are in your search pattern? Are you looking for a fixed string, a match for a basic regular expression, or a match for an extended regular expression? Are you hoping to get a count of matching lines, a list of the matching lines, the line numbers of lines that match?

What options are you giving to grep when you use it to search for a string in each log file in a directory?

What operating system and shell are you using?

That won't work! The /S/ will print lines containing the character S. If the search string doesn't contain any characters that are special in an ERE, the following should work:

Code:

awk -v S="$SEARCH" '$0 ~ S' *.log

if the goal is just to print the matching lines and the target OS is not a Solaris/SunOS system.

the characters vary. sometimes we need to find a particular transaction id, look for errors, or a particular word.

Code:

> uname -a
Linux xx.xx.xx.com 2.6.18-371.6.1.el5 #1 SMP Tue Feb 18 11:42:11 EST 2014 x86_64 x86_64 x86_64 GNU/Linux

> ps -p $$
  PID TTY          TIME CMD
 5140 pts/0    00:00:00 ksh

I repeat: What grep command-line are you using to get the output you want?

here are some examples

Code:

zgrep 11263511 *.trace.log*
grep 1215456 *.log

We are not sure which grep command is what crashes the server. Our performance team can only see that its a grep command that causes the spike in swaps space.