Is it better to grep and pipe to awk, or to seach with awk itself Post: 302243994

Sponsored Content

Top Forums Shell Programming and Scripting Is it better to grep and pipe to awk, or to seach with awk itself Post 302243994 by DeCoTwc on Tuesday 7th of October 2008 02:24:25 AM

10-07-2008

Registered User

Quote:

Originally Posted by cfajohnson

Why are you using command substitution instead of a straight assignment? In all shells except ksh93, it forks a new process and is almost as slow as an external command. Use:
Code:
user=$UID

The search code in grep is much faster than that in all versions of awk except mawk, but it will not make much difference except on very large files.

The authors of AWK, in their book, The AWK Programming Language, recommend using grep to search and piping the results though awk for processing rather than doing it all in AWK.

The UID thing...because I'm for lack of a better word...a newb.

Thanks for the info regarding the awk V. grep. The log I'm searching gets to be rather large as it only rolls every 12 hours. And even 45 minutes into it's cycle there was already a 3 second difference in running with grep as opposed to grep.

My second question is an extension of the first. Is there any benefit to giving grep more or less to search for?

is it better to grep for 0x12, as that's the error code, or to grep for "AddResCnf response (0x12) is not 'OK'" which is the entire error? In my mind I could think of logical reasons why I could think of why a longer search term is better than a short one...but as I said, I'm kind of a new.

DeCoTwc

View Public Profile for DeCoTwc

Find all posts by DeCoTwc

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

AWK seach for exact word in certain column

Can anyone help me how I will extract all lines in a file where the word "worker" or "co-worker" in 2nd column exist. There are also word in 2nd column like "workers" or "worker2" but I don't want to display those lines. Appreciate any help in advance! Thank you!

2. Shell Programming and Scripting

pipe'ing grep output to awk

This script is supposed to find out if tomcat is running or not. #!/bin/sh if netstat -a | grep `grep ${1}: /tomcat/bases | awk -F: '{print $3}'` > /dev/null then echo Tomcat for $1 running else echo Tomcat for $1 NOT running fi the /tomcat/bases is a file that...

3. Shell Programming and Scripting

Read content between xml tags with awk, grep, awk or what ever...

Hello, I trying to extract text that is surrounded by xml-tags. I tried this cat tst.xml | egrep "<SERVER>.*</SERVER>" |sed -e "s/<SERVER>$.*$<\/SERVER>/\1/"|tr "|" " " which works perfect, if the start-tag and the end-tag are in the same line, e.g.: <tag1>Hello Linux-Users</tag1> ...

4. Shell Programming and Scripting

Pipe to awk to variable

Hi! If I'm trying something like: echo "hello world" | myvar=`awk -F "world" '{print $1}'` echo $myvar myvar is always empty :confused: I googled for houres now and don't understand why it isn't working... Trying it in normal bash. Can someone explain it to me so I can say "Of course!...

5. Shell Programming and Scripting

help with sed or awk with less pipe

<tr><th align=right valign=top>Faulty_Part</th><td align=left valign=top>readhat version 6.0</td></tr> <tr><th align=right valign=top>Submit_Date</th><td align=left valign=top>2011-04-28 02:08:02</td></tr> .......(a long string) I want to get all the field between "left valign=top>" and "...

6. Shell Programming and Scripting

Find files and seach limit with grep

Hi, I'm testing nginx-cache-purge and notice that grep searches all file content, and since the cache key is in the second line I would like to limit grep searching. This is the script I'm using: github.com/perusio/nginx-cache-purge The line I would like to change and limit grep search...

7. Shell Programming and Scripting

awk print pipe

Hey fellas, I wrote an script which its output is like this: a 1 T a 1 T a 2 A b 5 G b 5 G b 5 G I wanna print $1 $2 and the total number of $2 value as the third column and after that $3. Sth like this: a 1 2 T a 2 1 A b 5 3 G I know how to do it with a given input...

8. Shell Programming and Scripting

Use less pipe for grep or awk sed to print the line not include xx yy zz

cat file |grep -v "xx" | grep -v "yy" |grep -v "zz"

9. Shell Programming and Scripting

awk seach and printing a particular pattern

Hi i need a help for this. the output of a command is like below 1 /tmp/x 2.2K /tmp/y 3.2k /tmp/z 1G /tmp/a/b 2.2G /tmp/c 3.4k /tmp/d Now i need to grep for the paths which are in GB..like below 1G /tmp/a/b 2.2G /tmp/c pls suggest me, how can i...

10. Shell Programming and Scripting

awk pipe to sort

In the below awk to add a sort by smallest to largest should it be added after the END? Thank you :). BEGIN { FS="*" } # Read search terms from file1 into 's' FNR==NR { s next } { # Check if $5 matches one of the search terms for(i in s) { if($5 ~ i) { ...

LEARN ABOUT BSD

zgrep

ZGREP(1)						      General Commands Manual							  ZGREP(1)

NAME

       zgrep - search possibly compressed files for a regular expression

SYNOPSIS

       zgrep [ grep_options ] [ -e ] pattern filename...

DESCRIPTION

       Zgrep  invokes  grep  on  compressed  or  gzipped  files.   These  grep	options  will  cause  zgrep  to  terminate  with  an  error  code:
       (-[drRzZ]|--di*|--exc*|--inc*|--rec*|--nu*).  All other options specified are passed directly to grep.  If no file is specified,  then  the
       standard input is decompressed if necessary and fed to grep.  Otherwise the given files are uncompressed if necessary and fed to grep.

       If the GREP environment variable is set, zgrep uses it as the grep program to be invoked.

EXIT CODE

       2 - An option that is not supported was specified.

AUTHOR

       Charles Levert (charles@comm.polymtl.ca)

SEE ALSO

       grep(1), gzexe(1), gzip(1), zdiff(1), zforce(1), zmore(1), znew(1)

																	  ZGREP(1)