One instance of comparing grep and awk


Login or Register for Dates, Times and to Reply

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting One instance of comparing grep and awk
# 1  
One instance of comparing grep and awk

Hi.

In thread Grouping and counting rovf and I had a mini-discussion on grep and awk.

Here is a demo script that compares the awk and grep approaches for this single problem:
Code:
#!/usr/bin/env bash

# @(#) s2       Demonstrate group and count with standard commands.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
LC_ALL=C ; LANG=C ; export LC_ALL LANG
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C awk grep cut sort uniq

FILE=${1-data1}

pl " Input data file $FILE:"
cat $FILE

pl " Results, awk filter:"
awk -F'|' '$4 == "\""1"\"" && $5=="\"""Y""\"" ' $FILE |
cut -f8 -d'|' | sort | uniq -c

pl " Results, grep filter:"
grep -E '^([^|]+[|]){3}.1...Y' $FILE |
cut -f8 -d'|' | sort | uniq -c

exit 0

producing:
Code:
$ ./s2

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 3.16.0-4-amd64, x86_64
Distribution        : Debian 8.4 (jessie) 
bash GNU bash 4.3.30
awk GNU Awk 4.1.1, API: 1.1 (GNU MPFR 3.1.2-p3, GNU MP 6.0.0)
grep (GNU grep) 2.20
cut (GNU coreutils) 8.23
sort (GNU coreutils) 8.23
uniq (GNU coreutils) 8.23

-----
 Input data file data1:
1111|"ZZZ"|"Date"|"1"|"Y"|"ABC"|""|CC
1234|"ZZZ"|"Date"|"1"|"Y"|"ABC"|""|AA
ABCD|"ZZZ"|"Date"|"1"|"Y"|"ABC"|""|AA
EFGH|"ZZZ"|"Date"|"1"|"Y"|"ABC"|""|BB
IJKLM|"ZZZ"|"Date"|"1"|"Y"|"ABC"|""|BB
NOPQ|"ZZZ"|"Date"|"1"|"Y"|"ABC"|""|BB
foox|"YYY"|"Date"|"2"|"Y"|"ABC"|""|BB
barx|"YYY"|"Date"|"1"|"N"|"ABC"|""|BB
bazx|"YYY"|"Date"|"3"|"N"|"ABC"|""|CC

-----
 Results, awk filter:
      2 AA
      3 BB
      1 CC

-----
 Results, grep filter:
      2 AA
      3 BB
      1 CC

Both outputs are the same. The suggestion of rovf is quite correct, as illustrated, you can use grep to match field content. However, it also illustrates why I do not usually do that. Both commands are more than really basic, perhaps intermediate-complex. The grep makes use of a regular expression that is beyond many people's ability to understand it, much less create it. The awk is also not basic, but the form suggests what we are looking for, namely the content of fields 4 and 5. The extra "baggage" is to take care of the quotes. There are probably other ways of doing that, but this one seems straight-forward when one realizes how to quote an escaped quote.

So from a results-oriented POV, the solutions are the same.

However, an additional point is that, if one uses awk as the first step, one might also make the entire solution awk-based. I used standard commands because the problem decomposed easily into steps, until the extra requirements of of field contents arose.

I don't think the issue of speed would come up, but generally grep is somewhat fast than awk in similar situations. I have posted elsewhere some results comparing utilities and languages for matching.

So, rovf is correct, but the issue of transparency is more important, in my opinion.

Thanks to rovf for pointing our that grep and regular expressions are very powerful ... cheers, drl
These 4 Users Gave Thanks to drl For This Post:
# 2  
I prefer awk, because I know it well, so there is little effort to make it precise and efficient.
For extreme robustness (e.g. long input lines) I go with perl. But usually takes longer till it does what I want.
For a "work on fields" task grep is often unprecise, or the readability and extend-ability are worse.
Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #51
Difficulty: Easy
The term 'IoT' means the 'Internet of Tomorrow'.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk line instance counter

I Have a text file with several thousand lines of text. Occasionally there will be a "sysAlive" line of text (every so often) What would be an awk command to print every line of text, and to put in incrementing counter ONLY on the "sysAlive" lines For example: >cat file.txt lineAAA a b c d... (4 Replies)
Discussion started by: ajp7701
4 Replies

2. UNIX for Advanced & Expert Users

Grep the only instance name

Hi, I want to get the only application name from the server. Ex: if i give $ ps -ef | grep bw. It will show all BW process with entire path. It will little confuse to list out the process. Can anyone have syntax to get only the instance name. I need this for be, hawk,ems also. Please... (2 Replies)
Discussion started by: ckchelladurai
2 Replies

3. Windows & DOS: Issues & Discussions

win2008R2 using gnu grep/awk to find only the I instance out of the command

Ok, running tsm on 2008R2, and I have the gnu awk and grep installed...I want to only capture inactive (I) files inside of the backup...I can get this to work on linux, but because windows interprets " ' ` differently, i can't get it to print out right...Here's what I have: so basically... (8 Replies)
Discussion started by: nextreme
8 Replies

4. Shell Programming and Scripting

How can I just grep one instance of a word in the file

I want to grep some information out of the dmidecode but when I type dmidecode | grep Memory I get several instances of the word. Is there a way I can just choose which instance I want to display? (8 Replies)
Discussion started by: jcnewton13
8 Replies

5. Shell Programming and Scripting

grep for a string until instance of a space

Hey guys, I'm having a bit of trouble getting this to work using either sed or grep. It's possible awk might be the ticket I need as well, but my regulat expression skills aren't quite up to the task for doing this. I'm looking to grep for the string ERROR from the following log up until any... (6 Replies)
Discussion started by: terrell
6 Replies

6. Shell Programming and Scripting

grep second instance of same string

Hi all, i am new to unix scripting in ksh or any shell for that matter. I have downloaded a xml file from a website and saved on my local harddrive. inside the xml, the same tag is listed multiple times. <title>Tonight</title> <title>Thursday</title> <title>Friday</title>... (6 Replies)
Discussion started by: scubasteve39
6 Replies

7. Shell Programming and Scripting

Search text file, then grep next instance of string

I need to be able to search for a beginning line header, then use grep or something else to get the very next instance of a particular string, which will ALWAYS be in "Line5". What I have is some data that appears like this: Line1 Line2 Line3 Line4 Line5 Line6 Line7 Line1 Line2 ...... (4 Replies)
Discussion started by: Akilleez
4 Replies

8. Shell Programming and Scripting

awk - last instance of a string

How do I use awk to find the last occurence of a string in a file? (3 Replies)
Discussion started by: locoroco
3 Replies

9. Shell Programming and Scripting

awk - find first instance of string after NR==10

How do I use awk to find the NR of first instance of a specific string after eg NR==10? I need to find the first instance of the word "class" after a specific NR. (2 Replies)
Discussion started by: locoroco
2 Replies

10. Shell Programming and Scripting

sed/awk: Delete matching words leaving only the first instance

I have an input text that looks like this (comes already sorted): on Caturday 22 at 10:15, some event on Caturday 22 at 10:15, some other event on Caturday 22 at 21:30, even more events on Funday 23 at 11:00, yet another event I need to delete all the matching words between the lines, from... (2 Replies)
Discussion started by: GrinningArmor
2 Replies

Featured Tech Videos