UNIX for Dummies Questions & Answers

Hi all,

I have problem with searching hundreds of CSV files, the problem is that search is lasting too long (over 5min).

Csv files are "," delimited, and have 30 fields each line, but I always grep same 4 fields - so is there a way to grep just those 4 fields to speed-up search.

Example:
1,22,11,44,55,7,45,55,55,555,55,66,6,66,book12horror,book34horror,book24horror,book45horror,22,44,55 ..etc.

grep -h "book34" /home/data/books/*

Also I wanted to know how to optimize grep on Solaris10, what option to use to speed up searches, what tweaks can I do on Solaris? Maybe awk can help me on this one?

You can try all of these, and see which one is the fastest:

Code:

awk -F, '$15$16$17$18~"book34"' /home/data/books/*

Code:

 perl -F, -anle 'print $_ if ($F[14].$F[15].$F[16].$F[17])=~/book34/' /home/data/books/*

Am not really sure, how this would make the search faster than the usual grep? There is no bypass or pruning to make it faster.

I could not think of a better apart approach from pruning the "literal search in the record" once a match is found or not found in the record at the expected field.
For ex:
When searching for 10th field in a record with 20 fields, don't continue searching for the pattern space even after the 10th field just prune the search and continue with the next record.
I agree this is not a great way but it will definitely make the search a bit faster.

Yes you are right, above example is not much faster (if any) :/

@matrixmadhan how tell perl or grep search to stop searching after 13 field and skip to next line, example ? (oneliner ofc)

Frankly I don't know whether such tools or programs exist. Best thing is to modify grep code that suits your needs, it shouldn't be that difficult.

Do you know any tutorial where can I learn to make custom search codes?
What I need is normal grep + modification so that grep skips to next line when only one match is found in line or when reached 13 field in csv.

Try this
Cross Reference: /onnv/onnv-gate/usr/src/cmd/grep_xpg4/grep.c

---------- Post updated at 03:58 PM ---------- Previous update was at 03:54 PM ----------

Before you could start writing your own version, try with this option in grep

Code:

grep -iP