How to make grep stop at first match


 
Thread Tools Search this Thread
Operating Systems AIX How to make grep stop at first match
# 1  
Old 04-01-2010
Question How to make grep stop at first match

I use grep to check for a string that validates data in a file, it works great but the problem is that the file is becoming too big and gerp has started hurting the response time to users. Since I only need to find the first occurrence I have been looking for ways to stop grep for scanning the rest of the file (6gb) but I haven't had much luck.
One way is using the -m parameter, unfortunately thta is not avaialble in the grep version we have in our AIX (not sure if I can update it). I also found a small scrip that worked fine at the beginning but in the long run is takes even more time that grep:

Code:
while read line
do
   echo $line | grep mystring
   if [ "$?" -eq "0" ]; then 
                  break
   fi
done < mytextfile

this works great if the matches are found at the beginning of the file but when I tested with a string that is at the middle or end it took even more time than grep alone to return an asnwer and stop.

I've researched in several palces but I haven found anything else (in a simple way) that could do the trick, I tried awk and also takes longer that grep when the match is in the middle of the file.

Any suggestion would be great.

Last edited by Franklin52; 04-01-2010 at 12:27 PM.. Reason: Please use code tags!
# 2  
Old 04-01-2010
Have you tried something like this with awk?
Code:
awk '$0 ~ var{print;exit}' var=$string mytextfile

# 3  
Old 04-01-2010
Hi.

GNU grep has the -m option for that, but AIX grep doesn't have such a feature, I think.

You can use sed, if that's quick enough. Something like

Code:
$ sed -n "/something/{p;q;}" mytextfile

or

$ sed "/something/{!d;q;}" mytextfile

Your original grep solution would probably be OK if you ran it as:
Code:
grep "mystring" mytextfile

instead of in a while loop.
# 4  
Old 04-01-2010
It might help to suppress greps output if you just want to know if a certain string is in your file or not. Use the "-q" option of grep for this and check the exit code (0 means "found", >0 means "not found").

You might consider using sed instead of grep. You could tell sed to stop at the first occurrence, here is an example:

standard grep way:
Code:
if [ -n "$(grep "blabla" /path/to/bigfile)" ] ; then
     echo "found it"
else
     echo "not found it"
fi

grep with "-q" option:

Code:
if [ $(grep -q "blabla" /path/to/bigfile) -eq 0 ] ; then
     echo "found it"
else
     echo "not found it"
fi

sed instead of grep:
Code:
if [ -n "$(sed -n '/blabla/ {;p;q;}' /path/to/bigfile)" ] ; then
     echo "found it"
else
     echo "not found it"
fi

The sed command applies only to lines containing "blabla" (the search string) and will print this line ("p"), then quit the processing ("q").

I'd be really grateful if you could provide the runtime statistics of all the three variants. It would be very interesting to see how these different approaches compare.

I hope this helps.

bakunin
# 5  
Old 04-01-2010
I tried all the suggestions with the same string that should locate a record at the ~17% of the file (~record number 170,000), the best response time still goes to grep :

grep took 17 seconds, 3 to show the match the res 14 scanning the rest of the file.

Code:
grep mystring myfile

awk took 20 seconds

Code:
awk '/mystring{1}/' myfile.txt


then sed as scottn suggested took 28 seconds

tried awk as franklin52 suggested and took 18 seconds

so in summary I'm still good with grep but my users are not happy to wait once in a while for 17 seconds to get their screen back with the response (neither my server performance if I have many users using the same look up).
I would be so nice to find a way to stop grep after that first match, response time would go down to 3 seconds! I guess I'll have to find another source for grep that supports -m and see if I can install it.
# 6  
Old 04-01-2010
You could pipe grep to "head -1" - might even improve the speed :-)
# 7  
Old 04-01-2010
Try (notice the -l ):

Code:
while read line
do
   echo $line | grep -l mystring
   if [ "$?" -eq "0" ]; then 
        break
   fi
done < mytextfile


Last edited by Franklin52; 04-02-2010 at 02:16 PM.. Reason: Please indent your code and use code tags, thank you
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need help on grep for particular match

Hi, Need help on grep for a particular match. cat testfile xx.xx.xx.xx:/share4 /nfsshare15 nfs defaults yes no xx.xx.xx.xx:/share5 /nfsshare15/sharedir1 nfs defaults 0 0 xx.xx.xx.xx:/share6 /nfsshare15/sharedir2 nfs defaults 0 0 Scenario... (6 Replies)
Discussion started by: sumanthupar
6 Replies

2. UNIX for Dummies Questions & Answers

Using grep to make a header

How would I do the following : Records other than ”ATOM”,”CONNECT”, ”HETATM”, ”TER” and ”END” are considered header records which describe the metadata about the molecule. Use grep to generate the header. I have this chemistry database. On the attachment. But I am not sure how to use... (4 Replies)
Discussion started by: homeylova223
4 Replies

3. UNIX for Dummies Questions & Answers

how to make this grep command

/usr/xpg4/bin/grep -e "Type" / datarecords.txt output datarecords.txt: male | datarecords.txt: male | datarecords.txt: female i wanna the output to be :male | :male | :female at the end not to appear the filename fom grep command :D :D (3 Replies)
Discussion started by: teefa
3 Replies

4. Shell Programming and Scripting

Grep Stop status from the output of script and send an eamil alert.

Hello Team, I have script which gives below output. Server clustServer11 is in a STARTED state Server clustServer12 is in a STOPPED state Server clustServer21 is in a STOPPED state I would like to prepare script which will grep stop word from the above output and send an email alert. (5 Replies)
Discussion started by: coolguyamy
5 Replies

5. AIX

Stop grep in /proc

Hello, I want to kill a root process: # find/ | xargs grep - L “G-string” The grep is in /proc and obviously the kill -9 does not stop it. How to stop it without shutdown -Fr ? Thank you. (1 Reply)
Discussion started by: Trunk
1 Replies

6. Shell Programming and Scripting

Need help to grep for a title match and then make some queries after the match

Here is the sample of my file address.txt Address 1 1234 Drive way New Orleans, LA Zipcode :- 12345 Address 2 4567 Spring way Chicago, IL Zipcode :- 67890 I would like to grep for an Address title (Ex :- Address 2) , then get its zipcode and echo both in a single line. Ex :- ... (3 Replies)
Discussion started by: leo.maveriick
3 Replies

7. UNIX for Dummies Questions & Answers

grep question: stop search from finding entire line

Sorry for the title, I really don't know how to word this question or what to even search for. I tried "grep one match", "grep 1 match", "stop grep" on both google and here and haven't found something that helps, so here I go: I have a file that's about 1.5 million lines long, every line looks... (3 Replies)
Discussion started by: rmoakler
3 Replies

8. UNIX for Dummies Questions & Answers

What can make Cronjobs stop working??

Up until two days ago they were working junt fine, then stoped working with out me doing anything. Yesterday they started working again...and then stoped working, at about the same time of day that they stoped working the day before. service crond restart did nothing. All SSH signs point to... (0 Replies)
Discussion started by: Nintendo
0 Replies

9. Shell Programming and Scripting

Make grep -c display like grep -n?

Hey Guys, Wondering if there is a way to do the following I have a file called test.txt abc def abc abc def I have a pattern file called pattern.txt containing the following abc def I want to do a count, but have it display the count value preceeding each line like grep -n (2 Replies)
Discussion started by: Jerrad
2 Replies

10. HP-UX

Need to make disk device files match

Hi, I was wondering if there was a way to change the disk device files ex. /dev/dsk/cxtxd0 ? What I have are two HPUX 11.0 servers using MC Service Guard 11.13. A consultant attached a SAN and both servers had created the same identical LUN device files. Now I could begin creating my volume... (0 Replies)
Discussion started by: mvizza
0 Replies
Login or Register to Ask a Question

Featured Tech Videos