Efficient way to grep


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Efficient way to grep
# 1  
Old 06-28-2011
Efficient way to grep

Hi Experts,

I've been trying simple grep to search for a string in a huge number of files in a directory.

grep <pattern> *

this gives the search results as well as the following -

grep: <filename>: Permission denied
grep: <filename>: Permission denied

for files which I don't have read permissions on.

I have three questions-
1) is there any way to eliminate these messages ?
2) is there any way to search in those files as well ?
3) is there any different approach to search for a string in a heap of files ( apart from sed)
# 2  
Old 06-28-2011
To dump the errors
Code:
% grep <pattern> * 2>/dev/null

# 3  
Old 06-28-2011
  1. you can redirect STDERR to /dev/null thus
    Code:
    grep string files 2>/dev/null

  2. If you have an account with the right to read them, issue the command from that account
  3. grep is about the fastest string search available, however your approach will eventually result in a "Too many arguments" failure, you should try grep -r or using a command to isolate the files you need to search, eg.
    Code:
    find ./ -type f -name *.pl | xargs grep 'use strict'  2>/dev/null

# 4  
Old 06-28-2011
Try this, it will check only the files having permissions and we can also avoid the errors.

find . -type f -perm 666 -exec grep "pattern" {} \;

Change the permission, as what ever you want.
These 2 Users Gave Thanks to Rksiva For This Post:
# 5  
Old 06-28-2011
Another way.
Code:
# First eliminate errors from directories which you cannot read
# Then check that you can read each file before using grep
find . -type f -print 2>/dev/null | while read FILENAME
do
     if [ -r "${FILENAME}" ]
     then
            grep "string" "${FILENAME}"
     fi
done

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with Efficient Looping

Hello guys My requirement is to read a file with parent-child relationship we need to iterate through each row to find its latest child. for eg. parent child ABC PQR PQR DEF DEF XYZ Expected Output ABC XYZ PQR XYZ DEF XYZ Script Logic : read parent from file seach child... (4 Replies)
Discussion started by: joshiamit
4 Replies

2. UNIX for Dummies Questions & Answers

Is this regex efficient?

I want to match the red portion: 9784323456787-Unknown Phrase with punctuation "Some other PhrASE." Is this the best regex to match this? '978\{10\}-*' (4 Replies)
Discussion started by: glev2005
4 Replies

3. UNIX for Advanced & Expert Users

efficient repace

some of the data i receive has been typed in manually due to which there are often places where i find 8 instead of ( and the incorrect use of case what according to you is the best way to correct such data. The data has around 20,000 records. The value i want to change is in the 4th field.... (2 Replies)
Discussion started by: VGR
2 Replies

4. Shell Programming and Scripting

efficient search

Hi, i have 2 files each with 200K lines. Each line contains a number. Now, i need to get the list of numbers existing in one fine and NOT in other file. I'm doing this by reading each number from 1 file and grepping on other file. But this taking LOT of time. Is there any efficient way of doing... (14 Replies)
Discussion started by: prvnrk
14 Replies

5. Shell Programming and Scripting

Efficient way to grep multiple strings

I have a script which searches a huge log file for the existence of a specified string and if the string is not present i receive an alert mail. Here's an extract: STRING=$(grep 'warning' logfile | tail -1 | wc -l) if (( ${STRING} > 0 )); then print -- "---- Warning etc.... (3 Replies)
Discussion started by: Moxy
3 Replies

6. Shell Programming and Scripting

help on most efficient search

Hello, We have a directory with 15 sub-directories where each sub-directory contains 1.5 to 2 lakhs of files in it. Daily, around 300-500 files will be uploaded to each sub-directory. Now, i need to get the list of files received today in most efficient way. I tried using "find with newer... (16 Replies)
Discussion started by: prvnrk
16 Replies

7. Shell Programming and Scripting

Can you suggest a more efficient way for this?

Hi I have the following at the end of a service shutdown script used in part of an active-passive failover setup: ### # Shutdown all primary Network Interfaces # associated with failover ### # get interface names based on IP's # and shut them down to simulate loss of # heartbeatd ... (1 Reply)
Discussion started by: mikie
1 Replies

8. Shell Programming and Scripting

Efficient way of Awk

Hi, Can someone let me know if the below AWK can be made much simpler / efficient ? I have 200 fields, I need to substr only the last fields. So i'm printing awk -F~ 'print {$1, $2, $3....................................$196,$197 , susbstr($198,1,3999), substr($199,1,3999)..}' Is there a... (4 Replies)
Discussion started by: braindrain
4 Replies

9. Shell Programming and Scripting

Need more efficient log file grep

I'm writing a script that at one point needs to check the contents of another script's log file to determine how to proceed. An example record from the log file is: "mcref04152006","060417","ANTH0415","282","272","476,983.37","465,268.44","loaded" I want my script to return this record if: ... (3 Replies)
Discussion started by: Glenn Arndt
3 Replies

10. Shell Programming and Scripting

Is there a more efficient way?

I'm using korn shell to connect to oracle, retrieve certain values, put them in a list, and iterate through them. While this method works, I can't help but think there is an easier method. If you know of one, please suggest a shorter, more efficient method. ############### FUNCTIONS ... (6 Replies)
Discussion started by: SelectSplat
6 Replies
Login or Register to Ask a Question