Login or Register to Ask a Question and Join Our Community


UNIX for Advanced & Expert Users

Help in faster and quicker grepping

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Help in faster and quicker grepping
# 1  
Old 06-03-2013
Help in faster and quicker grepping
Hi,

i have a log file of 8 GB size , i need to grep the count of a word which i give as an input , i mean to say to find the occurance of a word on that file.

grep is taking too much time ,can you please give me any command so that i can grep the word in a quicker way..

thanks,

senthil
# 2  
Old 06-03-2013
What do you have right now?
grep is usually very fast, if you don't slow it down with (too many) .* things.
Also word boundaries -w or \< \> are slow in some grep implementations.
Further, grep is slow if the lines are very long, e.g. if your file is not a text file.
# 3  
Old 06-03-2013
hi

thnaks for the response.

am using this command -> grep "string" filename |wc -l

the string will be of 10 to 12 words long.

is there any alternate command ..?
# 4  
Old 06-03-2013
Could you please give an example for "string".
What gives the following?:
Code:
file "logfile"
uname
type grep

# 5  
Old 06-03-2013
Hi,

file "logfile"
App.log: ASCII English text, with very long lines

uname
Linux

type grep
grep is /bin/grep



---------- Post updated at 07:02 PM ---------- Previous update was at 06:56 PM ----------

one of the example for the string is below

"'FLOW - Unable to read the message from MQ within Time Out Period "
# 6  
Old 06-03-2013
Does the word

FLOW

start at the very beginning of the line?

If true and FLOW occurs rarely then,
Code:
grep '^FLOW' filename  | grep  'FLOW - Unable to read the message from MQ within Time Out Period'

is faster because it looks only at the first 4 characters of the line
# 7  
Old 06-03-2013
Code:
grep -c "string" App.log

is faster than
Code:
grep "string" App.log | wc -l

Last edited by MadeInGermany; 06-03-2013 at 11:50 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Which system is faster?

i'm trying to decide if to move operations from one of these hosts to the other. but i cant decide which one of them is the most powerful. each host has 8 cpus. HOSTA processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 44 model name : Intel(R) Xeon(R) CPU ... (6 Replies)
Discussion started by: SkySmart
6 Replies

2. UNIX for Advanced & Expert Users

any quicker way to list disc usage by users?

Hi: it takes a long time for "du -sh list_of_users" to give you the output. Is there a quicker way to get this info? Thanks! N.B. Phil (4 Replies)
Discussion started by: phil518
4 Replies

3. UNIX for Dummies Questions & Answers

Why is RAID0 faster?

I have read anecdotes about people installing RAID0 (RAID - Wikipedia, the free encyclopedia) on some of their machines because it gives a performance boost. Because bandwidth on the motherboard is limited, can someone explain exactly why it should be faster? (7 Replies)
Discussion started by: figaro
7 Replies

4. UNIX for Dummies Questions & Answers

Which command will be faster? y?

i)wc -c/etc/passwd|awk'{print $1}' ii)ls -al/etc/passwd|awk'{print $5}' (4 Replies)
Discussion started by: karthi_g
4 Replies

5. Shell Programming and Scripting

Which is faster AWK or CUT

If I just wanted to get andred08 from the following ldap dn would I be best to use AWK or CUT? uid=andred08,ou=People,o=example,dc=com It doesn't make a difference if it's just one ldap search I am getting it from but when there's a couple of hundred people in the group that retruns all... (10 Replies)
Discussion started by: dopple
10 Replies

6. UNIX for Dummies Questions & Answers

How to grep faster ?

Hi I have to grep for 2000 strings in a file one after the other.Say the file name is Snxx.out which has these strings. I have to search for all the strings in the file Snxx.out one after the other. What is the fastest way to do it ?? Note:The current grep process is taking lot of time per... (7 Replies)
Discussion started by: preethgideon
7 Replies

7. Shell Programming and Scripting

Faster then cp ?

Hi , I need to copy every day about 35GB of files from one file system to another. Im using the cp command and its toke me about 25 min. I also tried to use dd command but its toke much more. Is there better option ? Regards. (6 Replies)
Discussion started by: yoavbe
6 Replies

8. UNIX for Advanced & Expert Users

faster way to loop?

Sample Log file IP.address Date&TimeStamp GET/POST URL ETC 123.45.67.89 MMDDYYYYHHMM GET myURL http://ABC.com 123.45.67.90 MMDDYYYYHHMM GET myURL http://XYZ.com I have a very huge web server log file (about 1.3GB) that contains entries like the one above. I need to get the last entries of... (9 Replies)
Discussion started by: tads98
9 Replies

9. IP Networking

Mandrake should be faster.

For some reason 8.1 Mandrake Linux seems much slower than Windows 2000 with my cable modem. DSL reports test says they conferable speed with Windows2 though. This is consistant slow with both of my boxes, at the same time. Linux used to be faster, but not with Mandrake. Any way to fix this? (17 Replies)
Discussion started by: lancest
17 Replies
Login or Register to Ask a Question