Searching mutiple word - Tuning grep command - UNIX for Dummies Questions & Answers

senthilkumar_ak · #1 (**permalink**) 4 Weeks Ago

Hi all,

I have a log file which is more than 1GB, i need to take count from the log file for two strings. i am using the below command but it take a long time to excetue, i need to tune this. Please help me

cat /logs/gcbs/gcbsTrace.log | grep -i "ViewStatementBusinessLogic" | grep -c -i "MYCACommunicationException"

similary i need to check the count for other execption too. Is there any simple way is avialable.

Regards,
Senthilkumar AK

era · #2 (**permalink**) 4 Weeks Ago

It's probably better if you post realistic examples and keep all the discussion in a single thread. See Please suggest some changes in my code and Guidance needed for quick script

As such, I would recommend that you combine all your grep statements to a single sed script.

Code:

sed -n '/regex1/s/.*\(regex2\).*/\1/p
/regex3/s/.*\(regex4\).*/p
/regex5/s/.*\(regex6\).*/p' /logs/gcbs/gcbsTrace.log | sort | uniq -c

That way, you only scan the whole file once, and spend less time waiting for the disk.

senthilkumar_ak · #3 (**permalink**) 4 Weeks Ago

Thanks era,

The Sed command also takes the same time as the grep takes, also i take the suggestion which you posted and continue in this thread alone.

fpmurphy · #4 (**permalink**) 4 Weeks Ago

If you really need to improve performance, you need to move away from a shell script
and write something in C or C++

senthilkumar_ak · #5 (**permalink**) 4 Weeks Ago

ok is it i can write a c program and use this in Linux machine..? I searched in the machine and found perl is there.

era · #6 (**permalink**) 3 Weeks Ago

The point with the sed is that it takes more time than a single grep, but you can replace all the greps with a single sed script.

Unless you are a skilled C programmer, I doubt that you can make this a lot faster by switching languages. C programs are compiled, you don't need the C compiler on the box itself in order to run the resulting binary.

Franklin52 · #7 (**permalink**) 3 Weeks Ago

You can give awk a try:

Code:

awk '{$0=tolower($0)}
/viewstatementbusinesslogic/ && /mycacommunicationexception/{c++}END{print c++}
' file

Regards

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
MySQL Tuning Tools with mysqltuner.pl and tuning-primer.sh	Neo	Web Programming, Web 2.0 and Mashups	1	4 Weeks Ago 07:49 AM
Searching word in a file with awk	studieu	Shell Programming and Scripting	3	07-09-2008 08:28 AM
Find Exactly word in grep command	koti_rama	UNIX for Dummies Questions & Answers	4	08-23-2007 03:52 AM
searching word in files	naamas03	UNIX for Dummies Questions & Answers	2	11-06-2006 01:26 AM
Searching for key word within a file	moradwan	UNIX for Dummies Questions & Answers	1	03-07-2006 11:31 PM

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode