UNIX for Dummies Questions & Answers

Hi all,

I have a log file which is more than 1GB, i need to take count from the log file for two strings. i am using the below command but it take a long time to excetue, i need to tune this. Please help me


cat /logs/gcbs/gcbsTrace.log | grep -i "ViewStatementBusinessLogic" | grep -c -i "MYCACommunicationException"

similary i need to check the count for other execption too. Is there any simple way is avialable.

Regards,
Senthilkumar AK

It's probably better if you post realistic examples and keep all the discussion in a single thread. See https://www.unix.com/shell-programmin...s-my-code.html and https://www.unix.com/unix-advanced-ex...ck-script.html

As such, I would recommend that you combine all your grep statements to a single sed script.

Code:

sed -n '/regex1/s/.*\(regex2\).*/\1/p
/regex3/s/.*\(regex4\).*/p
/regex5/s/.*\(regex6\).*/p' /logs/gcbs/gcbsTrace.log | sort | uniq -c

That way, you only scan the whole file once, and spend less time waiting for the disk.

Thanks era,

The Sed command also takes the same time as the grep takes, also i take the suggestion which you posted and continue in this thread alone.

If you really need to improve performance, you need to move away from a shell script
and write something in C or C++

ok is it i can write a c program and use this in Linux machine..? I searched in the machine and found perl is there.

The point with the sed is that it takes more time than a single grep, but you can replace all the greps with a single sed script.

Unless you are a skilled C programmer, I doubt that you can make this a lot faster by switching languages. C programs are compiled, you don't need the C compiler on the box itself in order to run the resulting binary.

You can give awk a try:

Code:

awk '{$0=tolower($0)}
/viewstatementbusinesslogic/ && /mycacommunicationexception/{c++}END{print c++}
' file

Regards