Shell Programming and Scripting

Hi,

I have a huge log file (~5 GB) generated in my server. I have to search through this file for the pattern matching "ERROR" and print only those lines. There is a Perl script available in the server and I have to add a new subroutine so that it will look for this regular expression pattern in an efficient way. I am not supposed to use 'grep' inside perl.

Since I am new to perl, what I have tried so far is:

Code:

#!/usr/bin/perl -w

sub printFun() {
    while(<>) {
       if ($_=~/ERROR/) {
           print $_;
        }
    }
}

printFun ./loghttpd-05212011

But, I am getting an error "Search pattern not terminated at test.pl line 11" when I run this script. Please help me to solve this issue and get the favored output.

Also, please let me know how to optimize this function (in terms of time consumption) in a better way so that it handles this big data file efficiently

First, just passing a filename as a parameter doesn't automatically open it for reading. You'll have to open it yourself. And you have to enclose any string in quotes (for Perl, ./loghttpd-05212011 is very different from "./loghttpd-05212011").

As for optimizing: don't read line-by-line, define the search regex as strict as possible (can "ERROR" occur anywhere in the line? Or at a specific position? Is there any fixed, or well defined, text leading up to it?), and if possible use the built-in grep function.

A simple example:

Code:

#!/usr/bin/perl -w

use strict;
use warnings;

sub printFun {
    my ($file) = @_;
    open my $INPUT, '<', $file or die "Can't open $file for reading: $!\n";
    print grep { /ERROR/ } <$INPUT>;
    close $INPUT;
}

printFun "./loghttpd-05212011";

Thanks a ton Pludi !! Is that possible to knock off 'grep' keyword in perl and can use something native to perl?

For your questions, yes, the pattern 'ERROR' can occur anywhere in the line. It is like "ERROR: " followed by some error messages.

This grep is native to Perl. Just follow the link I gave for it, it's a native Perl built-in, it doesn't rely on any external tools.