UNIX for Dummies Questions & Answers

I've prepared perl script, which will not show repetitious messages. It looks for every incoming line, remember all new messages and do not show them, if they appeared previously. Message is the string after ">" sign. There is no "grep" operation, so you have to grep for particular lines. I'm not sure, if it's something that you need. I don't want to "sort" lines, because I guess your input is some kind of log, in this context sorting will be not good.

Code:

#!/usr/bin/perl

# kindofuniq script

my %appeared;

while (<STDIN>) {
        /.*\> (.*)/;
        my $message = "$1\n";
        unless (exists $appeared{$message}) {
                print $_;
                $appeared{$message} = 1;
        }
}

Example of usage:

Code:

kindofuniq < abc.txt | grep Error

You can use in a way of "live watching logs", but remember, that the same message will not show twice:

Code:

tail -f abc.txt | kindofuniq | grep Error