debian man page for plucene::analysis::tokenizer

Plucene::Analysis::Tokenizer(3pm)			User Contributed Perl Documentation			 Plucene::Analysis::Tokenizer(3pm)

NAME
       Plucene::Analysis::Tokenizer - base class for tokenizers

SYNOPSIS
	       my $tokenizer = Plucene::Analysis::Tokenizer::Subclass
		       ->new({ reader => $reader });

DESCRIPTION
       This is an abstract base class for tokenizers.

       A Tokenizer is a TokenStream whose input is a Reader.

METHODS
   new
	       my $tokenizer = Plucene::Analysis::Tokenizer::Subclass
		       ->new({ reader => $reader });

       This will create a new tokenizer.

   next
       This must be defined in a subclass

   close
       Close the input reader.

perl v5.12.4							    2011-08-14					 Plucene::Analysis::Tokenizer(3pm)