Uses of Class
org.apache.lucene.analysis.TokenStream

Packages that use TokenStream

org.apache.lucene.analysis	API and code to convert text into indexable tokens.
org.apache.lucene.analysis.de	Support for indexing and searching of German text.
org.apache.lucene.analysis.standard	A grammar-based tokenizer constructed with JavaCC.

Uses of TokenStream in org.apache.lucene.analysis

Subclasses of TokenStream in org.apache.lucene.analysis

class	CharTokenizer An abstract base class for simple, character-oriented tokenizers.
class	LetterTokenizer A LetterTokenizer is a tokenizer that divides text at non-letters.
class	LowerCaseFilter Normalizes token text to lower case.
class	LowerCaseTokenizer LowerCaseTokenizer performs the function of LetterTokenizer and LowerCaseFilter together.
class	PorterStemFilter Transforms the token stream as per the Porter stemming algorithm.
class	StopFilter Removes stop words from a token stream.
class	TokenFilter A TokenFilter is a TokenStream whose input is another token stream.
class	Tokenizer A Tokenizer is a TokenStream whose input is a Reader.
class	WhitespaceTokenizer A WhitespaceTokenizer is a tokenizer that divides text at whitespace.

Fields in org.apache.lucene.analysis declared as TokenStream

protected TokenStream

TokenFilter.input The source of tokens for this filter.

Methods in org.apache.lucene.analysis that return TokenStream

TokenStream	Analyzer.tokenStream(String fieldName, Reader reader) Creates a TokenStream which tokenizes all the text in the provided Reader.
TokenStream	Analyzer.tokenStream(Reader reader) Deprecated. use tokenStream(String, Reader) instead.
TokenStream	WhitespaceAnalyzer.tokenStream(String fieldName, Reader reader)
TokenStream	StopAnalyzer.tokenStream(String fieldName, Reader reader) Filters LowerCaseTokenizer with StopFilter.
TokenStream	SimpleAnalyzer.tokenStream(String fieldName, Reader reader)

Constructors in org.apache.lucene.analysis with parameters of type TokenStream

PorterStemFilter(TokenStream in)

LowerCaseFilter(TokenStream in)

StopFilter(TokenStream in, String[] stopWords)
Constructs a filter which removes words from the input TokenStream that are named in the array of words.

StopFilter(TokenStream in, Hashtable stopTable)
Constructs a filter which removes words from the input TokenStream that are named in the Hashtable.

Uses of TokenStream in org.apache.lucene.analysis.de

Subclasses of TokenStream in org.apache.lucene.analysis.de

class

GermanStemFilter A filter that stems German words.

Methods in org.apache.lucene.analysis.de that return TokenStream

TokenStream

GermanAnalyzer.tokenStream(String fieldName, Reader reader) Creates a TokenStream which tokenizes all the text in the provided Reader.

Constructors in org.apache.lucene.analysis.de with parameters of type TokenStream

GermanStemFilter(TokenStream in)

GermanStemFilter(TokenStream in, Hashtable exclusiontable)
Builds a GermanStemFilter that uses an exclusiontable.

Uses of TokenStream in org.apache.lucene.analysis.standard

Subclasses of TokenStream in org.apache.lucene.analysis.standard

class	StandardFilter Normalizes tokens extracted with StandardTokenizer.
class	StandardTokenizer A grammar-based tokenizer constructed with JavaCC.

Methods in org.apache.lucene.analysis.standard that return TokenStream

TokenStream

StandardAnalyzer.tokenStream(String fieldName, Reader reader) Constructs a StandardTokenizer filtered by a StandardFilter, a LowerCaseFilter and a StopFilter.

Constructors in org.apache.lucene.analysis.standard with parameters of type TokenStream

StandardFilter(TokenStream in)
Construct filtering in.