org.apache.lucene.analysis
Class WhitespaceTokenizer
java.lang.Object
|
+--org.apache.lucene.analysis.TokenStream
|
+--org.apache.lucene.analysis.Tokenizer
|
+--org.apache.lucene.analysis.CharTokenizer
|
+--org.apache.lucene.analysis.WhitespaceTokenizer
- public class WhitespaceTokenizer
- extends CharTokenizer
A WhitespaceTokenizer is a tokenizer that divides text at whitespace.
Adjacent sequences of non-Whitespace characters form tokens.
Fields inherited from class org.apache.lucene.analysis.Tokenizer |
input |
Method Summary |
protected boolean |
isTokenChar(char c)
Collects only characters which do not satisfy
Character#isWhitespace(char) . |
Methods inherited from class org.apache.lucene.analysis.Tokenizer |
close |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
WhitespaceTokenizer
public WhitespaceTokenizer(Reader in)
- Construct a new WhitespaceTokenizer.
isTokenChar
protected boolean isTokenChar(char c)
- Collects only characters which do not satisfy
Character#isWhitespace(char)
.
- Overrides:
isTokenChar
in class CharTokenizer
Copyright © 2000-2002 Apache Software Foundation. All Rights Reserved.