org.apache.lucene.analysis
Class WhitespaceTokenizer
public
class
WhitespaceTokenizer
extends CharTokenizer
A WhitespaceTokenizer is a tokenizer that divides text at whitespace.
Adjacent sequences of non-Whitespace characters form tokens.
Method Summary |
protected boolean | isTokenChar(char c) Collects only characters which do not satisfy
{@link Character#isWhitespace(char)}. |
public WhitespaceTokenizer(Reader in)
Construct a new WhitespaceTokenizer.
protected boolean isTokenChar(char c)
Collects only characters which do not satisfy
{@link Character#isWhitespace(char)}.
Copyright © 2000-2007 Apache Software Foundation. All Rights Reserved.