public final class IcuTokenizer
extends org.apache.lucene.analysis.Tokenizer
IcuTokenizerConfig
.Constructor and Description |
---|
IcuTokenizer()
Construct a new ICUTokenizer that breaks text into words from the given
Reader.
|
IcuTokenizer(org.apache.lucene.util.AttributeFactory factory,
IcuTokenizerConfig config)
Construct a new ICUTokenizer that breaks text into words from the given
Reader, using a tailored BreakIterator configuration.
|
IcuTokenizer(IcuTokenizerConfig config)
Construct a new ICUTokenizer that breaks text into words from the given
Reader, using a tailored BreakIterator configuration.
|
Modifier and Type | Method and Description |
---|---|
void |
end() |
boolean |
equals(java.lang.Object object) |
int |
hashCode() |
boolean |
incrementToken() |
void |
reset() |
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, endAttributes, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString
public IcuTokenizer()
DefaultIcuTokenizerConfig
public IcuTokenizer(IcuTokenizerConfig config)
config
- Tailored BreakIterator configurationpublic IcuTokenizer(org.apache.lucene.util.AttributeFactory factory, IcuTokenizerConfig config)
factory
- AttributeFactory to useconfig
- Tailored BreakIterator configurationpublic boolean incrementToken() throws java.io.IOException
incrementToken
in class org.apache.lucene.analysis.TokenStream
java.io.IOException
public void reset() throws java.io.IOException
reset
in class org.apache.lucene.analysis.Tokenizer
java.io.IOException
public void end() throws java.io.IOException
end
in class org.apache.lucene.analysis.TokenStream
java.io.IOException
public boolean equals(java.lang.Object object)
equals
in class org.apache.lucene.util.AttributeSource
public int hashCode()
hashCode
in class org.apache.lucene.util.AttributeSource