org.xbib.elasticsearch.index.analysis.icu

Class IcuTransformTokenFilter

java.lang.Object

org.apache.lucene.util.AttributeSource

org.apache.lucene.analysis.TokenStream

org.apache.lucene.analysis.TokenFilter

org.xbib.elasticsearch.index.analysis.icu.IcuTransformTokenFilter

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable

public final class IcuTransformTokenFilter
extends org.apache.lucene.analysis.TokenFilter

A TokenFilter that transforms text with ICU.

ICU provides text-transformation functionality via its Transliteration API. Although script conversion is its most common use, a Transliterator can actually perform a more general class of tasks. In fact, Transliterator defines a very general API which specifies only that a segment of the input text is replaced by new text. The particulars of this conversion are determined entirely by subclasses of Transliterator.

Some useful transformations for search are built-in:

• Conversion from Traditional to Simplified Chinese characters
• Conversion from Hiragana to Katakana
• Conversion from Fullwidth to Halfwidth forms.
• Script conversions, for example Serbian Cyrillic to Latin

Example usage:

stream = new ICUTransformFilter(stream, Transliterator.getInstance("Traditional-Simplified"));

For more details, see the ICU User Guide.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

org.apache.lucene.util.AttributeSource.State

Field Summary

Fields inherited from class org.apache.lucene.analysis.TokenFilter

input

Fields inherited from class org.apache.lucene.analysis.TokenStream

DEFAULT_TOKEN_ATTRIBUTE_FACTORY

Constructor Summary

Constructors

Constructor and Description
IcuTransformTokenFilter(org.apache.lucene.analysis.TokenStream input, com.ibm.icu.text.Transliterator transliterator) Create a new IcuTransformFilter that transforms text on the given stream.

Method Summary

Modifier and Type	Method and Description
boolean	equals(java.lang.Object object)
int	hashCode()
boolean	incrementToken()

Methods inherited from class org.apache.lucene.analysis.TokenFilter

close, end, reset

Methods inherited from class org.apache.lucene.util.AttributeSource

addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, endAttributes, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

IcuTransformTokenFilter

public IcuTransformTokenFilter(org.apache.lucene.analysis.TokenStream input,
                               com.ibm.icu.text.Transliterator transliterator)

Create a new IcuTransformFilter that transforms text on the given stream.

Parameters:

input - TokenStream to filter.

transliterator - Transliterator to transform the text.

Method Detail

incrementToken

public boolean incrementToken()
                       throws java.io.IOException

Specified by:

incrementToken in class org.apache.lucene.analysis.TokenStream

Throws:

java.io.IOException

equals

public boolean equals(java.lang.Object object)

Overrides:

equals in class org.apache.lucene.util.AttributeSource

hashCode

public int hashCode()

Overrides:

hashCode in class org.apache.lucene.util.AttributeSource