de.ipcon.db.fulltext.lucene

Class SplitTokenFilter

java.lang.Object

org.apache.lucene.util.AttributeSource

org.apache.lucene.analysis.TokenStream

org.apache.lucene.analysis.TokenFilter

de.ipcon.db.fulltext.lucene.SplitTokenFilter

public class SplitTokenFilter
extends org.apache.lucene.analysis.TokenFilter

Filter that will split tokens using a given regular expression. Using a regex of "[^\p{Alnum}]": Example 1: "WTA1234" will result in token "WTA1234". Example 2: "WTA+1234" will result in tokens "WTA+1234", "WTA" and "1234". Example 3: "WTA-1234 XYZ" will result in token "WTA-1234 XYZ", "WTA", "1234" and "XYZ".

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

org.apache.lucene.util.AttributeSource.AttributeFactory, org.apache.lucene.util.AttributeSource.State

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	TOKEN_TYPE

Constructor Summary

Constructors

Constructor and Description
SplitTokenFilter(org.apache.lucene.analysis.TokenStream _in, java.lang.String _regex)

Method Summary

Modifier and Type	Method and Description
org.apache.lucene.analysis.Token	next()

Methods inherited from class org.apache.lucene.analysis.TokenFilter

close, end, reset

Methods inherited from class org.apache.lucene.analysis.TokenStream

getOnlyUseNewAPI, incrementToken, next, setOnlyUseNewAPI

Methods inherited from class org.apache.lucene.util.AttributeSource

addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, restoreState, toString

Methods inherited from class java.lang.Object

getClass, notify, notifyAll, wait, wait, wait

Field Detail

TOKEN_TYPE

public static final java.lang.String TOKEN_TYPE

See Also:

Constant Field Values

Constructor Detail

SplitTokenFilter

public SplitTokenFilter(org.apache.lucene.analysis.TokenStream _in,
                        java.lang.String _regex)

Parameters:

_in - the "parent" TokenStream from which the "original" Tokens are read.

_regex - a regular expression; the tokens will be splitted using this regex (see String.split() for example).

Method Detail

next

public org.apache.lucene.analysis.Token next()
                                      throws java.io.IOException

Overrides:

next in class org.apache.lucene.analysis.TokenStream

Throws:

java.io.IOException