org.deeplearning4j.text.tokenization.tokenizer.preprocessor

Class CommonPreprocessor

java.lang.Object

org.deeplearning4j.text.tokenization.tokenizer.preprocessor.CommonPreprocessor

All Implemented Interfaces:

TokenPreProcess

Direct Known Subclasses:

CustomStemmingPreprocessor, StemmingPreprocessor

public class CommonPreprocessor
extends java.lang.Object
implements TokenPreProcess

Constructor Summary

Constructors

Constructor and Description
CommonPreprocessor()

Method Summary

Modifier and Type	Method and Description
java.lang.String	preProcess(java.lang.String token) Pre process a token

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CommonPreprocessor

public CommonPreprocessor()

Method Detail

preProcess

public java.lang.String preProcess(java.lang.String token)

Description copied from interface: TokenPreProcess

Pre process a token

Specified by:

preProcess in interface TokenPreProcess

Parameters:

token - the token to pre process

Returns:

the preprocessed token