org.datavec.nlp.tokenization.tokenizer

Interface TokenPreProcess

All Known Implementing Classes:

EndingPreProcessor

public interface TokenPreProcess

Token preprocessing

Method Summary

Modifier and Type	Method and Description
java.lang.String	preProcess(java.lang.String token) Pre process a token

Method Detail

preProcess

java.lang.String preProcess(java.lang.String token)

Pre process a token

Parameters:

token - the token to pre process

Returns:

the preprocessed token