Class | Description |
---|---|
MultiThreadedTokenizerTest | |
RandomizedInputTest | |
SearchTokenizerTest | |
Token |
IPADIC token produced by the IPADIC tokenizer with various morphological features
|
Tokenizer |
A tokenizer based on the IPADIC dictionary
|
Tokenizer.Builder |
Builder class for creating a customized tokenizer instance
|
TokenizerTest | |
UserDictionaryTokenizerTest |