org.datavec.nlp.vectorizer

Class TfidfVectorizer

java.lang.Object

org.datavec.nlp.vectorizer.TextVectorizer<VECTOR_TYPE>

org.datavec.nlp.vectorizer.AbstractTfidfVectorizer<org.nd4j.linalg.api.ndarray.INDArray>

org.datavec.nlp.vectorizer.TfidfVectorizer

All Implemented Interfaces:

Vectorizer<org.nd4j.linalg.api.ndarray.INDArray>

public class TfidfVectorizer
extends AbstractTfidfVectorizer<org.nd4j.linalg.api.ndarray.INDArray>

Nd4j tfidf vectorizer

Nested Class Summary

Nested classes/interfaces inherited from interface org.datavec.api.vector.Vectorizer

Vectorizer.RecordCallBack

Field Summary

Fields inherited from class org.datavec.nlp.vectorizer.TextVectorizer

cache, MIN_WORD_FREQUENCY, minWordFrequency, STOP_WORDS, stopWords, TOKENIZER, tokenizerFactory, VOCAB_CACHE

Constructor Summary

Constructors

Constructor and Description
TfidfVectorizer()

Method Summary

Modifier and Type	Method and Description
org.nd4j.linalg.api.ndarray.INDArray	createVector(java.lang.Object[] args) Create a vector based on the given arguments
org.nd4j.linalg.api.ndarray.INDArray	fitTransform(RecordReader reader) Fit based on a record reader
org.nd4j.linalg.api.ndarray.INDArray	fitTransform(RecordReader reader, Vectorizer.RecordCallBack callBack) Fit based on a record reader
org.nd4j.linalg.api.ndarray.INDArray	transform(Record record) Transform a record in to a vector

Methods inherited from class org.datavec.nlp.vectorizer.AbstractTfidfVectorizer

createTokenizerFactory, doWithTokens

Methods inherited from class org.datavec.nlp.vectorizer.TextVectorizer

fit, fit, initialize, toString, wordFrequenciesForRecord

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TfidfVectorizer

public TfidfVectorizer()

Method Detail

createVector

public org.nd4j.linalg.api.ndarray.INDArray createVector(java.lang.Object[] args)

Description copied from interface: Vectorizer

Create a vector based on the given arguments

Specified by:

createVector in interface Vectorizer<org.nd4j.linalg.api.ndarray.INDArray>

Specified by:

createVector in class AbstractTfidfVectorizer<org.nd4j.linalg.api.ndarray.INDArray>

Parameters:

args - the arguments to create a vector with

Returns:

the created vector

fitTransform

public org.nd4j.linalg.api.ndarray.INDArray fitTransform(RecordReader reader)

Description copied from interface: Vectorizer

Fit based on a record reader

Specified by:

fitTransform in interface Vectorizer<org.nd4j.linalg.api.ndarray.INDArray>

Specified by:

fitTransform in class AbstractTfidfVectorizer<org.nd4j.linalg.api.ndarray.INDArray>

fitTransform

public org.nd4j.linalg.api.ndarray.INDArray fitTransform(RecordReader reader,
                                                         Vectorizer.RecordCallBack callBack)

Description copied from interface: Vectorizer

Fit based on a record reader

transform

public org.nd4j.linalg.api.ndarray.INDArray transform(Record record)

Description copied from interface: Vectorizer

Transform a record in to a vector

Specified by:

transform in interface Vectorizer<org.nd4j.linalg.api.ndarray.INDArray>

Specified by:

transform in class AbstractTfidfVectorizer<org.nd4j.linalg.api.ndarray.INDArray>

Parameters:

record - the record to write

Returns: