public class TfidfVectorizer extends BaseTextVectorizer
Modifier and Type | Class and Description |
---|---|
static class |
TfidfVectorizer.Builder |
index, isParallel, iterator, labelsSource, minWordFrequency, stopWords, tokenizerFactory, vocabCache
Constructor and Description |
---|
TfidfVectorizer() |
Modifier and Type | Method and Description |
---|---|
double |
tfidfWord(java.lang.String word,
long wordCount,
long documentLength) |
org.nd4j.linalg.api.ndarray.INDArray |
transform(java.util.List<java.lang.String> tokens)
Transforms the matrix
|
org.nd4j.linalg.api.ndarray.INDArray |
transform(java.lang.String text)
Transforms the matrix
|
org.nd4j.linalg.dataset.DataSet |
vectorize()
Vectorizes the input source in to a dataset
|
org.nd4j.linalg.dataset.DataSet |
vectorize(java.io.File input,
java.lang.String label) |
org.nd4j.linalg.dataset.DataSet |
vectorize(java.io.InputStream is,
java.lang.String label)
Text coming from an input stream considered as one document
|
org.nd4j.linalg.dataset.DataSet |
vectorize(java.lang.String text,
java.lang.String label)
Vectorizes the passed in text treating it as one document
|
buildVocab, fit, getLabelsSource, numWordsEncountered
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
getIndex, getVocabCache
public org.nd4j.linalg.dataset.DataSet vectorize(java.io.InputStream is, java.lang.String label)
is
- the input stream to read fromlabel
- the label to assignpublic org.nd4j.linalg.dataset.DataSet vectorize(java.lang.String text, java.lang.String label)
text
- the text to vectorizelabel
- the label of the textpublic org.nd4j.linalg.dataset.DataSet vectorize(java.io.File input, java.lang.String label)
input
- the text to vectorizelabel
- the label of the textDataSet
with a applyTransformToDestination of
weights(relative to impl; could be word counts or tfidf scores)public org.nd4j.linalg.api.ndarray.INDArray transform(java.lang.String text)
text
- text to transformINDArray
public org.nd4j.linalg.api.ndarray.INDArray transform(java.util.List<java.lang.String> tokens)
TextVectorizer
public double tfidfWord(java.lang.String word, long wordCount, long documentLength)
public org.nd4j.linalg.dataset.DataSet vectorize()