org.deeplearning4j.spark.models.paragraphvectors

Class SparkParagraphVectors

java.lang.Object

org.deeplearning4j.models.embeddings.wordvectors.WordVectorsImpl<T>

org.deeplearning4j.models.sequencevectors.SequenceVectors<T>

org.deeplearning4j.spark.models.sequencevectors.SparkSequenceVectors<VocabWord>

org.deeplearning4j.spark.models.paragraphvectors.SparkParagraphVectors

All Implemented Interfaces:

java.io.Serializable, WordVectors

public class SparkParagraphVectors
extends SparkSequenceVectors<VocabWord>

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.deeplearning4j.spark.models.sequencevectors.SparkSequenceVectors

SparkSequenceVectors.Builder<T extends SequenceElement>

Nested classes/interfaces inherited from class org.deeplearning4j.models.sequencevectors.SequenceVectors

SequenceVectors.AsyncSequencer

Field Summary

Fields inherited from class org.deeplearning4j.spark.models.sequencevectors.SparkSequenceVectors

configurationBroadcast, ela, elementsFreqAccum, elementsFreqAccumExtra, exporter, isAutoDiscoveryMode, isEnvironmentReady, paramServerConfiguration, shallowVocabCache, shallowVocabCacheBroadcast, sla, storageLevel, vocabCacheBroadcast

Fields inherited from class org.deeplearning4j.models.sequencevectors.SequenceVectors

configuration, configured, elementsLearningAlgorithm, enableScavenger, eventListeners, existingModel, iterator, log, scoreElements, scoreSequences, sequenceLearningAlgorithm, unknownElement

Fields inherited from class org.deeplearning4j.models.embeddings.wordvectors.WordVectorsImpl

batchSize, DEFAULT_UNK, layerSize, learningRate, learningRateDecayWords, lookupTable, minLearningRate, minWordFrequency, modelUtils, negative, numEpochs, numIterations, resetModel, sampling, seed, stopWords, trainElementsVectors, trainSequenceVectors, useAdeGrad, useUnknown, variableWindows, vocab, window, workers

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	SparkParagraphVectors()

Method Summary

Modifier and Type	Method and Description
void	fitLabelledDocuments(org.apache.spark.api.java.JavaRDD<LabelledDocument> documentsRdd) This method builds ParagraphVectors model, expecting JavaRDD.
void	fitMultipleFiles(org.apache.spark.api.java.JavaPairRDD<java.lang.String,java.lang.String> documentsRdd) This method builds ParagraphVectors model, expecting JavaPairRDD with key as label, and value as document-in-a-string.
protected VocabCache<ShallowSequenceElement>	getShallowVocabCache()
protected void	validateConfiguration()

Methods inherited from class org.deeplearning4j.spark.models.sequencevectors.SparkSequenceVectors

broadcastEnvironment, buildShallowVocabCache, fit, fitLists, fitSequences, getCounter

Methods inherited from class org.deeplearning4j.models.sequencevectors.SequenceVectors

buildVocab, getElementsScore, getSequencesScore, getUNK, getWordVectorMatrix, initLearners, trainSequence

Methods inherited from class org.deeplearning4j.models.embeddings.wordvectors.WordVectorsImpl

accuracy, getLayerSize, getWordVector, getWordVectorMatrixNormalized, getWordVectors, getWordVectorsMean, hasWord, indexOf, lookupTable, setLookupTable, setModelUtils, setVocab, similarity, similarWordsInVocabTo, update, update, vocab, wordsNearest, wordsNearest, wordsNearest, wordsNearestSum, wordsNearestSum, wordsNearestSum

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.deeplearning4j.models.embeddings.wordvectors.WordVectors

accuracy, getWordVector, getWordVectorMatrixNormalized, getWordVectors, getWordVectorsMean, hasWord, indexOf, lookupTable, setModelUtils, setUNK, similarity, similarWordsInVocabTo, vocab, wordsNearest, wordsNearest, wordsNearest, wordsNearestSum, wordsNearestSum, wordsNearestSum

Constructor Detail

SparkParagraphVectors

protected SparkParagraphVectors()

Method Detail

getShallowVocabCache

protected VocabCache<ShallowSequenceElement> getShallowVocabCache()

Overrides:

getShallowVocabCache in class SparkSequenceVectors<VocabWord>

validateConfiguration

protected void validateConfiguration()

Overrides:

validateConfiguration in class SparkSequenceVectors<VocabWord>

fitMultipleFiles

public void fitMultipleFiles(org.apache.spark.api.java.JavaPairRDD<java.lang.String,java.lang.String> documentsRdd)

This method builds ParagraphVectors model, expecting JavaPairRDD with key as label, and value as document-in-a-string.

Parameters:

documentsRdd -

fitLabelledDocuments

public void fitLabelledDocuments(org.apache.spark.api.java.JavaRDD<LabelledDocument> documentsRdd)

This method builds ParagraphVectors model, expecting JavaRDD. It can be either non-tokenized documents, or tokenized.

Parameters:

documentsRdd -