org.deeplearning4j.spark.models.word2vec

Class SparkWord2Vec

java.lang.Object

org.deeplearning4j.models.embeddings.wordvectors.WordVectorsImpl<T>

org.deeplearning4j.models.sequencevectors.SequenceVectors<T>

org.deeplearning4j.spark.models.sequencevectors.SparkSequenceVectors<VocabWord>

org.deeplearning4j.spark.models.word2vec.SparkWord2Vec

All Implemented Interfaces:

java.io.Serializable, WordVectors

public class SparkWord2Vec
extends SparkSequenceVectors<VocabWord>

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SparkWord2Vec.Builder

Nested classes/interfaces inherited from class org.deeplearning4j.models.sequencevectors.SequenceVectors

SequenceVectors.AsyncSequencer

Field Summary

Fields inherited from class org.deeplearning4j.spark.models.sequencevectors.SparkSequenceVectors

configurationBroadcast, ela, elementsFreqAccum, elementsFreqAccumExtra, exporter, isAutoDiscoveryMode, isEnvironmentReady, paramServerConfiguration, shallowVocabCache, shallowVocabCacheBroadcast, sla, storageLevel, vocabCacheBroadcast

Fields inherited from class org.deeplearning4j.models.sequencevectors.SequenceVectors

configuration, configured, elementsLearningAlgorithm, enableScavenger, eventListeners, existingModel, iterator, log, scoreElements, scoreSequences, sequenceLearningAlgorithm, unknownElement

Fields inherited from class org.deeplearning4j.models.embeddings.wordvectors.WordVectorsImpl

batchSize, DEFAULT_UNK, layerSize, learningRate, learningRateDecayWords, lookupTable, minLearningRate, minWordFrequency, modelUtils, negative, numEpochs, numIterations, resetModel, sampling, seed, stopWords, trainElementsVectors, trainSequenceVectors, useAdeGrad, useUnknown, variableWindows, vocab, window, workers

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	SparkWord2Vec()
	SparkWord2Vec(org.nd4j.parameterserver.distributed.conf.VoidConfiguration psConfiguration, VectorsConfiguration configuration)

Method Summary

Modifier and Type	Method and Description
void	fit() Deprecated.
void	fitSentences(org.apache.spark.api.java.JavaRDD<java.lang.String> sentences)
protected VocabCache<ShallowSequenceElement>	getShallowVocabCache()
protected void	validateConfiguration()

Methods inherited from class org.deeplearning4j.spark.models.sequencevectors.SparkSequenceVectors

broadcastEnvironment, buildShallowVocabCache, fitLists, fitSequences, getCounter

Methods inherited from class org.deeplearning4j.models.sequencevectors.SequenceVectors

buildVocab, getElementsScore, getSequencesScore, getUNK, getWordVectorMatrix, initLearners, trainSequence

Methods inherited from class org.deeplearning4j.models.embeddings.wordvectors.WordVectorsImpl

accuracy, getLayerSize, getWordVector, getWordVectorMatrixNormalized, getWordVectors, getWordVectorsMean, hasWord, indexOf, lookupTable, setLookupTable, setModelUtils, setVocab, similarity, similarWordsInVocabTo, update, update, vocab, wordsNearest, wordsNearest, wordsNearest, wordsNearestSum, wordsNearestSum, wordsNearestSum

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.deeplearning4j.models.embeddings.wordvectors.WordVectors

accuracy, getWordVector, getWordVectorMatrixNormalized, getWordVectors, getWordVectorsMean, hasWord, indexOf, lookupTable, setModelUtils, setUNK, similarity, similarWordsInVocabTo, vocab, wordsNearest, wordsNearest, wordsNearest, wordsNearestSum, wordsNearestSum, wordsNearestSum

Constructor Detail

SparkWord2Vec

protected SparkWord2Vec()

SparkWord2Vec

public SparkWord2Vec(@NonNull
                     org.nd4j.parameterserver.distributed.conf.VoidConfiguration psConfiguration,
                     @NonNull
                     VectorsConfiguration configuration)

Method Detail

getShallowVocabCache

protected VocabCache<ShallowSequenceElement> getShallowVocabCache()

Overrides:

getShallowVocabCache in class SparkSequenceVectors<VocabWord>

validateConfiguration

protected void validateConfiguration()

Overrides:

validateConfiguration in class SparkSequenceVectors<VocabWord>

fit

@Deprecated
public void fit()

Deprecated.

PLEASE NOTE: This method isn't supported for Spark implementation. Consider using fitLists() or fitSequences() instead.

Overrides:

fit in class SparkSequenceVectors<VocabWord>

fitSentences

public void fitSentences(org.apache.spark.api.java.JavaRDD<java.lang.String> sentences)