Word2Vec.Builder

java.lang.Object
- org.deeplearning4j.models.sequencevectors.SequenceVectors.Builder<VocabWord>
- - org.deeplearning4j.models.word2vec.Word2Vec.Builder

Direct Known Subclasses:: ParagraphVectors.Builder

Enclosing class:: Word2Vec

public static class Word2Vec.Builder
extends SequenceVectors.Builder<VocabWord>

Field Summary

Fields
Modifier and Type	Field and Description
`protected boolean`	`allowParallelTokenization`
`protected LabelAwareIterator`	`labelAwareIterator`
`protected SentenceIterator`	`sentenceIterator`
`protected TokenizerFactory`	`tokenizerFactory`

Fields inherited from class org.deeplearning4j.models.sequencevectors.SequenceVectors.Builder
batchSize, configuration, elementsLearningAlgorithm, enableScavenger, existingVectors, hugeModelExpected, iterations, iterator, layerSize, learningRate, learningRateDecayWords, lookupTable, minLearningRate, minWordFrequency, modelUtils, negative, numEpochs, preciseWeightInit, resetModel, sampling, seed, sequenceLearningAlgorithm, STOP, stopWords, trainElementsVectors, trainSequenceVectors, UNK, unknownElement, useAdaGrad, useHierarchicSoftmax, useUnknown, variableWindows, vectorsListeners, vocabCache, window, workers

Constructor Summary

Constructors
Constructor and Description

Builder()

Builder(VectorsConfiguration configuration)

Constructors
Constructor and Description
`Builder()`
`Builder(VectorsConfiguration configuration)`

Method Summary

All Methods Instance Methods Concrete Methods Deprecated Methods
Modifier and Type	Method and Description
`Word2Vec.Builder`	`allowParallelTokenization(boolean allow)` This method enables/disables parallel tokenization.
`Word2Vec.Builder`	`batchSize(int batchSize)` This method defines mini-batch size
`Word2Vec`	`build()` Build SequenceVectors instance with defined settings/options
`Word2Vec.Builder`	`elementsLearningAlgorithm(ElementsLearningAlgorithm<VocabWord> algorithm)` * Sets specific LearningAlgorithm as Elements Learning Algorithm
`Word2Vec.Builder`	`elementsLearningAlgorithm(java.lang.String algorithm)` * Sets specific LearningAlgorithm as Elements Learning Algorithm
`Word2Vec.Builder`	`enableScavenger(boolean reallyEnable)` This method ebables/disables periodical vocab truncation during construction Default value: disabled
`Word2Vec.Builder`	`epochs(int numEpochs)` This method defines number of epochs (iterations over whole training corpus) for training
`Word2Vec.Builder`	`index(InvertedIndex<VocabWord> index)` Deprecated.
`Word2Vec.Builder`	`iterate(DocumentIterator iterator)`
`Word2Vec.Builder`	`iterate(LabelAwareIterator iterator)` This method used to feed LabelAwareIterator, that is usually used
`Word2Vec.Builder`	`iterate(SentenceIterator iterator)` This method used to feed SentenceIterator, that contains training corpus, into ParagraphVectors
`Word2Vec.Builder`	`iterate(SequenceIterator<VocabWord> iterator)` This method used to feed SequenceIterator, that contains training corpus, into ParagraphVectors
`Word2Vec.Builder`	`iterations(int iterations)` This method defines number of iterations done for each mini-batch during training
`Word2Vec.Builder`	`layerSize(int layerSize)` This method defines number of dimensions for output vectors
`Word2Vec.Builder`	`learningRate(double learningRate)` This method defines initial learning rate for model training
`Word2Vec.Builder`	`lookupTable(WeightLookupTable<VocabWord> lookupTable)` This method allows to define external WeightLookupTable to be used
`Word2Vec.Builder`	`minLearningRate(double minLearningRate)` This method defines minimal learning rate value for training
`Word2Vec.Builder`	`minWordFrequency(int minWordFrequency)` This method defines minimal word frequency in training corpus.
`Word2Vec.Builder`	`modelUtils(ModelUtils<VocabWord> modelUtils)` Sets ModelUtils that gonna be used as provider for utility methods: similarity(), wordsNearest(), accuracy(), etc
`Word2Vec.Builder`	`negativeSample(double negative)` This method defines whether negative sampling should be used or not
`Word2Vec.Builder`	`resetModel(boolean reallyReset)` This method defines whether model should be totally wiped out prior building, or not
`Word2Vec.Builder`	`sampling(double sampling)` This method defines whether subsampling should be used or not
`Word2Vec.Builder`	`seed(long randomSeed)` This method defines random seed for random numbers generator
`Word2Vec.Builder`	`setVectorsListeners(java.util.Collection<VectorsListener<VocabWord>> vectorsListeners)` This method sets VectorsListeners for this SequenceVectors model
`Word2Vec.Builder`	`stopWords(java.util.Collection<VocabWord> stopList)` This method defines stop words that should be ignored during training
`Word2Vec.Builder`	`stopWords(java.util.List<java.lang.String> stopList)` This method defines stop words that should be ignored during training
`Word2Vec.Builder`	`tokenizerFactory(TokenizerFactory tokenizerFactory)` This method defines TokenizerFactory to be used for strings tokenization during training PLEASE NOTE: If external VocabCache is used, the same TokenizerFactory should be used to keep derived tokens equal.
`Word2Vec.Builder`	`trainElementsRepresentation(boolean trainElements)` This method is hardcoded to TRUE, since that's whole point of Word2Vec
`Word2Vec.Builder`	`trainSequencesRepresentation(boolean trainSequences)` This method is hardcoded to FALSE, since that's whole point of Word2Vec
`Word2Vec.Builder`	`unknownElement(VocabWord element)` This method allows you to specify SequenceElement that will be used as UNK element, if UNK is used
`Word2Vec.Builder`	`useAdaGrad(boolean reallyUse)` This method defines whether adaptive gradients should be used or not
`protected Word2Vec.Builder`	`useExistingWordVectors(WordVectors vec)` This method has no effect for Word2Vec
`Word2Vec.Builder`	`useHierarchicSoftmax(boolean reallyUse)` Enable/disable hierarchic softmax
`Word2Vec.Builder`	`usePreciseWeightInit(boolean reallyUse)` If set to true, initial weights for elements/sequences will be derived from elements themself.
`Word2Vec.Builder`	`useUnknown(boolean reallyUse)` This method allows you to specify, if UNK word should be used internally
`Word2Vec.Builder`	`useVariableWindow(int... windows)` This method allows to use variable window size.
`Word2Vec.Builder`	`vocabCache(VocabCache<VocabWord> vocabCache)` This method allows to define external VocabCache to be used
`Word2Vec.Builder`	`windowSize(int windowSize)` This method defines context window size
`Word2Vec.Builder`	`workers(int numWorkers)` This method defines maximum number of concurrent threads available for training

Methods inherited from class org.deeplearning4j.models.sequencevectors.SequenceVectors.Builder
presetTables, sequenceLearningAlgorithm, sequenceLearningAlgorithm

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - sentenceIterator
```
protected SentenceIterator sentenceIterator
```
  - labelAwareIterator
```
protected LabelAwareIterator labelAwareIterator
```
  - tokenizerFactory
```
protected TokenizerFactory tokenizerFactory
```
  - allowParallelTokenization
```
protected boolean allowParallelTokenization
```
- Constructor Detail
  - Builder
```
public Builder()
```
  - Builder
```
public Builder(@NonNull
               VectorsConfiguration configuration)
```
- Method Detail
  - useExistingWordVectors
```
protected Word2Vec.Builder useExistingWordVectors(@NonNull
                                                  WordVectors vec)
```
    This method has no effect for Word2Vec
    
    Overrides:
    
    useExistingWordVectors in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    vec - existing WordVectors model
    
    Returns:
  - iterate
```
public Word2Vec.Builder iterate(@NonNull
                                DocumentIterator iterator)
```
  - iterate
```
public Word2Vec.Builder iterate(@NonNull
                                SentenceIterator iterator)
```
    This method used to feed SentenceIterator, that contains training corpus, into ParagraphVectors
    
    Parameters:
    
    iterator -
    
    Returns:
  - tokenizerFactory
```
public Word2Vec.Builder tokenizerFactory(@NonNull
                                         TokenizerFactory tokenizerFactory)
```
    This method defines TokenizerFactory to be used for strings tokenization during training PLEASE NOTE: If external VocabCache is used, the same TokenizerFactory should be used to keep derived tokens equal.
    
    Parameters:
    
    tokenizerFactory -
    
    Returns:
  - index
```
@Deprecated
public Word2Vec.Builder index(@NonNull
                                          InvertedIndex<VocabWord> index)
```
    Deprecated.
  - iterate
```
public Word2Vec.Builder iterate(@NonNull
                                SequenceIterator<VocabWord> iterator)
```
    This method used to feed SequenceIterator, that contains training corpus, into ParagraphVectors
    
    Overrides:
    
    iterate in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    iterator -
    
    Returns:
  - iterate
```
public Word2Vec.Builder iterate(@NonNull
                                LabelAwareIterator iterator)
```
    This method used to feed LabelAwareIterator, that is usually used
    
    Parameters:
    
    iterator -
    
    Returns:
  - batchSize
```
public Word2Vec.Builder batchSize(int batchSize)
```
    This method defines mini-batch size
    
    Overrides:
    
    batchSize in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    batchSize -
    
    Returns:
  - iterations
```
public Word2Vec.Builder iterations(int iterations)
```
    This method defines number of iterations done for each mini-batch during training
    
    Overrides:
    
    iterations in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    iterations -
    
    Returns:
  - epochs
```
public Word2Vec.Builder epochs(int numEpochs)
```
    This method defines number of epochs (iterations over whole training corpus) for training
    
    Overrides:
    
    epochs in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    numEpochs -
    
    Returns:
  - layerSize
```
public Word2Vec.Builder layerSize(int layerSize)
```
    This method defines number of dimensions for output vectors
    
    Overrides:
    
    layerSize in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    layerSize -
    
    Returns:
  - learningRate
```
public Word2Vec.Builder learningRate(double learningRate)
```
    This method defines initial learning rate for model training
    
    Overrides:
    
    learningRate in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    learningRate -
    
    Returns:
  - minWordFrequency
```
public Word2Vec.Builder minWordFrequency(int minWordFrequency)
```
    This method defines minimal word frequency in training corpus. All words below this threshold will be removed prior model training
    
    Overrides:
    
    minWordFrequency in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    minWordFrequency -
    
    Returns:
  - minLearningRate
```
public Word2Vec.Builder minLearningRate(double minLearningRate)
```
    This method defines minimal learning rate value for training
    
    Overrides:
    
    minLearningRate in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    minLearningRate -
    
    Returns:
  - resetModel
```
public Word2Vec.Builder resetModel(boolean reallyReset)
```
    This method defines whether model should be totally wiped out prior building, or not
    
    Overrides:
    
    resetModel in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    reallyReset -
    
    Returns:
  - vocabCache
```
public Word2Vec.Builder vocabCache(@NonNull
                                   VocabCache<VocabWord> vocabCache)
```
    This method allows to define external VocabCache to be used
    
    Overrides:
    
    vocabCache in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    vocabCache -
    
    Returns:
  - lookupTable
```
public Word2Vec.Builder lookupTable(@NonNull
                                    WeightLookupTable<VocabWord> lookupTable)
```
    This method allows to define external WeightLookupTable to be used
    
    Overrides:
    
    lookupTable in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    lookupTable -
    
    Returns:
  - sampling
```
public Word2Vec.Builder sampling(double sampling)
```
    This method defines whether subsampling should be used or not
    
    Overrides:
    
    sampling in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    sampling - set > 0 to subsampling argument, or 0 to disable
    
    Returns:
  - useAdaGrad
```
public Word2Vec.Builder useAdaGrad(boolean reallyUse)
```
    This method defines whether adaptive gradients should be used or not
    
    Overrides:
    
    useAdaGrad in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    reallyUse -
    
    Returns:
  - negativeSample
```
public Word2Vec.Builder negativeSample(double negative)
```
    This method defines whether negative sampling should be used or not
    
    Overrides:
    
    negativeSample in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    negative - set > 0 as negative sampling argument, or 0 to disable
    
    Returns:
  - stopWords
```
public Word2Vec.Builder stopWords(@NonNull
                                  java.util.List<java.lang.String> stopList)
```
    This method defines stop words that should be ignored during training
    
    Overrides:
    
    stopWords in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    stopList -
    
    Returns:
  - trainElementsRepresentation
```
public Word2Vec.Builder trainElementsRepresentation(boolean trainElements)
```
    This method is hardcoded to TRUE, since that's whole point of Word2Vec
    
    Overrides:
    
    trainElementsRepresentation in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    trainElements -
    
    Returns:
  - trainSequencesRepresentation
```
public Word2Vec.Builder trainSequencesRepresentation(boolean trainSequences)
```
    This method is hardcoded to FALSE, since that's whole point of Word2Vec
    
    Overrides:
    
    trainSequencesRepresentation in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    trainSequences -
    
    Returns:
  - stopWords
```
public Word2Vec.Builder stopWords(@NonNull
                                  java.util.Collection<VocabWord> stopList)
```
    This method defines stop words that should be ignored during training
    
    Overrides:
    
    stopWords in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    stopList -
    
    Returns:
  - windowSize
```
public Word2Vec.Builder windowSize(int windowSize)
```
    This method defines context window size
    
    Overrides:
    
    windowSize in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    windowSize -
    
    Returns:
  - seed
```
public Word2Vec.Builder seed(long randomSeed)
```
    This method defines random seed for random numbers generator
    
    Overrides:
    
    seed in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    randomSeed -
    
    Returns:
  - workers
```
public Word2Vec.Builder workers(int numWorkers)
```
    This method defines maximum number of concurrent threads available for training
    
    Overrides:
    
    workers in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    numWorkers -
    
    Returns:
  - modelUtils
```
public Word2Vec.Builder modelUtils(@NonNull
                                   ModelUtils<VocabWord> modelUtils)
```
    Sets ModelUtils that gonna be used as provider for utility methods: similarity(), wordsNearest(), accuracy(), etc
    
    Overrides:
    
    modelUtils in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    modelUtils - model utils to be used
    
    Returns:
  - useVariableWindow
```
public Word2Vec.Builder useVariableWindow(int... windows)
```
    This method allows to use variable window size. In this case, every batch gets processed using one of predefined window sizes
    
    Overrides:
    
    useVariableWindow in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    windows -
    
    Returns:
  - unknownElement
```
public Word2Vec.Builder unknownElement(VocabWord element)
```
    This method allows you to specify SequenceElement that will be used as UNK element, if UNK is used
    
    Overrides:
    
    unknownElement in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    element -
    
    Returns:
  - useUnknown
```
public Word2Vec.Builder useUnknown(boolean reallyUse)
```
    This method allows you to specify, if UNK word should be used internally
    
    Overrides:
    
    useUnknown in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    reallyUse -
    
    Returns:
  - setVectorsListeners
```
public Word2Vec.Builder setVectorsListeners(@NonNull
                                            java.util.Collection<VectorsListener<VocabWord>> vectorsListeners)
```
    This method sets VectorsListeners for this SequenceVectors model
    
    Overrides:
    
    setVectorsListeners in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    vectorsListeners -
    
    Returns:
  - elementsLearningAlgorithm
```
public Word2Vec.Builder elementsLearningAlgorithm(@NonNull
                                                  java.lang.String algorithm)
```
    Description copied from class: SequenceVectors.Builder
    
    * Sets specific LearningAlgorithm as Elements Learning Algorithm
    
    Overrides:
    
    elementsLearningAlgorithm in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    algorithm - fully qualified class name
    
    Returns:
  - elementsLearningAlgorithm
```
public Word2Vec.Builder elementsLearningAlgorithm(@NonNull
                                                  ElementsLearningAlgorithm<VocabWord> algorithm)
```
    Description copied from class: SequenceVectors.Builder
    
    * Sets specific LearningAlgorithm as Elements Learning Algorithm
    
    Overrides:
    
    elementsLearningAlgorithm in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    algorithm - ElementsLearningAlgorithm implementation
    
    Returns:
  - allowParallelTokenization
```
public Word2Vec.Builder allowParallelTokenization(boolean allow)
```
    This method enables/disables parallel tokenization. Default value: TRUE
    
    Parameters:
    
    allow -
    
    Returns:
  - enableScavenger
```
public Word2Vec.Builder enableScavenger(boolean reallyEnable)
```
    This method ebables/disables periodical vocab truncation during construction Default value: disabled
    
    Overrides:
    
    enableScavenger in class SequenceVectors.Builder<VocabWord>
    
    Parameters:
    
    reallyEnable -
    
    Returns:
  - useHierarchicSoftmax
```
public Word2Vec.Builder useHierarchicSoftmax(boolean reallyUse)
```
    Description copied from class: SequenceVectors.Builder
    
    Enable/disable hierarchic softmax
    
    Overrides:
    
    useHierarchicSoftmax in class SequenceVectors.Builder<VocabWord>
    
    Returns:
  - usePreciseWeightInit
```
public Word2Vec.Builder usePreciseWeightInit(boolean reallyUse)
```
    Description copied from class: SequenceVectors.Builder
    
    If set to true, initial weights for elements/sequences will be derived from elements themself. However, this implies additional cycle through input iterator. Default value: FALSE
    
    Overrides:
    
    usePreciseWeightInit in class SequenceVectors.Builder<VocabWord>
    
    Returns:
  - build
```
public Word2Vec build()
```
    Description copied from class: SequenceVectors.Builder
    
    Build SequenceVectors instance with defined settings/options
    
    Overrides:
    
    build in class SequenceVectors.Builder<VocabWord>
    
    Returns:

Class Word2Vec.Builder

Field Summary

Fields inherited from class org.deeplearning4j.models.sequencevectors.SequenceVectors.Builder

Constructor Summary

Method Summary

Methods inherited from class org.deeplearning4j.models.sequencevectors.SequenceVectors.Builder

Methods inherited from class java.lang.Object

Field Detail

sentenceIterator

labelAwareIterator

tokenizerFactory

allowParallelTokenization

Constructor Detail

Builder

Builder

Method Detail

useExistingWordVectors

iterate

iterate

tokenizerFactory

index

iterate

iterate

batchSize

iterations

epochs

layerSize

learningRate

minWordFrequency

minLearningRate

resetModel

vocabCache

lookupTable

sampling

useAdaGrad

negativeSample

stopWords

trainElementsRepresentation

trainSequencesRepresentation

stopWords

windowSize

seed

workers

modelUtils

useVariableWindow

unknownElement

useUnknown

setVectorsListeners

elementsLearningAlgorithm

elementsLearningAlgorithm

allowParallelTokenization

enableScavenger

useHierarchicSoftmax

usePreciseWeightInit

build