org.deeplearning4j.models.word2vec.wordstore

Class VocabularyHolder.Builder

java.lang.Object

org.deeplearning4j.models.word2vec.wordstore.VocabularyHolder.Builder

Enclosing class:

VocabularyHolder

public static class VocabularyHolder.Builder
extends java.lang.Object

Constructor Summary

Constructors

Constructor and Description
Builder()

Method Summary

Modifier and Type	Method and Description
VocabularyHolder	build()
VocabularyHolder.Builder	externalCache(VocabCache cache)
VocabularyHolder.Builder	hugeModelExpected(boolean reallyExpected) With this argument set to true, you'll have your vocab scanned for low-freq words periodically.
VocabularyHolder.Builder	minWordFrequency(int threshold)
VocabularyHolder.Builder	scavengerActivationThreshold(int threshold) Activation threshold defines, how ofter scavenger will be executed, to throw away low-frequency keywords.
VocabularyHolder.Builder	scavengerRetentionDelay(int delay) Retention delay defines, how long low-freq word will be kept in vocab, during building.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Builder

public Builder()

Method Detail

externalCache

public VocabularyHolder.Builder externalCache(@NonNull
                                              VocabCache cache)

minWordFrequency

public VocabularyHolder.Builder minWordFrequency(int threshold)

hugeModelExpected

public VocabularyHolder.Builder hugeModelExpected(boolean reallyExpected)

With this argument set to true, you'll have your vocab scanned for low-freq words periodically. Please note: this is incompatible with SPECIAL mechanics.

Parameters:

reallyExpected -

Returns:

scavengerActivationThreshold

public VocabularyHolder.Builder scavengerActivationThreshold(int threshold)

Activation threshold defines, how ofter scavenger will be executed, to throw away low-frequency keywords. Good values to start mostly depends on your workstation. Something like 1000000 looks pretty nice to start with. Too low values can lead to undesired removal of words from vocab. Please note: this is incompatible with SPECIAL mechanics.

Parameters:

threshold -

Returns:

scavengerRetentionDelay

public VocabularyHolder.Builder scavengerRetentionDelay(int delay)

Retention delay defines, how long low-freq word will be kept in vocab, during building. Good values to start with: 3,4,5. Not too high, and not too low. Please note: this is incompatible with SPECIAL mechanics.

Parameters:

delay -

Returns:

build

public VocabularyHolder build()