org.deeplearning4j.models.embeddings.reader.impl

Class BasicModelUtils<T extends SequenceElement>

java.lang.Object

org.deeplearning4j.models.embeddings.reader.impl.BasicModelUtils<T>

All Implemented Interfaces:

ModelUtils<T>

Direct Known Subclasses:

FlatModelUtils, TreeModelUtils

public class BasicModelUtils<T extends SequenceElement>
extends java.lang.Object
implements ModelUtils<T>

Basic implementation for ModelUtils interface, suited for standalone use. PLEASE NOTE: This reader applies normalization to underlying lookup table.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BasicModelUtils.ArrayComparator
static class	BasicModelUtils.SimilarityComparator
static class	BasicModelUtils.WordSimilarity

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	CORRECT
static java.lang.String	EXISTS
protected WeightLookupTable<T>	lookupTable
protected boolean	normalized
protected VocabCache<T>	vocabCache
static java.lang.String	WRONG

Constructor Summary

Constructors

Constructor and Description
BasicModelUtils()

Method Summary

Modifier and Type	Method and Description
java.util.Map<java.lang.String,java.lang.Double>	accuracy(java.util.List<java.lang.String> questions) Accuracy based on questions which are a space separated list of strings where the first word is the query word, the next 2 words are negative, and the last word is the predicted word to be nearest
static java.util.List<java.lang.String>	getLabels(java.util.List<BasicModelUtils.WordSimilarity> results, int limit)
void	init(WeightLookupTable<T> lookupTable) This method implementations should accept given lookup table, and use them in further calls to interface methods
double	similarity(java.lang.String label1, java.lang.String label2) Returns the similarity of 2 words.
java.util.List<java.lang.String>	similarWordsInVocabTo(java.lang.String word, double accuracy) Find all words with a similar characters in the vocab
java.util.Collection<java.lang.String>	wordsNearest(java.util.Collection<java.lang.String> positive, java.util.Collection<java.lang.String> negative, int top) Words nearest based on positive and negative words
java.util.Collection<java.lang.String>	wordsNearest(org.nd4j.linalg.api.ndarray.INDArray words, int top) Words nearest based on positive and negative words * @param top the top n words
java.util.Collection<java.lang.String>	wordsNearest(java.lang.String label, int n) This method implementations should return N nearest elements labels to given element's label
java.util.Collection<java.lang.String>	wordsNearestSum(java.util.Collection<java.lang.String> positive, java.util.Collection<java.lang.String> negative, int top) Words nearest based on positive and negative words
java.util.Collection<java.lang.String>	wordsNearestSum(org.nd4j.linalg.api.ndarray.INDArray words, int top) Words nearest based on positive and negative words * @param top the top n words
java.util.Collection<java.lang.String>	wordsNearestSum(java.lang.String word, int n) Get the top n words most similar to the given word

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

EXISTS

public static final java.lang.String EXISTS

See Also:

Constant Field Values

CORRECT

public static final java.lang.String CORRECT

See Also:

Constant Field Values

WRONG

public static final java.lang.String WRONG

See Also:

Constant Field Values

vocabCache

protected volatile VocabCache<T extends SequenceElement> vocabCache

lookupTable

protected volatile WeightLookupTable<T extends SequenceElement> lookupTable

normalized

protected volatile boolean normalized

Constructor Detail

BasicModelUtils

public BasicModelUtils()

Method Detail

init

public void init(@NonNull
                 WeightLookupTable<T> lookupTable)

Description copied from interface: ModelUtils

This method implementations should accept given lookup table, and use them in further calls to interface methods

Specified by:

init in interface ModelUtils<T extends SequenceElement>

similarity

public double similarity(@NonNull
                         java.lang.String label1,
                         @NonNull
                         java.lang.String label2)

Returns the similarity of 2 words. Result value will be in range [-1,1], where -1.0 is exact opposite similarity, i.e. NO similarity, and 1.0 is total match of two word vectors. However, most of time you'll see values in range [0,1], but that's something depends of training corpus. Returns NaN if any of labels not exists in vocab, or any label is null

Specified by:

similarity in interface ModelUtils<T extends SequenceElement>

Parameters:

label1 - the first word

label2 - the second word

Returns:

a normalized similarity (cosine similarity)

wordsNearest

public java.util.Collection<java.lang.String> wordsNearest(java.lang.String label,
                                                           int n)

Description copied from interface: ModelUtils

This method implementations should return N nearest elements labels to given element's label

Specified by:

wordsNearest in interface ModelUtils<T extends SequenceElement>

Parameters:

label - label to return nearest elements for

n - number of nearest words to return

Returns:

accuracy

public java.util.Map<java.lang.String,java.lang.Double> accuracy(java.util.List<java.lang.String> questions)

Accuracy based on questions which are a space separated list of strings where the first word is the query word, the next 2 words are negative, and the last word is the predicted word to be nearest

Specified by:

accuracy in interface ModelUtils<T extends SequenceElement>

Parameters:

questions - the questions to ask

Returns:

the accuracy based on these questions

similarWordsInVocabTo

public java.util.List<java.lang.String> similarWordsInVocabTo(java.lang.String word,
                                                              double accuracy)

Find all words with a similar characters in the vocab

Specified by:

similarWordsInVocabTo in interface ModelUtils<T extends SequenceElement>

Parameters:

word - the word to compare

accuracy - the accuracy: 0 to 1

Returns:

the list of words that are similar in the vocab

wordsNearest

public java.util.Collection<java.lang.String> wordsNearest(@NonNull
                                                           java.util.Collection<java.lang.String> positive,
                                                           @NonNull
                                                           java.util.Collection<java.lang.String> negative,
                                                           int top)

Description copied from interface: ModelUtils

Words nearest based on positive and negative words

Specified by:

wordsNearest in interface ModelUtils<T extends SequenceElement>

Parameters:

positive - the positive words

negative - the negative words

top - the top n words

Returns:

the words nearest the mean of the words

wordsNearestSum

public java.util.Collection<java.lang.String> wordsNearestSum(java.lang.String word,
                                                              int n)

Get the top n words most similar to the given word

Specified by:

wordsNearestSum in interface ModelUtils<T extends SequenceElement>

Parameters:

word - the word to compare

n - the n to get

Returns:

the top n words

wordsNearest

public java.util.Collection<java.lang.String> wordsNearest(org.nd4j.linalg.api.ndarray.INDArray words,
                                                           int top)

Words nearest based on positive and negative words * @param top the top n words

Specified by:

wordsNearest in interface ModelUtils<T extends SequenceElement>

Returns:

the words nearest the mean of the words

wordsNearestSum

public java.util.Collection<java.lang.String> wordsNearestSum(org.nd4j.linalg.api.ndarray.INDArray words,
                                                              int top)

Words nearest based on positive and negative words * @param top the top n words

Specified by:

wordsNearestSum in interface ModelUtils<T extends SequenceElement>

Returns:

the words nearest the mean of the words

wordsNearestSum

public java.util.Collection<java.lang.String> wordsNearestSum(java.util.Collection<java.lang.String> positive,
                                                              java.util.Collection<java.lang.String> negative,
                                                              int top)

Words nearest based on positive and negative words

Specified by:

wordsNearestSum in interface ModelUtils<T extends SequenceElement>

Parameters:

positive - the positive words

negative - the negative words

top - the top n words

Returns:

the words nearest the mean of the words

getLabels

public static java.util.List<java.lang.String> getLabels(java.util.List<BasicModelUtils.WordSimilarity> results,
                                                         int limit)