org.deeplearning4j.models.glove

Class AbstractCoOccurrences<T extends SequenceElement>

java.lang.Object

org.deeplearning4j.models.glove.AbstractCoOccurrences<T>

All Implemented Interfaces:

java.io.Serializable

public class AbstractCoOccurrences<T extends SequenceElement>
extends java.lang.Object
implements java.io.Serializable

This class implements building cooccurrence map for abstract training corpus. However it's performance rather low, due to exsessive IO that happens in ShadowCopyThread PLEASE NOTE: Current implementation involves massive IO, and it should be rewritter as soon as ND4j gets sparse arrays support

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	AbstractCoOccurrences.Builder<T extends SequenceElement>

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.concurrent.locks.ReentrantReadWriteLock	lock
protected static org.slf4j.Logger	logger
protected long	memory_threshold
protected SequenceIterator<T>	sequenceIterator
protected boolean	symmetric
protected java.io.File	targetFile
protected VocabCache<T>	vocabCache
protected int	windowSize
protected int	workers

Method Summary

Modifier and Type	Method and Description
void	fit()
double	getCoOccurrenceCount(T element1, T element2) This method returns cooccurrence distance weights for two SequenceElements
protected long	getMemoryFootprint() This method returns estimated memory footrpint, based on current CountMap content
protected long	getMemoryThreshold() This memory returns memory threshold, defined as 1/2 of memory allowed for allocation
java.util.Iterator<Pair<Pair<T,T>,java.lang.Double>>	iterator() This method returns iterator with elements pairs and their weights.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

symmetric

protected boolean symmetric

windowSize

protected int windowSize

vocabCache

protected VocabCache<T extends SequenceElement> vocabCache

sequenceIterator

protected SequenceIterator<T extends SequenceElement> sequenceIterator

workers

protected int workers

targetFile

protected java.io.File targetFile

lock

protected java.util.concurrent.locks.ReentrantReadWriteLock lock

memory_threshold

protected long memory_threshold

logger

protected static final org.slf4j.Logger logger

Method Detail

getCoOccurrenceCount

public double getCoOccurrenceCount(@NonNull
                                   T element1,
                                   @NonNull
                                   T element2)

This method returns cooccurrence distance weights for two SequenceElements

Parameters:

element1 -

element2 -

Returns:

distance weight

getMemoryFootprint

protected long getMemoryFootprint()

This method returns estimated memory footrpint, based on current CountMap content

Returns:

getMemoryThreshold

protected long getMemoryThreshold()

This memory returns memory threshold, defined as 1/2 of memory allowed for allocation

Returns:

fit

public void fit()

iterator

public java.util.Iterator<Pair<Pair<T,T>,java.lang.Double>> iterator()

This method returns iterator with elements pairs and their weights. Resulting iterator is safe to use in multi-threaded environment. Developer's note: thread safety on received iterator is delegated to PrefetchedSentenceIterator

Returns: