org.deeplearning4j.models.sequencevectors.transformers.impl

Class SentenceTransformer

java.lang.Object

org.deeplearning4j.models.sequencevectors.transformers.impl.SentenceTransformer

All Implemented Interfaces:

java.lang.Iterable<Sequence<VocabWord>>, SequenceTransformer<VocabWord,java.lang.String>

public class SentenceTransformer
extends java.lang.Object
implements SequenceTransformer<VocabWord,java.lang.String>, java.lang.Iterable<Sequence<VocabWord>>

This simple class is responsible for conversion lines of text to Sequences of SequenceElements to fit them into SequenceVectors model

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SentenceTransformer.Builder

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	allowMultithreading
protected BasicTransformerIterator	currentIterator
protected LabelAwareIterator	iterator
protected static org.slf4j.Logger	log
protected boolean	readOnly
protected java.util.concurrent.atomic.AtomicInteger	sentenceCounter
protected TokenizerFactory	tokenizerFactory

Method Summary

Modifier and Type	Method and Description
java.util.Iterator<Sequence<VocabWord>>	iterator()
void	reset()
Sequence<VocabWord>	transformToSequence(java.lang.String object) This is generic method for transformation data from any format to Sequence of SequenceElement.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Field Detail

tokenizerFactory

protected TokenizerFactory tokenizerFactory

iterator

protected LabelAwareIterator iterator

readOnly

protected boolean readOnly

sentenceCounter

protected java.util.concurrent.atomic.AtomicInteger sentenceCounter

allowMultithreading

protected boolean allowMultithreading

currentIterator

protected BasicTransformerIterator currentIterator

log

protected static final org.slf4j.Logger log

Method Detail

transformToSequence

public Sequence<VocabWord> transformToSequence(java.lang.String object)

Description copied from interface: SequenceTransformer

This is generic method for transformation data from any format to Sequence of SequenceElement. It will be used both in Vocab building, and in training process

Specified by:

transformToSequence in interface SequenceTransformer<VocabWord,java.lang.String>

Parameters:

object - - Object to be transformed into Sequence

Returns:

iterator

public java.util.Iterator<Sequence<VocabWord>> iterator()

Specified by:

iterator in interface java.lang.Iterable<Sequence<VocabWord>>

reset

public void reset()

Specified by:

reset in interface SequenceTransformer<VocabWord,java.lang.String>