org.deeplearning4j.datasets.iterator

Class AbstractDataSetIterator<T>

java.lang.Object

org.deeplearning4j.datasets.iterator.AbstractDataSetIterator<T>

All Implemented Interfaces:

java.io.Serializable, java.util.Iterator<org.nd4j.linalg.dataset.DataSet>, org.nd4j.linalg.dataset.api.iterator.DataSetIterator

Direct Known Subclasses:

DoublesDataSetIterator, FloatsDataSetIterator, INDArrayDataSetIterator

public abstract class AbstractDataSetIterator<T>
extends java.lang.Object
implements org.nd4j.linalg.dataset.api.iterator.DataSetIterator

This is simple DataSetIterator implementation, that builds DataSetIterator out of INDArray/float[]/double[] pairs. Suitable for model feeding with externally originated data. PLEASE NOTE: If total number of input elements % batchSize != 0, reminder will be ignored

See Also:

Serialized Form

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	AbstractDataSetIterator(java.lang.Iterable<Pair<T,T>> iterable, int batchSize)

Method Summary

Modifier and Type	Method and Description
boolean	asyncSupported()
int	batch() Batch size
int	cursor() The current cursor if applicable
protected void	fillQueue()
java.util.List<java.lang.String>	getLabels() Get dataset iterator record reader labels
org.nd4j.linalg.dataset.api.DataSetPreProcessor	getPreProcessor()
boolean	hasNext() Returns true if the iteration has more elements.
int	inputColumns() Input columns for the dataset
org.nd4j.linalg.dataset.DataSet	next() Returns the next element in the iteration.
org.nd4j.linalg.dataset.DataSet	next(int num) Like the standard next method but allows a customizable number of examples returned
int	numExamples() Total number of examples in the dataset
void	remove() Removes from the underlying collection the last element returned by this iterator (optional operation).
void	reset() Resets the iterator back to the beginning
boolean	resetSupported()
void	setPreProcessor(org.nd4j.linalg.dataset.api.DataSetPreProcessor preProcessor) Set a pre processor
int	totalExamples() Total examples in the iterator
int	totalOutcomes() The number of labels for the dataset

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Iterator

forEachRemaining

Constructor Detail

AbstractDataSetIterator

protected AbstractDataSetIterator(@NonNull
                                  java.lang.Iterable<Pair<T,T>> iterable,
                                  int batchSize)

Method Detail

next

public org.nd4j.linalg.dataset.DataSet next(int num)

Like the standard next method but allows a customizable number of examples returned

Specified by:

next in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

Parameters:

num - the number of examples

Returns:

the next data applyTransformToDestination

totalExamples

public int totalExamples()

Total examples in the iterator

Specified by:

totalExamples in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

Returns:

inputColumns

public int inputColumns()

Input columns for the dataset

Specified by:

inputColumns in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

Returns:

totalOutcomes

public int totalOutcomes()

The number of labels for the dataset

Specified by:

totalOutcomes in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

Returns:

resetSupported

public boolean resetSupported()

Specified by:

resetSupported in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

asyncSupported

public boolean asyncSupported()

Specified by:

asyncSupported in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

reset

public void reset()

Resets the iterator back to the beginning

Specified by:

reset in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

batch

public int batch()

Batch size

Specified by:

batch in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

Returns:

cursor

public int cursor()

The current cursor if applicable

Specified by:

cursor in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

Returns:

numExamples

public int numExamples()

Total number of examples in the dataset

Specified by:

numExamples in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

Returns:

setPreProcessor

public void setPreProcessor(org.nd4j.linalg.dataset.api.DataSetPreProcessor preProcessor)

Set a pre processor

Specified by:

setPreProcessor in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

Parameters:

preProcessor - a pre processor to set

getLabels

public java.util.List<java.lang.String> getLabels()

Get dataset iterator record reader labels

Specified by:

getLabels in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator

hasNext

public boolean hasNext()

Returns true if the iteration has more elements. (In other words, returns true if next(int) would return an element rather than throwing an exception.)

Specified by:

hasNext in interface java.util.Iterator<org.nd4j.linalg.dataset.DataSet>

Returns:

true if the iteration has more elements

fillQueue

protected void fillQueue()

next

public org.nd4j.linalg.dataset.DataSet next()
                                     throws java.util.NoSuchElementException

Returns the next element in the iteration.

Specified by:

next in interface java.util.Iterator<org.nd4j.linalg.dataset.DataSet>

Returns:

the next element in the iteration

Throws:

java.util.NoSuchElementException - if the iteration has no more elements

remove

public void remove()

Removes from the underlying collection the last element returned by this iterator (optional operation). This method can be called only once per call to next(int). The behavior of an iterator is unspecified if the underlying collection is modified while the iteration is in progress in any way other than by calling this method.

Specified by:

remove in interface java.util.Iterator<org.nd4j.linalg.dataset.DataSet>

Throws:

java.lang.UnsupportedOperationException - if the remove operation is not supported by this iterator

java.lang.IllegalStateException - if the next method has not yet been called, or the remove method has already been called after the last call to the next method

getPreProcessor

public org.nd4j.linalg.dataset.api.DataSetPreProcessor getPreProcessor()

Specified by:

getPreProcessor in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator