org.deeplearning4j.spark.iterator

Class PathSparkMultiDataSetIterator

java.lang.Object

org.deeplearning4j.spark.iterator.PathSparkMultiDataSetIterator

All Implemented Interfaces:

java.io.Serializable, java.util.Iterator<org.nd4j.linalg.dataset.api.MultiDataSet>, org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator

public class PathSparkMultiDataSetIterator
extends java.lang.Object
implements org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator

A DataSetIterator that loads serialized DataSet objects (saved with MultiDataSet.save(OutputStream)) from a String that represents the path (for example, on HDFS)

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static int	BUFFER_SIZE

Constructor Summary

Constructors

Constructor and Description
PathSparkMultiDataSetIterator(java.util.Collection<java.lang.String> dataSetStreams)
PathSparkMultiDataSetIterator(java.util.Iterator<java.lang.String> iter)

Method Summary

Modifier and Type	Method and Description
boolean	asyncSupported()
boolean	hasNext()
org.nd4j.linalg.dataset.api.MultiDataSet	next()
org.nd4j.linalg.dataset.api.MultiDataSet	next(int num)
void	remove()
void	reset()
boolean	resetSupported()
void	setPreProcessor(org.nd4j.linalg.dataset.api.MultiDataSetPreProcessor preProcessor)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Iterator

forEachRemaining

Field Detail

BUFFER_SIZE

public static final int BUFFER_SIZE

See Also:

Constant Field Values

Constructor Detail

PathSparkMultiDataSetIterator

public PathSparkMultiDataSetIterator(java.util.Iterator<java.lang.String> iter)

PathSparkMultiDataSetIterator

public PathSparkMultiDataSetIterator(java.util.Collection<java.lang.String> dataSetStreams)

Method Detail

next

public org.nd4j.linalg.dataset.api.MultiDataSet next(int num)

Specified by:

next in interface org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator

resetSupported

public boolean resetSupported()

Specified by:

resetSupported in interface org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator

asyncSupported

public boolean asyncSupported()

Specified by:

asyncSupported in interface org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator

reset

public void reset()

Specified by:

reset in interface org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator

setPreProcessor

public void setPreProcessor(org.nd4j.linalg.dataset.api.MultiDataSetPreProcessor preProcessor)

Specified by:

setPreProcessor in interface org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator

hasNext

public boolean hasNext()

Specified by:

hasNext in interface java.util.Iterator<org.nd4j.linalg.dataset.api.MultiDataSet>

next

public org.nd4j.linalg.dataset.api.MultiDataSet next()

Specified by:

next in interface java.util.Iterator<org.nd4j.linalg.dataset.api.MultiDataSet>

remove

public void remove()

Specified by:

remove in interface java.util.Iterator<org.nd4j.linalg.dataset.api.MultiDataSet>