org.deeplearning4j.spark.iterator

Class PortableDataStreamMultiDataSetIterator

java.lang.Object

org.deeplearning4j.spark.iterator.PortableDataStreamMultiDataSetIterator

All Implemented Interfaces:

java.io.Serializable, java.util.Iterator<org.nd4j.linalg.dataset.api.MultiDataSet>, org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator

public class PortableDataStreamMultiDataSetIterator
extends java.lang.Object
implements org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator

A DataSetIterator that loads serialized MultiDataSet objects (saved with MultiDataSet.save(OutputStream)) from a PortableDataStream, usually obtained from SparkContext.binaryFiles()

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
PortableDataStreamMultiDataSetIterator(java.util.Collection<org.apache.spark.input.PortableDataStream> dataSetStreams)
PortableDataStreamMultiDataSetIterator(java.util.Iterator<org.apache.spark.input.PortableDataStream> iter)

Method Summary

Modifier and Type	Method and Description
boolean	asyncSupported()
boolean	hasNext()
org.nd4j.linalg.dataset.api.MultiDataSet	next()
org.nd4j.linalg.dataset.api.MultiDataSet	next(int num)
void	remove()
void	reset()
boolean	resetSupported()
void	setPreProcessor(org.nd4j.linalg.dataset.api.MultiDataSetPreProcessor preProcessor)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Iterator

forEachRemaining

Constructor Detail

PortableDataStreamMultiDataSetIterator

public PortableDataStreamMultiDataSetIterator(java.util.Iterator<org.apache.spark.input.PortableDataStream> iter)

PortableDataStreamMultiDataSetIterator

public PortableDataStreamMultiDataSetIterator(java.util.Collection<org.apache.spark.input.PortableDataStream> dataSetStreams)

Method Detail

next

public org.nd4j.linalg.dataset.api.MultiDataSet next(int num)

Specified by:

next in interface org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator

resetSupported

public boolean resetSupported()

Specified by:

resetSupported in interface org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator

asyncSupported

public boolean asyncSupported()

Specified by:

asyncSupported in interface org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator

reset

public void reset()

Specified by:

reset in interface org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator

setPreProcessor

public void setPreProcessor(org.nd4j.linalg.dataset.api.MultiDataSetPreProcessor preProcessor)

Specified by:

setPreProcessor in interface org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator

hasNext

public boolean hasNext()

Specified by:

hasNext in interface java.util.Iterator<org.nd4j.linalg.dataset.api.MultiDataSet>

next

public org.nd4j.linalg.dataset.api.MultiDataSet next()

Specified by:

next in interface java.util.Iterator<org.nd4j.linalg.dataset.api.MultiDataSet>

remove

public void remove()

Specified by:

remove in interface java.util.Iterator<org.nd4j.linalg.dataset.api.MultiDataSet>