org.deeplearning4j.spark.data

Class DataSetExportFunction

java.lang.Object

org.deeplearning4j.spark.data.DataSetExportFunction

All Implemented Interfaces:

java.io.Serializable, org.apache.spark.api.java.function.VoidFunction<java.util.Iterator<org.nd4j.linalg.dataset.DataSet>>

public class DataSetExportFunction
extends java.lang.Object
implements org.apache.spark.api.java.function.VoidFunction<java.util.Iterator<org.nd4j.linalg.dataset.DataSet>>

A function (used in forEachPartition) to save DataSet objects to disk/HDFS. Each DataSet object is given a random and (probably) unique name, starting with "dataset_" and ending with ".bin".
Use with JavaRDD<DataSet>.foreachPartition()

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
DataSetExportFunction(java.net.URI outputDir)

Method Summary

Modifier and Type	Method and Description
void	call(java.util.Iterator<org.nd4j.linalg.dataset.DataSet> iter)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DataSetExportFunction

public DataSetExportFunction(java.net.URI outputDir)

Method Detail

call

public void call(java.util.Iterator<org.nd4j.linalg.dataset.DataSet> iter)
          throws java.lang.Exception

Specified by:

call in interface org.apache.spark.api.java.function.VoidFunction<java.util.Iterator<org.nd4j.linalg.dataset.DataSet>>

Throws:

java.lang.Exception