org.deeplearning4j.spark.data

Class SplitDataSetsFunction

java.lang.Object

org.datavec.spark.transform.BaseFlatMapFunctionAdaptee<java.util.Iterator<org.nd4j.linalg.dataset.DataSet>,org.nd4j.linalg.dataset.DataSet>

org.deeplearning4j.spark.data.SplitDataSetsFunction

All Implemented Interfaces:

java.io.Serializable, org.apache.spark.api.java.function.FlatMapFunction<java.util.Iterator<org.nd4j.linalg.dataset.DataSet>,org.nd4j.linalg.dataset.DataSet>

public class SplitDataSetsFunction
extends org.datavec.spark.transform.BaseFlatMapFunctionAdaptee<java.util.Iterator<org.nd4j.linalg.dataset.DataSet>,org.nd4j.linalg.dataset.DataSet>

Take an existing DataSet object, and split it into multiple DataSet objects with one example in each Usage:

 
      RDD<DataSet> myBatchedExampleDataSets = ...;
      RDD<DataSet> singleExamlpeDataSets = myBatchedExampleDataSets.mapPartitions(new SplitDataSets(batchSize));
 
 

See Also:

Serialized Form

Field Summary

Fields inherited from class org.datavec.spark.transform.BaseFlatMapFunctionAdaptee

adapter

Constructor Summary

Constructors

Constructor and Description
SplitDataSetsFunction()

Method Summary

Methods inherited from class org.datavec.spark.transform.BaseFlatMapFunctionAdaptee

call

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SplitDataSetsFunction

public SplitDataSetsFunction()