org.deeplearning4j.spark.data

Class BatchDataSetsFunction

java.lang.Object

org.datavec.spark.transform.BaseFlatMapFunctionAdaptee<java.util.Iterator<org.nd4j.linalg.dataset.DataSet>,org.nd4j.linalg.dataset.DataSet>

org.deeplearning4j.spark.data.BatchDataSetsFunction

All Implemented Interfaces:

java.io.Serializable, org.apache.spark.api.java.function.FlatMapFunction<java.util.Iterator<org.nd4j.linalg.dataset.DataSet>,org.nd4j.linalg.dataset.DataSet>

public class BatchDataSetsFunction
extends org.datavec.spark.transform.BaseFlatMapFunctionAdaptee<java.util.Iterator<org.nd4j.linalg.dataset.DataSet>,org.nd4j.linalg.dataset.DataSet>

Function used to batch DataSet objects together. Typically used to combine singe-example DataSet objects out of something like DataVecDataSetFunction together into minibatches.
Usage:

 
      RDD<DataSet> mySingleExampleDataSets = ...;
      RDD<DataSet> batchData = mySingleExampleDataSets.mapPartitions(new BatchDataSetsFunction(batchSize));
 
 

See Also:

Serialized Form

Field Summary

Fields inherited from class org.datavec.spark.transform.BaseFlatMapFunctionAdaptee

adapter

Constructor Summary

Constructors

Constructor and Description
BatchDataSetsFunction(int minibatchSize)

Method Summary

Methods inherited from class org.datavec.spark.transform.BaseFlatMapFunctionAdaptee

call

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BatchDataSetsFunction

public BatchDataSetsFunction(int minibatchSize)