org.deeplearning4j.spark.data.shuffle

Class SplitDataSetExamplesPairFlatMapFunction

java.lang.Object

org.deeplearning4j.spark.util.BasePairFlatMapFunctionAdaptee<org.nd4j.linalg.dataset.DataSet,java.lang.Integer,org.nd4j.linalg.dataset.DataSet>

org.deeplearning4j.spark.data.shuffle.SplitDataSetExamplesPairFlatMapFunction

All Implemented Interfaces:

java.io.Serializable, org.apache.spark.api.java.function.PairFlatMapFunction<org.nd4j.linalg.dataset.DataSet,java.lang.Integer,org.nd4j.linalg.dataset.DataSet>

public class SplitDataSetExamplesPairFlatMapFunction
extends BasePairFlatMapFunctionAdaptee<org.nd4j.linalg.dataset.DataSet,java.lang.Integer,org.nd4j.linalg.dataset.DataSet>

A PairFlatMapFunction that splits each example in a DataSet object into its own DataSet. Also adds a random key (integer value) in the range 0 to maxKeyIndex-1.
Used in SparkUtils.shuffleExamples(JavaRDD, int, int)

See Also:

Serialized Form

Field Summary

Fields inherited from class org.deeplearning4j.spark.util.BasePairFlatMapFunctionAdaptee

adapter

Constructor Summary

Constructors

Constructor and Description
SplitDataSetExamplesPairFlatMapFunction(int maxKeyIndex)

Method Summary

Methods inherited from class org.deeplearning4j.spark.util.BasePairFlatMapFunctionAdaptee

call

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SplitDataSetExamplesPairFlatMapFunction

public SplitDataSetExamplesPairFlatMapFunction(int maxKeyIndex)