Interface | Description |
---|---|
DataSetProvider |
A provider for an
DataSet
rdd. |
Class | Description |
---|---|
BatchAndExportDataSetsFunction |
Function used with
RDD<DataSet>.mapPartitionsWithIndex . |
BatchAndExportMultiDataSetsFunction |
Function used with
RDD<MultiDataSet>.mapPartitionsWithIndex . |
BatchDataSetsFunction |
Function used to batch DataSet objects together.
|
DataSetExportFunction |
A function (used in forEachPartition) to save DataSet objects to disk/HDFS.
|
MultiDataSetExportFunction |
A function (used in forEachPartition) to save MultiDataSet objects to disk/HDFS.
|
PathToDataSetFunction |
Simple function used to load DataSets (serialized with DataSet.save()) from a given Path (as a String)
to a DataSet object - i.e.,
RDD<String> to RDD<DataSet> |
PathToMultiDataSetFunction |
Simple function used to load MultiDataSets (serialized with MultiDataSet.save()) from a given Path (as a String)
to a MultiDataSet object - i.e.,
RDD<String> to RDD<MultiDataSet> |
SplitDataSetsFunction |
Take an existing DataSet object, and split it into multiple DataSet objects with one example in each
Usage:
|
TestShuffleExamples |
Created by Alex on 06/01/2017.
|