org.deeplearning4j.spark.datavec

Class DataVecSequenceDataSetFunction

java.lang.Object

org.deeplearning4j.spark.datavec.DataVecSequenceDataSetFunction

All Implemented Interfaces:

java.io.Serializable, org.apache.spark.api.java.function.Function<java.util.List<java.util.List<org.datavec.api.writable.Writable>>,org.nd4j.linalg.dataset.DataSet>

public class DataVecSequenceDataSetFunction
extends java.lang.Object
implements org.apache.spark.api.java.function.Function<java.util.List<java.util.List<org.datavec.api.writable.Writable>>,org.nd4j.linalg.dataset.DataSet>, java.io.Serializable

Map Collection<Collection<Writable>> objects (out of a datavec-spark sequence record reader function) to DataSet objects for Spark training. Analogous to SequenceRecordReaderDataSetIterator, but in the context of Spark. Supports loading data from a single source only (hence no masknig arrays, many-to-one etc here) see DataVecSequencePairDataSetFunction for the separate collections for input and labels version

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
DataVecSequenceDataSetFunction(int labelIndex, int numPossibleLabels, boolean regression)
DataVecSequenceDataSetFunction(int labelIndex, int numPossibleLabels, boolean regression, org.nd4j.linalg.dataset.api.DataSetPreProcessor preProcessor, org.datavec.api.io.WritableConverter converter)

Method Summary

Modifier and Type	Method and Description
org.nd4j.linalg.dataset.DataSet	call(java.util.List<java.util.List<org.datavec.api.writable.Writable>> input)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DataVecSequenceDataSetFunction

public DataVecSequenceDataSetFunction(int labelIndex,
                                      int numPossibleLabels,
                                      boolean regression)

Parameters:

labelIndex - Index of the label column

numPossibleLabels - Number of classes for classification (not used if regression = true)

regression - False for classification, true for regression

DataVecSequenceDataSetFunction

public DataVecSequenceDataSetFunction(int labelIndex,
                                      int numPossibleLabels,
                                      boolean regression,
                                      org.nd4j.linalg.dataset.api.DataSetPreProcessor preProcessor,
                                      org.datavec.api.io.WritableConverter converter)

Parameters:

labelIndex - Index of the label column

numPossibleLabels - Number of classes for classification (not used if regression = true)

regression - False for classification, true for regression

preProcessor - DataSetPreprocessor (may be null)

converter - WritableConverter (may be null)

Method Detail

call

public org.nd4j.linalg.dataset.DataSet call(java.util.List<java.util.List<org.datavec.api.writable.Writable>> input)
                                     throws java.lang.Exception

Specified by:

call in interface org.apache.spark.api.java.function.Function<java.util.List<java.util.List<org.datavec.api.writable.Writable>>,org.nd4j.linalg.dataset.DataSet>

Throws:

java.lang.Exception