org.deeplearning4j.spark.impl.layer

Class SparkDl4jLayer

java.lang.Object

org.deeplearning4j.spark.impl.layer.SparkDl4jLayer

All Implemented Interfaces:

java.io.Serializable

public class SparkDl4jLayer
extends java.lang.Object
implements java.io.Serializable

Master class for org.deeplearning4j.spark layers

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
SparkDl4jLayer(org.apache.spark.api.java.JavaSparkContext sc, NeuralNetConfiguration conf)
SparkDl4jLayer(org.apache.spark.SparkContext sparkContext, NeuralNetConfiguration conf)

Method Summary

Modifier and Type	Method and Description
Layer	fit(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaRDD<org.apache.spark.mllib.regression.LabeledPoint> rdd) Fit the given rdd given the context.
Layer	fit(java.lang.String path, int labelIndex, org.datavec.api.records.reader.RecordReader recordReader) Fit the layer based on the specified org.deeplearning4j.spark context text file
Layer	fitDataSet(org.apache.spark.api.java.JavaRDD<org.nd4j.linalg.dataset.DataSet> rdd) Fit a java rdd of dataset
org.apache.spark.mllib.linalg.Matrix	predict(org.apache.spark.mllib.linalg.Matrix features) Predict the given feature matrix
org.apache.spark.mllib.linalg.Vector	predict(org.apache.spark.mllib.linalg.Vector point) Predict the given vector
static Layer	train(org.apache.spark.api.java.JavaRDD<org.apache.spark.mllib.regression.LabeledPoint> data, NeuralNetConfiguration conf) Train a multi layer network

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SparkDl4jLayer

public SparkDl4jLayer(org.apache.spark.SparkContext sparkContext,
                      NeuralNetConfiguration conf)

SparkDl4jLayer

public SparkDl4jLayer(org.apache.spark.api.java.JavaSparkContext sc,
                      NeuralNetConfiguration conf)

Method Detail

fit

public Layer fit(java.lang.String path,
                 int labelIndex,
                 org.datavec.api.records.reader.RecordReader recordReader)

Fit the layer based on the specified org.deeplearning4j.spark context text file

Parameters:

path - the path to the text file

labelIndex - the index of the label

recordReader - the record reader

Returns:

the fit layer

fit

public Layer fit(org.apache.spark.api.java.JavaSparkContext sc,
                 org.apache.spark.api.java.JavaRDD<org.apache.spark.mllib.regression.LabeledPoint> rdd)

Fit the given rdd given the context. This will convert the labeled points to the internal dl4j format and train the model on that

Parameters:

sc - the org.deeplearning4j.spark context

rdd - the rdd to fitDataSet

Returns:

the multi layer network that was fitDataSet

fitDataSet

public Layer fitDataSet(org.apache.spark.api.java.JavaRDD<org.nd4j.linalg.dataset.DataSet> rdd)

Fit a java rdd of dataset

Parameters:

rdd - the rdd to fit

Returns:

the fit layer

predict

public org.apache.spark.mllib.linalg.Matrix predict(org.apache.spark.mllib.linalg.Matrix features)

Predict the given feature matrix

Parameters:

features - the given feature matrix

Returns:

the predictions

predict

public org.apache.spark.mllib.linalg.Vector predict(org.apache.spark.mllib.linalg.Vector point)

Predict the given vector

Parameters:

point - the vector to predict

Returns:

the predicted vector

train

public static Layer train(org.apache.spark.api.java.JavaRDD<org.apache.spark.mllib.regression.LabeledPoint> data,
                          NeuralNetConfiguration conf)

Train a multi layer network

Parameters:

data - the data to train on

conf - the configuration of the network

Returns:

the fit multi layer network