public class MultiLayerNetwork extends java.lang.Object implements java.io.Serializable, Classifier, Layer
ComputationGraph
which allows for an arbitrary directed acyclic graph connection structure between layers.
MultiLayerNetwork is trainable via backprop, with optional pretraining, depending on the type of layers it contains.Layer.TrainingMode, Layer.Type| Modifier and Type | Field and Description |
|---|---|
protected NeuralNetConfiguration |
defaultConfiguration |
protected org.nd4j.linalg.api.ndarray.INDArray |
epsilon |
protected org.nd4j.linalg.api.ndarray.INDArray |
flattenedGradients |
protected org.nd4j.linalg.api.ndarray.INDArray |
flattenedParams |
protected Gradient |
gradient |
protected boolean |
initCalled |
protected boolean |
initDone |
protected org.nd4j.linalg.api.ndarray.INDArray |
input |
protected org.nd4j.linalg.api.ndarray.INDArray |
labels |
protected int |
layerIndex |
protected java.util.LinkedHashMap<java.lang.String,Layer> |
layerMap |
protected Layer[] |
layers |
protected MultiLayerConfiguration |
layerWiseConfigurations |
protected org.nd4j.linalg.api.ndarray.INDArray |
mask |
protected double |
score |
protected Solver |
solver |
| Constructor and Description |
|---|
MultiLayerNetwork(MultiLayerConfiguration conf) |
MultiLayerNetwork(MultiLayerConfiguration conf,
org.nd4j.linalg.api.ndarray.INDArray params)
Initialize the network based on the configuraiton
|
MultiLayerNetwork(java.lang.String conf,
org.nd4j.linalg.api.ndarray.INDArray params)
Initialize the network based on the configuration
|
| Modifier and Type | Method and Description |
|---|---|
void |
accumulateScore(double accum)
Sets a rolling tally for the score.
|
org.nd4j.linalg.api.ndarray.INDArray |
activate()
Triggers the activation of the last hidden layer ie: not logistic regression
|
org.nd4j.linalg.api.ndarray.INDArray |
activate(boolean training)
Trigger an activation with the last specified input
|
org.nd4j.linalg.api.ndarray.INDArray |
activate(org.nd4j.linalg.api.ndarray.INDArray input)
Initialize the layer with the given input
and return the activation for this layer
given this input
|
org.nd4j.linalg.api.ndarray.INDArray |
activate(org.nd4j.linalg.api.ndarray.INDArray input,
boolean training)
Initialize the layer with the given input
and return the activation for this layer
given this input
|
org.nd4j.linalg.api.ndarray.INDArray |
activate(org.nd4j.linalg.api.ndarray.INDArray input,
Layer.TrainingMode training)
Initialize the layer with the given input
and return the activation for this layer
given this input
|
org.nd4j.linalg.api.ndarray.INDArray |
activate(int layer)
Triggers the activation for a given layer
|
org.nd4j.linalg.api.ndarray.INDArray |
activate(int layer,
org.nd4j.linalg.api.ndarray.INDArray input)
Triggers the activation of the given layer
|
org.nd4j.linalg.api.ndarray.INDArray |
activate(Layer.TrainingMode training)
Trigger an activation with the last specified input
|
org.nd4j.linalg.api.ndarray.INDArray |
activateSelectedLayers(int from,
int to,
org.nd4j.linalg.api.ndarray.INDArray input)
Calculate activation for few layers at once.
|
org.nd4j.linalg.api.ndarray.INDArray |
activationFromPrevLayer(int curr,
org.nd4j.linalg.api.ndarray.INDArray input,
boolean training)
Calculate activation from previous layer including pre processing where necessary
|
org.nd4j.linalg.api.ndarray.INDArray |
activationMean()
Calculate the mean representation
for the activation for this layer
|
void |
applyLearningRateScoreDecay()
Update learningRate using for this model.
|
protected void |
backprop()
Calculate and set gradients for MultiLayerNetwork, based on OutputLayer and labels
|
Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray> |
backpropGradient(org.nd4j.linalg.api.ndarray.INDArray epsilon)
Calculate the gradient relative to the error in the next layer
|
int |
batchSize()
The current inputs batch size
|
protected Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray> |
calcBackpropGradients(org.nd4j.linalg.api.ndarray.INDArray epsilon,
boolean withOutputLayer)
Calculate gradients and errors.
|
Gradient |
calcGradient(Gradient layerError,
org.nd4j.linalg.api.ndarray.INDArray activation)
Calculate the gradient
|
double |
calcL1(boolean backpropParamsOnly)
Calculate the l1 regularization term
0.0 if regularization is not used. |
double |
calcL2(boolean backpropParamsOnly)
Calculate the l2 regularization term
0.0 if regularization is not used. |
void |
clear()
Clear the inputs.
|
void |
clearLayerMaskArrays()
Remove the mask arrays from all layers.
See setLayerMaskArrays(INDArray, INDArray) for details on mask arrays. |
MultiLayerNetwork |
clone()
Clone the layer
|
void |
computeGradientAndScore()
Update the score
|
java.util.List<org.nd4j.linalg.api.ndarray.INDArray> |
computeZ(boolean training)
* Compute input linear transformation (z) of the output layer
|
java.util.List<org.nd4j.linalg.api.ndarray.INDArray> |
computeZ(org.nd4j.linalg.api.ndarray.INDArray input,
boolean training)
Compute activations from input to output of the output layer
|
NeuralNetConfiguration |
conf()
The configuration for the neural network
|
org.nd4j.linalg.api.ndarray.INDArray |
derivativeActivation(org.nd4j.linalg.api.ndarray.INDArray input)
Take the derivative of the given input
based on the activation
|
<T extends IEvaluation> |
doEvaluation(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator,
T evaluation)
Perform evaluation using an arbitrary IEvaluation instance.
|
protected void |
doTruncatedBPTT(org.nd4j.linalg.api.ndarray.INDArray input,
org.nd4j.linalg.api.ndarray.INDArray labels,
org.nd4j.linalg.api.ndarray.INDArray featuresMaskArray,
org.nd4j.linalg.api.ndarray.INDArray labelsMaskArray) |
org.nd4j.linalg.api.ndarray.INDArray |
epsilon() |
Gradient |
error(org.nd4j.linalg.api.ndarray.INDArray errorSignal)
Calculate error with respect to the
current layer.
|
Evaluation |
evaluate(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator)
Evaluate the network (classification performance)
|
Evaluation |
evaluate(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator,
java.util.List<java.lang.String> labelsList)
Evaluate the network on the provided data set.
|
Evaluation |
evaluate(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator,
java.util.List<java.lang.String> labelsList,
int topN)
Evaluate the network (for classification) on the provided data set, with top N accuracy in addition to standard accuracy.
|
RegressionEvaluation |
evaluateRegression(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator)
Evaluate the network for regression performance
|
ROC |
evaluateROC(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator,
int rocThresholdSteps)
Evaluate the network (must be a binary classifier) on the specified data, using the
ROC class |
ROCMultiClass |
evaluateROCMultiClass(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator,
int rocThresholdSteps)
Evaluate the network on the specified data, using the
ROCMultiClass class |
double |
f1Score(org.nd4j.linalg.dataset.api.DataSet data)
Sets the input and labels and returns a score for the prediction
wrt true labels
|
double |
f1Score(org.nd4j.linalg.api.ndarray.INDArray input,
org.nd4j.linalg.api.ndarray.INDArray labels)
Sets the input and labels and returns a score for the prediction
wrt true labels
|
java.util.List<org.nd4j.linalg.api.ndarray.INDArray> |
feedForward()
Compute activations from input to output of the output layer
|
java.util.List<org.nd4j.linalg.api.ndarray.INDArray> |
feedForward(boolean train)
Compute activations from input to output of the output layer
|
java.util.List<org.nd4j.linalg.api.ndarray.INDArray> |
feedForward(org.nd4j.linalg.api.ndarray.INDArray input)
Compute activations from input to output of the output layer
|
java.util.List<org.nd4j.linalg.api.ndarray.INDArray> |
feedForward(org.nd4j.linalg.api.ndarray.INDArray input,
boolean train)
Compute activations from input to output of the output layer
|
java.util.List<org.nd4j.linalg.api.ndarray.INDArray> |
feedForward(org.nd4j.linalg.api.ndarray.INDArray input,
org.nd4j.linalg.api.ndarray.INDArray featuresMask,
org.nd4j.linalg.api.ndarray.INDArray labelsMask)
Compute the activations from the input to the output layer, given mask arrays (that may be null)
The masking arrays are used in situations such an one-to-many and many-to-one rucerrent neural network (RNN)
designs, as well as for supporting time series of varying lengths within the same minibatch for RNNs.
|
Pair<org.nd4j.linalg.api.ndarray.INDArray,MaskState> |
feedForwardMaskArray(org.nd4j.linalg.api.ndarray.INDArray maskArray,
MaskState currentMaskState,
int minibatchSize)
Feed forward the input mask array, setting in in the layer as appropriate.
|
java.util.List<org.nd4j.linalg.api.ndarray.INDArray> |
feedForwardToLayer(int layerNum,
boolean train)
Compute the activations from the input to the specified layer, using the currently set input for the network.
To compute activations for all layers, use feedForward(...) methods Note: output list includes the original input. |
java.util.List<org.nd4j.linalg.api.ndarray.INDArray> |
feedForwardToLayer(int layerNum,
org.nd4j.linalg.api.ndarray.INDArray input)
Compute the activations from the input to the specified layer.
To compute activations for all layers, use feedForward(...) methods Note: output list includes the original input. |
java.util.List<org.nd4j.linalg.api.ndarray.INDArray> |
feedForwardToLayer(int layerNum,
org.nd4j.linalg.api.ndarray.INDArray input,
boolean train)
Compute the activations from the input to the specified layer.
To compute activations for all layers, use feedForward(...) methods Note: output list includes the original input. |
void |
finetune()
Run SGD based on the given labels
|
void |
fit()
All models have a fit method
|
void |
fit(org.nd4j.linalg.dataset.api.DataSet data)
Fit the model
|
void |
fit(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator)
Train the model based on the datasetiterator
|
void |
fit(org.nd4j.linalg.api.ndarray.INDArray data)
Fit the unsupervised model
|
void |
fit(org.nd4j.linalg.api.ndarray.INDArray data,
org.nd4j.linalg.api.ndarray.INDArray labels)
Fit the model
|
void |
fit(org.nd4j.linalg.api.ndarray.INDArray features,
org.nd4j.linalg.api.ndarray.INDArray labels,
org.nd4j.linalg.api.ndarray.INDArray featuresMask,
org.nd4j.linalg.api.ndarray.INDArray labelsMask)
Fit the model
|
void |
fit(org.nd4j.linalg.api.ndarray.INDArray examples,
int[] labels)
Fit the model
|
NeuralNetConfiguration |
getDefaultConfiguration() |
int |
getIndex()
Get the layer index.
|
org.nd4j.linalg.api.ndarray.INDArray |
getInput() |
int |
getInputMiniBatchSize()
Get current/last input mini-batch size, as set by setInputMiniBatchSize(int)
|
org.nd4j.linalg.api.ndarray.INDArray |
getLabels() |
Layer |
getLayer(int i) |
Layer |
getLayer(java.lang.String name) |
java.util.List<java.lang.String> |
getLayerNames() |
Layer[] |
getLayers() |
MultiLayerConfiguration |
getLayerWiseConfigurations() |
java.util.Collection<IterationListener> |
getListeners()
Get the iteration listeners for this layer.
|
org.nd4j.linalg.api.ndarray.INDArray |
getMask() |
org.nd4j.linalg.api.ndarray.INDArray |
getMaskArray() |
int |
getnLayers()
Get the number of layers in the network
|
ConvexOptimizer |
getOptimizer()
Returns this models optimizer
|
Layer |
getOutputLayer()
Get the output layer
|
org.nd4j.linalg.api.ndarray.INDArray |
getParam(java.lang.String param)
Get the parameter
|
Updater |
getUpdater()
Get the updater for this MultiLayerNetwork
|
Gradient |
gradient()
Calculate a gradient
|
Pair<Gradient,java.lang.Double> |
gradientAndScore()
Get the gradient and score
|
void |
init()
Initialize the MultiLayerNetwork.
|
void |
init(org.nd4j.linalg.api.ndarray.INDArray parameters,
boolean cloneParametersArray)
Initialize the MultiLayerNetwork, optionally with an existing parameters array.
|
void |
initGradientsView()
This method: initializes the flattened gradients array (used in backprop) and sets the appropriate subset in all layers.
|
void |
initialize(org.nd4j.linalg.dataset.DataSet data)
Sets the input and labels from this dataset
|
void |
initializeLayers(org.nd4j.linalg.api.ndarray.INDArray input)
Base class for initializing the neuralNets based on the input.
|
void |
initParams()
Initialize the parameters
|
org.nd4j.linalg.api.ndarray.INDArray |
input()
The input/feature matrix for the model
|
protected void |
intializeConfigurations() |
boolean |
isInitCalled() |
boolean |
isPretrainLayer()
Returns true if the layer can be trained in an unsupervised/pretrain manner (VAE, RBMs etc)
|
void |
iterate(org.nd4j.linalg.api.ndarray.INDArray input)
Run one iteration
|
org.nd4j.linalg.api.ndarray.INDArray |
labelProbabilities(org.nd4j.linalg.api.ndarray.INDArray examples)
Returns the probabilities for each label
for each example row wise
|
void |
merge(Layer layer,
int batchSize)
Deprecated.
Not supported and not used
|
void |
merge(MultiLayerNetwork network,
int batchSize)
Deprecated.
As of 0.7.3 - Feb 2017. No longer used; parameter averaging is performed via alternative means/methods
|
int |
numLabels()
Returns the number of possible labels
|
int |
numParams()
Returns a 1 x m vector where the vector is composed of
a flattened vector of all of the weights for the
various neuralNets and output layer
|
int |
numParams(boolean backwards)
the number of parameters for the model
|
org.nd4j.linalg.api.ndarray.INDArray |
output(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator) |
org.nd4j.linalg.api.ndarray.INDArray |
output(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator,
boolean train)
Label the probabilities of the input
|
org.nd4j.linalg.api.ndarray.INDArray |
output(org.nd4j.linalg.api.ndarray.INDArray input)
Label the probabilities of the input
|
org.nd4j.linalg.api.ndarray.INDArray |
output(org.nd4j.linalg.api.ndarray.INDArray input,
boolean train)
Label the probabilities of the input
|
org.nd4j.linalg.api.ndarray.INDArray |
output(org.nd4j.linalg.api.ndarray.INDArray input,
boolean train,
org.nd4j.linalg.api.ndarray.INDArray featuresMask,
org.nd4j.linalg.api.ndarray.INDArray labelsMask)
Calculate the output of the network, with masking arrays.
|
org.nd4j.linalg.api.ndarray.INDArray |
output(org.nd4j.linalg.api.ndarray.INDArray input,
Layer.TrainingMode train)
Label the probabilities of the input
|
org.nd4j.linalg.api.ndarray.INDArray |
params()
Returns a 1 x m vector where the vector is composed of
a flattened vector of all of the weights for the
various neuralNets(w,hbias NOT VBIAS) and output layer
|
org.nd4j.linalg.api.ndarray.INDArray |
params(boolean backwardOnly)
Returns a 1 x m vector where the vector is composed of
a flattened vector of all of the weights for the
various neuralNets(w,hbias NOT VBIAS) and output layer
|
java.util.Map<java.lang.String,org.nd4j.linalg.api.ndarray.INDArray> |
paramTable()
The param table
|
java.util.Map<java.lang.String,org.nd4j.linalg.api.ndarray.INDArray> |
paramTable(boolean backpropParamsOnly)
Table of parameters by key, for backprop
For many models (dense layers, etc) - all parameters are backprop parameters
|
java.util.List<java.lang.String> |
predict(org.nd4j.linalg.dataset.api.DataSet dataSet)
Return predicted label names
|
int[] |
predict(org.nd4j.linalg.api.ndarray.INDArray d)
Returns the predictions for each example in the dataset
|
org.nd4j.linalg.api.ndarray.INDArray |
preOutput(org.nd4j.linalg.api.ndarray.INDArray x)
Raw activations
|
org.nd4j.linalg.api.ndarray.INDArray |
preOutput(org.nd4j.linalg.api.ndarray.INDArray x,
boolean training)
Raw activations
|
org.nd4j.linalg.api.ndarray.INDArray |
preOutput(org.nd4j.linalg.api.ndarray.INDArray x,
Layer.TrainingMode training)
Raw activations
|
void |
pretrain(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iter)
Perform layerwise pretraining on all pre-trainable layers in the network (VAEs, RBMs, Autoencoders, etc)
Note that pretraining will be performed on one layer after the other, resetting the DataSetIterator between iterations. For multiple epochs per layer, appropriately wrap the iterator (for example, a MultipleEpochsIterator) or train each layer manually using pretrainLayer(int, DataSetIterator) |
void |
pretrain(org.nd4j.linalg.api.ndarray.INDArray input)
Deprecated.
use
pretrain(DataSetIterator) or pretrainLayer(int, DataSetIterator) or pretrainLayer(int, INDArray).
Pretraining each layer in a row on a single minibatch (as per this method) instead of N epochs per layer is not advisable. |
void |
pretrainLayer(int layerIdx,
org.nd4j.linalg.dataset.api.iterator.DataSetIterator iter)
Perform layerwise unsupervised training on a single pre-trainable layer in the network (VAEs, RBMs, Autoencoders, etc)
If the specified layer index (0 to numLayers - 1) is not a pretrainable layer, this is a no-op. |
void |
pretrainLayer(int layerIdx,
org.nd4j.linalg.api.ndarray.INDArray features)
Perform layerwise unsupervised training on a single pre-trainable layer in the network (VAEs, RBMs, Autoencoders, etc)
If the specified layer index (0 to numLayers - 1) is not a pretrainable layer, this is a no-op. |
void |
printConfiguration()
Prints the configuration
|
org.nd4j.linalg.api.ndarray.INDArray |
reconstruct(org.nd4j.linalg.api.ndarray.INDArray x,
int layerNum)
Reconstructs the input.
|
java.util.List<org.nd4j.linalg.api.ndarray.INDArray> |
rnnActivateUsingStoredState(org.nd4j.linalg.api.ndarray.INDArray input,
boolean training,
boolean storeLastForTBPTT)
Similar to rnnTimeStep and feedForward() methods.
|
void |
rnnClearPreviousState()
Clear the previous state of the RNN layers (if any).
|
java.util.Map<java.lang.String,org.nd4j.linalg.api.ndarray.INDArray> |
rnnGetPreviousState(int layer)
Get the state of the RNN layer, as used in rnnTimeStep().
|
void |
rnnSetPreviousState(int layer,
java.util.Map<java.lang.String,org.nd4j.linalg.api.ndarray.INDArray> state)
Set the state of the RNN layer.
|
org.nd4j.linalg.api.ndarray.INDArray |
rnnTimeStep(org.nd4j.linalg.api.ndarray.INDArray input)
If this MultiLayerNetwork contains one or more RNN layers: conduct forward pass (prediction)
but using previous stored state for any RNN layers.
|
double |
score()
Score of the model (relative to the objective function)
|
double |
score(org.nd4j.linalg.dataset.DataSet data)
Sets the input and labels and returns a score for the prediction with respect to the true labels
This is equivalent to score(DataSet, boolean) with training==true. |
double |
score(org.nd4j.linalg.dataset.DataSet data,
boolean training)
Calculate the score (loss function) of the prediction with respect to the true labels
|
org.nd4j.linalg.api.ndarray.INDArray |
scoreExamples(org.nd4j.linalg.dataset.DataSet data,
boolean addRegularizationTerms)
Calculate the score for each example in a DataSet individually.
|
org.nd4j.linalg.api.ndarray.INDArray |
scoreExamples(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iter,
boolean addRegularizationTerms) |
void |
setBackpropGradientsViewArray(org.nd4j.linalg.api.ndarray.INDArray gradients)
Set the gradients array as a view of the full (backprop) network parameters
NOTE: this is intended to be used internally in MultiLayerNetwork and ComputationGraph, not by users.
|
void |
setConf(NeuralNetConfiguration conf)
Setter for the configuration
|
void |
setIndex(int index)
Set the layer index.
|
void |
setInput(org.nd4j.linalg.api.ndarray.INDArray input)
Note that if input isn't null
and the neuralNets are null, this is a way
of initializing the neural network
|
void |
setInputMiniBatchSize(int size)
Set current/last input mini-batch size.
Used for score and gradient calculations. |
void |
setLabels(org.nd4j.linalg.api.ndarray.INDArray labels) |
void |
setLayerMaskArrays(org.nd4j.linalg.api.ndarray.INDArray featuresMaskArray,
org.nd4j.linalg.api.ndarray.INDArray labelsMaskArray)
Set the mask arrays for features and labels.
|
void |
setLayers(Layer[] layers) |
void |
setLayerWiseConfigurations(MultiLayerConfiguration layerWiseConfigurations) |
void |
setListeners(java.util.Collection<IterationListener> listeners)
Set the IterationListeners for the ComputationGraph (and all layers in the network)
|
void |
setListeners(IterationListener... listeners)
Set the IterationListeners for the ComputationGraph (and all layers in the network)
|
void |
setMask(org.nd4j.linalg.api.ndarray.INDArray mask) |
void |
setMaskArray(org.nd4j.linalg.api.ndarray.INDArray maskArray)
Set the mask array.
|
void |
setParam(java.lang.String key,
org.nd4j.linalg.api.ndarray.INDArray val)
Set the parameter with a new ndarray
|
void |
setParameters(org.nd4j.linalg.api.ndarray.INDArray params)
Sets parameters for the model.
|
void |
setParams(org.nd4j.linalg.api.ndarray.INDArray params)
Set the parameters for this model.
|
void |
setParamsViewArray(org.nd4j.linalg.api.ndarray.INDArray params)
Set the initial parameters array as a view of the full (backprop) network parameters
NOTE: this is intended to be used internally in MultiLayerNetwork and ComputationGraph, not by users.
|
void |
setParamTable(java.util.Map<java.lang.String,org.nd4j.linalg.api.ndarray.INDArray> paramTable)
Setter for the param table
|
void |
setScore(double score) |
void |
setUpdater(Updater updater)
Set the updater for the MultiLayerNetwork
|
java.lang.String |
summary()
String detailing the architecture of the multilayernetwork.
|
Layer |
transpose()
Return a transposed copy of the weights/bias
(this means reverse the number of inputs and outputs on the weights)
|
protected void |
truncatedBPTTGradient()
Equivalent to backprop(), but calculates gradient for truncated BPTT instead.
|
Layer.Type |
type()
Returns the layer type
|
void |
update(Gradient gradient)
Update layer weights and biases with gradient change
|
void |
update(org.nd4j.linalg.api.ndarray.INDArray gradient,
java.lang.String paramType)
Perform one update applying the gradient
|
void |
update(MultiLayerNetwork network)
Assigns the parameters of this model to the ones specified by this
network.
|
void |
updateRnnStateWithTBPTTState() |
void |
validateInput()
Validate the input
|
org.nd4j.linalg.api.ndarray.INDArray |
zFromPrevLayer(int curr,
org.nd4j.linalg.api.ndarray.INDArray input,
boolean training)
Compute input linear transformation (z) from previous layer
Apply pre processing transformation where necessary
|
protected Layer[] layers
protected java.util.LinkedHashMap<java.lang.String,Layer> layerMap
protected org.nd4j.linalg.api.ndarray.INDArray input
protected org.nd4j.linalg.api.ndarray.INDArray labels
protected boolean initCalled
protected NeuralNetConfiguration defaultConfiguration
protected MultiLayerConfiguration layerWiseConfigurations
protected Gradient gradient
protected org.nd4j.linalg.api.ndarray.INDArray epsilon
protected double score
protected boolean initDone
protected org.nd4j.linalg.api.ndarray.INDArray flattenedParams
protected transient org.nd4j.linalg.api.ndarray.INDArray flattenedGradients
protected org.nd4j.linalg.api.ndarray.INDArray mask
protected int layerIndex
protected transient Solver solver
public MultiLayerNetwork(MultiLayerConfiguration conf)
public MultiLayerNetwork(java.lang.String conf,
org.nd4j.linalg.api.ndarray.INDArray params)
conf - the configuration jsonparams - the parameterspublic MultiLayerNetwork(MultiLayerConfiguration conf, org.nd4j.linalg.api.ndarray.INDArray params)
conf - the configurationparams - the parametersprotected void intializeConfigurations()
public void pretrain(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iter)
pretrainLayer(int, DataSetIterator)iter - Training datapublic void pretrainLayer(int layerIdx,
org.nd4j.linalg.dataset.api.iterator.DataSetIterator iter)
layerIdx - Index of the layer to train (0 to numLayers-1)iter - Training datapublic void pretrainLayer(int layerIdx,
org.nd4j.linalg.api.ndarray.INDArray features)
layerIdx - Index of the layer to train (0 to numLayers-1)features - Training data array@Deprecated public void pretrain(org.nd4j.linalg.api.ndarray.INDArray input)
pretrain(DataSetIterator) or pretrainLayer(int, DataSetIterator) or pretrainLayer(int, INDArray).
Pretraining each layer in a row on a single minibatch (as per this method) instead of N epochs per layer is not advisable.public int batchSize()
Modelpublic NeuralNetConfiguration conf()
Modelpublic void setConf(NeuralNetConfiguration conf)
Modelpublic org.nd4j.linalg.api.ndarray.INDArray input()
Modelpublic void validateInput()
ModelvalidateInput in interface Modelpublic ConvexOptimizer getOptimizer()
ModelgetOptimizer in interface Modelpublic org.nd4j.linalg.api.ndarray.INDArray getParam(java.lang.String param)
Modelpublic void initParams()
ModelinitParams in interface Modelpublic java.util.Map<java.lang.String,org.nd4j.linalg.api.ndarray.INDArray> paramTable()
ModelparamTable in interface Modelpublic java.util.Map<java.lang.String,org.nd4j.linalg.api.ndarray.INDArray> paramTable(boolean backpropParamsOnly)
ModelparamTable in interface ModelbackpropParamsOnly - If true, return backprop params only. If false: return all params (equivalent to
paramsTable())public void setParamTable(java.util.Map<java.lang.String,org.nd4j.linalg.api.ndarray.INDArray> paramTable)
ModelsetParamTable in interface Modelpublic void setParam(java.lang.String key,
org.nd4j.linalg.api.ndarray.INDArray val)
Modelpublic MultiLayerConfiguration getLayerWiseConfigurations()
public void setLayerWiseConfigurations(MultiLayerConfiguration layerWiseConfigurations)
public void initializeLayers(org.nd4j.linalg.api.ndarray.INDArray input)
input - the input matrix for trainingpublic void init()
public void init(org.nd4j.linalg.api.ndarray.INDArray parameters,
boolean cloneParametersArray)
parameters - Network parameter. May be null. If null: randomly initialize.cloneParametersArray - Whether the parameter array (if any) should be cloned, or used directlypublic boolean isInitCalled()
public void initGradientsView()
public org.nd4j.linalg.api.ndarray.INDArray activate()
public org.nd4j.linalg.api.ndarray.INDArray activate(int layer)
layer - the layer to activate onpublic org.nd4j.linalg.api.ndarray.INDArray activate(org.nd4j.linalg.api.ndarray.INDArray input)
Layerpublic org.nd4j.linalg.api.ndarray.INDArray activate(int layer,
org.nd4j.linalg.api.ndarray.INDArray input)
layer - the layer to trigger oninput - the input to the hidden layerpublic org.nd4j.linalg.api.ndarray.INDArray activationMean()
LayeractivationMean in interface Layerpublic void initialize(org.nd4j.linalg.dataset.DataSet data)
data - the dataset to initialize withpublic org.nd4j.linalg.api.ndarray.INDArray zFromPrevLayer(int curr,
org.nd4j.linalg.api.ndarray.INDArray input,
boolean training)
curr - the current layerinput - the inputtraining - training or test modepublic org.nd4j.linalg.api.ndarray.INDArray activationFromPrevLayer(int curr,
org.nd4j.linalg.api.ndarray.INDArray input,
boolean training)
curr - the current layerinput - the inputpublic org.nd4j.linalg.api.ndarray.INDArray activateSelectedLayers(int from,
int to,
org.nd4j.linalg.api.ndarray.INDArray input)
from - first layer to be activated, inclusiveto - last layer to be activated, inclusivepublic java.util.List<org.nd4j.linalg.api.ndarray.INDArray> computeZ(boolean training)
public java.util.List<org.nd4j.linalg.api.ndarray.INDArray> computeZ(org.nd4j.linalg.api.ndarray.INDArray input,
boolean training)
public java.util.List<org.nd4j.linalg.api.ndarray.INDArray> feedForward(org.nd4j.linalg.api.ndarray.INDArray input,
boolean train)
public java.util.List<org.nd4j.linalg.api.ndarray.INDArray> feedForward(boolean train)
public java.util.List<org.nd4j.linalg.api.ndarray.INDArray> feedForwardToLayer(int layerNum,
org.nd4j.linalg.api.ndarray.INDArray input)
layerNum - Index of the last layer to calculate activations for. Layers are zero-indexed.
feedForwardToLayer(i,input) will return the activations for layers 0..i (inclusive)input - Input to the networkpublic java.util.List<org.nd4j.linalg.api.ndarray.INDArray> feedForwardToLayer(int layerNum,
org.nd4j.linalg.api.ndarray.INDArray input,
boolean train)
layerNum - Index of the last layer to calculate activations for. Layers are zero-indexed.
feedForwardToLayer(i,input) will return the activations for layers 0..i (inclusive)input - Input to the networktrain - true for training, false for test (i.e., false if using network after training)public java.util.List<org.nd4j.linalg.api.ndarray.INDArray> feedForwardToLayer(int layerNum,
boolean train)
layerNum - Index of the last layer to calculate activations for. Layers are zero-indexed.
feedForwardToLayer(i,input) will return the activations for layers 0..i (inclusive)train - true for training, false for test (i.e., false if using network after training)public java.util.List<org.nd4j.linalg.api.ndarray.INDArray> feedForward()
public java.util.List<org.nd4j.linalg.api.ndarray.INDArray> feedForward(org.nd4j.linalg.api.ndarray.INDArray input)
public java.util.List<org.nd4j.linalg.api.ndarray.INDArray> feedForward(org.nd4j.linalg.api.ndarray.INDArray input,
org.nd4j.linalg.api.ndarray.INDArray featuresMask,
org.nd4j.linalg.api.ndarray.INDArray labelsMask)
public org.nd4j.linalg.api.ndarray.INDArray epsilon()
public Pair<Gradient,java.lang.Double> gradientAndScore()
ModelgradientAndScore in interface Modelpublic MultiLayerNetwork clone()
Layerpublic org.nd4j.linalg.api.ndarray.INDArray params(boolean backwardOnly)
public org.nd4j.linalg.api.ndarray.INDArray params()
public void setParams(org.nd4j.linalg.api.ndarray.INDArray params)
public void setParamsViewArray(org.nd4j.linalg.api.ndarray.INDArray params)
ModelsetParamsViewArray in interface Modelparams - a 1 x nParams row vector that is a view of the larger (MLN/CG) parameters arraypublic void setBackpropGradientsViewArray(org.nd4j.linalg.api.ndarray.INDArray gradients)
ModelsetBackpropGradientsViewArray in interface Modelgradients - a 1 x nParams row vector that is a view of the larger (MLN/CG) gradients arraypublic int numParams()
public int numParams(boolean backwards)
Modelpublic double f1Score(org.nd4j.linalg.dataset.api.DataSet data)
f1Score in interface Classifierdata - the data to scorepublic void fit(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator)
Classifierfit in interface Classifieriterator - the iterator to train onprotected void backprop()
protected Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray> calcBackpropGradients(org.nd4j.linalg.api.ndarray.INDArray epsilon, boolean withOutputLayer)
epsilon - Errors (technically errors .* activations). Not used if withOutputLayer = truewithOutputLayer - if true: assume last layer is output layer, and calculate errors based on labels. In this
case, the epsilon input is not used (may/should be null).
If false: calculate backprop gradientsprotected void doTruncatedBPTT(org.nd4j.linalg.api.ndarray.INDArray input,
org.nd4j.linalg.api.ndarray.INDArray labels,
org.nd4j.linalg.api.ndarray.INDArray featuresMaskArray,
org.nd4j.linalg.api.ndarray.INDArray labelsMaskArray)
public void updateRnnStateWithTBPTTState()
protected void truncatedBPTTGradient()
public java.util.Collection<IterationListener> getListeners()
LayergetListeners in interface Layerpublic void setListeners(java.util.Collection<IterationListener> listeners)
ModelsetListeners in interface LayersetListeners in interface Modelpublic void setListeners(IterationListener... listeners)
ModelsetListeners in interface LayersetListeners in interface Modelpublic void finetune()
public int[] predict(org.nd4j.linalg.api.ndarray.INDArray d)
predict in interface Classifierd - the matrix to predictpublic java.util.List<java.lang.String> predict(org.nd4j.linalg.dataset.api.DataSet dataSet)
predict in interface ClassifierdataSet - to predictpublic org.nd4j.linalg.api.ndarray.INDArray labelProbabilities(org.nd4j.linalg.api.ndarray.INDArray examples)
labelProbabilities in interface Classifierexamples - the examples to classify (one example in each row)public void fit(org.nd4j.linalg.api.ndarray.INDArray data,
org.nd4j.linalg.api.ndarray.INDArray labels)
fit in interface Classifierdata - the examples to classify (one example in each row)labels - the example labels(a binary outcome matrix)public void fit(org.nd4j.linalg.api.ndarray.INDArray features,
org.nd4j.linalg.api.ndarray.INDArray labels,
org.nd4j.linalg.api.ndarray.INDArray featuresMask,
org.nd4j.linalg.api.ndarray.INDArray labelsMask)
features - the examples to classify (one example in each row)labels - the example labels(a binary outcome matrix)featuresMask - The mask array for the features (used for variable length time series, etc). May be null.labelsMask - The mask array for the labels (used for variable length time series, etc). May be null.public void fit(org.nd4j.linalg.api.ndarray.INDArray data)
public void iterate(org.nd4j.linalg.api.ndarray.INDArray input)
Modelpublic void fit(org.nd4j.linalg.dataset.api.DataSet data)
fit in interface Classifierdata - the data to train onpublic void fit(org.nd4j.linalg.api.ndarray.INDArray examples,
int[] labels)
fit in interface Classifierexamples - the examples to classify (one example in each row)labels - the labels for each example (the number of labels must matchpublic org.nd4j.linalg.api.ndarray.INDArray output(org.nd4j.linalg.api.ndarray.INDArray input,
Layer.TrainingMode train)
input - the input to labeltrain - whether the output
is test or train. This mainly
affect hyper parameters such as
drop out where certain things should
be applied with activationsThis is typically of the form: [0.5, 0.5] or some other probability distribution summing to one
public org.nd4j.linalg.api.ndarray.INDArray output(org.nd4j.linalg.api.ndarray.INDArray input,
boolean train)
input - the input to labeltrain - whether the output
is test or train. This mainly
affect hyper parameters such as
drop out where certain things should
be applied with activationsThis is typically of the form: [0.5, 0.5] or some other probability distribution summing to one
public org.nd4j.linalg.api.ndarray.INDArray output(org.nd4j.linalg.api.ndarray.INDArray input,
boolean train,
org.nd4j.linalg.api.ndarray.INDArray featuresMask,
org.nd4j.linalg.api.ndarray.INDArray labelsMask)
public org.nd4j.linalg.api.ndarray.INDArray output(org.nd4j.linalg.api.ndarray.INDArray input)
input - the input to labelThis is typically of the form: [0.5, 0.5] or some other probability distribution summing to one
public org.nd4j.linalg.api.ndarray.INDArray output(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator,
boolean train)
iterator - test data to evaluateThis is typically of the form: [0.5, 0.5] or some other probability distribution summing to one
public org.nd4j.linalg.api.ndarray.INDArray output(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator)
public org.nd4j.linalg.api.ndarray.INDArray reconstruct(org.nd4j.linalg.api.ndarray.INDArray x,
int layerNum)
x - the input to transformlayerNum - the layer to output for encodingThis is typically of the form: [0.5, 0.5] or some other probability distribution summing to one
public void printConfiguration()
public void update(MultiLayerNetwork network)
network - the network to getFromOrigin parameters frompublic double f1Score(org.nd4j.linalg.api.ndarray.INDArray input,
org.nd4j.linalg.api.ndarray.INDArray labels)
f1Score in interface Classifierinput - the input to scorelabels - the true labelspublic int numLabels()
numLabels in interface Classifierpublic double score(org.nd4j.linalg.dataset.DataSet data)
score(DataSet, boolean) with training==true.data - the data to scorescore(DataSet, boolean)public double score(org.nd4j.linalg.dataset.DataSet data,
boolean training)
data - data to calculate score fortraining - If true: score during training. If false: score at test time. This can affect the application of
certain features, such as dropout and dropconnect (which are applied at training time only)public org.nd4j.linalg.api.ndarray.INDArray scoreExamples(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iter,
boolean addRegularizationTerms)
public org.nd4j.linalg.api.ndarray.INDArray scoreExamples(org.nd4j.linalg.dataset.DataSet data,
boolean addRegularizationTerms)
score(DataSet) and score(DataSet, boolean)
this method does not average/sum over examples. This method allows for examples to be scored individually (at test time only), which
may be useful for example for autoencoder architectures and the like.data - The data to scoreaddRegularizationTerms - If true: add l1/l2 regularization terms (if any) to the score. If false: don't add regularization termspublic void fit()
Modelpublic void update(org.nd4j.linalg.api.ndarray.INDArray gradient,
java.lang.String paramType)
Modelpublic double score()
public void setScore(double score)
public void computeGradientAndScore()
ModelcomputeGradientAndScore in interface Modelpublic void accumulateScore(double accum)
ModelaccumulateScore in interface Modelaccum - the amount to accumpublic void clear()
@Deprecated public void merge(Layer layer, int batchSize)
@Deprecated public void merge(MultiLayerNetwork network, int batchSize)
network - the network to merge withbatchSize - the batch size (number of training examples)
to average bypublic void setInput(org.nd4j.linalg.api.ndarray.INDArray input)
public Layer getOutputLayer()
public void setParameters(org.nd4j.linalg.api.ndarray.INDArray params)
params - a parameter vector equal 1,numParameterspublic void applyLearningRateScoreDecay()
ModelapplyLearningRateScoreDecay in interface Modelpublic NeuralNetConfiguration getDefaultConfiguration()
public org.nd4j.linalg.api.ndarray.INDArray getLabels()
public org.nd4j.linalg.api.ndarray.INDArray getInput()
public void setLabels(org.nd4j.linalg.api.ndarray.INDArray labels)
labels - public int getnLayers()
public Layer[] getLayers()
public Layer getLayer(int i)
public Layer getLayer(java.lang.String name)
public java.util.List<java.lang.String> getLayerNames()
public void setLayers(Layer[] layers)
public org.nd4j.linalg.api.ndarray.INDArray getMask()
public void setMask(org.nd4j.linalg.api.ndarray.INDArray mask)
public org.nd4j.linalg.api.ndarray.INDArray getMaskArray()
getMaskArray in interface Layerpublic boolean isPretrainLayer()
LayerisPretrainLayer in interface Layerpublic Pair<org.nd4j.linalg.api.ndarray.INDArray,MaskState> feedForwardMaskArray(org.nd4j.linalg.api.ndarray.INDArray maskArray, MaskState currentMaskState, int minibatchSize)
LayerfeedForwardMaskArray in interface LayermaskArray - Mask array to setcurrentMaskState - Current state of the mask - see MaskStateminibatchSize - Current minibatch size. Needs to be known as it cannot always be inferred from the activations
array due to reshaping (such as a DenseLayer within a recurrent neural network)public Gradient error(org.nd4j.linalg.api.ndarray.INDArray errorSignal)
Layerpublic Layer.Type type()
Layerpublic org.nd4j.linalg.api.ndarray.INDArray derivativeActivation(org.nd4j.linalg.api.ndarray.INDArray input)
LayerderivativeActivation in interface Layerinput - the input to take the derivative ofpublic Gradient calcGradient(Gradient layerError, org.nd4j.linalg.api.ndarray.INDArray activation)
LayercalcGradient in interface LayerlayerError - the layer errorpublic org.nd4j.linalg.api.ndarray.INDArray preOutput(org.nd4j.linalg.api.ndarray.INDArray x)
Layerpublic org.nd4j.linalg.api.ndarray.INDArray preOutput(org.nd4j.linalg.api.ndarray.INDArray x,
Layer.TrainingMode training)
Layerpublic org.nd4j.linalg.api.ndarray.INDArray activate(Layer.TrainingMode training)
Layerpublic org.nd4j.linalg.api.ndarray.INDArray activate(org.nd4j.linalg.api.ndarray.INDArray input,
Layer.TrainingMode training)
Layerpublic Layer transpose()
Layerpublic Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray> backpropGradient(org.nd4j.linalg.api.ndarray.INDArray epsilon)
LayerbackpropGradient in interface Layerepsilon - w^(L+1)*delta^(L+1). Or, equiv: dC/da, i.e., (dC/dz)*(dz/da) = dC/da, where C
is cost function a=sigma(z) is activation.public void setIndex(int index)
Layerpublic int getIndex()
Layerpublic double calcL2(boolean backpropParamsOnly)
Layerpublic double calcL1(boolean backpropParamsOnly)
Layerpublic void update(Gradient gradient)
Modelpublic org.nd4j.linalg.api.ndarray.INDArray preOutput(org.nd4j.linalg.api.ndarray.INDArray x,
boolean training)
Layerpublic org.nd4j.linalg.api.ndarray.INDArray activate(boolean training)
Layerpublic org.nd4j.linalg.api.ndarray.INDArray activate(org.nd4j.linalg.api.ndarray.INDArray input,
boolean training)
Layerpublic void setInputMiniBatchSize(int size)
LayersetInputMiniBatchSize in interface Layerpublic int getInputMiniBatchSize()
LayergetInputMiniBatchSize in interface LayerLayer.setInputMiniBatchSize(int)public void setMaskArray(org.nd4j.linalg.api.ndarray.INDArray maskArray)
LayerLayer.feedForwardMaskArray(INDArray, MaskState, int) should be used in
preference to this.setMaskArray in interface LayermaskArray - Mask array to setpublic org.nd4j.linalg.api.ndarray.INDArray rnnTimeStep(org.nd4j.linalg.api.ndarray.INDArray input)
input - Input to network. May be for one or multiple time steps. For single time step:
input has shape [miniBatchSize,inputSize] or [miniBatchSize,inputSize,1]. miniBatchSize=1 for single example.public java.util.Map<java.lang.String,org.nd4j.linalg.api.ndarray.INDArray> rnnGetPreviousState(int layer)
layer - Number/index of the layer.public void rnnSetPreviousState(int layer,
java.util.Map<java.lang.String,org.nd4j.linalg.api.ndarray.INDArray> state)
layer - The number/index of the layer.state - The state to set the specified layer topublic void rnnClearPreviousState()
public java.util.List<org.nd4j.linalg.api.ndarray.INDArray> rnnActivateUsingStoredState(org.nd4j.linalg.api.ndarray.INDArray input,
boolean training,
boolean storeLastForTBPTT)
input - Input to networktraining - Whether training or notstoreLastForTBPTT - set to true if used as part of truncated BPTT trainingpublic Updater getUpdater()
public void setUpdater(Updater updater)
public void setLayerMaskArrays(org.nd4j.linalg.api.ndarray.INDArray featuresMaskArray,
org.nd4j.linalg.api.ndarray.INDArray labelsMaskArray)
feedForward(INDArray, INDArray, INDArray)
and output(INDArray, boolean, INDArray, INDArray) handle setting of masking internally.featuresMaskArray - Mask array for features (input)labelsMaskArray - Mask array for labels (output)clearLayerMaskArrays()public void clearLayerMaskArrays()
setLayerMaskArrays(INDArray, INDArray) for details on mask arrays.public Evaluation evaluate(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator)
iterator - Iterator to evaluate onpublic RegressionEvaluation evaluateRegression(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator)
iterator - Data to evaluate onpublic ROC evaluateROC(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator, int rocThresholdSteps)
ROC classiterator - Data to evaluate onrocThresholdSteps - Number of threshold steps to use with ROCpublic ROCMultiClass evaluateROCMultiClass(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator, int rocThresholdSteps)
ROCMultiClass classiterator - Data to evaluate onrocThresholdSteps - Number of threshold steps to use with ROCMultiClasspublic <T extends IEvaluation> T doEvaluation(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator, T evaluation)
iterator - data to evaluate onevaluation - IEvaluation instance to perform evaluation withpublic Evaluation evaluate(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator, java.util.List<java.lang.String> labelsList)
iterator - Data to undertake evaluation onpublic Evaluation evaluate(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator, java.util.List<java.lang.String> labelsList, int topN)
iterator - Iterator (data) to evaluate onlabelsList - List of labels. May be null.topN - N value for top N accuracy evaluationpublic java.lang.String summary()