public class MultiLayerNetwork extends java.lang.Object implements java.io.Serializable, Classifier, Layer
ComputationGraph
which allows for an arbitrary directed acyclic graph connection structure between layers.
MultiLayerNetwork is trainable via backprop, with optional pretraining, depending on the type of layers it contains.Layer.TrainingMode, Layer.Type
Modifier and Type | Field and Description |
---|---|
protected NeuralNetConfiguration |
defaultConfiguration |
protected org.nd4j.linalg.api.ndarray.INDArray |
epsilon |
protected org.nd4j.linalg.api.ndarray.INDArray |
flattenedGradients |
protected org.nd4j.linalg.api.ndarray.INDArray |
flattenedParams |
protected Gradient |
gradient |
protected boolean |
initCalled |
protected boolean |
initDone |
protected org.nd4j.linalg.api.ndarray.INDArray |
input |
protected org.nd4j.linalg.api.ndarray.INDArray |
labels |
protected int |
layerIndex |
protected java.util.LinkedHashMap<java.lang.String,Layer> |
layerMap |
protected Layer[] |
layers |
protected MultiLayerConfiguration |
layerWiseConfigurations |
protected org.nd4j.linalg.api.ndarray.INDArray |
mask |
protected double |
score |
protected Solver |
solver |
Constructor and Description |
---|
MultiLayerNetwork(MultiLayerConfiguration conf) |
MultiLayerNetwork(MultiLayerConfiguration conf,
org.nd4j.linalg.api.ndarray.INDArray params)
Initialize the network based on the configuraiton
|
MultiLayerNetwork(java.lang.String conf,
org.nd4j.linalg.api.ndarray.INDArray params)
Initialize the network based on the configuration
|
Modifier and Type | Method and Description |
---|---|
void |
accumulateScore(double accum)
Sets a rolling tally for the score.
|
org.nd4j.linalg.api.ndarray.INDArray |
activate()
Triggers the activation of the last hidden layer ie: not logistic regression
|
org.nd4j.linalg.api.ndarray.INDArray |
activate(boolean training)
Trigger an activation with the last specified input
|
org.nd4j.linalg.api.ndarray.INDArray |
activate(org.nd4j.linalg.api.ndarray.INDArray input)
Initialize the layer with the given input
and return the activation for this layer
given this input
|
org.nd4j.linalg.api.ndarray.INDArray |
activate(org.nd4j.linalg.api.ndarray.INDArray input,
boolean training)
Initialize the layer with the given input
and return the activation for this layer
given this input
|
org.nd4j.linalg.api.ndarray.INDArray |
activate(org.nd4j.linalg.api.ndarray.INDArray input,
Layer.TrainingMode training)
Initialize the layer with the given input
and return the activation for this layer
given this input
|
org.nd4j.linalg.api.ndarray.INDArray |
activate(int layer)
Triggers the activation for a given layer
|
org.nd4j.linalg.api.ndarray.INDArray |
activate(int layer,
org.nd4j.linalg.api.ndarray.INDArray input)
Triggers the activation of the given layer
|
org.nd4j.linalg.api.ndarray.INDArray |
activate(Layer.TrainingMode training)
Trigger an activation with the last specified input
|
org.nd4j.linalg.api.ndarray.INDArray |
activateSelectedLayers(int from,
int to,
org.nd4j.linalg.api.ndarray.INDArray input)
Calculate activation for few layers at once.
|
org.nd4j.linalg.api.ndarray.INDArray |
activationFromPrevLayer(int curr,
org.nd4j.linalg.api.ndarray.INDArray input,
boolean training)
Calculate activation from previous layer including pre processing where necessary
|
org.nd4j.linalg.api.ndarray.INDArray |
activationMean()
Calculate the mean representation
for the activation for this layer
|
void |
applyLearningRateScoreDecay()
Update learningRate using for this model.
|
protected void |
backprop()
Calculate and set gradients for MultiLayerNetwork, based on OutputLayer and labels
|
Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray> |
backpropGradient(org.nd4j.linalg.api.ndarray.INDArray epsilon)
Calculate the gradient relative to the error in the next layer
|
int |
batchSize()
The current inputs batch size
|
protected Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray> |
calcBackpropGradients(org.nd4j.linalg.api.ndarray.INDArray epsilon,
boolean withOutputLayer)
Calculate gradients and errors.
|
Gradient |
calcGradient(Gradient layerError,
org.nd4j.linalg.api.ndarray.INDArray activation)
Calculate the gradient
|
double |
calcL1(boolean backpropParamsOnly)
Calculate the l1 regularization term
0.0 if regularization is not used. |
double |
calcL2(boolean backpropParamsOnly)
Calculate the l2 regularization term
0.0 if regularization is not used. |
void |
clear()
Clear the inputs.
|
void |
clearLayerMaskArrays()
Remove the mask arrays from all layers.
See setLayerMaskArrays(INDArray, INDArray) for details on mask arrays. |
MultiLayerNetwork |
clone()
Clone the layer
|
void |
computeGradientAndScore()
Update the score
|
java.util.List<org.nd4j.linalg.api.ndarray.INDArray> |
computeZ(boolean training)
* Compute input linear transformation (z) of the output layer
|
java.util.List<org.nd4j.linalg.api.ndarray.INDArray> |
computeZ(org.nd4j.linalg.api.ndarray.INDArray input,
boolean training)
Compute activations from input to output of the output layer
|
NeuralNetConfiguration |
conf()
The configuration for the neural network
|
org.nd4j.linalg.api.ndarray.INDArray |
derivativeActivation(org.nd4j.linalg.api.ndarray.INDArray input)
Take the derivative of the given input
based on the activation
|
<T extends IEvaluation> |
doEvaluation(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator,
T evaluation)
Perform evaluation using an arbitrary IEvaluation instance.
|
protected void |
doTruncatedBPTT(org.nd4j.linalg.api.ndarray.INDArray input,
org.nd4j.linalg.api.ndarray.INDArray labels,
org.nd4j.linalg.api.ndarray.INDArray featuresMaskArray,
org.nd4j.linalg.api.ndarray.INDArray labelsMaskArray) |
org.nd4j.linalg.api.ndarray.INDArray |
epsilon() |
Gradient |
error(org.nd4j.linalg.api.ndarray.INDArray errorSignal)
Calculate error with respect to the
current layer.
|
Evaluation |
evaluate(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator)
Evaluate the network (classification performance)
|
Evaluation |
evaluate(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator,
java.util.List<java.lang.String> labelsList)
Evaluate the network on the provided data set.
|
Evaluation |
evaluate(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator,
java.util.List<java.lang.String> labelsList,
int topN)
Evaluate the network (for classification) on the provided data set, with top N accuracy in addition to standard accuracy.
|
RegressionEvaluation |
evaluateRegression(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator)
Evaluate the network for regression performance
|
ROC |
evaluateROC(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator,
int rocThresholdSteps)
Evaluate the network (must be a binary classifier) on the specified data, using the
ROC class |
ROCMultiClass |
evaluateROCMultiClass(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator,
int rocThresholdSteps)
Evaluate the network on the specified data, using the
ROCMultiClass class |
double |
f1Score(org.nd4j.linalg.dataset.api.DataSet data)
Sets the input and labels and returns a score for the prediction
wrt true labels
|
double |
f1Score(org.nd4j.linalg.api.ndarray.INDArray input,
org.nd4j.linalg.api.ndarray.INDArray labels)
Sets the input and labels and returns a score for the prediction
wrt true labels
|
java.util.List<org.nd4j.linalg.api.ndarray.INDArray> |
feedForward()
Compute activations from input to output of the output layer
|
java.util.List<org.nd4j.linalg.api.ndarray.INDArray> |
feedForward(boolean train)
Compute activations from input to output of the output layer
|
java.util.List<org.nd4j.linalg.api.ndarray.INDArray> |
feedForward(org.nd4j.linalg.api.ndarray.INDArray input)
Compute activations from input to output of the output layer
|
java.util.List<org.nd4j.linalg.api.ndarray.INDArray> |
feedForward(org.nd4j.linalg.api.ndarray.INDArray input,
boolean train)
Compute activations from input to output of the output layer
|
java.util.List<org.nd4j.linalg.api.ndarray.INDArray> |
feedForward(org.nd4j.linalg.api.ndarray.INDArray input,
org.nd4j.linalg.api.ndarray.INDArray featuresMask,
org.nd4j.linalg.api.ndarray.INDArray labelsMask)
Compute the activations from the input to the output layer, given mask arrays (that may be null)
The masking arrays are used in situations such an one-to-many and many-to-one rucerrent neural network (RNN)
designs, as well as for supporting time series of varying lengths within the same minibatch for RNNs.
|
Pair<org.nd4j.linalg.api.ndarray.INDArray,MaskState> |
feedForwardMaskArray(org.nd4j.linalg.api.ndarray.INDArray maskArray,
MaskState currentMaskState,
int minibatchSize)
Feed forward the input mask array, setting in in the layer as appropriate.
|
java.util.List<org.nd4j.linalg.api.ndarray.INDArray> |
feedForwardToLayer(int layerNum,
boolean train)
Compute the activations from the input to the specified layer, using the currently set input for the network.
To compute activations for all layers, use feedForward(...) methods Note: output list includes the original input. |
java.util.List<org.nd4j.linalg.api.ndarray.INDArray> |
feedForwardToLayer(int layerNum,
org.nd4j.linalg.api.ndarray.INDArray input)
Compute the activations from the input to the specified layer.
To compute activations for all layers, use feedForward(...) methods Note: output list includes the original input. |
java.util.List<org.nd4j.linalg.api.ndarray.INDArray> |
feedForwardToLayer(int layerNum,
org.nd4j.linalg.api.ndarray.INDArray input,
boolean train)
Compute the activations from the input to the specified layer.
To compute activations for all layers, use feedForward(...) methods Note: output list includes the original input. |
void |
finetune()
Run SGD based on the given labels
|
void |
fit()
All models have a fit method
|
void |
fit(org.nd4j.linalg.dataset.api.DataSet data)
Fit the model
|
void |
fit(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator)
Train the model based on the datasetiterator
|
void |
fit(org.nd4j.linalg.api.ndarray.INDArray data)
Fit the unsupervised model
|
void |
fit(org.nd4j.linalg.api.ndarray.INDArray data,
org.nd4j.linalg.api.ndarray.INDArray labels)
Fit the model
|
void |
fit(org.nd4j.linalg.api.ndarray.INDArray features,
org.nd4j.linalg.api.ndarray.INDArray labels,
org.nd4j.linalg.api.ndarray.INDArray featuresMask,
org.nd4j.linalg.api.ndarray.INDArray labelsMask)
Fit the model
|
void |
fit(org.nd4j.linalg.api.ndarray.INDArray examples,
int[] labels)
Fit the model
|
NeuralNetConfiguration |
getDefaultConfiguration() |
int |
getIndex()
Get the layer index.
|
org.nd4j.linalg.api.ndarray.INDArray |
getInput() |
int |
getInputMiniBatchSize()
Get current/last input mini-batch size, as set by setInputMiniBatchSize(int)
|
org.nd4j.linalg.api.ndarray.INDArray |
getLabels() |
Layer |
getLayer(int i) |
Layer |
getLayer(java.lang.String name) |
java.util.List<java.lang.String> |
getLayerNames() |
Layer[] |
getLayers() |
MultiLayerConfiguration |
getLayerWiseConfigurations() |
java.util.Collection<IterationListener> |
getListeners()
Get the iteration listeners for this layer.
|
org.nd4j.linalg.api.ndarray.INDArray |
getMask() |
org.nd4j.linalg.api.ndarray.INDArray |
getMaskArray() |
int |
getnLayers()
Get the number of layers in the network
|
ConvexOptimizer |
getOptimizer()
Returns this models optimizer
|
Layer |
getOutputLayer()
Get the output layer
|
org.nd4j.linalg.api.ndarray.INDArray |
getParam(java.lang.String param)
Get the parameter
|
Updater |
getUpdater()
Get the updater for this MultiLayerNetwork
|
Gradient |
gradient()
Calculate a gradient
|
Pair<Gradient,java.lang.Double> |
gradientAndScore()
Get the gradient and score
|
void |
init()
Initialize the MultiLayerNetwork.
|
void |
init(org.nd4j.linalg.api.ndarray.INDArray parameters,
boolean cloneParametersArray)
Initialize the MultiLayerNetwork, optionally with an existing parameters array.
|
void |
initGradientsView()
This method: initializes the flattened gradients array (used in backprop) and sets the appropriate subset in all layers.
|
void |
initialize(org.nd4j.linalg.dataset.DataSet data)
Sets the input and labels from this dataset
|
void |
initializeLayers(org.nd4j.linalg.api.ndarray.INDArray input)
Base class for initializing the neuralNets based on the input.
|
void |
initParams()
Initialize the parameters
|
org.nd4j.linalg.api.ndarray.INDArray |
input()
The input/feature matrix for the model
|
protected void |
intializeConfigurations() |
boolean |
isInitCalled() |
boolean |
isPretrainLayer()
Returns true if the layer can be trained in an unsupervised/pretrain manner (VAE, RBMs etc)
|
void |
iterate(org.nd4j.linalg.api.ndarray.INDArray input)
Run one iteration
|
org.nd4j.linalg.api.ndarray.INDArray |
labelProbabilities(org.nd4j.linalg.api.ndarray.INDArray examples)
Returns the probabilities for each label
for each example row wise
|
void |
merge(Layer layer,
int batchSize)
Deprecated.
Not supported and not used
|
void |
merge(MultiLayerNetwork network,
int batchSize)
Deprecated.
As of 0.7.3 - Feb 2017. No longer used; parameter averaging is performed via alternative means/methods
|
int |
numLabels()
Returns the number of possible labels
|
int |
numParams()
Returns a 1 x m vector where the vector is composed of
a flattened vector of all of the weights for the
various neuralNets and output layer
|
int |
numParams(boolean backwards)
the number of parameters for the model
|
org.nd4j.linalg.api.ndarray.INDArray |
output(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator) |
org.nd4j.linalg.api.ndarray.INDArray |
output(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator,
boolean train)
Label the probabilities of the input
|
org.nd4j.linalg.api.ndarray.INDArray |
output(org.nd4j.linalg.api.ndarray.INDArray input)
Label the probabilities of the input
|
org.nd4j.linalg.api.ndarray.INDArray |
output(org.nd4j.linalg.api.ndarray.INDArray input,
boolean train)
Label the probabilities of the input
|
org.nd4j.linalg.api.ndarray.INDArray |
output(org.nd4j.linalg.api.ndarray.INDArray input,
boolean train,
org.nd4j.linalg.api.ndarray.INDArray featuresMask,
org.nd4j.linalg.api.ndarray.INDArray labelsMask)
Calculate the output of the network, with masking arrays.
|
org.nd4j.linalg.api.ndarray.INDArray |
output(org.nd4j.linalg.api.ndarray.INDArray input,
Layer.TrainingMode train)
Label the probabilities of the input
|
org.nd4j.linalg.api.ndarray.INDArray |
params()
Returns a 1 x m vector where the vector is composed of
a flattened vector of all of the weights for the
various neuralNets(w,hbias NOT VBIAS) and output layer
|
org.nd4j.linalg.api.ndarray.INDArray |
params(boolean backwardOnly)
Returns a 1 x m vector where the vector is composed of
a flattened vector of all of the weights for the
various neuralNets(w,hbias NOT VBIAS) and output layer
|
java.util.Map<java.lang.String,org.nd4j.linalg.api.ndarray.INDArray> |
paramTable()
The param table
|
java.util.Map<java.lang.String,org.nd4j.linalg.api.ndarray.INDArray> |
paramTable(boolean backpropParamsOnly)
Table of parameters by key, for backprop
For many models (dense layers, etc) - all parameters are backprop parameters
|
java.util.List<java.lang.String> |
predict(org.nd4j.linalg.dataset.api.DataSet dataSet)
Return predicted label names
|
int[] |
predict(org.nd4j.linalg.api.ndarray.INDArray d)
Returns the predictions for each example in the dataset
|
org.nd4j.linalg.api.ndarray.INDArray |
preOutput(org.nd4j.linalg.api.ndarray.INDArray x)
Raw activations
|
org.nd4j.linalg.api.ndarray.INDArray |
preOutput(org.nd4j.linalg.api.ndarray.INDArray x,
boolean training)
Raw activations
|
org.nd4j.linalg.api.ndarray.INDArray |
preOutput(org.nd4j.linalg.api.ndarray.INDArray x,
Layer.TrainingMode training)
Raw activations
|
void |
pretrain(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iter)
Perform layerwise pretraining on all pre-trainable layers in the network (VAEs, RBMs, Autoencoders, etc)
Note that pretraining will be performed on one layer after the other, resetting the DataSetIterator between iterations. For multiple epochs per layer, appropriately wrap the iterator (for example, a MultipleEpochsIterator) or train each layer manually using pretrainLayer(int, DataSetIterator) |
void |
pretrain(org.nd4j.linalg.api.ndarray.INDArray input)
Deprecated.
use
pretrain(DataSetIterator) or pretrainLayer(int, DataSetIterator) or pretrainLayer(int, INDArray) .
Pretraining each layer in a row on a single minibatch (as per this method) instead of N epochs per layer is not advisable. |
void |
pretrainLayer(int layerIdx,
org.nd4j.linalg.dataset.api.iterator.DataSetIterator iter)
Perform layerwise unsupervised training on a single pre-trainable layer in the network (VAEs, RBMs, Autoencoders, etc)
If the specified layer index (0 to numLayers - 1) is not a pretrainable layer, this is a no-op. |
void |
pretrainLayer(int layerIdx,
org.nd4j.linalg.api.ndarray.INDArray features)
Perform layerwise unsupervised training on a single pre-trainable layer in the network (VAEs, RBMs, Autoencoders, etc)
If the specified layer index (0 to numLayers - 1) is not a pretrainable layer, this is a no-op. |
void |
printConfiguration()
Prints the configuration
|
org.nd4j.linalg.api.ndarray.INDArray |
reconstruct(org.nd4j.linalg.api.ndarray.INDArray x,
int layerNum)
Reconstructs the input.
|
java.util.List<org.nd4j.linalg.api.ndarray.INDArray> |
rnnActivateUsingStoredState(org.nd4j.linalg.api.ndarray.INDArray input,
boolean training,
boolean storeLastForTBPTT)
Similar to rnnTimeStep and feedForward() methods.
|
void |
rnnClearPreviousState()
Clear the previous state of the RNN layers (if any).
|
java.util.Map<java.lang.String,org.nd4j.linalg.api.ndarray.INDArray> |
rnnGetPreviousState(int layer)
Get the state of the RNN layer, as used in rnnTimeStep().
|
void |
rnnSetPreviousState(int layer,
java.util.Map<java.lang.String,org.nd4j.linalg.api.ndarray.INDArray> state)
Set the state of the RNN layer.
|
org.nd4j.linalg.api.ndarray.INDArray |
rnnTimeStep(org.nd4j.linalg.api.ndarray.INDArray input)
If this MultiLayerNetwork contains one or more RNN layers: conduct forward pass (prediction)
but using previous stored state for any RNN layers.
|
double |
score()
Score of the model (relative to the objective function)
|
double |
score(org.nd4j.linalg.dataset.DataSet data)
Sets the input and labels and returns a score for the prediction with respect to the true labels
This is equivalent to score(DataSet, boolean) with training==true. |
double |
score(org.nd4j.linalg.dataset.DataSet data,
boolean training)
Calculate the score (loss function) of the prediction with respect to the true labels
|
org.nd4j.linalg.api.ndarray.INDArray |
scoreExamples(org.nd4j.linalg.dataset.DataSet data,
boolean addRegularizationTerms)
Calculate the score for each example in a DataSet individually.
|
org.nd4j.linalg.api.ndarray.INDArray |
scoreExamples(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iter,
boolean addRegularizationTerms) |
void |
setBackpropGradientsViewArray(org.nd4j.linalg.api.ndarray.INDArray gradients)
Set the gradients array as a view of the full (backprop) network parameters
NOTE: this is intended to be used internally in MultiLayerNetwork and ComputationGraph, not by users.
|
void |
setConf(NeuralNetConfiguration conf)
Setter for the configuration
|
void |
setIndex(int index)
Set the layer index.
|
void |
setInput(org.nd4j.linalg.api.ndarray.INDArray input)
Note that if input isn't null
and the neuralNets are null, this is a way
of initializing the neural network
|
void |
setInputMiniBatchSize(int size)
Set current/last input mini-batch size.
Used for score and gradient calculations. |
void |
setLabels(org.nd4j.linalg.api.ndarray.INDArray labels) |
void |
setLayerMaskArrays(org.nd4j.linalg.api.ndarray.INDArray featuresMaskArray,
org.nd4j.linalg.api.ndarray.INDArray labelsMaskArray)
Set the mask arrays for features and labels.
|
void |
setLayers(Layer[] layers) |
void |
setLayerWiseConfigurations(MultiLayerConfiguration layerWiseConfigurations) |
void |
setListeners(java.util.Collection<IterationListener> listeners)
Set the IterationListeners for the ComputationGraph (and all layers in the network)
|
void |
setListeners(IterationListener... listeners)
Set the IterationListeners for the ComputationGraph (and all layers in the network)
|
void |
setMask(org.nd4j.linalg.api.ndarray.INDArray mask) |
void |
setMaskArray(org.nd4j.linalg.api.ndarray.INDArray maskArray)
Set the mask array.
|
void |
setParam(java.lang.String key,
org.nd4j.linalg.api.ndarray.INDArray val)
Set the parameter with a new ndarray
|
void |
setParameters(org.nd4j.linalg.api.ndarray.INDArray params)
Sets parameters for the model.
|
void |
setParams(org.nd4j.linalg.api.ndarray.INDArray params)
Set the parameters for this model.
|
void |
setParamsViewArray(org.nd4j.linalg.api.ndarray.INDArray params)
Set the initial parameters array as a view of the full (backprop) network parameters
NOTE: this is intended to be used internally in MultiLayerNetwork and ComputationGraph, not by users.
|
void |
setParamTable(java.util.Map<java.lang.String,org.nd4j.linalg.api.ndarray.INDArray> paramTable)
Setter for the param table
|
void |
setScore(double score) |
void |
setUpdater(Updater updater)
Set the updater for the MultiLayerNetwork
|
java.lang.String |
summary()
String detailing the architecture of the multilayernetwork.
|
Layer |
transpose()
Return a transposed copy of the weights/bias
(this means reverse the number of inputs and outputs on the weights)
|
protected void |
truncatedBPTTGradient()
Equivalent to backprop(), but calculates gradient for truncated BPTT instead.
|
Layer.Type |
type()
Returns the layer type
|
void |
update(Gradient gradient)
Update layer weights and biases with gradient change
|
void |
update(org.nd4j.linalg.api.ndarray.INDArray gradient,
java.lang.String paramType)
Perform one update applying the gradient
|
void |
update(MultiLayerNetwork network)
Assigns the parameters of this model to the ones specified by this
network.
|
void |
updateRnnStateWithTBPTTState() |
void |
validateInput()
Validate the input
|
org.nd4j.linalg.api.ndarray.INDArray |
zFromPrevLayer(int curr,
org.nd4j.linalg.api.ndarray.INDArray input,
boolean training)
Compute input linear transformation (z) from previous layer
Apply pre processing transformation where necessary
|
protected Layer[] layers
protected java.util.LinkedHashMap<java.lang.String,Layer> layerMap
protected org.nd4j.linalg.api.ndarray.INDArray input
protected org.nd4j.linalg.api.ndarray.INDArray labels
protected boolean initCalled
protected NeuralNetConfiguration defaultConfiguration
protected MultiLayerConfiguration layerWiseConfigurations
protected Gradient gradient
protected org.nd4j.linalg.api.ndarray.INDArray epsilon
protected double score
protected boolean initDone
protected org.nd4j.linalg.api.ndarray.INDArray flattenedParams
protected transient org.nd4j.linalg.api.ndarray.INDArray flattenedGradients
protected org.nd4j.linalg.api.ndarray.INDArray mask
protected int layerIndex
protected transient Solver solver
public MultiLayerNetwork(MultiLayerConfiguration conf)
public MultiLayerNetwork(java.lang.String conf, org.nd4j.linalg.api.ndarray.INDArray params)
conf
- the configuration jsonparams
- the parameterspublic MultiLayerNetwork(MultiLayerConfiguration conf, org.nd4j.linalg.api.ndarray.INDArray params)
conf
- the configurationparams
- the parametersprotected void intializeConfigurations()
public void pretrain(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iter)
pretrainLayer(int, DataSetIterator)
iter
- Training datapublic void pretrainLayer(int layerIdx, org.nd4j.linalg.dataset.api.iterator.DataSetIterator iter)
layerIdx
- Index of the layer to train (0 to numLayers-1)iter
- Training datapublic void pretrainLayer(int layerIdx, org.nd4j.linalg.api.ndarray.INDArray features)
layerIdx
- Index of the layer to train (0 to numLayers-1)features
- Training data array@Deprecated public void pretrain(org.nd4j.linalg.api.ndarray.INDArray input)
pretrain(DataSetIterator)
or pretrainLayer(int, DataSetIterator)
or pretrainLayer(int, INDArray)
.
Pretraining each layer in a row on a single minibatch (as per this method) instead of N epochs per layer is not advisable.public int batchSize()
Model
public NeuralNetConfiguration conf()
Model
public void setConf(NeuralNetConfiguration conf)
Model
public org.nd4j.linalg.api.ndarray.INDArray input()
Model
public void validateInput()
Model
validateInput
in interface Model
public ConvexOptimizer getOptimizer()
Model
getOptimizer
in interface Model
public org.nd4j.linalg.api.ndarray.INDArray getParam(java.lang.String param)
Model
public void initParams()
Model
initParams
in interface Model
public java.util.Map<java.lang.String,org.nd4j.linalg.api.ndarray.INDArray> paramTable()
Model
paramTable
in interface Model
public java.util.Map<java.lang.String,org.nd4j.linalg.api.ndarray.INDArray> paramTable(boolean backpropParamsOnly)
Model
paramTable
in interface Model
backpropParamsOnly
- If true, return backprop params only. If false: return all params (equivalent to
paramsTable())public void setParamTable(java.util.Map<java.lang.String,org.nd4j.linalg.api.ndarray.INDArray> paramTable)
Model
setParamTable
in interface Model
public void setParam(java.lang.String key, org.nd4j.linalg.api.ndarray.INDArray val)
Model
public MultiLayerConfiguration getLayerWiseConfigurations()
public void setLayerWiseConfigurations(MultiLayerConfiguration layerWiseConfigurations)
public void initializeLayers(org.nd4j.linalg.api.ndarray.INDArray input)
input
- the input matrix for trainingpublic void init()
public void init(org.nd4j.linalg.api.ndarray.INDArray parameters, boolean cloneParametersArray)
parameters
- Network parameter. May be null. If null: randomly initialize.cloneParametersArray
- Whether the parameter array (if any) should be cloned, or used directlypublic boolean isInitCalled()
public void initGradientsView()
public org.nd4j.linalg.api.ndarray.INDArray activate()
public org.nd4j.linalg.api.ndarray.INDArray activate(int layer)
layer
- the layer to activate onpublic org.nd4j.linalg.api.ndarray.INDArray activate(org.nd4j.linalg.api.ndarray.INDArray input)
Layer
public org.nd4j.linalg.api.ndarray.INDArray activate(int layer, org.nd4j.linalg.api.ndarray.INDArray input)
layer
- the layer to trigger oninput
- the input to the hidden layerpublic org.nd4j.linalg.api.ndarray.INDArray activationMean()
Layer
activationMean
in interface Layer
public void initialize(org.nd4j.linalg.dataset.DataSet data)
data
- the dataset to initialize withpublic org.nd4j.linalg.api.ndarray.INDArray zFromPrevLayer(int curr, org.nd4j.linalg.api.ndarray.INDArray input, boolean training)
curr
- the current layerinput
- the inputtraining
- training or test modepublic org.nd4j.linalg.api.ndarray.INDArray activationFromPrevLayer(int curr, org.nd4j.linalg.api.ndarray.INDArray input, boolean training)
curr
- the current layerinput
- the inputpublic org.nd4j.linalg.api.ndarray.INDArray activateSelectedLayers(int from, int to, org.nd4j.linalg.api.ndarray.INDArray input)
from
- first layer to be activated, inclusiveto
- last layer to be activated, inclusivepublic java.util.List<org.nd4j.linalg.api.ndarray.INDArray> computeZ(boolean training)
public java.util.List<org.nd4j.linalg.api.ndarray.INDArray> computeZ(org.nd4j.linalg.api.ndarray.INDArray input, boolean training)
public java.util.List<org.nd4j.linalg.api.ndarray.INDArray> feedForward(org.nd4j.linalg.api.ndarray.INDArray input, boolean train)
public java.util.List<org.nd4j.linalg.api.ndarray.INDArray> feedForward(boolean train)
public java.util.List<org.nd4j.linalg.api.ndarray.INDArray> feedForwardToLayer(int layerNum, org.nd4j.linalg.api.ndarray.INDArray input)
layerNum
- Index of the last layer to calculate activations for. Layers are zero-indexed.
feedForwardToLayer(i,input) will return the activations for layers 0..i (inclusive)input
- Input to the networkpublic java.util.List<org.nd4j.linalg.api.ndarray.INDArray> feedForwardToLayer(int layerNum, org.nd4j.linalg.api.ndarray.INDArray input, boolean train)
layerNum
- Index of the last layer to calculate activations for. Layers are zero-indexed.
feedForwardToLayer(i,input) will return the activations for layers 0..i (inclusive)input
- Input to the networktrain
- true for training, false for test (i.e., false if using network after training)public java.util.List<org.nd4j.linalg.api.ndarray.INDArray> feedForwardToLayer(int layerNum, boolean train)
layerNum
- Index of the last layer to calculate activations for. Layers are zero-indexed.
feedForwardToLayer(i,input) will return the activations for layers 0..i (inclusive)train
- true for training, false for test (i.e., false if using network after training)public java.util.List<org.nd4j.linalg.api.ndarray.INDArray> feedForward()
public java.util.List<org.nd4j.linalg.api.ndarray.INDArray> feedForward(org.nd4j.linalg.api.ndarray.INDArray input)
public java.util.List<org.nd4j.linalg.api.ndarray.INDArray> feedForward(org.nd4j.linalg.api.ndarray.INDArray input, org.nd4j.linalg.api.ndarray.INDArray featuresMask, org.nd4j.linalg.api.ndarray.INDArray labelsMask)
public org.nd4j.linalg.api.ndarray.INDArray epsilon()
public Pair<Gradient,java.lang.Double> gradientAndScore()
Model
gradientAndScore
in interface Model
public MultiLayerNetwork clone()
Layer
public org.nd4j.linalg.api.ndarray.INDArray params(boolean backwardOnly)
public org.nd4j.linalg.api.ndarray.INDArray params()
public void setParams(org.nd4j.linalg.api.ndarray.INDArray params)
public void setParamsViewArray(org.nd4j.linalg.api.ndarray.INDArray params)
Model
setParamsViewArray
in interface Model
params
- a 1 x nParams row vector that is a view of the larger (MLN/CG) parameters arraypublic void setBackpropGradientsViewArray(org.nd4j.linalg.api.ndarray.INDArray gradients)
Model
setBackpropGradientsViewArray
in interface Model
gradients
- a 1 x nParams row vector that is a view of the larger (MLN/CG) gradients arraypublic int numParams()
public int numParams(boolean backwards)
Model
public double f1Score(org.nd4j.linalg.dataset.api.DataSet data)
f1Score
in interface Classifier
data
- the data to scorepublic void fit(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator)
Classifier
fit
in interface Classifier
iterator
- the iterator to train onprotected void backprop()
protected Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray> calcBackpropGradients(org.nd4j.linalg.api.ndarray.INDArray epsilon, boolean withOutputLayer)
epsilon
- Errors (technically errors .* activations). Not used if withOutputLayer = truewithOutputLayer
- if true: assume last layer is output layer, and calculate errors based on labels. In this
case, the epsilon input is not used (may/should be null).
If false: calculate backprop gradientsprotected void doTruncatedBPTT(org.nd4j.linalg.api.ndarray.INDArray input, org.nd4j.linalg.api.ndarray.INDArray labels, org.nd4j.linalg.api.ndarray.INDArray featuresMaskArray, org.nd4j.linalg.api.ndarray.INDArray labelsMaskArray)
public void updateRnnStateWithTBPTTState()
protected void truncatedBPTTGradient()
public java.util.Collection<IterationListener> getListeners()
Layer
getListeners
in interface Layer
public void setListeners(java.util.Collection<IterationListener> listeners)
Model
setListeners
in interface Layer
setListeners
in interface Model
public void setListeners(IterationListener... listeners)
Model
setListeners
in interface Layer
setListeners
in interface Model
public void finetune()
public int[] predict(org.nd4j.linalg.api.ndarray.INDArray d)
predict
in interface Classifier
d
- the matrix to predictpublic java.util.List<java.lang.String> predict(org.nd4j.linalg.dataset.api.DataSet dataSet)
predict
in interface Classifier
dataSet
- to predictpublic org.nd4j.linalg.api.ndarray.INDArray labelProbabilities(org.nd4j.linalg.api.ndarray.INDArray examples)
labelProbabilities
in interface Classifier
examples
- the examples to classify (one example in each row)public void fit(org.nd4j.linalg.api.ndarray.INDArray data, org.nd4j.linalg.api.ndarray.INDArray labels)
fit
in interface Classifier
data
- the examples to classify (one example in each row)labels
- the example labels(a binary outcome matrix)public void fit(org.nd4j.linalg.api.ndarray.INDArray features, org.nd4j.linalg.api.ndarray.INDArray labels, org.nd4j.linalg.api.ndarray.INDArray featuresMask, org.nd4j.linalg.api.ndarray.INDArray labelsMask)
features
- the examples to classify (one example in each row)labels
- the example labels(a binary outcome matrix)featuresMask
- The mask array for the features (used for variable length time series, etc). May be null.labelsMask
- The mask array for the labels (used for variable length time series, etc). May be null.public void fit(org.nd4j.linalg.api.ndarray.INDArray data)
public void iterate(org.nd4j.linalg.api.ndarray.INDArray input)
Model
public void fit(org.nd4j.linalg.dataset.api.DataSet data)
fit
in interface Classifier
data
- the data to train onpublic void fit(org.nd4j.linalg.api.ndarray.INDArray examples, int[] labels)
fit
in interface Classifier
examples
- the examples to classify (one example in each row)labels
- the labels for each example (the number of labels must matchpublic org.nd4j.linalg.api.ndarray.INDArray output(org.nd4j.linalg.api.ndarray.INDArray input, Layer.TrainingMode train)
input
- the input to labeltrain
- whether the output
is test or train. This mainly
affect hyper parameters such as
drop out where certain things should
be applied with activationsThis is typically of the form: [0.5, 0.5] or some other probability distribution summing to one
public org.nd4j.linalg.api.ndarray.INDArray output(org.nd4j.linalg.api.ndarray.INDArray input, boolean train)
input
- the input to labeltrain
- whether the output
is test or train. This mainly
affect hyper parameters such as
drop out where certain things should
be applied with activationsThis is typically of the form: [0.5, 0.5] or some other probability distribution summing to one
public org.nd4j.linalg.api.ndarray.INDArray output(org.nd4j.linalg.api.ndarray.INDArray input, boolean train, org.nd4j.linalg.api.ndarray.INDArray featuresMask, org.nd4j.linalg.api.ndarray.INDArray labelsMask)
public org.nd4j.linalg.api.ndarray.INDArray output(org.nd4j.linalg.api.ndarray.INDArray input)
input
- the input to labelThis is typically of the form: [0.5, 0.5] or some other probability distribution summing to one
public org.nd4j.linalg.api.ndarray.INDArray output(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator, boolean train)
iterator
- test data to evaluateThis is typically of the form: [0.5, 0.5] or some other probability distribution summing to one
public org.nd4j.linalg.api.ndarray.INDArray output(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator)
public org.nd4j.linalg.api.ndarray.INDArray reconstruct(org.nd4j.linalg.api.ndarray.INDArray x, int layerNum)
x
- the input to transformlayerNum
- the layer to output for encodingThis is typically of the form: [0.5, 0.5] or some other probability distribution summing to one
public void printConfiguration()
public void update(MultiLayerNetwork network)
network
- the network to getFromOrigin parameters frompublic double f1Score(org.nd4j.linalg.api.ndarray.INDArray input, org.nd4j.linalg.api.ndarray.INDArray labels)
f1Score
in interface Classifier
input
- the input to scorelabels
- the true labelspublic int numLabels()
numLabels
in interface Classifier
public double score(org.nd4j.linalg.dataset.DataSet data)
score(DataSet, boolean)
with training==true.data
- the data to scorescore(DataSet, boolean)
public double score(org.nd4j.linalg.dataset.DataSet data, boolean training)
data
- data to calculate score fortraining
- If true: score during training. If false: score at test time. This can affect the application of
certain features, such as dropout and dropconnect (which are applied at training time only)public org.nd4j.linalg.api.ndarray.INDArray scoreExamples(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iter, boolean addRegularizationTerms)
public org.nd4j.linalg.api.ndarray.INDArray scoreExamples(org.nd4j.linalg.dataset.DataSet data, boolean addRegularizationTerms)
score(DataSet)
and score(DataSet, boolean)
this method does not average/sum over examples. This method allows for examples to be scored individually (at test time only), which
may be useful for example for autoencoder architectures and the like.data
- The data to scoreaddRegularizationTerms
- If true: add l1/l2 regularization terms (if any) to the score. If false: don't add regularization termspublic void fit()
Model
public void update(org.nd4j.linalg.api.ndarray.INDArray gradient, java.lang.String paramType)
Model
public double score()
public void setScore(double score)
public void computeGradientAndScore()
Model
computeGradientAndScore
in interface Model
public void accumulateScore(double accum)
Model
accumulateScore
in interface Model
accum
- the amount to accumpublic void clear()
@Deprecated public void merge(Layer layer, int batchSize)
@Deprecated public void merge(MultiLayerNetwork network, int batchSize)
network
- the network to merge withbatchSize
- the batch size (number of training examples)
to average bypublic void setInput(org.nd4j.linalg.api.ndarray.INDArray input)
public Layer getOutputLayer()
public void setParameters(org.nd4j.linalg.api.ndarray.INDArray params)
params
- a parameter vector equal 1,numParameterspublic void applyLearningRateScoreDecay()
Model
applyLearningRateScoreDecay
in interface Model
public NeuralNetConfiguration getDefaultConfiguration()
public org.nd4j.linalg.api.ndarray.INDArray getLabels()
public org.nd4j.linalg.api.ndarray.INDArray getInput()
public void setLabels(org.nd4j.linalg.api.ndarray.INDArray labels)
labels
- public int getnLayers()
public Layer[] getLayers()
public Layer getLayer(int i)
public Layer getLayer(java.lang.String name)
public java.util.List<java.lang.String> getLayerNames()
public void setLayers(Layer[] layers)
public org.nd4j.linalg.api.ndarray.INDArray getMask()
public void setMask(org.nd4j.linalg.api.ndarray.INDArray mask)
public org.nd4j.linalg.api.ndarray.INDArray getMaskArray()
getMaskArray
in interface Layer
public boolean isPretrainLayer()
Layer
isPretrainLayer
in interface Layer
public Pair<org.nd4j.linalg.api.ndarray.INDArray,MaskState> feedForwardMaskArray(org.nd4j.linalg.api.ndarray.INDArray maskArray, MaskState currentMaskState, int minibatchSize)
Layer
feedForwardMaskArray
in interface Layer
maskArray
- Mask array to setcurrentMaskState
- Current state of the mask - see MaskState
minibatchSize
- Current minibatch size. Needs to be known as it cannot always be inferred from the activations
array due to reshaping (such as a DenseLayer within a recurrent neural network)public Gradient error(org.nd4j.linalg.api.ndarray.INDArray errorSignal)
Layer
public Layer.Type type()
Layer
public org.nd4j.linalg.api.ndarray.INDArray derivativeActivation(org.nd4j.linalg.api.ndarray.INDArray input)
Layer
derivativeActivation
in interface Layer
input
- the input to take the derivative ofpublic Gradient calcGradient(Gradient layerError, org.nd4j.linalg.api.ndarray.INDArray activation)
Layer
calcGradient
in interface Layer
layerError
- the layer errorpublic org.nd4j.linalg.api.ndarray.INDArray preOutput(org.nd4j.linalg.api.ndarray.INDArray x)
Layer
public org.nd4j.linalg.api.ndarray.INDArray preOutput(org.nd4j.linalg.api.ndarray.INDArray x, Layer.TrainingMode training)
Layer
public org.nd4j.linalg.api.ndarray.INDArray activate(Layer.TrainingMode training)
Layer
public org.nd4j.linalg.api.ndarray.INDArray activate(org.nd4j.linalg.api.ndarray.INDArray input, Layer.TrainingMode training)
Layer
public Layer transpose()
Layer
public Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray> backpropGradient(org.nd4j.linalg.api.ndarray.INDArray epsilon)
Layer
backpropGradient
in interface Layer
epsilon
- w^(L+1)*delta^(L+1). Or, equiv: dC/da, i.e., (dC/dz)*(dz/da) = dC/da, where C
is cost function a=sigma(z) is activation.public void setIndex(int index)
Layer
public int getIndex()
Layer
public double calcL2(boolean backpropParamsOnly)
Layer
public double calcL1(boolean backpropParamsOnly)
Layer
public void update(Gradient gradient)
Model
public org.nd4j.linalg.api.ndarray.INDArray preOutput(org.nd4j.linalg.api.ndarray.INDArray x, boolean training)
Layer
public org.nd4j.linalg.api.ndarray.INDArray activate(boolean training)
Layer
public org.nd4j.linalg.api.ndarray.INDArray activate(org.nd4j.linalg.api.ndarray.INDArray input, boolean training)
Layer
public void setInputMiniBatchSize(int size)
Layer
setInputMiniBatchSize
in interface Layer
public int getInputMiniBatchSize()
Layer
getInputMiniBatchSize
in interface Layer
Layer.setInputMiniBatchSize(int)
public void setMaskArray(org.nd4j.linalg.api.ndarray.INDArray maskArray)
Layer
Layer.feedForwardMaskArray(INDArray, MaskState, int)
should be used in
preference to this.setMaskArray
in interface Layer
maskArray
- Mask array to setpublic org.nd4j.linalg.api.ndarray.INDArray rnnTimeStep(org.nd4j.linalg.api.ndarray.INDArray input)
input
- Input to network. May be for one or multiple time steps. For single time step:
input has shape [miniBatchSize,inputSize] or [miniBatchSize,inputSize,1]. miniBatchSize=1 for single example.public java.util.Map<java.lang.String,org.nd4j.linalg.api.ndarray.INDArray> rnnGetPreviousState(int layer)
layer
- Number/index of the layer.public void rnnSetPreviousState(int layer, java.util.Map<java.lang.String,org.nd4j.linalg.api.ndarray.INDArray> state)
layer
- The number/index of the layer.state
- The state to set the specified layer topublic void rnnClearPreviousState()
public java.util.List<org.nd4j.linalg.api.ndarray.INDArray> rnnActivateUsingStoredState(org.nd4j.linalg.api.ndarray.INDArray input, boolean training, boolean storeLastForTBPTT)
input
- Input to networktraining
- Whether training or notstoreLastForTBPTT
- set to true if used as part of truncated BPTT trainingpublic Updater getUpdater()
public void setUpdater(Updater updater)
public void setLayerMaskArrays(org.nd4j.linalg.api.ndarray.INDArray featuresMaskArray, org.nd4j.linalg.api.ndarray.INDArray labelsMaskArray)
feedForward(INDArray, INDArray, INDArray)
and output(INDArray, boolean, INDArray, INDArray)
handle setting of masking internally.featuresMaskArray
- Mask array for features (input)labelsMaskArray
- Mask array for labels (output)clearLayerMaskArrays()
public void clearLayerMaskArrays()
setLayerMaskArrays(INDArray, INDArray)
for details on mask arrays.public Evaluation evaluate(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator)
iterator
- Iterator to evaluate onpublic RegressionEvaluation evaluateRegression(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator)
iterator
- Data to evaluate onpublic ROC evaluateROC(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator, int rocThresholdSteps)
ROC
classiterator
- Data to evaluate onrocThresholdSteps
- Number of threshold steps to use with ROC
public ROCMultiClass evaluateROCMultiClass(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator, int rocThresholdSteps)
ROCMultiClass
classiterator
- Data to evaluate onrocThresholdSteps
- Number of threshold steps to use with ROCMultiClass
public <T extends IEvaluation> T doEvaluation(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator, T evaluation)
iterator
- data to evaluate onevaluation
- IEvaluation instance to perform evaluation withpublic Evaluation evaluate(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator, java.util.List<java.lang.String> labelsList)
iterator
- Data to undertake evaluation onpublic Evaluation evaluate(org.nd4j.linalg.dataset.api.iterator.DataSetIterator iterator, java.util.List<java.lang.String> labelsList, int topN)
iterator
- Iterator (data) to evaluate onlabelsList
- List of labels. May be null.topN
- N value for top N accuracy evaluationpublic java.lang.String summary()