org.deeplearning4j.nn.api

Interface Layer

All Superinterfaces:

java.lang.Cloneable, Model, java.io.Serializable

All Known Subinterfaces:

IOutputLayer, RecurrentLayer

All Known Implementing Classes:

ActivationLayer, AutoEncoder, BaseLayer, BaseOutputLayer, BasePretrainNetwork, BaseRecurrentLayer, BatchNormalization, CenterLossOutputLayer, Convolution1DLayer, ConvolutionLayer, CustomLayerImpl, CustomLayerImpl, CustomOutputLayerImpl, DenseLayer, DropoutLayer, EmbeddingLayer, FrozenLayer, GlobalPoolingLayer, GravesBidirectionalLSTM, GravesLSTM, LocalResponseNormalization, LossLayer, MultiLayerNetwork, OutputLayer, RBM, RnnOutputLayer, Subsampling1DLayer, SubsamplingLayer, VariationalAutoencoder, ZeroPaddingLayer

public interface Layer
extends java.io.Serializable, java.lang.Cloneable, Model

Interface for a layer of a neural network. This has an activation function, an input and output size, weights, and a bias

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	Layer.TrainingMode
static class	Layer.Type

Method Summary

Modifier and Type	Method and Description
org.nd4j.linalg.api.ndarray.INDArray	activate() Trigger an activation with the last specified input
org.nd4j.linalg.api.ndarray.INDArray	activate(boolean training) Trigger an activation with the last specified input
org.nd4j.linalg.api.ndarray.INDArray	activate(org.nd4j.linalg.api.ndarray.INDArray input) Initialize the layer with the given input and return the activation for this layer given this input
org.nd4j.linalg.api.ndarray.INDArray	activate(org.nd4j.linalg.api.ndarray.INDArray input, boolean training) Initialize the layer with the given input and return the activation for this layer given this input
org.nd4j.linalg.api.ndarray.INDArray	activate(org.nd4j.linalg.api.ndarray.INDArray input, Layer.TrainingMode training) Initialize the layer with the given input and return the activation for this layer given this input
org.nd4j.linalg.api.ndarray.INDArray	activate(Layer.TrainingMode training) Trigger an activation with the last specified input
org.nd4j.linalg.api.ndarray.INDArray	activationMean() Deprecated. As of 0.7.3 - Feb 2017. No longer used.
Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray>	backpropGradient(org.nd4j.linalg.api.ndarray.INDArray epsilon) Calculate the gradient relative to the error in the next layer
Gradient	calcGradient(Gradient layerError, org.nd4j.linalg.api.ndarray.INDArray indArray) Deprecated. As of 0.7.3 - Feb 2017. No longer used.
double	calcL1(boolean backpropOnlyParams) Calculate the l1 regularization term 0.0 if regularization is not used.
double	calcL2(boolean backpropOnlyParams) Calculate the l2 regularization term 0.0 if regularization is not used.
Layer	clone() Clone the layer
org.nd4j.linalg.api.ndarray.INDArray	derivativeActivation(org.nd4j.linalg.api.ndarray.INDArray input) Deprecated. As of 0.7.3 - Feb 2017. No longer used.
Gradient	error(org.nd4j.linalg.api.ndarray.INDArray input) Deprecated. As of 0.7.3 - Feb 2017. No longer used.
Pair<org.nd4j.linalg.api.ndarray.INDArray,MaskState>	feedForwardMaskArray(org.nd4j.linalg.api.ndarray.INDArray maskArray, MaskState currentMaskState, int minibatchSize) Feed forward the input mask array, setting in in the layer as appropriate.
int	getIndex() Get the layer index.
int	getInputMiniBatchSize() Get current/last input mini-batch size, as set by setInputMiniBatchSize(int)
java.util.Collection<IterationListener>	getListeners() Get the iteration listeners for this layer.
org.nd4j.linalg.api.ndarray.INDArray	getMaskArray()
boolean	isPretrainLayer() Returns true if the layer can be trained in an unsupervised/pretrain manner (VAE, RBMs etc)
void	merge(Layer layer, int batchSize) Deprecated. As of 0.7.3 - Feb 2017. No longer used. Merging (for parameter averaging) done via alternative means
org.nd4j.linalg.api.ndarray.INDArray	preOutput(org.nd4j.linalg.api.ndarray.INDArray x) Raw activations
org.nd4j.linalg.api.ndarray.INDArray	preOutput(org.nd4j.linalg.api.ndarray.INDArray x, boolean training) Raw activations
org.nd4j.linalg.api.ndarray.INDArray	preOutput(org.nd4j.linalg.api.ndarray.INDArray x, Layer.TrainingMode training) Raw activations
void	setIndex(int index) Set the layer index.
void	setInput(org.nd4j.linalg.api.ndarray.INDArray input) Get the layer input.
void	setInputMiniBatchSize(int size) Set current/last input mini-batch size. Used for score and gradient calculations.
void	setListeners(java.util.Collection<IterationListener> listeners) Set the iteration listeners for this layer.
void	setListeners(IterationListener... listeners) Set the iteration listeners for this layer.
void	setMaskArray(org.nd4j.linalg.api.ndarray.INDArray maskArray) Set the mask array.
Layer	transpose() Return a transposed copy of the weights/bias (this means reverse the number of inputs and outputs on the weights)
Layer.Type	type() Returns the layer type

Methods inherited from interface org.deeplearning4j.nn.api.Model

accumulateScore, applyLearningRateScoreDecay, batchSize, clear, computeGradientAndScore, conf, fit, fit, getOptimizer, getParam, gradient, gradientAndScore, init, initParams, input, iterate, numParams, numParams, params, paramTable, paramTable, score, setBackpropGradientsViewArray, setConf, setParam, setParams, setParamsViewArray, setParamTable, update, update, validateInput

Method Detail

calcL2

double calcL2(boolean backpropOnlyParams)

Calculate the l2 regularization term
0.0 if regularization is not used. Or 0.5 * l2Coeff * l2Magnitude otherwise.
Note that this does not divide by mini-batch size

Parameters:

backpropOnlyParams - If true: calculate L2 based on backprop params only. If false: calculate based on all params (including pretrain params, if any)

Returns:

the l2 regularization term for this layer.

calcL1

double calcL1(boolean backpropOnlyParams)

Calculate the l1 regularization term
0.0 if regularization is not used. Or l1Coeff * l1Magnitude otherwise.
Note that this does not divide by mini-batch size

Parameters:

backpropOnlyParams - If true: calculate L1 based on backprop params only. If false: calculate based on all params (including pretrain params, if any)

Returns:

the l1 regularization term for this layer.

type

Layer.Type type()

Returns the layer type

Returns:

error

@Deprecated
Gradient error(org.nd4j.linalg.api.ndarray.INDArray input)

Deprecated. As of 0.7.3 - Feb 2017. No longer used.

Calculate error with respect to the current layer. This gradient will contain the error signal

Parameters:

input - the gradient for the forward layer If this is the final layer, it will start with the error from the output. This is on the user to initialize.

Returns:

the gradient wrt the parameters on the current layer

derivativeActivation

@Deprecated
org.nd4j.linalg.api.ndarray.INDArray derivativeActivation(org.nd4j.linalg.api.ndarray.INDArray input)

Deprecated. As of 0.7.3 - Feb 2017. No longer used.

Take the derivative of the given input based on the activation

Parameters:

input - the input to take the derivative of

Returns:

the derivative of the action

calcGradient

@Deprecated
Gradient calcGradient(Gradient layerError,
                                  org.nd4j.linalg.api.ndarray.INDArray indArray)

Deprecated. As of 0.7.3 - Feb 2017. No longer used.

Calculate the gradient

Parameters:

layerError - the layer error

indArray -

Returns:

the gradient

backpropGradient

Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray> backpropGradient(org.nd4j.linalg.api.ndarray.INDArray epsilon)

Calculate the gradient relative to the error in the next layer

Parameters:

epsilon - w^(L+1)*delta^(L+1). Or, equiv: dC/da, i.e., (dC/dz)*(dz/da) = dC/da, where C is cost function a=sigma(z) is activation.

Returns:

Pair where Gradient is gradient for this layer, INDArray is epsilon needed by next layer, but before element-wise multiply by sigmaPrime(z). So for standard feed-forward layer, if this layer is L, then return.getSecond() == (w^(L)*(delta^(L))^T)^T

merge

@Deprecated
void merge(Layer layer,
                       int batchSize)

Deprecated. As of 0.7.3 - Feb 2017. No longer used. Merging (for parameter averaging) done via alternative means

Parameter averaging

Parameters:

layer - the layer to merge

batchSize - the batch size to merge on

activationMean

@Deprecated
org.nd4j.linalg.api.ndarray.INDArray activationMean()

Deprecated. As of 0.7.3 - Feb 2017. No longer used.

Calculate the mean representation for the activation for this layer

Returns:

the activation mean for this layer

preOutput

org.nd4j.linalg.api.ndarray.INDArray preOutput(org.nd4j.linalg.api.ndarray.INDArray x)

Raw activations

Parameters:

x - the input to transform

Returns:

the raw activation for this layer

preOutput

org.nd4j.linalg.api.ndarray.INDArray preOutput(org.nd4j.linalg.api.ndarray.INDArray x,
                                               Layer.TrainingMode training)

Raw activations

Parameters:

x - the input to transform

Returns:

the raw activation for this layer

activate

org.nd4j.linalg.api.ndarray.INDArray activate(Layer.TrainingMode training)

Trigger an activation with the last specified input

Parameters:

training - training or test mode

Returns:

the activation of the last specified input

activate

org.nd4j.linalg.api.ndarray.INDArray activate(org.nd4j.linalg.api.ndarray.INDArray input,
                                              Layer.TrainingMode training)

Initialize the layer with the given input and return the activation for this layer given this input

Parameters:

input - the input to use

training - train or test mode

Returns:

preOutput

org.nd4j.linalg.api.ndarray.INDArray preOutput(org.nd4j.linalg.api.ndarray.INDArray x,
                                               boolean training)

Raw activations

Parameters:

x - the input to transform

Returns:

the raw activation for this layer

activate

org.nd4j.linalg.api.ndarray.INDArray activate(boolean training)

Trigger an activation with the last specified input

Parameters:

training - training or test mode

Returns:

the activation of the last specified input

activate

org.nd4j.linalg.api.ndarray.INDArray activate(org.nd4j.linalg.api.ndarray.INDArray input,
                                              boolean training)

Initialize the layer with the given input and return the activation for this layer given this input

Parameters:

input - the input to use

training - train or test mode

Returns:

activate

org.nd4j.linalg.api.ndarray.INDArray activate()

Trigger an activation with the last specified input

Returns:

the activation of the last specified input

activate

org.nd4j.linalg.api.ndarray.INDArray activate(org.nd4j.linalg.api.ndarray.INDArray input)

Initialize the layer with the given input and return the activation for this layer given this input

Parameters:

input - the input to use

Returns:

transpose

Layer transpose()

Return a transposed copy of the weights/bias (this means reverse the number of inputs and outputs on the weights)

Returns:

the transposed layer

clone

Layer clone()

Clone the layer

Returns:

getListeners

java.util.Collection<IterationListener> getListeners()

Get the iteration listeners for this layer.

setListeners

void setListeners(IterationListener... listeners)

Set the iteration listeners for this layer.

Specified by:

setListeners in interface Model

setListeners

void setListeners(java.util.Collection<IterationListener> listeners)

Set the iteration listeners for this layer.

Specified by:

setListeners in interface Model

setIndex

void setIndex(int index)

Set the layer index.

getIndex

int getIndex()

Get the layer index.

setInput

void setInput(org.nd4j.linalg.api.ndarray.INDArray input)

Get the layer input.

setInputMiniBatchSize

void setInputMiniBatchSize(int size)

Set current/last input mini-batch size.
Used for score and gradient calculations. Mini batch size may be different from getInput().size(0) due to reshaping operations - for example, when using RNNs with DenseLayer and OutputLayer. Called automatically during forward pass.

getInputMiniBatchSize

int getInputMiniBatchSize()

Get current/last input mini-batch size, as set by setInputMiniBatchSize(int)

See Also:

setInputMiniBatchSize(int)

setMaskArray

void setMaskArray(org.nd4j.linalg.api.ndarray.INDArray maskArray)

Set the mask array. Note: In general, feedForwardMaskArray(INDArray, MaskState, int) should be used in preference to this.

Parameters:

maskArray - Mask array to set

getMaskArray

org.nd4j.linalg.api.ndarray.INDArray getMaskArray()

isPretrainLayer

boolean isPretrainLayer()

Returns true if the layer can be trained in an unsupervised/pretrain manner (VAE, RBMs etc)

Returns:

true if the layer can be pretrained (using fit(INDArray), false otherwise

feedForwardMaskArray

Pair<org.nd4j.linalg.api.ndarray.INDArray,MaskState> feedForwardMaskArray(org.nd4j.linalg.api.ndarray.INDArray maskArray,
                                                                          MaskState currentMaskState,
                                                                          int minibatchSize)

Feed forward the input mask array, setting in in the layer as appropriate. This allows different layers to handle masks differently - for example, bidirectional RNNs and normal RNNs operate differently with masks (the former sets activations to 0 outside of the data present region (and keeps the mask active for future layers like dense layers), whereas normal RNNs don't zero out the activations/errors )instead relying on backpropagated error arrays to handle the variable length case.
This is also used for example for networks that contain global pooling layers, arbitrary preprocessors, etc.

Parameters:

maskArray - Mask array to set

currentMaskState - Current state of the mask - see MaskState

minibatchSize - Current minibatch size. Needs to be known as it cannot always be inferred from the activations array due to reshaping (such as a DenseLayer within a recurrent neural network)

Returns:

New mask array after this layer, along with the new mask state.