public class GravesBidirectionalLSTM extends BaseRecurrentLayer<GravesBidirectionalLSTM>
Layer.TrainingMode, Layer.Type
stateMap, tBpttStateMap
conf, dropoutApplied, dropoutMask, gradient, gradientsFlattened, gradientViews, index, input, iterationListeners, maskArray, maskState, optimizer, params, paramsFlattened, score, solver
Constructor and Description |
---|
GravesBidirectionalLSTM(NeuralNetConfiguration conf) |
GravesBidirectionalLSTM(NeuralNetConfiguration conf,
org.nd4j.linalg.api.ndarray.INDArray input) |
Modifier and Type | Method and Description |
---|---|
org.nd4j.linalg.api.ndarray.INDArray |
activate()
Trigger an activation with the last specified input
|
org.nd4j.linalg.api.ndarray.INDArray |
activate(boolean training)
Trigger an activation with the last specified input
|
org.nd4j.linalg.api.ndarray.INDArray |
activate(org.nd4j.linalg.api.ndarray.INDArray input)
Initialize the layer with the given input
and return the activation for this layer
given this input
|
org.nd4j.linalg.api.ndarray.INDArray |
activate(org.nd4j.linalg.api.ndarray.INDArray input,
boolean training)
Initialize the layer with the given input
and return the activation for this layer
given this input
|
org.nd4j.linalg.api.ndarray.INDArray |
activationMean()
Calculate the mean representation
for the activation for this layer
|
Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray> |
backpropGradient(org.nd4j.linalg.api.ndarray.INDArray epsilon)
Calculate the gradient relative to the error in the next layer
|
Gradient |
calcGradient(Gradient layerError,
org.nd4j.linalg.api.ndarray.INDArray activation)
Calculate the gradient
|
double |
calcL1(boolean backpropParamsOnly)
Calculate the l1 regularization term
0.0 if regularization is not used. |
double |
calcL2(boolean backpropParamsOnly)
Calculate the l2 regularization term
0.0 if regularization is not used. |
Pair<org.nd4j.linalg.api.ndarray.INDArray,MaskState> |
feedForwardMaskArray(org.nd4j.linalg.api.ndarray.INDArray maskArray,
MaskState currentMaskState,
int minibatchSize)
Feed forward the input mask array, setting in in the layer as appropriate.
|
Gradient |
gradient()
Calculate a gradient
|
boolean |
isPretrainLayer()
Returns true if the layer can be trained in an unsupervised/pretrain manner (VAE, RBMs etc)
|
org.nd4j.linalg.api.ndarray.INDArray |
preOutput(org.nd4j.linalg.api.ndarray.INDArray x)
Classify input
|
org.nd4j.linalg.api.ndarray.INDArray |
preOutput(org.nd4j.linalg.api.ndarray.INDArray x,
boolean training)
Raw activations
|
org.nd4j.linalg.api.ndarray.INDArray |
rnnActivateUsingStoredState(org.nd4j.linalg.api.ndarray.INDArray input,
boolean training,
boolean storeLastForTBPTT)
Similar to rnnTimeStep, this method is used for activations using the state
stored in the stateMap as the initialization.
|
org.nd4j.linalg.api.ndarray.INDArray |
rnnTimeStep(org.nd4j.linalg.api.ndarray.INDArray input)
Do one or more time steps using the previous time step state stored in stateMap.
Can be used to efficiently do forward pass one or n-steps at a time (instead of doing forward pass always from t=0) If stateMap is empty, default initialization (usually zeros) is used Implementations also update stateMap at the end of this method |
Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray> |
tbpttBackpropGradient(org.nd4j.linalg.api.ndarray.INDArray epsilon,
int tbpttBackwardLength)
Truncated BPTT equivalent of Layer.backpropGradient().
|
Layer |
transpose()
Return a transposed copy of the weights/bias
(this means reverse the number of inputs and outputs on the weights)
|
Layer.Type |
type()
Returns the layer type
|
rnnClearPreviousState, rnnGetPreviousState, rnnGetTBPTTState, rnnSetPreviousState, rnnSetTBPTTState
accumulateScore, activate, activate, applyDropOutIfNecessary, applyLearningRateScoreDecay, applyMask, batchSize, clear, clone, computeGradientAndScore, conf, createGradient, derivativeActivation, error, fit, fit, getIndex, getInput, getInputMiniBatchSize, getListeners, getMaskArray, getOptimizer, getParam, gradientAndScore, init, initParams, input, iterate, layerConf, layerNameAndIndex, merge, numParams, numParams, params, paramTable, paramTable, preOutput, preOutput, score, setBackpropGradientsViewArray, setConf, setIndex, setInput, setInputMiniBatchSize, setListeners, setListeners, setMaskArray, setParam, setParams, setParams, setParamsViewArray, setParamTable, setScoreWithZ, toString, update, update, validateInput
equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
activate, activate, clone, derivativeActivation, error, getIndex, getInputMiniBatchSize, getListeners, getMaskArray, merge, preOutput, setIndex, setInput, setInputMiniBatchSize, setListeners, setListeners, setMaskArray
accumulateScore, applyLearningRateScoreDecay, batchSize, clear, computeGradientAndScore, conf, fit, fit, getOptimizer, getParam, gradientAndScore, init, initParams, input, iterate, numParams, numParams, params, paramTable, paramTable, score, setBackpropGradientsViewArray, setConf, setParam, setParams, setParamsViewArray, setParamTable, update, update, validateInput
public GravesBidirectionalLSTM(NeuralNetConfiguration conf)
public GravesBidirectionalLSTM(NeuralNetConfiguration conf, org.nd4j.linalg.api.ndarray.INDArray input)
public Gradient gradient()
Model
gradient
in interface Model
gradient
in class BaseLayer<GravesBidirectionalLSTM>
public Gradient calcGradient(Gradient layerError, org.nd4j.linalg.api.ndarray.INDArray activation)
Layer
calcGradient
in interface Layer
calcGradient
in class BaseLayer<GravesBidirectionalLSTM>
layerError
- the layer errorpublic Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray> backpropGradient(org.nd4j.linalg.api.ndarray.INDArray epsilon)
Layer
backpropGradient
in interface Layer
backpropGradient
in class BaseLayer<GravesBidirectionalLSTM>
epsilon
- w^(L+1)*delta^(L+1). Or, equiv: dC/da, i.e., (dC/dz)*(dz/da) = dC/da, where C
is cost function a=sigma(z) is activation.public Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray> tbpttBackpropGradient(org.nd4j.linalg.api.ndarray.INDArray epsilon, int tbpttBackwardLength)
RecurrentLayer
public org.nd4j.linalg.api.ndarray.INDArray preOutput(org.nd4j.linalg.api.ndarray.INDArray x)
BaseLayer
preOutput
in interface Layer
preOutput
in class BaseLayer<GravesBidirectionalLSTM>
x
- the input (can either be a matrix or vector)
If it's a matrix, each row is considered an example
and associated rows are classified accordingly.
Each row will be the likelihood of a label given that examplepublic org.nd4j.linalg.api.ndarray.INDArray preOutput(org.nd4j.linalg.api.ndarray.INDArray x, boolean training)
Layer
preOutput
in interface Layer
preOutput
in class BaseLayer<GravesBidirectionalLSTM>
x
- the input to transformpublic org.nd4j.linalg.api.ndarray.INDArray activate(org.nd4j.linalg.api.ndarray.INDArray input, boolean training)
Layer
activate
in interface Layer
activate
in class BaseLayer<GravesBidirectionalLSTM>
input
- the input to usetraining
- train or test modepublic org.nd4j.linalg.api.ndarray.INDArray activate(org.nd4j.linalg.api.ndarray.INDArray input)
Layer
activate
in interface Layer
activate
in class BaseLayer<GravesBidirectionalLSTM>
input
- the input to usepublic org.nd4j.linalg.api.ndarray.INDArray activate(boolean training)
Layer
activate
in interface Layer
activate
in class BaseLayer<GravesBidirectionalLSTM>
training
- training or test modepublic org.nd4j.linalg.api.ndarray.INDArray activate()
Layer
activate
in interface Layer
activate
in class BaseLayer<GravesBidirectionalLSTM>
public org.nd4j.linalg.api.ndarray.INDArray activationMean()
Layer
activationMean
in interface Layer
activationMean
in class BaseLayer<GravesBidirectionalLSTM>
public Layer.Type type()
Layer
type
in interface Layer
type
in class BaseLayer<GravesBidirectionalLSTM>
public Layer transpose()
Layer
transpose
in interface Layer
transpose
in class BaseLayer<GravesBidirectionalLSTM>
public boolean isPretrainLayer()
Layer
public double calcL2(boolean backpropParamsOnly)
Layer
calcL2
in interface Layer
calcL2
in class BaseLayer<GravesBidirectionalLSTM>
backpropParamsOnly
- If true: calculate L2 based on backprop params only. If false: calculate
based on all params (including pretrain params, if any)public double calcL1(boolean backpropParamsOnly)
Layer
calcL1
in interface Layer
calcL1
in class BaseLayer<GravesBidirectionalLSTM>
backpropParamsOnly
- If true: calculate L1 based on backprop params only. If false: calculate
based on all params (including pretrain params, if any)public org.nd4j.linalg.api.ndarray.INDArray rnnTimeStep(org.nd4j.linalg.api.ndarray.INDArray input)
RecurrentLayer
input
- Input to this layerpublic org.nd4j.linalg.api.ndarray.INDArray rnnActivateUsingStoredState(org.nd4j.linalg.api.ndarray.INDArray input, boolean training, boolean storeLastForTBPTT)
RecurrentLayer
input
- Layer inputtraining
- if true: training. Otherwise: teststoreLastForTBPTT
- If true: store the final state in tBpttStateMap for use in truncated BPTT trainingpublic Pair<org.nd4j.linalg.api.ndarray.INDArray,MaskState> feedForwardMaskArray(org.nd4j.linalg.api.ndarray.INDArray maskArray, MaskState currentMaskState, int minibatchSize)
Layer
feedForwardMaskArray
in interface Layer
feedForwardMaskArray
in class BaseLayer<GravesBidirectionalLSTM>
maskArray
- Mask array to setcurrentMaskState
- Current state of the mask - see MaskState
minibatchSize
- Current minibatch size. Needs to be known as it cannot always be inferred from the activations
array due to reshaping (such as a DenseLayer within a recurrent neural network)