org.deeplearning4j.nn.layers.recurrent

Class LSTMHelpers

java.lang.Object

org.deeplearning4j.nn.layers.recurrent.LSTMHelpers

public class LSTMHelpers
extends java.lang.Object

RNN tutorial: http://deeplearning4j.org/usingrnns.html READ THIS FIRST if you want to understand what the heck is happening here. Shared code for the standard "forwards" LSTM RNN and the bidirectional LSTM RNN This was extracted from GravesLSTM and refactored into static helper functions. The general reasoning for this was so we only have math in one place, instead of two. Based on Graves: Supervised Sequence Labelling with Recurrent Neural Networks http://www.cs.toronto.edu/~graves/phd.pdf See also for full/vectorized equations (and a comparison to other LSTM variants): Greff et al. 2015, "LSTM: A Search Space Odyssey", pg11. This is the "vanilla" variant in said paper http://arxiv.org/pdf/1503.04069.pdf Please note that truncated backpropagation through time (BPTT) will not work with the bidirectional layer as-is. Additionally, variable length data sets will also not work with the bidirectional layer.

Method Summary

Modifier and Type	Method and Description
static FwdPassReturn	activateHelper(Layer layer, NeuralNetConfiguration conf, org.nd4j.linalg.activations.IActivation gateActivationFn, org.nd4j.linalg.api.ndarray.INDArray input, org.nd4j.linalg.api.ndarray.INDArray recurrentWeights, org.nd4j.linalg.api.ndarray.INDArray originalInputWeights, org.nd4j.linalg.api.ndarray.INDArray biases, boolean training, org.nd4j.linalg.api.ndarray.INDArray originalPrevOutputActivations, org.nd4j.linalg.api.ndarray.INDArray originalPrevMemCellState, boolean forBackprop, boolean forwards, java.lang.String inputWeightKey, org.nd4j.linalg.api.ndarray.INDArray maskArray) Returns FwdPassReturn object with activations/INDArrays.
static Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray>	backpropGradientHelper(NeuralNetConfiguration conf, org.nd4j.linalg.activations.IActivation gateActivationFn, org.nd4j.linalg.api.ndarray.INDArray input, org.nd4j.linalg.api.ndarray.INDArray recurrentWeights, org.nd4j.linalg.api.ndarray.INDArray inputWeights, org.nd4j.linalg.api.ndarray.INDArray epsilon, boolean truncatedBPTT, int tbpttBackwardLength, FwdPassReturn fwdPass, boolean forwards, java.lang.String inputWeightKey, java.lang.String recurrentWeightKey, java.lang.String biasWeightKey, java.util.Map<java.lang.String,org.nd4j.linalg.api.ndarray.INDArray> gradientViews, org.nd4j.linalg.api.ndarray.INDArray maskArray)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

activateHelper

public static FwdPassReturn activateHelper(Layer layer,
                                           NeuralNetConfiguration conf,
                                           org.nd4j.linalg.activations.IActivation gateActivationFn,
                                           org.nd4j.linalg.api.ndarray.INDArray input,
                                           org.nd4j.linalg.api.ndarray.INDArray recurrentWeights,
                                           org.nd4j.linalg.api.ndarray.INDArray originalInputWeights,
                                           org.nd4j.linalg.api.ndarray.INDArray biases,
                                           boolean training,
                                           org.nd4j.linalg.api.ndarray.INDArray originalPrevOutputActivations,
                                           org.nd4j.linalg.api.ndarray.INDArray originalPrevMemCellState,
                                           boolean forBackprop,
                                           boolean forwards,
                                           java.lang.String inputWeightKey,
                                           org.nd4j.linalg.api.ndarray.INDArray maskArray)

Returns FwdPassReturn object with activations/INDArrays. Allows activateHelper to be used for forward pass, backward pass and rnnTimeStep whilst being reasonably efficient for all

backpropGradientHelper

public static Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray> backpropGradientHelper(NeuralNetConfiguration conf,
                                                                                         org.nd4j.linalg.activations.IActivation gateActivationFn,
                                                                                         org.nd4j.linalg.api.ndarray.INDArray input,
                                                                                         org.nd4j.linalg.api.ndarray.INDArray recurrentWeights,
                                                                                         org.nd4j.linalg.api.ndarray.INDArray inputWeights,
                                                                                         org.nd4j.linalg.api.ndarray.INDArray epsilon,
                                                                                         boolean truncatedBPTT,
                                                                                         int tbpttBackwardLength,
                                                                                         FwdPassReturn fwdPass,
                                                                                         boolean forwards,
                                                                                         java.lang.String inputWeightKey,
                                                                                         java.lang.String recurrentWeightKey,
                                                                                         java.lang.String biasWeightKey,
                                                                                         java.util.Map<java.lang.String,org.nd4j.linalg.api.ndarray.INDArray> gradientViews,
                                                                                         org.nd4j.linalg.api.ndarray.INDArray maskArray)