public class LocalResponseNormalization extends BaseLayer<LocalResponseNormalization>
For a^i_{x,y} the activity of a neuron computed by applying kernel i at position (x,y) and applying ReLU nonlinearity, the response normalized activation b^i_{x,y} is given by:
x^2 = (a^j_{x,y})^2 unitScale = (k + alpha * sum_{j=max(0, i - n/2)}^{max(N-1, i + n/2)} (a^j_{x,y})^2 ) y = b^i_{x,y} = x * unitScale**-beta
gy = epsilon (aka deltas from previous layer) sumPart = sum(a^j_{x,y} * gb^j_{x,y}) gx = gy * unitScale**-beta - 2 * alpha * beta * sumPart/unitScale * a^i_{x,y}
Reference: http://www.cs.toronto.edu/~fritz/absps/imagenet.pdf https://github.com/vlfeat/matconvnet/issues/10 Chainer
Created by nyghtowl on 10/29/15.
Layer.TrainingMode, Layer.Type| Modifier and Type | Field and Description |
|---|---|
protected static org.slf4j.Logger |
log |
conf, dropoutApplied, dropoutMask, gradient, gradientsFlattened, gradientViews, index, input, iterationListeners, maskArray, maskState, optimizer, params, paramsFlattened, score, solver| Constructor and Description |
|---|
LocalResponseNormalization(NeuralNetConfiguration conf) |
LocalResponseNormalization(NeuralNetConfiguration conf,
org.nd4j.linalg.api.ndarray.INDArray input) |
| Modifier and Type | Method and Description |
|---|---|
org.nd4j.linalg.api.ndarray.INDArray |
activate(boolean training)
Trigger an activation with the last specified input
|
Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray> |
backpropGradient(org.nd4j.linalg.api.ndarray.INDArray epsilon)
Calculate the gradient relative to the error in the next layer
|
Gradient |
calcGradient(Gradient layerError,
org.nd4j.linalg.api.ndarray.INDArray indArray)
Calculate the gradient
|
double |
calcL1(boolean backpropParamsOnly)
Calculate the l1 regularization term
0.0 if regularization is not used. |
double |
calcL2(boolean backpropParamsOnly)
Calculate the l2 regularization term
0.0 if regularization is not used. |
void |
fit(org.nd4j.linalg.api.ndarray.INDArray input)
Fit the model to the given data
|
org.nd4j.linalg.api.ndarray.INDArray |
getParam(java.lang.String param)
Get the parameter
|
boolean |
isPretrainLayer()
Returns true if the layer can be trained in an unsupervised/pretrain manner (VAE, RBMs etc)
|
void |
merge(Layer layer,
int batchSize)
Averages the given logistic regression from a mini batch into this layer
|
org.nd4j.linalg.api.ndarray.INDArray |
params()
Returns the parameters of the neural network as a flattened row vector
|
void |
setParams(org.nd4j.linalg.api.ndarray.INDArray params)
Set the parameters for this model.
|
Layer |
transpose()
Return a transposed copy of the weights/bias
(this means reverse the number of inputs and outputs on the weights)
|
Layer.Type |
type()
Returns the layer type
|
accumulateScore, activate, activate, activate, activate, activate, activationMean, applyDropOutIfNecessary, applyLearningRateScoreDecay, applyMask, batchSize, clear, clone, computeGradientAndScore, conf, createGradient, derivativeActivation, error, feedForwardMaskArray, fit, getIndex, getInput, getInputMiniBatchSize, getListeners, getMaskArray, getOptimizer, gradient, gradientAndScore, init, initParams, input, iterate, layerConf, layerNameAndIndex, numParams, numParams, paramTable, paramTable, preOutput, preOutput, preOutput, preOutput, score, setBackpropGradientsViewArray, setConf, setIndex, setInput, setInputMiniBatchSize, setListeners, setListeners, setMaskArray, setParam, setParams, setParamsViewArray, setParamTable, setScoreWithZ, toString, update, update, validateInputpublic LocalResponseNormalization(NeuralNetConfiguration conf, org.nd4j.linalg.api.ndarray.INDArray input)
public LocalResponseNormalization(NeuralNetConfiguration conf)
public double calcL2(boolean backpropParamsOnly)
LayercalcL2 in interface LayercalcL2 in class BaseLayer<LocalResponseNormalization>backpropParamsOnly - If true: calculate L2 based on backprop params only. If false: calculate
based on all params (including pretrain params, if any)public double calcL1(boolean backpropParamsOnly)
LayercalcL1 in interface LayercalcL1 in class BaseLayer<LocalResponseNormalization>backpropParamsOnly - If true: calculate L1 based on backprop params only. If false: calculate
based on all params (including pretrain params, if any)public Layer.Type type()
Layertype in interface Layertype in class BaseLayer<LocalResponseNormalization>public void fit(org.nd4j.linalg.api.ndarray.INDArray input)
Modelfit in interface Modelfit in class BaseLayer<LocalResponseNormalization>input - the data to fit the model topublic Pair<Gradient,org.nd4j.linalg.api.ndarray.INDArray> backpropGradient(org.nd4j.linalg.api.ndarray.INDArray epsilon)
LayerbackpropGradient in interface LayerbackpropGradient in class BaseLayer<LocalResponseNormalization>epsilon - w^(L+1)*delta^(L+1). Or, equiv: dC/da, i.e., (dC/dz)*(dz/da) = dC/da, where C
is cost function a=sigma(z) is activation.public org.nd4j.linalg.api.ndarray.INDArray activate(boolean training)
Layeractivate in interface Layeractivate in class BaseLayer<LocalResponseNormalization>training - training or test modepublic Layer transpose()
Layertranspose in interface Layertranspose in class BaseLayer<LocalResponseNormalization>public boolean isPretrainLayer()
Layerpublic Gradient calcGradient(Gradient layerError, org.nd4j.linalg.api.ndarray.INDArray indArray)
LayercalcGradient in interface LayercalcGradient in class BaseLayer<LocalResponseNormalization>layerError - the layer errorpublic void merge(Layer layer, int batchSize)
BaseLayermerge in interface Layermerge in class BaseLayer<LocalResponseNormalization>layer - the logistic regression layer to average into this layerbatchSize - the batch sizepublic org.nd4j.linalg.api.ndarray.INDArray params()
BaseLayerparams in interface Modelparams in class BaseLayer<LocalResponseNormalization>public org.nd4j.linalg.api.ndarray.INDArray getParam(java.lang.String param)
ModelgetParam in interface ModelgetParam in class BaseLayer<LocalResponseNormalization>param - the key of the parameterpublic void setParams(org.nd4j.linalg.api.ndarray.INDArray params)
ModelsetParams in interface ModelsetParams in class BaseLayer<LocalResponseNormalization>params - the parameters for the model