org.deeplearning4j.nn.conf.layers.variational

Class GaussianReconstructionDistribution

java.lang.Object

org.deeplearning4j.nn.conf.layers.variational.GaussianReconstructionDistribution

All Implemented Interfaces:

java.io.Serializable, ReconstructionDistribution

public class GaussianReconstructionDistribution
extends java.lang.Object
implements ReconstructionDistribution

Gaussian reconstruction distribution for variational autoencoder.
Outputs are modelled by a Gaussian distribution, with the mean and variances (diagonal covariance matrix) for each output determined by the network forward pass.

Specifically, the GaussianReconstructionDistribution models mean and log(stdev^2). This parameterization gives log(1) = 0, and inputs can be in range (-infinity,infinity). Other parameterizations for variance are of course possible but may be problematic with respect to the average pre-activation function values and activation function ranges.
For activation functions, identity and perhaps tanh are typical - though tanh (unlike identity) implies a minimum/maximum possible value for mean and log variance. Asymmetric activation functions such as sigmoid or relu should be avoided.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
GaussianReconstructionDistribution() Create a GaussianReconstructionDistribution with the default identity activation function.
GaussianReconstructionDistribution(org.nd4j.linalg.activations.Activation activationFn)
GaussianReconstructionDistribution(org.nd4j.linalg.activations.IActivation activationFn)
GaussianReconstructionDistribution(java.lang.String activationFn) Deprecated. Use GaussianReconstructionDistribution(Activation)

Method Summary

Modifier and Type	Method and Description
int	distributionInputSize(int dataSize) Get the number of distribution parameters for the given input data size.
org.nd4j.linalg.api.ndarray.INDArray	exampleNegLogProbability(org.nd4j.linalg.api.ndarray.INDArray x, org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams) Calculate the negative log probability for each example individually
org.nd4j.linalg.api.ndarray.INDArray	generateAtMean(org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams) Generate a sample from P(x|z), where x = E[P(x|z)] i.e., return the mean value for the distribution
org.nd4j.linalg.api.ndarray.INDArray	generateRandom(org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams) Randomly sample from P(x|z) using the specified distribution parameters
org.nd4j.linalg.api.ndarray.INDArray	gradient(org.nd4j.linalg.api.ndarray.INDArray x, org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams) Calculate the gradient of the negative log probability with respect to the preOutDistributionParams
boolean	hasLossFunction() Does this reconstruction distribution has a standard neural network loss function (such as mean squared error, which is deterministic) or is it a standard VAE with a probabilistic reconstruction distribution?
double	negLogProbability(org.nd4j.linalg.api.ndarray.INDArray x, org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams, boolean average) Calculate the negative log probability (summed or averaged over each example in the minibatch)
java.lang.String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

GaussianReconstructionDistribution

public GaussianReconstructionDistribution()

Create a GaussianReconstructionDistribution with the default identity activation function.

GaussianReconstructionDistribution

@Deprecated
public GaussianReconstructionDistribution(java.lang.String activationFn)

Deprecated. Use GaussianReconstructionDistribution(Activation)

GaussianReconstructionDistribution

public GaussianReconstructionDistribution(org.nd4j.linalg.activations.Activation activationFn)

Parameters:

activationFn - Activation function for the reconstruction distribution. Typically identity or tanh.

GaussianReconstructionDistribution

public GaussianReconstructionDistribution(org.nd4j.linalg.activations.IActivation activationFn)

Parameters:

activationFn - Activation function for the reconstruction distribution. Typically identity or tanh.

Method Detail

hasLossFunction

public boolean hasLossFunction()

Description copied from interface: ReconstructionDistribution

Does this reconstruction distribution has a standard neural network loss function (such as mean squared error, which is deterministic) or is it a standard VAE with a probabilistic reconstruction distribution?

Specified by:

hasLossFunction in interface ReconstructionDistribution

Returns:

true if the reconstruction distribution has a loss function only (and no probabilistic reconstruction distribution)

distributionInputSize

public int distributionInputSize(int dataSize)

Description copied from interface: ReconstructionDistribution

Get the number of distribution parameters for the given input data size. For example, a Gaussian distribution has 2 parameters value (mean and log(variance)) for each data value, whereas a Bernoulli distribution has only 1 parameter value (probability) for each data value.

Specified by:

distributionInputSize in interface ReconstructionDistribution

Parameters:

dataSize - Size of the data. i.e., nIn value

Returns:

Number of distribution parameters for the given reconstruction distribution

negLogProbability

public double negLogProbability(org.nd4j.linalg.api.ndarray.INDArray x,
                                org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams,
                                boolean average)

Description copied from interface: ReconstructionDistribution

Calculate the negative log probability (summed or averaged over each example in the minibatch)

Specified by:

negLogProbability in interface ReconstructionDistribution

Parameters:

x - Data to be modelled (reconstructions)

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian)

average - Whether the log probability should be averaged over the minibatch, or simply summed.

Returns:

Average or sum of negative log probability of the reconstruction given the distribution parameters

exampleNegLogProbability

public org.nd4j.linalg.api.ndarray.INDArray exampleNegLogProbability(org.nd4j.linalg.api.ndarray.INDArray x,
                                                                     org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams)

Description copied from interface: ReconstructionDistribution

Calculate the negative log probability for each example individually

Specified by:

exampleNegLogProbability in interface ReconstructionDistribution

Parameters:

x - Data to be modelled (reconstructions)

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian) - before applying activation function

Returns:

Negative log probability of the reconstruction given the distribution parameters, for each example individually. Column vector, shape [numExamples, 1]

gradient

public org.nd4j.linalg.api.ndarray.INDArray gradient(org.nd4j.linalg.api.ndarray.INDArray x,
                                                     org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams)

Description copied from interface: ReconstructionDistribution

Calculate the gradient of the negative log probability with respect to the preOutDistributionParams

Specified by:

gradient in interface ReconstructionDistribution

Parameters:

x - Data

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian) - before applying activation function

Returns:

Gradient with respect to the preOutDistributionParams

generateRandom

public org.nd4j.linalg.api.ndarray.INDArray generateRandom(org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams)

Description copied from interface: ReconstructionDistribution

Randomly sample from P(x|z) using the specified distribution parameters

Specified by:

generateRandom in interface ReconstructionDistribution

Parameters:

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian) - before applying activation function

Returns:

A random sample of x given the distribution parameters

generateAtMean

public org.nd4j.linalg.api.ndarray.INDArray generateAtMean(org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams)

Description copied from interface: ReconstructionDistribution

Generate a sample from P(x|z), where x = E[P(x|z)] i.e., return the mean value for the distribution

Specified by:

generateAtMean in interface ReconstructionDistribution

Parameters:

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian) - before applying activation function

Returns:

A deterministic sample of x (mean/expected value) given the distribution parameters

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object