org.deeplearning4j.nn.conf.layers.variational

Interface ReconstructionDistribution

All Superinterfaces:

java.io.Serializable

All Known Implementing Classes:

BernoulliReconstructionDistribution, CompositeReconstructionDistribution, ExponentialReconstructionDistribution, GaussianReconstructionDistribution, LossFunctionWrapper

public interface ReconstructionDistribution
extends java.io.Serializable

The ReconstructionDistribution is used with variational autoencoders VariationalAutoencoder to specify the form of the distribution p(data|x). For example, real-valued data could be modelled by a GaussianReconstructionDistribution, whereas binary data could be modelled by a BernoulliReconstructionDistribution.

To model multiple types of data in the one data vector, use CompositeReconstructionDistribution.

Method Summary

Modifier and Type	Method and Description
int	distributionInputSize(int dataSize) Get the number of distribution parameters for the given input data size.
org.nd4j.linalg.api.ndarray.INDArray	exampleNegLogProbability(org.nd4j.linalg.api.ndarray.INDArray x, org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams) Calculate the negative log probability for each example individually
org.nd4j.linalg.api.ndarray.INDArray	generateAtMean(org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams) Generate a sample from P(x|z), where x = E[P(x|z)] i.e., return the mean value for the distribution
org.nd4j.linalg.api.ndarray.INDArray	generateRandom(org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams) Randomly sample from P(x|z) using the specified distribution parameters
org.nd4j.linalg.api.ndarray.INDArray	gradient(org.nd4j.linalg.api.ndarray.INDArray x, org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams) Calculate the gradient of the negative log probability with respect to the preOutDistributionParams
boolean	hasLossFunction() Does this reconstruction distribution has a standard neural network loss function (such as mean squared error, which is deterministic) or is it a standard VAE with a probabilistic reconstruction distribution?
double	negLogProbability(org.nd4j.linalg.api.ndarray.INDArray x, org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams, boolean average) Calculate the negative log probability (summed or averaged over each example in the minibatch)

Method Detail

hasLossFunction

boolean hasLossFunction()

Does this reconstruction distribution has a standard neural network loss function (such as mean squared error, which is deterministic) or is it a standard VAE with a probabilistic reconstruction distribution?

Returns:

true if the reconstruction distribution has a loss function only (and no probabilistic reconstruction distribution)

distributionInputSize

int distributionInputSize(int dataSize)

Get the number of distribution parameters for the given input data size. For example, a Gaussian distribution has 2 parameters value (mean and log(variance)) for each data value, whereas a Bernoulli distribution has only 1 parameter value (probability) for each data value.

Parameters:

dataSize - Size of the data. i.e., nIn value

Returns:

Number of distribution parameters for the given reconstruction distribution

negLogProbability

double negLogProbability(org.nd4j.linalg.api.ndarray.INDArray x,
                         org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams,
                         boolean average)

Calculate the negative log probability (summed or averaged over each example in the minibatch)

Parameters:

x - Data to be modelled (reconstructions)

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian)

average - Whether the log probability should be averaged over the minibatch, or simply summed.

Returns:

Average or sum of negative log probability of the reconstruction given the distribution parameters

exampleNegLogProbability

org.nd4j.linalg.api.ndarray.INDArray exampleNegLogProbability(org.nd4j.linalg.api.ndarray.INDArray x,
                                                              org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams)

Calculate the negative log probability for each example individually

Parameters:

x - Data to be modelled (reconstructions)

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian) - before applying activation function

Returns:

Negative log probability of the reconstruction given the distribution parameters, for each example individually. Column vector, shape [numExamples, 1]

gradient

org.nd4j.linalg.api.ndarray.INDArray gradient(org.nd4j.linalg.api.ndarray.INDArray x,
                                              org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams)

Calculate the gradient of the negative log probability with respect to the preOutDistributionParams

Parameters:

x - Data

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian) - before applying activation function

Returns:

Gradient with respect to the preOutDistributionParams

generateRandom

org.nd4j.linalg.api.ndarray.INDArray generateRandom(org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams)

Randomly sample from P(x|z) using the specified distribution parameters

Parameters:

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian) - before applying activation function

Returns:

A random sample of x given the distribution parameters

generateAtMean

org.nd4j.linalg.api.ndarray.INDArray generateAtMean(org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams)

Generate a sample from P(x|z), where x = E[P(x|z)] i.e., return the mean value for the distribution

Parameters:

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian) - before applying activation function

Returns:

A deterministic sample of x (mean/expected value) given the distribution parameters