org.deeplearning4j.nn.conf.layers.variational

Class LossFunctionWrapper

java.lang.Object

org.deeplearning4j.nn.conf.layers.variational.LossFunctionWrapper

All Implemented Interfaces:

java.io.Serializable, ReconstructionDistribution

public class LossFunctionWrapper
extends java.lang.Object
implements ReconstructionDistribution

LossFunctionWrapper allows training of a VAE model with a standard (possibly deterministic) neural network loss function for the reconstruction, instead of using a ReconstructionDistribution as would normally be done with a VAE model.

Note: most functionality is supported, but clearly reconstruction log probability cannot be calculated when using LossFunctionWrapper, as ILossFunction instances do not have either (a) a probabilistic interpretation, or (b) a means of calculating the negative log probability.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
LossFunctionWrapper(org.nd4j.linalg.activations.Activation activation, org.nd4j.linalg.lossfunctions.ILossFunction lossFunction)
LossFunctionWrapper(org.nd4j.linalg.activations.IActivation activationFn, org.nd4j.linalg.lossfunctions.ILossFunction lossFunction)

Method Summary

Modifier and Type	Method and Description
int	distributionInputSize(int dataSize) Get the number of distribution parameters for the given input data size.
org.nd4j.linalg.api.ndarray.INDArray	exampleNegLogProbability(org.nd4j.linalg.api.ndarray.INDArray x, org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams) Calculate the negative log probability for each example individually
org.nd4j.linalg.api.ndarray.INDArray	generateAtMean(org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams) Generate a sample from P(x|z), where x = E[P(x|z)] i.e., return the mean value for the distribution
org.nd4j.linalg.api.ndarray.INDArray	generateRandom(org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams) Randomly sample from P(x|z) using the specified distribution parameters
org.nd4j.linalg.api.ndarray.INDArray	gradient(org.nd4j.linalg.api.ndarray.INDArray x, org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams) Calculate the gradient of the negative log probability with respect to the preOutDistributionParams
boolean	hasLossFunction() Does this reconstruction distribution has a standard neural network loss function (such as mean squared error, which is deterministic) or is it a standard VAE with a probabilistic reconstruction distribution?
double	negLogProbability(org.nd4j.linalg.api.ndarray.INDArray x, org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams, boolean average) Calculate the negative log probability (summed or averaged over each example in the minibatch)
java.lang.String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

LossFunctionWrapper

public LossFunctionWrapper(org.nd4j.linalg.activations.IActivation activationFn,
                           org.nd4j.linalg.lossfunctions.ILossFunction lossFunction)

LossFunctionWrapper

public LossFunctionWrapper(org.nd4j.linalg.activations.Activation activation,
                           org.nd4j.linalg.lossfunctions.ILossFunction lossFunction)

Method Detail

hasLossFunction

public boolean hasLossFunction()

Description copied from interface: ReconstructionDistribution

Does this reconstruction distribution has a standard neural network loss function (such as mean squared error, which is deterministic) or is it a standard VAE with a probabilistic reconstruction distribution?

Specified by:

hasLossFunction in interface ReconstructionDistribution

Returns:

true if the reconstruction distribution has a loss function only (and no probabilistic reconstruction distribution)

distributionInputSize

public int distributionInputSize(int dataSize)

Description copied from interface: ReconstructionDistribution

Get the number of distribution parameters for the given input data size. For example, a Gaussian distribution has 2 parameters value (mean and log(variance)) for each data value, whereas a Bernoulli distribution has only 1 parameter value (probability) for each data value.

Specified by:

distributionInputSize in interface ReconstructionDistribution

Parameters:

dataSize - Size of the data. i.e., nIn value

Returns:

Number of distribution parameters for the given reconstruction distribution

negLogProbability

public double negLogProbability(org.nd4j.linalg.api.ndarray.INDArray x,
                                org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams,
                                boolean average)

Description copied from interface: ReconstructionDistribution

Calculate the negative log probability (summed or averaged over each example in the minibatch)

Specified by:

negLogProbability in interface ReconstructionDistribution

Parameters:

x - Data to be modelled (reconstructions)

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian)

average - Whether the log probability should be averaged over the minibatch, or simply summed.

Returns:

Average or sum of negative log probability of the reconstruction given the distribution parameters

exampleNegLogProbability

public org.nd4j.linalg.api.ndarray.INDArray exampleNegLogProbability(org.nd4j.linalg.api.ndarray.INDArray x,
                                                                     org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams)

Description copied from interface: ReconstructionDistribution

Calculate the negative log probability for each example individually

Specified by:

exampleNegLogProbability in interface ReconstructionDistribution

Parameters:

x - Data to be modelled (reconstructions)

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian) - before applying activation function

Returns:

Negative log probability of the reconstruction given the distribution parameters, for each example individually. Column vector, shape [numExamples, 1]

gradient

public org.nd4j.linalg.api.ndarray.INDArray gradient(org.nd4j.linalg.api.ndarray.INDArray x,
                                                     org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams)

Description copied from interface: ReconstructionDistribution

Calculate the gradient of the negative log probability with respect to the preOutDistributionParams

Specified by:

gradient in interface ReconstructionDistribution

Parameters:

x - Data

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian) - before applying activation function

Returns:

Gradient with respect to the preOutDistributionParams

generateRandom

public org.nd4j.linalg.api.ndarray.INDArray generateRandom(org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams)

Description copied from interface: ReconstructionDistribution

Randomly sample from P(x|z) using the specified distribution parameters

Specified by:

generateRandom in interface ReconstructionDistribution

Parameters:

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian) - before applying activation function

Returns:

A random sample of x given the distribution parameters

generateAtMean

public org.nd4j.linalg.api.ndarray.INDArray generateAtMean(org.nd4j.linalg.api.ndarray.INDArray preOutDistributionParams)

Description copied from interface: ReconstructionDistribution

Generate a sample from P(x|z), where x = E[P(x|z)] i.e., return the mean value for the distribution

Specified by:

generateAtMean in interface ReconstructionDistribution

Parameters:

preOutDistributionParams - Distribution parameters used by this reconstruction distribution (for example, mean and log variance values for Gaussian) - before applying activation function

Returns:

A deterministic sample of x (mean/expected value) given the distribution parameters

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object