org.deeplearning4j.optimize.solvers

Class BaseOptimizer

java.lang.Object

org.deeplearning4j.optimize.solvers.BaseOptimizer

All Implemented Interfaces:

java.io.Serializable, ConvexOptimizer

Direct Known Subclasses:

ConjugateGradient, LBFGS, LineGradientDescent, StochasticGradientDescent

public abstract class BaseOptimizer
extends java.lang.Object
implements ConvexOptimizer

Base optimizer

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected ComputationGraphUpdater	computationGraphUpdater
protected NeuralNetConfiguration	conf
static java.lang.String	GRADIENT_KEY
protected java.util.Collection<IterationListener>	iterationListeners
protected BackTrackLineSearch	lineMaximizer
protected static org.slf4j.Logger	log
protected Model	model
protected double	oldScore
static java.lang.String	PARAMS_KEY
protected double	score
static java.lang.String	SCORE_KEY
static java.lang.String	SEARCH_DIR
protected java.util.Map<java.lang.String,java.lang.Object>	searchState
protected double	step
protected StepFunction	stepFunction
protected double	stepMax
protected java.util.Collection<TerminationCondition>	terminationConditions
protected Updater	updater

Constructor Summary

Constructors

Constructor and Description
BaseOptimizer(NeuralNetConfiguration conf, StepFunction stepFunction, java.util.Collection<IterationListener> iterationListeners, java.util.Collection<TerminationCondition> terminationConditions, Model model)
BaseOptimizer(NeuralNetConfiguration conf, StepFunction stepFunction, java.util.Collection<IterationListener> iterationListeners, Model model)

Method Summary

Modifier and Type	Method and Description
int	batchSize() The batch size for the optimizer
boolean	checkTerminalConditions(org.nd4j.linalg.api.ndarray.INDArray gradient, double oldScore, double score, int i) Check termination conditions setup a search state
ComputationGraphUpdater	getComputationGraphUpdater()
NeuralNetConfiguration	getConf()
static StepFunction	getDefaultStepFunctionForOptimizer(java.lang.Class<? extends ConvexOptimizer> optimizerClass)
static int	getIterationCount(Model model)
Updater	getUpdater()
Pair<Gradient,java.lang.Double>	gradientAndScore() The gradient and score for this optimizer
static void	incrementIterationCount(Model model, int incrementBy)
boolean	optimize() Optimize call.
protected void	postFirstStep(org.nd4j.linalg.api.ndarray.INDArray gradient)
void	postStep(org.nd4j.linalg.api.ndarray.INDArray gradient) Post step to update searchDirection with new gradient and parameter information
void	preProcessLine() Pre preProcess to setup initial searchDirection approximation
double	score() The score for the optimizer so far
void	setBatchSize(int batchSize) Set the batch size for the optimizer
void	setListeners(java.util.Collection<IterationListener> listeners)
void	setUpdater(Updater updater)
void	setUpdaterComputationGraph(ComputationGraphUpdater updater)
void	setupSearchState(Pair<Gradient,java.lang.Double> pair) Setup the initial search state
void	updateGradientAccordingToParams(Gradient gradient, Model model, int batchSize) Update the gradient according to the configuration such as adagrad, momentum, and sparsity

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

conf

protected NeuralNetConfiguration conf

log

protected static final org.slf4j.Logger log

stepFunction

protected StepFunction stepFunction

iterationListeners

protected java.util.Collection<IterationListener> iterationListeners

terminationConditions

protected java.util.Collection<TerminationCondition> terminationConditions

model

protected Model model

lineMaximizer

protected BackTrackLineSearch lineMaximizer

updater

protected Updater updater

computationGraphUpdater

protected ComputationGraphUpdater computationGraphUpdater

step

protected double step

score

protected double score

oldScore

protected double oldScore

stepMax

protected double stepMax

GRADIENT_KEY

public static final java.lang.String GRADIENT_KEY

See Also:

Constant Field Values

SCORE_KEY

public static final java.lang.String SCORE_KEY

See Also:

Constant Field Values

PARAMS_KEY

public static final java.lang.String PARAMS_KEY

See Also:

Constant Field Values

SEARCH_DIR

public static final java.lang.String SEARCH_DIR

See Also:

Constant Field Values

searchState

protected java.util.Map<java.lang.String,java.lang.Object> searchState

Constructor Detail

BaseOptimizer

public BaseOptimizer(NeuralNetConfiguration conf,
                     StepFunction stepFunction,
                     java.util.Collection<IterationListener> iterationListeners,
                     Model model)

Parameters:

conf -

stepFunction -

iterationListeners -

model -

BaseOptimizer

public BaseOptimizer(NeuralNetConfiguration conf,
                     StepFunction stepFunction,
                     java.util.Collection<IterationListener> iterationListeners,
                     java.util.Collection<TerminationCondition> terminationConditions,
                     Model model)

Parameters:

conf -

stepFunction -

iterationListeners -

terminationConditions -

model -

Method Detail

score

public double score()

Description copied from interface: ConvexOptimizer

The score for the optimizer so far

Specified by:

score in interface ConvexOptimizer

Returns:

the score for this optimizer so far

getUpdater

public Updater getUpdater()

Specified by:

getUpdater in interface ConvexOptimizer

setUpdater

public void setUpdater(Updater updater)

Specified by:

setUpdater in interface ConvexOptimizer

getComputationGraphUpdater

public ComputationGraphUpdater getComputationGraphUpdater()

Specified by:

getComputationGraphUpdater in interface ConvexOptimizer

setUpdaterComputationGraph

public void setUpdaterComputationGraph(ComputationGraphUpdater updater)

Specified by:

setUpdaterComputationGraph in interface ConvexOptimizer

setListeners

public void setListeners(java.util.Collection<IterationListener> listeners)

Specified by:

setListeners in interface ConvexOptimizer

getConf

public NeuralNetConfiguration getConf()

Specified by:

getConf in interface ConvexOptimizer

gradientAndScore

public Pair<Gradient,java.lang.Double> gradientAndScore()

Description copied from interface: ConvexOptimizer

The gradient and score for this optimizer

Specified by:

gradientAndScore in interface ConvexOptimizer

Returns:

the gradient and score for this optimizer

optimize

public boolean optimize()

Optimize call. This runs the optimizer.

Specified by:

optimize in interface ConvexOptimizer

Returns:

whether it converged or not

postFirstStep

protected void postFirstStep(org.nd4j.linalg.api.ndarray.INDArray gradient)

checkTerminalConditions

public boolean checkTerminalConditions(org.nd4j.linalg.api.ndarray.INDArray gradient,
                                       double oldScore,
                                       double score,
                                       int i)

Description copied from interface: ConvexOptimizer

Check termination conditions setup a search state

Specified by:

checkTerminalConditions in interface ConvexOptimizer

Parameters:

gradient - layer gradients

i - what iteration the optimizer is on

batchSize

public int batchSize()

Description copied from interface: ConvexOptimizer

The batch size for the optimizer

Specified by:

batchSize in interface ConvexOptimizer

Returns:

setBatchSize

public void setBatchSize(int batchSize)

Description copied from interface: ConvexOptimizer

Set the batch size for the optimizer

Specified by:

setBatchSize in interface ConvexOptimizer

preProcessLine

public void preProcessLine()

Pre preProcess to setup initial searchDirection approximation

Specified by:

preProcessLine in interface ConvexOptimizer

postStep

public void postStep(org.nd4j.linalg.api.ndarray.INDArray gradient)

Post step to update searchDirection with new gradient and parameter information

Specified by:

postStep in interface ConvexOptimizer

updateGradientAccordingToParams

public void updateGradientAccordingToParams(Gradient gradient,
                                            Model model,
                                            int batchSize)

Description copied from interface: ConvexOptimizer

Update the gradient according to the configuration such as adagrad, momentum, and sparsity

Specified by:

updateGradientAccordingToParams in interface ConvexOptimizer

Parameters:

gradient - the gradient to modify

model - the model with the parameters to update

batchSize - batchSize for update

setupSearchState

public void setupSearchState(Pair<Gradient,java.lang.Double> pair)

Setup the initial search state

Specified by:

setupSearchState in interface ConvexOptimizer

Parameters:

pair -

getDefaultStepFunctionForOptimizer

public static StepFunction getDefaultStepFunctionForOptimizer(java.lang.Class<? extends ConvexOptimizer> optimizerClass)

getIterationCount

public static int getIterationCount(Model model)

incrementIterationCount

public static void incrementIterationCount(Model model,
                                           int incrementBy)