org.deeplearning4j.eval

Class ROCMultiClass

java.lang.Object

org.deeplearning4j.eval.BaseEvaluation<ROCMultiClass>

org.deeplearning4j.eval.ROCMultiClass

All Implemented Interfaces:

java.io.Serializable, IEvaluation<ROCMultiClass>

public class ROCMultiClass
extends BaseEvaluation<ROCMultiClass>

ROC (Receiver Operating Characteristic) for multi-class classifiers, using the specified number of threshold steps.

The ROC curves are produced by treating the predictions as a set of one-vs-all classifiers, and then calculating ROC curves for each. In practice, this means for N classes, we get N ROC curves.

Some ROC implementations will automatically calculate the threshold points based on the data set to give a 'smoother' ROC curve (or optimal cut points for diagnostic purposes). This implementation currently uses fixed steps of size 1.0 / thresholdSteps, as this allows easy implementation for batched and distributed evaluation scenarios (where the full data set is not available in memory on any one machine at once).

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
ROCMultiClass(int thresholdSteps)

Method Summary

Modifier and Type	Method and Description
double	calculateAUC(int classIdx) Calculate the AUC - Area Under Curve Utilizes trapezoidal integration internally
double	calculateAverageAUC() Calculate the average (one-vs-all) AUC for all classes
void	eval(org.nd4j.linalg.api.ndarray.INDArray labels, org.nd4j.linalg.api.ndarray.INDArray predictions) Evaluate (collect statistics for) the given minibatch of data.
java.util.List<ROC.PrecisionRecallPoint>	getPrecisionRecallCurve(int classIndex)
java.util.List<ROC.ROCValue>	getResults(int classIdx) Get the ROC curve, as a set of points
double[][]	getResultsAsArray(int classIdx) Get the ROC curve, as a set of (falsePositive, truePositive) points
void	merge(ROCMultiClass other) Merge this ROCMultiClass instance with another.

Methods inherited from class org.deeplearning4j.eval.BaseEvaluation

eval, eval, evalTimeSeries, evalTimeSeries

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ROCMultiClass

public ROCMultiClass(int thresholdSteps)

Parameters:

thresholdSteps - Number of threshold steps to use for the ROC calculation

Method Detail

eval

public void eval(org.nd4j.linalg.api.ndarray.INDArray labels,
                 org.nd4j.linalg.api.ndarray.INDArray predictions)

Evaluate (collect statistics for) the given minibatch of data. For time series (3 dimensions) use BaseEvaluation.evalTimeSeries(INDArray, INDArray) or BaseEvaluation.evalTimeSeries(INDArray, INDArray, INDArray)

Parameters:

labels - Labels / true outcomes

predictions - Predictions

getResults

public java.util.List<ROC.ROCValue> getResults(int classIdx)

Get the ROC curve, as a set of points

Parameters:

classIdx - Index of the class to get the (one-vs-all) ROC cur

Returns:

ROC curve, as a list of points

getResultsAsArray

public double[][] getResultsAsArray(int classIdx)

Get the ROC curve, as a set of (falsePositive, truePositive) points

Returns a 2d array of {falsePositive, truePositive values}.
Size is [2][thresholdSteps], with out[0][.] being false positives, and out[1][.] being true positives

Returns:

ROC curve as double[][]

calculateAUC

public double calculateAUC(int classIdx)

Calculate the AUC - Area Under Curve
Utilizes trapezoidal integration internally

Returns:

AUC

calculateAverageAUC

public double calculateAverageAUC()

Calculate the average (one-vs-all) AUC for all classes

getPrecisionRecallCurve

public java.util.List<ROC.PrecisionRecallPoint> getPrecisionRecallCurve(int classIndex)

merge

public void merge(ROCMultiClass other)

Merge this ROCMultiClass instance with another. This ROCMultiClass instance is modified, by adding the stats from the other instance.

Parameters:

other - ROCMultiClass instance to combine with this one