Evaluation

java.lang.Object
- org.deeplearning4j.eval.BaseEvaluation<Evaluation>
- - org.deeplearning4j.eval.Evaluation

All Implemented Interfaces:

java.io.Serializable, IEvaluation<Evaluation>
```
public class Evaluation
extends BaseEvaluation<Evaluation>
```
Evaluation metrics: precision, recall, f1

See Also:

Serialized Form

Field Summary

Fields
Modifier and Type	Field and Description
`protected ConfusionMatrix<java.lang.Integer>`	`confusion`
`protected java.util.Map<Pair<java.lang.Integer,java.lang.Integer>,java.util.List<java.lang.Object>>`	`confusionMatrixMetaData`
`protected static double`	`DEFAULT_EDGE_VALUE`
`protected Counter<java.lang.Integer>`	`falseNegatives`
`protected Counter<java.lang.Integer>`	`falsePositives`
`protected java.util.List<java.lang.String>`	`labelsList`
`protected int`	`numRowCounter`
`protected int`	`topN`
`protected int`	`topNCorrectCount`
`protected int`	`topNTotalCount`
`protected Counter<java.lang.Integer>`	`trueNegatives`
`protected Counter<java.lang.Integer>`	`truePositives`

Constructor Summary

Constructors
Constructor and Description
`Evaluation()`
`Evaluation(int numClasses)` The number of classes to account for in the evaluation
`Evaluation(java.util.List<java.lang.String> labels)` The labels to include with the evaluation.
`Evaluation(java.util.List<java.lang.String> labels, int topN)` Constructor to use for top N accuracy
`Evaluation(java.util.Map<java.lang.Integer,java.lang.String> labels)` Use a map to generate labels Pass in a label index with the actual label you want to use for output

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`double`	`accuracy()` Accuracy: (TP + TN) / (P + N)
`void`	`addToConfusion(java.lang.Integer real, java.lang.Integer guess)` Adds to the confusion matrix
`int`	`classCount(java.lang.Integer clazz)` Returns the number of times the given label has actually occurred
`java.lang.String`	`confusionToString()` Get a String representation of the confusion matrix
`void`	`eval(org.nd4j.linalg.api.ndarray.INDArray realOutcomes, org.nd4j.linalg.api.ndarray.INDArray guesses)` Collects statistics on the real outcomes vs the guesses.
`void`	`eval(org.nd4j.linalg.api.ndarray.INDArray trueLabels, org.nd4j.linalg.api.ndarray.INDArray input, ComputationGraph network)` Evaluate the output using the given true labels, the input to the multi layer network and the multi layer network to use for evaluation
`void`	`eval(org.nd4j.linalg.api.ndarray.INDArray realOutcomes, org.nd4j.linalg.api.ndarray.INDArray guesses, java.util.List<? extends java.io.Serializable> recordMetaData)` Evaluate the network, with optional metadata
`void`	`eval(org.nd4j.linalg.api.ndarray.INDArray trueLabels, org.nd4j.linalg.api.ndarray.INDArray input, MultiLayerNetwork network)` Evaluate the output using the given true labels, the input to the multi layer network and the multi layer network to use for evaluation
`void`	`eval(int predictedIdx, int actualIdx)` Evaluate a single prediction (one prediction at a time)
`double`	`f1()` TP: true positive FP: False Positive FN: False Negative F1 score: 2 * TP / (2TP + FP + FN)
`double`	`f1(java.lang.Integer classLabel)` Calculate f1 score for a given class
`double`	`falseAlarmRate()` False Alarm Rate (FAR) reflects rate of misclassified to classified records http://ro.ecu.edu.au/cgi/viewcontent.cgi?article=1058&context=isw
`double`	`falseNegativeRate()` False negative rate based on guesses so far Takes into account all known classes and outputs average fnr across all of them
`double`	`falseNegativeRate(java.lang.Integer classLabel)` Returns the false negative rate for a given label
`double`	`falseNegativeRate(java.lang.Integer classLabel, double edgeCase)` Returns the false negative rate for a given label
`java.util.Map<java.lang.Integer,java.lang.Integer>`	`falseNegatives()` False negatives: correctly rejected
`double`	`falsePositiveRate()` False positive rate based on guesses so far Takes into account all known classes and outputs average fpr across all of them
`double`	`falsePositiveRate(java.lang.Integer classLabel)` Returns the false positive rate for a given label
`double`	`falsePositiveRate(java.lang.Integer classLabel, double edgeCase)` Returns the false positive rate for a given label
`java.util.Map<java.lang.Integer,java.lang.Integer>`	`falsePositives()` False positive: wrong guess
`java.lang.String`	`getClassLabel(java.lang.Integer clazz)`
`ConfusionMatrix<java.lang.Integer>`	`getConfusionMatrix()` Returns the confusion matrix variable
`int`	`getNumRowCounter()`
`java.util.List<Prediction>`	`getPredictionByPredictedClass(int predictedClass)` Get a list of predictions, for all data with the specified predicted class, regardless of the actual data class.
`java.util.List<Prediction>`	`getPredictionErrors()` Get a list of prediction errors, on a per-record basis
`java.util.List<Prediction>`	`getPredictions(int actualClass, int predictedClass)` Get a list of predictions in the specified confusion matrix entry (i.e., for the given actua/predicted class pair)
`java.util.List<Prediction>`	`getPredictionsByActualClass(int actualClass)` Get a list of predictions, for all data with the specified actual class, regardless of the predicted class.
`int`	`getTopNCorrectCount()` Return the number of correct predictions according to top N value.
`int`	`getTopNTotalCount()` Return the total number of top N evaluations.
`void`	`incrementFalseNegatives(java.lang.Integer classLabel)`
`void`	`incrementFalsePositives(java.lang.Integer classLabel)`
`void`	`incrementTrueNegatives(java.lang.Integer classLabel)`
`void`	`incrementTruePositives(java.lang.Integer classLabel)`
`void`	`merge(Evaluation other)` Merge the other evaluation object into this one.
`java.util.Map<java.lang.Integer,java.lang.Integer>`	`negative()` Total negatives true negatives + false negatives
`java.util.Map<java.lang.Integer,java.lang.Integer>`	`positive()` Returns all of the positive guesses: true positive + false negative
`double`	`precision()` Precision based on guesses so far Takes into account all known classes and outputs average precision across all of them
`double`	`precision(java.lang.Integer classLabel)` Returns the precision for a given label
`double`	`precision(java.lang.Integer classLabel, double edgeCase)` Returns the precision for a given label
`double`	`recall()` Recall based on guesses so far Takes into account all known classes and outputs average recall across all of them
`double`	`recall(java.lang.Integer classLabel)` Returns the recall for a given label
`double`	`recall(java.lang.Integer classLabel, double edgeCase)` Returns the recall for a given label
`java.lang.String`	`stats()`
`java.lang.String`	`stats(boolean suppressWarnings)` Method to obtain the classification report as a String
`double`	`topNAccuracy()` Top N accuracy of the predictions so far.
`java.util.Map<java.lang.Integer,java.lang.Integer>`	`trueNegatives()` True negatives: correctly rejected
`java.util.Map<java.lang.Integer,java.lang.Integer>`	`truePositives()` True positives: correctly rejected

Methods inherited from class org.deeplearning4j.eval.BaseEvaluation
eval, evalTimeSeries, evalTimeSeries

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - topN
```
protected final int topN
```
  - topNCorrectCount
```
protected int topNCorrectCount
```
  - topNTotalCount
```
protected int topNTotalCount
```
  - truePositives
```
protected Counter<java.lang.Integer> truePositives
```
  - falsePositives
```
protected Counter<java.lang.Integer> falsePositives
```
  - trueNegatives
```
protected Counter<java.lang.Integer> trueNegatives
```
  - falseNegatives
```
protected Counter<java.lang.Integer> falseNegatives
```
  - confusion
```
protected ConfusionMatrix<java.lang.Integer> confusion
```
  - numRowCounter
```
protected int numRowCounter
```
  - labelsList
```
protected java.util.List<java.lang.String> labelsList
```
  - DEFAULT_EDGE_VALUE
```
protected static final double DEFAULT_EDGE_VALUE
```
    See Also:
    
    Constant Field Values
  - confusionMatrixMetaData
```
protected java.util.Map<Pair<java.lang.Integer,java.lang.Integer>,java.util.List<java.lang.Object>> confusionMatrixMetaData
```
- Constructor Detail
  - Evaluation
```
public Evaluation()
```
  - Evaluation
```
public Evaluation(int numClasses)
```
    The number of classes to account for in the evaluation
    
    Parameters:
    
    numClasses - the number of classes to account for in the evaluation
  - Evaluation
```
public Evaluation(java.util.List<java.lang.String> labels)
```
    The labels to include with the evaluation. This constructor can be used for generating labeled output rather than just numbers for the labels
    
    Parameters:
    
    labels - the labels to use for the output
  - Evaluation
```
public Evaluation(java.util.Map<java.lang.Integer,java.lang.String> labels)
```
    Use a map to generate labels Pass in a label index with the actual label you want to use for output
    
    Parameters:
    
    labels - a map of label index to label value
  - Evaluation
```
public Evaluation(java.util.List<java.lang.String> labels,
                  int topN)
```
    Constructor to use for top N accuracy
    
    Parameters:
    
    labels - Labels for the classes (may be null)
    
    topN - Value to use for top N accuracy calculation (<=1: standard accuracy). Note that with top N accuracy, an example is considered 'correct' if the probability for the true class is one of the highest N values
- Method Detail
  - eval
```
public void eval(org.nd4j.linalg.api.ndarray.INDArray trueLabels,
                 org.nd4j.linalg.api.ndarray.INDArray input,
                 ComputationGraph network)
```
    Evaluate the output using the given true labels, the input to the multi layer network and the multi layer network to use for evaluation
    
    Parameters:
    
    trueLabels - the labels to ise
    
    input - the input to the network to use for evaluation
    
    network - the network to use for output
  - eval
```
public void eval(org.nd4j.linalg.api.ndarray.INDArray trueLabels,
                 org.nd4j.linalg.api.ndarray.INDArray input,
                 MultiLayerNetwork network)
```
    Evaluate the output using the given true labels, the input to the multi layer network and the multi layer network to use for evaluation
    
    Parameters:
    
    trueLabels - the labels to ise
    
    input - the input to the network to use for evaluation
    
    network - the network to use for output
  - eval
```
public void eval(org.nd4j.linalg.api.ndarray.INDArray realOutcomes,
                 org.nd4j.linalg.api.ndarray.INDArray guesses)
```
    Collects statistics on the real outcomes vs the guesses. This is for logistic outcome matrices.
    Note that an IllegalArgumentException is thrown if the two passed in matrices aren't the same length.
    
    Parameters:
    
    realOutcomes - the real outcomes (labels - usually binary)
    
    guesses - the guesses/prediction (usually a probability vector)
  - eval
```
public void eval(org.nd4j.linalg.api.ndarray.INDArray realOutcomes,
                 org.nd4j.linalg.api.ndarray.INDArray guesses,
                 java.util.List<? extends java.io.Serializable> recordMetaData)
```
    Evaluate the network, with optional metadata
    
    Specified by:
    
    eval in interface IEvaluation<Evaluation>
    
    Overrides:
    
    eval in class BaseEvaluation<Evaluation>
    
    Parameters:
    
    realOutcomes - Data labels
    
    guesses - Network predictions
    
    recordMetaData - Optional; may be null. If not null, should have size equal to the number of outcomes/guesses
  - eval
```
public void eval(int predictedIdx,
                 int actualIdx)
```
    Evaluate a single prediction (one prediction at a time)
    
    Parameters:
    
    predictedIdx - Index of class predicted by the network
    
    actualIdx - Index of actual class
  - stats
```
public java.lang.String stats()
```
  - stats
```
public java.lang.String stats(boolean suppressWarnings)
```
    Method to obtain the classification report as a String
    
    Parameters:
    
    suppressWarnings - whether or not to output warnings related to the evaluation results
    
    Returns:
    
    A (multi-line) String with accuracy, precision, recall, f1 score etc
  - precision
```
public double precision(java.lang.Integer classLabel)
```
    Returns the precision for a given label
    
    Parameters:
    
    classLabel - the label
    
    Returns:
    
    the precision for the label
  - precision
```
public double precision(java.lang.Integer classLabel,
                        double edgeCase)
```
    Returns the precision for a given label
    
    Parameters:
    
    classLabel - the label
    
    edgeCase - What to output in case of 0/0
    
    Returns:
    
    the precision for the label
  - precision
```
public double precision()
```
    Precision based on guesses so far Takes into account all known classes and outputs average precision across all of them
    
    Returns:
    
    the total precision based on guesses so far
  - recall
```
public double recall(java.lang.Integer classLabel)
```
    Returns the recall for a given label
    
    Parameters:
    
    classLabel - the label
    
    Returns:
    
    Recall rate as a double
  - recall
```
public double recall(java.lang.Integer classLabel,
                     double edgeCase)
```
    Returns the recall for a given label
    
    Parameters:
    
    classLabel - the label
    
    edgeCase - What to output in case of 0/0
    
    Returns:
    
    Recall rate as a double
  - recall
```
public double recall()
```
    Recall based on guesses so far Takes into account all known classes and outputs average recall across all of them
    
    Returns:
    
    the recall for the outcomes
  - falsePositiveRate
```
public double falsePositiveRate(java.lang.Integer classLabel)
```
    Returns the false positive rate for a given label
    
    Parameters:
    
    classLabel - the label
    
    Returns:
    
    fpr as a double
  - falsePositiveRate
```
public double falsePositiveRate(java.lang.Integer classLabel,
                                double edgeCase)
```
    Returns the false positive rate for a given label
    
    Parameters:
    
    classLabel - the label
    
    edgeCase - What to output in case of 0/0
    
    Returns:
    
    fpr as a double
  - falsePositiveRate
```
public double falsePositiveRate()
```
    False positive rate based on guesses so far Takes into account all known classes and outputs average fpr across all of them
    
    Returns:
    
    the fpr for the outcomes
  - falseNegativeRate
```
public double falseNegativeRate(java.lang.Integer classLabel)
```
    Returns the false negative rate for a given label
    
    Parameters:
    
    classLabel - the label
    
    Returns:
    
    fnr as a double
  - falseNegativeRate
```
public double falseNegativeRate(java.lang.Integer classLabel,
                                double edgeCase)
```
    Returns the false negative rate for a given label
    
    Parameters:
    
    classLabel - the label
    
    edgeCase - What to output in case of 0/0
    
    Returns:
    
    fnr as a double
  - falseNegativeRate
```
public double falseNegativeRate()
```
    False negative rate based on guesses so far Takes into account all known classes and outputs average fnr across all of them
    
    Returns:
    
    the fnr for the outcomes
  - falseAlarmRate
```
public double falseAlarmRate()
```
    False Alarm Rate (FAR) reflects rate of misclassified to classified records http://ro.ecu.edu.au/cgi/viewcontent.cgi?article=1058&context=isw
    
    Returns:
    
    the fpr for the outcomes
  - f1
```
public double f1(java.lang.Integer classLabel)
```
    Calculate f1 score for a given class
    
    Parameters:
    
    classLabel - the label to calculate f1 for
    
    Returns:
    
    the f1 score for the given label
  - f1
```
public double f1()
```
    TP: true positive FP: False Positive FN: False Negative F1 score: 2 * TP / (2TP + FP + FN)
    
    Returns:
    
    the f1 score or harmonic mean based on current guesses
  - accuracy
```
public double accuracy()
```
    Accuracy: (TP + TN) / (P + N)
    
    Returns:
    
    the accuracy of the guesses so far
  - topNAccuracy
```
public double topNAccuracy()
```
    Top N accuracy of the predictions so far. For top N = 1 (default), equivalent to accuracy()
    
    Returns:
    
    Top N accuracy
  - truePositives
```
public java.util.Map<java.lang.Integer,java.lang.Integer> truePositives()
```
    True positives: correctly rejected
    
    Returns:
    
    the total true positives so far
  - trueNegatives
```
public java.util.Map<java.lang.Integer,java.lang.Integer> trueNegatives()
```
    True negatives: correctly rejected
    
    Returns:
    
    the total true negatives so far
  - falsePositives
```
public java.util.Map<java.lang.Integer,java.lang.Integer> falsePositives()
```
    False positive: wrong guess
    
    Returns:
    
    the count of the false positives
  - falseNegatives
```
public java.util.Map<java.lang.Integer,java.lang.Integer> falseNegatives()
```
    False negatives: correctly rejected
    
    Returns:
    
    the total false negatives so far
  - negative
```
public java.util.Map<java.lang.Integer,java.lang.Integer> negative()
```
    Total negatives true negatives + false negatives
    
    Returns:
    
    the overall negative count
  - positive
```
public java.util.Map<java.lang.Integer,java.lang.Integer> positive()
```
    Returns all of the positive guesses: true positive + false negative
  - incrementTruePositives
```
public void incrementTruePositives(java.lang.Integer classLabel)
```
  - incrementTrueNegatives
```
public void incrementTrueNegatives(java.lang.Integer classLabel)
```
  - incrementFalseNegatives
```
public void incrementFalseNegatives(java.lang.Integer classLabel)
```
  - incrementFalsePositives
```
public void incrementFalsePositives(java.lang.Integer classLabel)
```
  - addToConfusion
```
public void addToConfusion(java.lang.Integer real,
                           java.lang.Integer guess)
```
    Adds to the confusion matrix
    
    Parameters:
    
    real - the actual guess
    
    guess - the system guess
  - classCount
```
public int classCount(java.lang.Integer clazz)
```
    Returns the number of times the given label has actually occurred
    
    Parameters:
    
    clazz - the label
    
    Returns:
    
    the number of times the label actually occurred
  - getNumRowCounter
```
public int getNumRowCounter()
```
  - getTopNCorrectCount
```
public int getTopNCorrectCount()
```
    Return the number of correct predictions according to top N value. For top N = 1 (default) this is equivalent to the number of correct predictions
    
    Returns:
    
    Number of correct top N predictions
  - getTopNTotalCount
```
public int getTopNTotalCount()
```
    Return the total number of top N evaluations. Most of the time, this is exactly equal to getNumRowCounter(), but may differ in the case of using eval(int, int) as top N accuracy cannot be calculated in that case (i.e., requires the full probability distribution, not just predicted/actual indices)
    
    Returns:
    
    Total number of top N predictions
  - getClassLabel
```
public java.lang.String getClassLabel(java.lang.Integer clazz)
```
  - getConfusionMatrix
```
public ConfusionMatrix<java.lang.Integer> getConfusionMatrix()
```
    Returns the confusion matrix variable
    
    Returns:
    
    confusion matrix variable for this evaluation
  - merge
```
public void merge(Evaluation other)
```
    Merge the other evaluation object into this one. The result is that this Evaluation instance contains the counts etc from both
    
    Parameters:
    
    other - Evaluation object to merge into this one.
  - confusionToString
```
public java.lang.String confusionToString()
```
    Get a String representation of the confusion matrix
  - getPredictionErrors
```
public java.util.List<Prediction> getPredictionErrors()
```
    Get a list of prediction errors, on a per-record basis
    
    Note: Prediction errors are ONLY available if the "evaluate with metadata" method is used: eval(INDArray, INDArray, List) Otherwise (if the metadata hasn't been recorded via that previously mentioned eval method), there is no value in splitting each prediction out into a separate Prediction object - instead, use the confusion matrix to get the counts, via getConfusionMatrix()
    
    Returns:
    
    A list of prediction errors, or null if no metadata has been recorded
  - getPredictionsByActualClass
```
public java.util.List<Prediction> getPredictionsByActualClass(int actualClass)
```
    Get a list of predictions, for all data with the specified actual class, regardless of the predicted class.
    Note: Prediction errors are ONLY available if the "evaluate with metadata" method is used: eval(INDArray, INDArray, List) Otherwise (if the metadata hasn't been recorded via that previously mentioned eval method), there is no value in splitting each prediction out into a separate Prediction object - instead, use the confusion matrix to get the counts, via getConfusionMatrix()
    
    Parameters:
    
    actualClass - Actual class to get predictions for
    
    Returns:
    
    List of predictions, or null if the "evaluate with metadata" method was not used
  - getPredictionByPredictedClass
```
public java.util.List<Prediction> getPredictionByPredictedClass(int predictedClass)
```
    Get a list of predictions, for all data with the specified predicted class, regardless of the actual data class.
    Note: Prediction errors are ONLY available if the "evaluate with metadata" method is used: eval(INDArray, INDArray, List) Otherwise (if the metadata hasn't been recorded via that previously mentioned eval method), there is no value in splitting each prediction out into a separate Prediction object - instead, use the confusion matrix to get the counts, via getConfusionMatrix()
    
    Parameters:
    
    predictedClass - Actual class to get predictions for
    
    Returns:
    
    List of predictions, or null if the "evaluate with metadata" method was not used
  - getPredictions
```
public java.util.List<Prediction> getPredictions(int actualClass,
                                                 int predictedClass)
```
    Get a list of predictions in the specified confusion matrix entry (i.e., for the given actua/predicted class pair)
    
    Parameters:
    
    actualClass - Actual class
    
    predictedClass - Predicted class
    
    Returns:
    
    List of predictions that match the specified actual/predicted classes, or null if the "evaluate with metadata" method was not used

Class Evaluation

Field Summary

Constructor Summary

Method Summary

Methods inherited from class org.deeplearning4j.eval.BaseEvaluation

Methods inherited from class java.lang.Object

Field Detail

topN

topNCorrectCount

topNTotalCount

truePositives

falsePositives

trueNegatives

falseNegatives

confusion

numRowCounter

labelsList

DEFAULT_EDGE_VALUE

confusionMatrixMetaData

Constructor Detail

Evaluation

Evaluation

Evaluation

Evaluation

Evaluation

Method Detail

eval

eval

eval

eval

eval

stats

stats

precision

precision

precision

recall

recall

recall

falsePositiveRate

falsePositiveRate

falsePositiveRate

falseNegativeRate

falseNegativeRate

falseNegativeRate

falseAlarmRate

f1

f1

accuracy

topNAccuracy

truePositives

trueNegatives

falsePositives

falseNegatives

negative

positive

incrementTruePositives

incrementTrueNegatives

incrementFalseNegatives

incrementFalsePositives

addToConfusion

classCount

getNumRowCounter

getTopNCorrectCount

getTopNTotalCount

getClassLabel

getConfusionMatrix

merge

confusionToString

getPredictionErrors

getPredictionsByActualClass

getPredictionByPredictedClass

getPredictions