org.deeplearning4j.spark.impl.paramavg.stats

Class ParameterAveragingTrainingWorkerStats

java.lang.Object

org.deeplearning4j.spark.impl.paramavg.stats.ParameterAveragingTrainingWorkerStats

All Implemented Interfaces:

java.io.Serializable, SparkTrainingStats

public class ParameterAveragingTrainingWorkerStats
extends java.lang.Object
implements SparkTrainingStats

Statistics collected by ParameterAveragingTrainingWorker instances

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ParameterAveragingTrainingWorkerStats.ParameterAveragingTrainingWorkerStatsHelper

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	DEFAULT_DELIMITER
static java.lang.String	FILENAME_BROADCAST_GET_STATS
static java.lang.String	FILENAME_FIT_STATS
static java.lang.String	FILENAME_INIT_STATS
static java.lang.String	PARAMETER_AVERAGING_WORKER_BROADCAST_GET_VALUE_TIME_MS
static java.lang.String	PARAMETER_AVERAGING_WORKER_FIT_TIMES_MS
static java.lang.String	PARAMETER_AVERAGING_WORKER_INIT_TIME_MS

Fields inherited from interface org.deeplearning4j.spark.api.stats.SparkTrainingStats

DEFAULT_PRINT_FORMAT, PRINT_INDENT

Constructor Summary

Constructors

Constructor and Description
ParameterAveragingTrainingWorkerStats(java.util.List<EventStats> parameterAveragingWorkerBroadcastGetValueTimeMs, java.util.List<EventStats> parameterAveragingWorkerInitTimeMs, java.util.List<EventStats> parameterAveragingWorkerFitTimesMs)

Method Summary

Modifier and Type	Method and Description
void	addOtherTrainingStats(SparkTrainingStats other) Combine the two training stats instances.
boolean	defaultIncludeInPlots(java.lang.String key) When plotting statistics, we don't necessarily want to plot everything.
void	exportStatFiles(java.lang.String outputPath, org.apache.spark.SparkContext sc) Export the stats as a collection of files.
java.util.Set<java.lang.String>	getKeySet()
SparkTrainingStats	getNestedTrainingStats() Return the nested training stats - if any.
java.lang.String	getShortNameForKey(java.lang.String key) Return a short (display) name for the given key.
java.util.List<EventStats>	getValue(java.lang.String key) Get the statistic value for this key
java.lang.String	statsAsString() Get a String representation of the stats.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_DELIMITER

public static final java.lang.String DEFAULT_DELIMITER

See Also:

Constant Field Values

FILENAME_BROADCAST_GET_STATS

public static final java.lang.String FILENAME_BROADCAST_GET_STATS

See Also:

Constant Field Values

FILENAME_INIT_STATS

public static final java.lang.String FILENAME_INIT_STATS

See Also:

Constant Field Values

FILENAME_FIT_STATS

public static final java.lang.String FILENAME_FIT_STATS

See Also:

Constant Field Values

PARAMETER_AVERAGING_WORKER_BROADCAST_GET_VALUE_TIME_MS

public static final java.lang.String PARAMETER_AVERAGING_WORKER_BROADCAST_GET_VALUE_TIME_MS

See Also:

Constant Field Values

PARAMETER_AVERAGING_WORKER_INIT_TIME_MS

public static final java.lang.String PARAMETER_AVERAGING_WORKER_INIT_TIME_MS

See Also:

Constant Field Values

PARAMETER_AVERAGING_WORKER_FIT_TIMES_MS

public static final java.lang.String PARAMETER_AVERAGING_WORKER_FIT_TIMES_MS

See Also:

Constant Field Values

Constructor Detail

ParameterAveragingTrainingWorkerStats

public ParameterAveragingTrainingWorkerStats(java.util.List<EventStats> parameterAveragingWorkerBroadcastGetValueTimeMs,
                                             java.util.List<EventStats> parameterAveragingWorkerInitTimeMs,
                                             java.util.List<EventStats> parameterAveragingWorkerFitTimesMs)

Method Detail

getKeySet

public java.util.Set<java.lang.String> getKeySet()

Specified by:

getKeySet in interface SparkTrainingStats

Returns:

Set of keys that can be used with SparkTrainingStats.getValue(String)

getValue

public java.util.List<EventStats> getValue(java.lang.String key)

Description copied from interface: SparkTrainingStats

Get the statistic value for this key

Specified by:

getValue in interface SparkTrainingStats

Parameters:

key - Key to get the value for

Returns:

Statistic for this key, or an exception if key is invalid

getShortNameForKey

public java.lang.String getShortNameForKey(java.lang.String key)

Description copied from interface: SparkTrainingStats

Return a short (display) name for the given key.

Specified by:

getShortNameForKey in interface SparkTrainingStats

Parameters:

key - Key

Returns:

Short/display name for key

defaultIncludeInPlots

public boolean defaultIncludeInPlots(java.lang.String key)

Description copied from interface: SparkTrainingStats

When plotting statistics, we don't necessarily want to plot everything. For example, some statistics/measurements are made up multiple smaller components; it does not always make sense to plot both the larger stat, and the components that make it up

Specified by:

defaultIncludeInPlots in interface SparkTrainingStats

Parameters:

key - Key to check for default plotting behaviour

Returns:

Whether the specified key should be included in plots by default or not

addOtherTrainingStats

public void addOtherTrainingStats(SparkTrainingStats other)

Description copied from interface: SparkTrainingStats

Combine the two training stats instances. Usually, the two objects must be of the same type

Specified by:

addOtherTrainingStats in interface SparkTrainingStats

Parameters:

other - Other training stats to return

getNestedTrainingStats

public SparkTrainingStats getNestedTrainingStats()

Description copied from interface: SparkTrainingStats

Return the nested training stats - if any.

Specified by:

getNestedTrainingStats in interface SparkTrainingStats

Returns:

The nested stats, if present/applicable, or null otherwise

statsAsString

public java.lang.String statsAsString()

Description copied from interface: SparkTrainingStats

Get a String representation of the stats. This functionality is implemented as a separate method (as opposed to toString()) as the resulting String can be very large.
NOTE: The String representation typically includes only duration information. To get full statistics (including machine IDs, etc) use SparkTrainingStats.getValue(String) or export full data via SparkTrainingStats.exportStatFiles(String, SparkContext)

Specified by:

statsAsString in interface SparkTrainingStats

Returns:

A String representation of the training statistics

exportStatFiles

public void exportStatFiles(java.lang.String outputPath,
                            org.apache.spark.SparkContext sc)
                     throws java.io.IOException

Description copied from interface: SparkTrainingStats

Export the stats as a collection of files. Stats are comma-delimited (CSV) with 1 header line

Specified by:

exportStatFiles in interface SparkTrainingStats

Parameters:

outputPath - Base directory to write files to

Throws:

java.io.IOException