org.deeplearning4j.ui.storage.sqlite

Class J7FileStatsStorage

java.lang.Object

org.deeplearning4j.ui.storage.sqlite.J7FileStatsStorage

All Implemented Interfaces:

StatsStorage, StatsStorageRouter

public class J7FileStatsStorage
extends java.lang.Object
implements StatsStorage

A Java 7 compatible file-based StatsStorage implementation, based on SQLite. Note: Where possible, use FileStatsStorage which should be faster (is based on MapDB). Obviously, the storage formats for J7FileStatsStorage and FileStatsStorage are incompatible.

Constructor Summary

Constructors

Constructor and Description
J7FileStatsStorage(java.io.File file)

Method Summary

Modifier and Type	Method and Description
protected java.util.List<StatsStorageEvent>	checkStorageEvents(Persistable p)
void	close() Close any open resources (files, etc)
void	deregisterStatsStorageListener(StatsStorageListener listener) Remove the specified listener, if it is present.
java.util.List<Persistable>	getAllStaticInfos(java.lang.String sessionID, java.lang.String typeID) Get all static informations for the given session and type ID
java.util.List<Persistable>	getAllUpdatesAfter(java.lang.String sessionID, java.lang.String typeID, long timestamp) Get all updates for the given session ID (all worker IDs), that occur after (not including) the given timestamp.
java.util.List<Persistable>	getAllUpdatesAfter(java.lang.String sessionID, java.lang.String typeID, java.lang.String workerID, long timestamp) Get all updates for the given session and worker ID, that occur after (not including) the given timestamp.
Persistable	getLatestUpdate(java.lang.String sessionID, java.lang.String typeID, java.lang.String workerID) Get the latest update record (i.e., update record with the largest timestamp value) for the specified session and worker IDs
java.util.List<Persistable>	getLatestUpdateAllWorkers(java.lang.String sessionID, java.lang.String typeID) Get the latest update for all workers, for the given session ID
java.util.List<StatsStorageListener>	getListeners() Get a list (shallow copy) of all listeners currently present
int	getNumUpdateRecordsFor(java.lang.String sessionID) Return the number of update records for the given session ID (all workers)
int	getNumUpdateRecordsFor(java.lang.String sessionID, java.lang.String typeID, java.lang.String workerID) Return the number of update records for the given session ID and worker ID
Persistable	getStaticInfo(java.lang.String sessionID, java.lang.String typeID, java.lang.String workerID) Get the static info for the given session and worker IDs, or null if no such static info has been reported
StorageMetaData	getStorageMetaData(java.lang.String sessionID, java.lang.String typeID) Get the session metadata, if any has been registered via StatsStorageRouter.putStorageMetaData(StorageMetaData)
Persistable	getUpdate(java.lang.String sessionID, java.lang.String typeId, java.lang.String workerID, long timestamp) Get the specified update (or null, if none exists for the given session/worker ids and timestamp)
boolean	isClosed()
java.util.List<java.lang.String>	listSessionIDs() Get a list of all sessions stored by this storage backend
java.util.List<java.lang.String>	listTypeIDsForSession(java.lang.String sessionID) Get the list of type IDs for the given session ID
java.util.List<java.lang.String>	listWorkerIDsForSession(java.lang.String sessionID) For a given session ID, list all of the known worker IDs
java.util.List<java.lang.String>	listWorkerIDsForSessionAndType(java.lang.String sessionID, java.lang.String typeID) For a given session ID and type ID, list all of the known worker IDs
protected void	notifyListeners(java.util.List<StatsStorageEvent> sses)
void	putStaticInfo(java.util.Collection<? extends Persistable> collection) Static info: reported once per session, upon initialization
void	putStaticInfo(Persistable staticInfo) Static info: reported once per session, upon initialization
void	putStorageMetaData(java.util.Collection<? extends StorageMetaData> collection)
void	putStorageMetaData(StorageMetaData storageMetaData) Method to store some additional metadata for each session.
void	putUpdate(java.util.Collection<? extends Persistable> collection) Updates: stored multiple times per session (periodically, for example)
void	putUpdate(Persistable update) Updates: stored multiple times per session (periodically, for example)
void	registerStatsStorageListener(StatsStorageListener listener) Add a new StatsStorageListener.
void	removeAllListeners() Remove all listeners from the StatsStorage instance
boolean	sessionExists(java.lang.String sessionID) Check if the specified session ID exists or not
java.lang.String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

J7FileStatsStorage

public J7FileStatsStorage(@NonNull
                          java.io.File file)

Parameters:

file - Storage location for the stats

Method Detail

checkStorageEvents

protected java.util.List<StatsStorageEvent> checkStorageEvents(Persistable p)

putStorageMetaData

public void putStorageMetaData(StorageMetaData storageMetaData)

Description copied from interface: StatsStorageRouter

Method to store some additional metadata for each session. Idea: record the classes used to serialize and deserialize the static info and updates (as a class name). This is mainly used for debugging and validation.

Specified by:

putStorageMetaData in interface StatsStorageRouter

Parameters:

storageMetaData - Storage metadata to store

putStorageMetaData

public void putStorageMetaData(java.util.Collection<? extends StorageMetaData> collection)

Specified by:

putStorageMetaData in interface StatsStorageRouter

putStaticInfo

public void putStaticInfo(Persistable staticInfo)

Description copied from interface: StatsStorageRouter

Static info: reported once per session, upon initialization

Specified by:

putStaticInfo in interface StatsStorageRouter

Parameters:

staticInfo - Static info to store

putStaticInfo

public void putStaticInfo(java.util.Collection<? extends Persistable> collection)

Description copied from interface: StatsStorageRouter

Static info: reported once per session, upon initialization

Specified by:

putStaticInfo in interface StatsStorageRouter

Parameters:

collection - Static info to store

putUpdate

public void putUpdate(Persistable update)

Description copied from interface: StatsStorageRouter

Updates: stored multiple times per session (periodically, for example)

Specified by:

putUpdate in interface StatsStorageRouter

Parameters:

update - Update info to store

putUpdate

public void putUpdate(java.util.Collection<? extends Persistable> collection)

Description copied from interface: StatsStorageRouter

Updates: stored multiple times per session (periodically, for example)

Specified by:

putUpdate in interface StatsStorageRouter

Parameters:

collection - Update info to store

close

public void close()
           throws java.io.IOException

Description copied from interface: StatsStorage

Close any open resources (files, etc)

Specified by:

close in interface StatsStorage

Throws:

java.io.IOException

isClosed

public boolean isClosed()

Specified by:

isClosed in interface StatsStorage

Returns:

Whether the StatsStorage implementation has been closed or not

listSessionIDs

public java.util.List<java.lang.String> listSessionIDs()

Description copied from interface: StatsStorage

Get a list of all sessions stored by this storage backend

Specified by:

listSessionIDs in interface StatsStorage

sessionExists

public boolean sessionExists(java.lang.String sessionID)

Description copied from interface: StatsStorage

Check if the specified session ID exists or not

Specified by:

sessionExists in interface StatsStorage

Parameters:

sessionID - Session ID to check

Returns:

true if session exists, false otherwise

getStaticInfo

public Persistable getStaticInfo(java.lang.String sessionID,
                                 java.lang.String typeID,
                                 java.lang.String workerID)

Description copied from interface: StatsStorage

Get the static info for the given session and worker IDs, or null if no such static info has been reported

Specified by:

getStaticInfo in interface StatsStorage

Parameters:

sessionID - Session ID

workerID - worker ID

Returns:

Static info, or null if none has been reported

getAllStaticInfos

public java.util.List<Persistable> getAllStaticInfos(java.lang.String sessionID,
                                                     java.lang.String typeID)

Description copied from interface: StatsStorage

Get all static informations for the given session and type ID

Specified by:

getAllStaticInfos in interface StatsStorage

Parameters:

sessionID - Session ID to get static info for

typeID - Type ID to get static info for

Returns:

All static info instances matching both the session and type IDs

listTypeIDsForSession

public java.util.List<java.lang.String> listTypeIDsForSession(java.lang.String sessionID)

Description copied from interface: StatsStorage

Get the list of type IDs for the given session ID

Specified by:

listTypeIDsForSession in interface StatsStorage

Parameters:

sessionID - Session ID to query

Returns:

List of type IDs

listWorkerIDsForSession

public java.util.List<java.lang.String> listWorkerIDsForSession(java.lang.String sessionID)

Description copied from interface: StatsStorage

For a given session ID, list all of the known worker IDs

Specified by:

listWorkerIDsForSession in interface StatsStorage

Parameters:

sessionID - Session ID

Returns:

List of worker IDs, or possibly null if session ID is unknown

listWorkerIDsForSessionAndType

public java.util.List<java.lang.String> listWorkerIDsForSessionAndType(java.lang.String sessionID,
                                                                       java.lang.String typeID)

Description copied from interface: StatsStorage

For a given session ID and type ID, list all of the known worker IDs

Specified by:

listWorkerIDsForSessionAndType in interface StatsStorage

Parameters:

sessionID - Session ID

typeID - Type ID

Returns:

List of worker IDs, or possibly null if session ID is unknown

getNumUpdateRecordsFor

public int getNumUpdateRecordsFor(java.lang.String sessionID)

Description copied from interface: StatsStorage

Return the number of update records for the given session ID (all workers)

Specified by:

getNumUpdateRecordsFor in interface StatsStorage

Parameters:

sessionID - Session ID

Returns:

number of update records

getNumUpdateRecordsFor

public int getNumUpdateRecordsFor(java.lang.String sessionID,
                                  java.lang.String typeID,
                                  java.lang.String workerID)

Description copied from interface: StatsStorage

Return the number of update records for the given session ID and worker ID

Specified by:

getNumUpdateRecordsFor in interface StatsStorage

Parameters:

sessionID - Session ID

workerID - Worker ID

Returns:

number of update records

getLatestUpdate

public Persistable getLatestUpdate(java.lang.String sessionID,
                                   java.lang.String typeID,
                                   java.lang.String workerID)

Description copied from interface: StatsStorage

Get the latest update record (i.e., update record with the largest timestamp value) for the specified session and worker IDs

Specified by:

getLatestUpdate in interface StatsStorage

Parameters:

sessionID - session ID

workerID - worker ID

Returns:

UpdateRecord containing the session/worker IDs, timestamp and content for the most recent update

getUpdate

public Persistable getUpdate(java.lang.String sessionID,
                             java.lang.String typeId,
                             java.lang.String workerID,
                             long timestamp)

Description copied from interface: StatsStorage

Get the specified update (or null, if none exists for the given session/worker ids and timestamp)

Specified by:

getUpdate in interface StatsStorage

Parameters:

sessionID - Session ID

workerID - Worker ID

timestamp - Timestamp

Returns:

Update

getLatestUpdateAllWorkers

public java.util.List<Persistable> getLatestUpdateAllWorkers(java.lang.String sessionID,
                                                             java.lang.String typeID)

Description copied from interface: StatsStorage

Get the latest update for all workers, for the given session ID

Specified by:

getLatestUpdateAllWorkers in interface StatsStorage

Parameters:

sessionID - Session ID

Returns:

List of updates for the given Session ID

getAllUpdatesAfter

public java.util.List<Persistable> getAllUpdatesAfter(java.lang.String sessionID,
                                                      java.lang.String typeID,
                                                      java.lang.String workerID,
                                                      long timestamp)

Description copied from interface: StatsStorage

Get all updates for the given session and worker ID, that occur after (not including) the given timestamp. Results should be sorted by time.

Specified by:

getAllUpdatesAfter in interface StatsStorage

Parameters:

sessionID - Session ID

workerID - Worker Id

timestamp - Timestamp

Returns:

List of records occurring after the given timestamp

getAllUpdatesAfter

public java.util.List<Persistable> getAllUpdatesAfter(java.lang.String sessionID,
                                                      java.lang.String typeID,
                                                      long timestamp)

Description copied from interface: StatsStorage

Get all updates for the given session ID (all worker IDs), that occur after (not including) the given timestamp. Results should be sorted by time.

Specified by:

getAllUpdatesAfter in interface StatsStorage

Parameters:

sessionID - Session ID

timestamp - Timestamp

Returns:

List of records occurring after the given timestamp

getStorageMetaData

public StorageMetaData getStorageMetaData(java.lang.String sessionID,
                                          java.lang.String typeID)

Description copied from interface: StatsStorage

Get the session metadata, if any has been registered via StatsStorageRouter.putStorageMetaData(StorageMetaData)

Specified by:

getStorageMetaData in interface StatsStorage

Parameters:

sessionID - Session ID to get metadat

Returns:

Session metadata, or null if none is available

registerStatsStorageListener

public void registerStatsStorageListener(StatsStorageListener listener)

Description copied from interface: StatsStorage

Add a new StatsStorageListener. The given listener will called whenever a state change occurs for the stats storage instance

Specified by:

registerStatsStorageListener in interface StatsStorage

Parameters:

listener - Listener to add

deregisterStatsStorageListener

public void deregisterStatsStorageListener(StatsStorageListener listener)

Description copied from interface: StatsStorage

Remove the specified listener, if it is present.

Specified by:

deregisterStatsStorageListener in interface StatsStorage

Parameters:

listener - Listener to remove

removeAllListeners

public void removeAllListeners()

Description copied from interface: StatsStorage

Remove all listeners from the StatsStorage instance

Specified by:

removeAllListeners in interface StatsStorage

getListeners

public java.util.List<StatsStorageListener> getListeners()

Description copied from interface: StatsStorage

Get a list (shallow copy) of all listeners currently present

Specified by:

getListeners in interface StatsStorage

Returns:

List of listeners

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

notifyListeners

protected void notifyListeners(java.util.List<StatsStorageEvent> sses)