org.datavec.dataframe.store

Class StorageManager

java.lang.Object

org.datavec.dataframe.store.StorageManager

public class StorageManager
extends java.lang.Object

A controller for reading and writing data in Tablesaw's own compressed, column-oriented file format

Constructor Summary

Constructors

Constructor and Description
StorageManager()

Method Summary

Modifier and Type	Method and Description
static BooleanColumn	readBooleanColumn(java.lang.String fileName, ColumnMetadata metadata)
static CategoryColumn	readCategoryColumn(java.lang.String fileName, ColumnMetadata metadata)
static FloatColumn	readFloatColumn(java.lang.String fileName, ColumnMetadata metadata)
static IntColumn	readIntColumn(java.lang.String fileName, ColumnMetadata metadata)
static DateColumn	readLocalDateColumn(java.lang.String fileName, ColumnMetadata metadata)
static DateTimeColumn	readLocalDateTimeColumn(java.lang.String fileName, ColumnMetadata metadata)
static TimeColumn	readLocalTimeColumn(java.lang.String fileName, ColumnMetadata metadata)
static LongColumn	readLongColumn(java.lang.String fileName, ColumnMetadata metadata)
static ShortColumn	readShortColumn(java.lang.String fileName, ColumnMetadata metadata)
static Table	readTable(java.lang.String path) Reads a tablesaw table into memory
static TableMetadata	readTableMetadata(java.lang.String fileName) Reads in a json-formatted file and creates a TableMetadata instance from it.
static java.lang.String	saveTable(java.lang.String folderName, Relation table) Saves the data from the given table in the location specified by folderName.
static void	writeColumn(java.lang.String fileName, BooleanColumn column)
static void	writeColumn(java.lang.String fileName, CategoryColumn column) Writes out the values of the category column encoded as ints to minimize the time required for subsequent reads
static void	writeColumn(java.lang.String fileName, DateColumn column)
static void	writeColumn(java.lang.String fileName, DateTimeColumn column)
static void	writeColumn(java.lang.String fileName, FloatColumn column)
static void	writeColumn(java.lang.String fileName, IntColumn column)
static void	writeColumn(java.lang.String fileName, LongColumn column)
static void	writeColumn(java.lang.String fileName, ShortColumn column)
static void	writeColumn(java.lang.String fileName, TimeColumn column)
static void	writeTableMetadata(java.lang.String fileName, Relation table) Writes out a json-formatted representation of the given table's metadata to the given file

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StorageManager

public StorageManager()

Method Detail

readTable

public static Table readTable(java.lang.String path)
                       throws java.io.IOException

Reads a tablesaw table into memory

Parameters:

path - The location of the table. It is interpreted as relative to the working directory if not fully specified. The path will typically end in ".saw", as in "mytables/nasdaq-2015.saw"

Throws:

java.io.IOException - if the file cannot be read

readFloatColumn

public static FloatColumn readFloatColumn(java.lang.String fileName,
                                          ColumnMetadata metadata)
                                   throws java.io.IOException

Throws:

java.io.IOException

readIntColumn

public static IntColumn readIntColumn(java.lang.String fileName,
                                      ColumnMetadata metadata)
                               throws java.io.IOException

Throws:

java.io.IOException

readShortColumn

public static ShortColumn readShortColumn(java.lang.String fileName,
                                          ColumnMetadata metadata)
                                   throws java.io.IOException

Throws:

java.io.IOException

readLongColumn

public static LongColumn readLongColumn(java.lang.String fileName,
                                        ColumnMetadata metadata)
                                 throws java.io.IOException

Throws:

java.io.IOException

readLocalDateColumn

public static DateColumn readLocalDateColumn(java.lang.String fileName,
                                             ColumnMetadata metadata)
                                      throws java.io.IOException

Throws:

java.io.IOException

readLocalDateTimeColumn

public static DateTimeColumn readLocalDateTimeColumn(java.lang.String fileName,
                                                     ColumnMetadata metadata)
                                              throws java.io.IOException

Throws:

java.io.IOException

readLocalTimeColumn

public static TimeColumn readLocalTimeColumn(java.lang.String fileName,
                                             ColumnMetadata metadata)
                                      throws java.io.IOException

Throws:

java.io.IOException

readCategoryColumn

public static CategoryColumn readCategoryColumn(java.lang.String fileName,
                                                ColumnMetadata metadata)
                                         throws java.io.IOException

Throws:

java.io.IOException

readBooleanColumn

public static BooleanColumn readBooleanColumn(java.lang.String fileName,
                                              ColumnMetadata metadata)
                                       throws java.io.IOException

Throws:

java.io.IOException

saveTable

public static java.lang.String saveTable(java.lang.String folderName,
                                         Relation table)
                                  throws java.io.IOException

Saves the data from the given table in the location specified by folderName. Within that folder each table has its own sub-folder, whose name is based on the name of the table.

NOTE: If you store a table with the same name in the same folder. The data in that folder will be over-written.

The storage format is the tablesaw compressed column-oriented format, which consists of a set of file in a folder. The name of the folder is based on the name of the table.

Parameters:

folderName - The location of the table (for example: "mytables")

table - The table to be saved

Returns:

The path and name of the table

Throws:

java.io.IOException

writeColumn

public static void writeColumn(java.lang.String fileName,
                               FloatColumn column)
                        throws java.io.IOException

Throws:

java.io.IOException

writeColumn

public static void writeColumn(java.lang.String fileName,
                               CategoryColumn column)
                        throws java.io.IOException

Writes out the values of the category column encoded as ints to minimize the time required for subsequent reads

The files are written Strings first, then the ints that encode them so they can be read in the opposite order

Throws:

java.io.IOException

writeColumn

public static void writeColumn(java.lang.String fileName,
                               IntColumn column)
                        throws java.io.IOException

Throws:

java.io.IOException

writeColumn

public static void writeColumn(java.lang.String fileName,
                               ShortColumn column)
                        throws java.io.IOException

Throws:

java.io.IOException

writeColumn

public static void writeColumn(java.lang.String fileName,
                               LongColumn column)
                        throws java.io.IOException

Throws:

java.io.IOException

writeColumn

public static void writeColumn(java.lang.String fileName,
                               DateColumn column)
                        throws java.io.IOException

Throws:

java.io.IOException

writeColumn

public static void writeColumn(java.lang.String fileName,
                               DateTimeColumn column)
                        throws java.io.IOException

Throws:

java.io.IOException

writeColumn

public static void writeColumn(java.lang.String fileName,
                               TimeColumn column)
                        throws java.io.IOException

Throws:

java.io.IOException

writeColumn

public static void writeColumn(java.lang.String fileName,
                               BooleanColumn column)
                        throws java.io.IOException

Throws:

java.io.IOException

writeTableMetadata

public static void writeTableMetadata(java.lang.String fileName,
                                      Relation table)
                               throws java.io.IOException

Writes out a json-formatted representation of the given table's metadata to the given file

Parameters:

fileName - Expected to be fully specified

Throws:

java.io.IOException - if the file can not be read

readTableMetadata

public static TableMetadata readTableMetadata(java.lang.String fileName)
                                       throws java.io.IOException

Reads in a json-formatted file and creates a TableMetadata instance from it. Files are expected to be in the format provided by TableMetadata}

Parameters:

fileName - Expected to be fully specified

Throws:

java.io.IOException - if the file can not be read