org.datavec.local.transforms

Class TableRecords

java.lang.Object

org.datavec.local.transforms.TableRecords

public class TableRecords
extends java.lang.Object

Util class for interop between normal datavec records and the dataframe

Constructor Summary

Constructors

Constructor and Description
TableRecords()

Method Summary

Modifier and Type	Method and Description
static org.nd4j.linalg.api.ndarray.INDArray	arrayFromTable(Table table) Create a matrix from a table where the matrix will be of n rows x m columns
static Column[]	columnsForSchema(Schema schema) Extract a column array from the given schema
static java.lang.Object	determineInput(int row, Column... inputColumns) Determine the input type based on the column metadata
static java.util.List<java.util.List<Writable>>	executeNonTimeSeries(java.util.List<java.util.List<Writable>> inputWritables, TransformProcess transformProcess) Execute the specified TransformProcess with the given input data Note: this method can only be used if the TransformProcess returns non-sequence data.
static Table	filterTable(Table toFilter, Filter filter) Implements a filter
static Table	fromRecordsAndSchema(java.util.List<java.util.List<Writable>> writable, Schema schema) Create a table from records and a given schema.
static java.util.List<java.util.List<Writable>>	fromTable(Table table) Convert an all numeric table to a list of records
static java.lang.Object	getEntry(Column column, int row) Get an entry from the given column
static Filter	mapFilterFromCondition(Condition condition, java.lang.String columnName, Schema output) Map a condition based on an input column and an output schema to a data frame filter
static Filter	mapToFilter(Filter toMap) Map a given filter to a dataframe filter
static Table	reduce(Table reduce, Reducer reducer) Run a reduce operation on the given table
static void	setEntry(Column column, int row, java.lang.Object value) Get an entry from the given column
static void	setEntryList(Column[] columns, int row, java.lang.Object value) Set an entry from the given columns
static Table	sortedRank(Table toRank, CalculateSortedRank rank)
static Table	tableFromSchema(Schema schema) Create a table from the given schema
static Table	transformNonTimeSeries(Table table, TransformProcess transformProcess) Apply a transformNonTimeSeries process to the given table
static Table	transformTable(Table table, Transform transform) Run a transformNonTimeSeries operation on the table

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TableRecords

public TableRecords()

Method Detail

executeNonTimeSeries

public static java.util.List<java.util.List<Writable>> executeNonTimeSeries(java.util.List<java.util.List<Writable>> inputWritables,
                                                                            TransformProcess transformProcess)

Execute the specified TransformProcess with the given input data
Note: this method can only be used if the TransformProcess returns non-sequence data. For TransformProcesses that return a sequence, use #executeToSequence(JavaRDD, TransformProcess)

Parameters:

inputWritables - Input data to process

transformProcess - TransformProcess to executeNonTimeSeries

Returns:

Processed data

transformNonTimeSeries

public static Table transformNonTimeSeries(Table table,
                                           TransformProcess transformProcess)

Apply a transformNonTimeSeries process to the given table

Parameters:

table - the table to apply this to

transformProcess - the transformNonTimeSeries process to apply

Returns:

the transformed table

reduce

public static Table reduce(Table reduce,
                           Reducer reducer)

Run a reduce operation on the given table

Parameters:

reduce - the reduce operation to run

reducer - the reducer to run

Returns:

sortedRank

public static Table sortedRank(Table toRank,
                               CalculateSortedRank rank)

Parameters:

toRank -

rank -

Returns:

mapFilterFromCondition

public static Filter mapFilterFromCondition(Condition condition,
                                            java.lang.String columnName,
                                            Schema output)

Map a condition based on an input column and an output schema to a data frame filter

Parameters:

condition - the condition to map

columnName - the input column name to map

output - the output schema to infer the type

Returns:

the appropriate dataframe filter

mapToFilter

public static Filter mapToFilter(Filter toMap)

Map a given filter to a dataframe filter

Parameters:

toMap - the filter to map

Returns:

an apporiate Filter mapping from the datavec api

filterTable

public static Table filterTable(Table toFilter,
                                Filter filter)

Implements a filter

Parameters:

toFilter -

filter -

Returns:

transformTable

public static Table transformTable(Table table,
                                   Transform transform)

Run a transformNonTimeSeries operation on the table

Parameters:

table - the table to run the transformNonTimeSeries operation on

transform - the transformNonTimeSeries to run

Returns:

determineInput

public static java.lang.Object determineInput(int row,
                                              Column... inputColumns)

Determine the input type based on the column metadata

Parameters:

row - the row of the column to get the input for

inputColumns - the input columns to get input for

Returns:

a list of the type for the given metadata

setEntryList

public static void setEntryList(Column[] columns,
                                int row,
                                java.lang.Object value)

Set an entry from the given columns

Parameters:

columns - the columns to set the entry for

row - the row to get the entry from

value - an object of type List

setEntry

public static void setEntry(Column column,
                            int row,
                            java.lang.Object value)

Get an entry from the given column

Parameters:

column - the column to get the entry from

row - the row to get the entry from

value - the value to set

getEntry

public static java.lang.Object getEntry(Column column,
                                        int row)

Get an entry from the given column

Parameters:

column - the column to get the entry from

row - the row to get the entry from

Returns:

the entry from the given column at the given row

arrayFromTable

public static org.nd4j.linalg.api.ndarray.INDArray arrayFromTable(Table table)

Create a matrix from a table where the matrix will be of n rows x m columns

Parameters:

table - the table to create

Returns:

the matrix created from this table

fromTable

public static java.util.List<java.util.List<Writable>> fromTable(Table table)

Convert an all numeric table to a list of records

Parameters:

table - the table to convert

Returns:

the list of records from the given table

fromRecordsAndSchema

public static Table fromRecordsAndSchema(java.util.List<java.util.List<Writable>> writable,
                                         Schema schema)

Create a table from records and a given schema. The records should either be writables such that each list should be interpreted as a row in the table. Optionally, you can also have singleton lists of NDArrayWritable to represent columns. The given ndarray must be a row of 1 x m where m is schema.numColumns()

Parameters:

writable - the records to create the table from

schema - the schema to use

Returns:

the created table

tableFromSchema

public static Table tableFromSchema(Schema schema)

Create a table from the given schema

Parameters:

schema - the schema to create the table from

Returns:

the created table

columnsForSchema

public static Column[] columnsForSchema(Schema schema)

Extract a column array from the given schema

Parameters:

schema - the schema to get columns from

Returns:

a column array based on the given schema