| Interface | Description |
|---|---|
| PathToKeyConverter |
PathToKeyConverter: Used to match up files based on their file names, for PairSequenceRecordReaderBytesFunction
For example, suppose we have files "/features_0.csv" and "/labels_0.csv", map both to same key: "0"
|
| Class | Description |
|---|---|
| BytesPairWritable |
A Hadoop writable class for a pair of byte arrays, plus the original URIs (as Strings) of the files they came from
|
| MapToBytesPairWritableFunction |
A function to read files (assuming exactly 2 per input) from a PortableDataStream and combine the contents into a BytesPairWritable
|
| PairSequenceRecordReaderBytesFunction |
SequenceRecordReaderBytesFunction: Converts two sets of binary data (in the form of a BytesPairWritable) to DataVec format data
(
Tuple2<List<List<<Writable>>,List<List<Writable>>) using two SequenceRecordReaders. |
| PathToKeyConverterFilename |
Convert the path to a key by taking the full file name (excluding the file extension and directories)
|
| PathToKeyConverterNumber |
A PathToKeyConverter that generates a key based on the file name.
|
| PathToKeyFunction |
Given a Tuple2
PathToKeyConverter),
second is an index, and third is the original data stream |