org.datavec.spark.functions

Class RecordReaderFunction

java.lang.Object

org.datavec.spark.functions.RecordReaderFunction

All Implemented Interfaces:

java.io.Serializable, org.apache.spark.api.java.function.Function<scala.Tuple2<java.lang.String,org.apache.spark.input.PortableDataStream>,java.util.List<Writable>>

public class RecordReaderFunction
extends java.lang.Object
implements org.apache.spark.api.java.function.Function<scala.Tuple2<java.lang.String,org.apache.spark.input.PortableDataStream>,java.util.List<Writable>>

RecordReaderFunction: Given a RecordReader and a file (via Spark PortableDataStream), load and parse the data into a Collection. NOTE: This is only useful for "one record per file" type situations (ImageRecordReader, etc)

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected RecordReader	recordReader

Constructor Summary

Constructors

Constructor and Description
RecordReaderFunction(RecordReader recordReader)

Method Summary

Modifier and Type	Method and Description
java.util.List<Writable>	call(scala.Tuple2<java.lang.String,org.apache.spark.input.PortableDataStream> value)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

recordReader

protected RecordReader recordReader

Constructor Detail

RecordReaderFunction

public RecordReaderFunction(RecordReader recordReader)

Method Detail

call

public java.util.List<Writable> call(scala.Tuple2<java.lang.String,org.apache.spark.input.PortableDataStream> value)
                              throws java.lang.Exception

Specified by:

call in interface org.apache.spark.api.java.function.Function<scala.Tuple2<java.lang.String,org.apache.spark.input.PortableDataStream>,java.util.List<Writable>>

Throws:

java.lang.Exception