org.datavec.spark.functions.data

Class FilesAsBytesFunction

java.lang.Object

org.datavec.spark.functions.data.FilesAsBytesFunction

All Implemented Interfaces:

java.io.Serializable, org.apache.spark.api.java.function.PairFunction<scala.Tuple2<java.lang.String,org.apache.spark.input.PortableDataStream>,org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable>

public class FilesAsBytesFunction
extends java.lang.Object
implements org.apache.spark.api.java.function.PairFunction<scala.Tuple2<java.lang.String,org.apache.spark.input.PortableDataStream>,org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable>

A PairFunction that simply loads bytes[] from a a PortableDataStream, and wraps it (and the String key) in Text and BytesWritable respectively.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
FilesAsBytesFunction()

Method Summary

Modifier and Type	Method and Description
scala.Tuple2<org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable>	call(scala.Tuple2<java.lang.String,org.apache.spark.input.PortableDataStream> in)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FilesAsBytesFunction

public FilesAsBytesFunction()

Method Detail

call

public scala.Tuple2<org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable> call(scala.Tuple2<java.lang.String,org.apache.spark.input.PortableDataStream> in)
                                                                                throws java.lang.Exception

Specified by:

call in interface org.apache.spark.api.java.function.PairFunction<scala.Tuple2<java.lang.String,org.apache.spark.input.PortableDataStream>,org.apache.hadoop.io.Text,org.apache.hadoop.io.BytesWritable>

Throws:

java.lang.Exception