org.datavec.api.split

Class InputStreamInputSplit

java.lang.Object

org.datavec.api.split.InputStreamInputSplit

All Implemented Interfaces:

java.io.Serializable, InputSplit, Writable

public class InputStreamInputSplit
extends java.lang.Object
implements InputSplit

Input stream input split. The normal pattern is reading the whole input stream and turning that in to a record. This is meant for streaming raw data rather than normal mini batch pre processing.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
InputStreamInputSplit(java.io.InputStream is)
InputStreamInputSplit(java.io.InputStream is, java.io.File path) Instantiate with the given file as a uri
InputStreamInputSplit(java.io.InputStream is, java.lang.String path) Instantiate with the given file as a uri
InputStreamInputSplit(java.io.InputStream is, java.net.URI path) Instantiate with the given file as a uri

Method Summary

Modifier and Type	Method and Description
java.io.InputStream	getIs()
long	length() Length of the split
java.net.URI[]	locations() Locations of the splits
java.util.Iterator<java.net.URI>	locationsIterator()
java.util.Iterator<java.lang.String>	locationsPathIterator()
void	readFields(java.io.DataInput in) Deserialize the fields of this object from in.
void	reset() Reset the InputSplit without reinitializing it from scratch.
void	setIs(java.io.InputStream is)
double	toDouble() Convert Writable to double.
float	toFloat() Convert Writable to float.
int	toInt() Convert Writable to int.
long	toLong() Convert Writable to long.
void	write(java.io.DataOutput out) Serialize the fields of this object to out.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

InputStreamInputSplit

public InputStreamInputSplit(java.io.InputStream is,
                             java.lang.String path)

Instantiate with the given file as a uri

Parameters:

is - the input stream to use

path - the path to use

InputStreamInputSplit

public InputStreamInputSplit(java.io.InputStream is,
                             java.io.File path)

Instantiate with the given file as a uri

Parameters:

is - the input stream to use

path - the path to use

InputStreamInputSplit

public InputStreamInputSplit(java.io.InputStream is,
                             java.net.URI path)

Instantiate with the given file as a uri

Parameters:

is - the input stream to use

path - the path to use

InputStreamInputSplit

public InputStreamInputSplit(java.io.InputStream is)

Method Detail

length

public long length()

Description copied from interface: InputSplit

Length of the split

Specified by:

length in interface InputSplit

Returns:

locations

public java.net.URI[] locations()

Description copied from interface: InputSplit

Locations of the splits

Specified by:

locations in interface InputSplit

Returns:

locationsIterator

public java.util.Iterator<java.net.URI> locationsIterator()

Specified by:

locationsIterator in interface InputSplit

locationsPathIterator

public java.util.Iterator<java.lang.String> locationsPathIterator()

Specified by:

locationsPathIterator in interface InputSplit

reset

public void reset()

Description copied from interface: InputSplit

Reset the InputSplit without reinitializing it from scratch. In many cases, this is a no-op. For InputSplits that have randomization: reset should shuffle the order.

Specified by:

reset in interface InputSplit

write

public void write(java.io.DataOutput out)
           throws java.io.IOException

Description copied from interface: Writable

Serialize the fields of this object to out.

Specified by:

write in interface Writable

Parameters:

out - DataOuput to serialize this object into.

Throws:

java.io.IOException

readFields

public void readFields(java.io.DataInput in)
                throws java.io.IOException

Description copied from interface: Writable

Deserialize the fields of this object from in.

For efficiency, implementations should attempt to re-use storage in the existing object where possible.

Specified by:

readFields in interface Writable

Parameters:

in - DataInput to deseriablize this object from.

Throws:

java.io.IOException

getIs

public java.io.InputStream getIs()

setIs

public void setIs(java.io.InputStream is)

toDouble

public double toDouble()

Description copied from interface: Writable

Convert Writable to double. Whether this is supported depends on the specific writable.

Specified by:

toDouble in interface Writable

toFloat

public float toFloat()

Description copied from interface: Writable

Convert Writable to float. Whether this is supported depends on the specific writable.

Specified by:

toFloat in interface Writable

toInt

public int toInt()

Description copied from interface: Writable

Convert Writable to int. Whether this is supported depends on the specific writable.

Specified by:

toInt in interface Writable

toLong

public long toLong()

Description copied from interface: Writable

Convert Writable to long. Whether this is supported depends on the specific writable.

Specified by:

toLong in interface Writable