org.datavec.api.split

Class NumberedFileInputSplit

java.lang.Object

org.datavec.api.split.NumberedFileInputSplit

All Implemented Interfaces:

java.io.Serializable, InputSplit, Writable

public class NumberedFileInputSplit
extends java.lang.Object
implements InputSplit

InputSplit for sequences of numbered files. Example usages:
Suppose files are sequenced according to "myFile_100.txt", "myFile_101.txt", ..., "myFile_200.txt" then use new NumberedFileInputSplit("myFile_%d.txt",100,200) NumberedFileInputSplit utilizes String.format(), hence the requirement for "%d" to represent the integer index.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
NumberedFileInputSplit(java.lang.String baseString, int minIdxInclusive, int maxIdxInclusive)

Method Summary

Modifier and Type	Method and Description
long	length() Length of the split
java.net.URI[]	locations() Locations of the splits
java.util.Iterator<java.net.URI>	locationsIterator()
java.util.Iterator<java.lang.String>	locationsPathIterator()
void	readFields(java.io.DataInput in) Deserialize the fields of this object from in.
void	reset() Reset the InputSplit without reinitializing it from scratch.
double	toDouble() Convert Writable to double.
float	toFloat() Convert Writable to float.
int	toInt() Convert Writable to int.
long	toLong() Convert Writable to long.
void	write(java.io.DataOutput out) Serialize the fields of this object to out.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

NumberedFileInputSplit

public NumberedFileInputSplit(java.lang.String baseString,
                              int minIdxInclusive,
                              int maxIdxInclusive)

Parameters:

baseString - String that defines file format. Must contain "%d", which will be replaced with the index of the file.

minIdxInclusive - Minimum index/number (starting number in sequence of files, inclusive)

maxIdxInclusive - Maximum index/number (last number in sequence of files, inclusive)

Method Detail

length

public long length()

Description copied from interface: InputSplit

Length of the split

Specified by:

length in interface InputSplit

Returns:

locations

public java.net.URI[] locations()

Description copied from interface: InputSplit

Locations of the splits

Specified by:

locations in interface InputSplit

Returns:

locationsIterator

public java.util.Iterator<java.net.URI> locationsIterator()

Specified by:

locationsIterator in interface InputSplit

locationsPathIterator

public java.util.Iterator<java.lang.String> locationsPathIterator()

Specified by:

locationsPathIterator in interface InputSplit

reset

public void reset()

Description copied from interface: InputSplit

Reset the InputSplit without reinitializing it from scratch. In many cases, this is a no-op. For InputSplits that have randomization: reset should shuffle the order.

Specified by:

reset in interface InputSplit

write

public void write(java.io.DataOutput out)
           throws java.io.IOException

Description copied from interface: Writable

Serialize the fields of this object to out.

Specified by:

write in interface Writable

Parameters:

out - DataOuput to serialize this object into.

Throws:

java.io.IOException

readFields

public void readFields(java.io.DataInput in)
                throws java.io.IOException

Description copied from interface: Writable

Deserialize the fields of this object from in.

For efficiency, implementations should attempt to re-use storage in the existing object where possible.

Specified by:

readFields in interface Writable

Parameters:

in - DataInput to deseriablize this object from.

Throws:

java.io.IOException

toDouble

public double toDouble()

Description copied from interface: Writable

Convert Writable to double. Whether this is supported depends on the specific writable.

Specified by:

toDouble in interface Writable

toFloat

public float toFloat()

Description copied from interface: Writable

Convert Writable to float. Whether this is supported depends on the specific writable.

Specified by:

toFloat in interface Writable

toInt

public int toInt()

Description copied from interface: Writable

Convert Writable to int. Whether this is supported depends on the specific writable.

Specified by:

toInt in interface Writable

toLong

public long toLong()

Description copied from interface: Writable

Convert Writable to long. Whether this is supported depends on the specific writable.

Specified by:

toLong in interface Writable