org.datavec.image.recordreader

Class BaseImageRecordReader

java.lang.Object

org.datavec.api.records.reader.BaseRecordReader

org.datavec.image.recordreader.BaseImageRecordReader

All Implemented Interfaces:

java.io.Closeable, java.io.Serializable, java.lang.AutoCloseable, Configurable, RecordReader

Direct Known Subclasses:

ImageRecordReader

public abstract class BaseImageRecordReader
extends BaseRecordReader

Base class for the image record reader

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.List<java.lang.String>	allPaths
protected boolean	appendLabel
protected int	channels
static java.lang.String	CHANNELS
protected Configuration	conf
static java.lang.String	CROP_IMAGE
protected boolean	cropImage
protected java.io.File	currentFile
protected java.util.Map<java.lang.String,java.lang.String>	fileNameMap
protected int	height
static java.lang.String	HEIGHT
protected boolean	hitImage
static java.lang.String	IMAGE_LOADER
protected BaseImageLoader	imageLoader
protected ImageTransform	imageTransform
protected InputSplit	inputSplit
protected java.util.Iterator<java.io.File>	iter
protected PathLabelGenerator	labelGenerator
protected java.util.List<java.lang.String>	labels
protected java.lang.String	pattern
protected int	patternPosition
protected java.util.List<Writable>	record
protected int	width
static java.lang.String	WIDTH

Fields inherited from class org.datavec.api.records.reader.BaseRecordReader

listeners

Fields inherited from interface org.datavec.api.records.reader.RecordReader

APPEND_LABEL, LABELS, NAME_SPACE

Constructor Summary

Constructors

Constructor and Description
BaseImageRecordReader()
BaseImageRecordReader(int height, int width, int channels, PathLabelGenerator labelGenerator)
BaseImageRecordReader(int height, int width, int channels, PathLabelGenerator labelGenerator, ImageTransform imageTransform)

Method Summary

Modifier and Type	Method and Description
protected void	accumulateLabel(java.lang.String path) Accumulate the label from the path
void	close()
protected boolean	containsFormat(java.lang.String format)
Configuration	getConf() Return the configuration used by this object.
java.io.File	getCurrentFile() Returns the file loaded last by next().
java.lang.String	getLabel(java.lang.String path) Get the label from the given path
java.util.List<java.lang.String>	getLabels() List of label strings
boolean	hasNext() Whether there are anymore records
void	initialize(Configuration conf, InputSplit split) Called once at initialization.
void	initialize(Configuration conf, InputSplit split, ImageTransform imageTransform) Called once at initialization.
void	initialize(InputSplit split) Called once at initialization.
void	initialize(InputSplit split, ImageTransform imageTransform) Called once at initialization.
java.util.List<Record>	loadFromMetaData(java.util.List<RecordMetaData> recordMetaDatas) Load multiple records from the given a list of RecordMetaData instances
Record	loadFromMetaData(RecordMetaData recordMetaData) Load a single record from the given RecordMetaData instance Note: that for data that isn't splittable (i.e., text data that needs to be scanned/split), it is more efficient to load multiple records at once using RecordReader.loadFromMetaData(List)
java.util.List<Writable>	next() Get the next record
Record	nextRecord() Similar to RecordReader.next(), but returns a Record object, that may include metadata such as the source of the data
int	numLabels() Returns getLabels().size().
java.util.List<Writable>	record(java.net.URI uri, java.io.DataInputStream dataInputStream) Load the record from the given DataInputStream Unlike RecordReader.next() the internal state of the RecordReader is not modified Implementations of this method should not close the DataInputStream
void	reset() Reset record reader iterator
void	setConf(Configuration conf) Set the configuration to be used by this object.
void	setCurrentFile(java.io.File currentFile) Sets manually the file returned by getCurrentFile().
void	setLabels(java.util.List<java.lang.String> labels)

Methods inherited from class org.datavec.api.records.reader.BaseRecordReader

getListeners, invokeListeners, setListeners, setListeners

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

allPaths

protected java.util.List<java.lang.String> allPaths

iter

protected java.util.Iterator<java.io.File> iter

conf

protected Configuration conf

currentFile

protected java.io.File currentFile

labelGenerator

protected PathLabelGenerator labelGenerator

labels

protected java.util.List<java.lang.String> labels

appendLabel

protected boolean appendLabel

record

protected java.util.List<Writable> record

hitImage

protected boolean hitImage

height

protected int height

width

protected int width

channels

protected int channels

cropImage

protected boolean cropImage

imageTransform

protected ImageTransform imageTransform

imageLoader

protected BaseImageLoader imageLoader

inputSplit

protected InputSplit inputSplit

fileNameMap

protected java.util.Map<java.lang.String,java.lang.String> fileNameMap

pattern

protected java.lang.String pattern

patternPosition

protected int patternPosition

HEIGHT

public static final java.lang.String HEIGHT

WIDTH

public static final java.lang.String WIDTH

CHANNELS

public static final java.lang.String CHANNELS

CROP_IMAGE

public static final java.lang.String CROP_IMAGE

IMAGE_LOADER

public static final java.lang.String IMAGE_LOADER

Constructor Detail

BaseImageRecordReader

public BaseImageRecordReader()

BaseImageRecordReader

public BaseImageRecordReader(int height,
                             int width,
                             int channels,
                             PathLabelGenerator labelGenerator)

BaseImageRecordReader

public BaseImageRecordReader(int height,
                             int width,
                             int channels,
                             PathLabelGenerator labelGenerator,
                             ImageTransform imageTransform)

Method Detail

containsFormat

protected boolean containsFormat(java.lang.String format)

initialize

public void initialize(InputSplit split)
                throws java.io.IOException

Description copied from interface: RecordReader

Called once at initialization.

Parameters:

split - the split that defines the range of records to read

Throws:

java.io.IOException

initialize

public void initialize(Configuration conf,
                       InputSplit split)
                throws java.io.IOException,
                       java.lang.InterruptedException

Description copied from interface: RecordReader

Called once at initialization.

Parameters:

conf - a configuration for initialization

split - the split that defines the range of records to read

Throws:

java.io.IOException

java.lang.InterruptedException

initialize

public void initialize(InputSplit split,
                       ImageTransform imageTransform)
                throws java.io.IOException

Called once at initialization.

Parameters:

split - the split that defines the range of records to read

imageTransform - the image transform to use to transform images while loading them

Throws:

java.io.IOException

initialize

public void initialize(Configuration conf,
                       InputSplit split,
                       ImageTransform imageTransform)
                throws java.io.IOException,
                       java.lang.InterruptedException

Called once at initialization.

Parameters:

conf - a configuration for initialization

split - the split that defines the range of records to read

imageTransform - the image transform to use to transform images while loading them

Throws:

java.io.IOException

java.lang.InterruptedException

next

public java.util.List<Writable> next()

Description copied from interface: RecordReader

Get the next record

Returns:

hasNext

public boolean hasNext()

Description copied from interface: RecordReader

Whether there are anymore records

Returns:

close

public void close()
           throws java.io.IOException

Throws:

java.io.IOException

setConf

public void setConf(Configuration conf)

Description copied from interface: Configurable

Set the configuration to be used by this object.

getConf

public Configuration getConf()

Description copied from interface: Configurable

Return the configuration used by this object.

getLabel

public java.lang.String getLabel(java.lang.String path)

Get the label from the given path

Parameters:

path - the path to get the label from

Returns:

the label for the given path

accumulateLabel

protected void accumulateLabel(java.lang.String path)

Accumulate the label from the path

Parameters:

path - the path to get the label from

getCurrentFile

public java.io.File getCurrentFile()

Returns the file loaded last by next().

setCurrentFile

public void setCurrentFile(java.io.File currentFile)

Sets manually the file returned by getCurrentFile().

getLabels

public java.util.List<java.lang.String> getLabels()

Description copied from interface: RecordReader

List of label strings

Returns:

setLabels

public void setLabels(java.util.List<java.lang.String> labels)

reset

public void reset()

Description copied from interface: RecordReader

Reset record reader iterator

numLabels

public int numLabels()

Returns getLabels().size().

record

public java.util.List<Writable> record(java.net.URI uri,
                                       java.io.DataInputStream dataInputStream)
                                throws java.io.IOException

Description copied from interface: RecordReader

Load the record from the given DataInputStream Unlike RecordReader.next() the internal state of the RecordReader is not modified Implementations of this method should not close the DataInputStream

Throws:

java.io.IOException - if error occurs during reading from the input stream

nextRecord

public Record nextRecord()

Description copied from interface: RecordReader

Similar to RecordReader.next(), but returns a Record object, that may include metadata such as the source of the data

Returns:

next record

loadFromMetaData

public Record loadFromMetaData(RecordMetaData recordMetaData)
                        throws java.io.IOException

Description copied from interface: RecordReader

Load a single record from the given RecordMetaData instance
Note: that for data that isn't splittable (i.e., text data that needs to be scanned/split), it is more efficient to load multiple records at once using RecordReader.loadFromMetaData(List)

Parameters:

recordMetaData - Metadata for the record that we want to load from

Returns:

Single record for the given RecordMetaData instance

Throws:

java.io.IOException - If I/O error occurs during loading

loadFromMetaData

public java.util.List<Record> loadFromMetaData(java.util.List<RecordMetaData> recordMetaDatas)
                                        throws java.io.IOException

Description copied from interface: RecordReader

Load multiple records from the given a list of RecordMetaData instances

Parameters:

recordMetaDatas - Metadata for the records that we want to load from

Returns:

Multiple records for the given RecordMetaData instances

Throws:

java.io.IOException - If I/O error occurs during loading