public class RecordReaderDataSetIterator
extends java.lang.Object
implements org.nd4j.linalg.dataset.api.iterator.DataSetIterator
Modifier and Type | Field and Description |
---|---|
protected int |
batchNum |
protected int |
batchSize |
protected org.datavec.api.io.WritableConverter |
converter |
protected int |
labelIndex |
protected int |
labelIndexTo |
protected org.nd4j.linalg.dataset.DataSet |
last |
protected int |
maxNumBatches |
protected int |
numPossibleLabels |
protected org.nd4j.linalg.dataset.api.DataSetPreProcessor |
preProcessor |
protected org.datavec.api.records.reader.RecordReader |
recordReader |
protected boolean |
regression |
protected java.util.Iterator<java.util.List<org.datavec.api.writable.Writable>> |
sequenceIter |
protected boolean |
useCurrent |
Constructor and Description |
---|
RecordReaderDataSetIterator(org.datavec.api.records.reader.RecordReader recordReader,
int batchSize) |
RecordReaderDataSetIterator(org.datavec.api.records.reader.RecordReader recordReader,
int batchSize,
int labelIndex,
int numPossibleLabels)
Main constructor for classification.
|
RecordReaderDataSetIterator(org.datavec.api.records.reader.RecordReader recordReader,
int batchSize,
int labelIndexFrom,
int labelIndexTo,
boolean regression)
Main constructor for multi-label regression (i.e., regression with multiple outputs)
|
RecordReaderDataSetIterator(org.datavec.api.records.reader.RecordReader recordReader,
int batchSize,
int labelIndex,
int numPossibleLabels,
int maxNumBatches) |
RecordReaderDataSetIterator(org.datavec.api.records.reader.RecordReader recordReader,
org.datavec.api.io.WritableConverter converter,
int batchSize) |
RecordReaderDataSetIterator(org.datavec.api.records.reader.RecordReader recordReader,
org.datavec.api.io.WritableConverter converter,
int batchSize,
int labelIndex,
int numPossibleLabels) |
RecordReaderDataSetIterator(org.datavec.api.records.reader.RecordReader recordReader,
org.datavec.api.io.WritableConverter converter,
int batchSize,
int labelIndex,
int numPossibleLabels,
boolean regression) |
RecordReaderDataSetIterator(org.datavec.api.records.reader.RecordReader recordReader,
org.datavec.api.io.WritableConverter converter,
int batchSize,
int labelIndex,
int numPossibleLabels,
int maxNumBatches,
boolean regression) |
RecordReaderDataSetIterator(org.datavec.api.records.reader.RecordReader recordReader,
org.datavec.api.io.WritableConverter converter,
int batchSize,
int labelIndexFrom,
int labelIndexTo,
int numPossibleLabels,
int maxNumBatches,
boolean regression)
Main constructor
|
Modifier and Type | Method and Description |
---|---|
boolean |
asyncSupported() |
int |
batch() |
int |
cursor() |
java.util.List<java.lang.String> |
getLabels() |
boolean |
hasNext() |
int |
inputColumns() |
org.nd4j.linalg.dataset.DataSet |
loadFromMetaData(java.util.List<org.datavec.api.records.metadata.RecordMetaData> list)
Load a multiple examples to a DataSet, using the provided RecordMetaData instances.
|
org.nd4j.linalg.dataset.DataSet |
loadFromMetaData(org.datavec.api.records.metadata.RecordMetaData recordMetaData)
Load a single example to a DataSet, using the provided RecordMetaData.
|
org.nd4j.linalg.dataset.DataSet |
next() |
org.nd4j.linalg.dataset.DataSet |
next(int num) |
int |
numExamples() |
void |
remove() |
void |
reset() |
boolean |
resetSupported() |
void |
setPreProcessor(org.nd4j.linalg.dataset.api.DataSetPreProcessor preProcessor) |
int |
totalExamples() |
int |
totalOutcomes() |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
protected org.datavec.api.records.reader.RecordReader recordReader
protected org.datavec.api.io.WritableConverter converter
protected int batchSize
protected int maxNumBatches
protected int batchNum
protected int labelIndex
protected int labelIndexTo
protected int numPossibleLabels
protected java.util.Iterator<java.util.List<org.datavec.api.writable.Writable>> sequenceIter
protected org.nd4j.linalg.dataset.DataSet last
protected boolean useCurrent
protected boolean regression
protected org.nd4j.linalg.dataset.api.DataSetPreProcessor preProcessor
public RecordReaderDataSetIterator(org.datavec.api.records.reader.RecordReader recordReader, org.datavec.api.io.WritableConverter converter, int batchSize)
public RecordReaderDataSetIterator(org.datavec.api.records.reader.RecordReader recordReader, int batchSize)
public RecordReaderDataSetIterator(org.datavec.api.records.reader.RecordReader recordReader, int batchSize, int labelIndex, int numPossibleLabels)
recordReader
- RecordReader: provides the source of the databatchSize
- Batch size (number of examples) for the output DataSet objectslabelIndex
- Index of the label Writable (usually an IntWritable), as obtained by recordReader.next()numPossibleLabels
- Number of classes (possible labels) for classificationpublic RecordReaderDataSetIterator(org.datavec.api.records.reader.RecordReader recordReader, org.datavec.api.io.WritableConverter converter, int batchSize, int labelIndex, int numPossibleLabels, boolean regression)
public RecordReaderDataSetIterator(org.datavec.api.records.reader.RecordReader recordReader, org.datavec.api.io.WritableConverter converter, int batchSize, int labelIndex, int numPossibleLabels)
public RecordReaderDataSetIterator(org.datavec.api.records.reader.RecordReader recordReader, int batchSize, int labelIndex, int numPossibleLabels, int maxNumBatches)
public RecordReaderDataSetIterator(org.datavec.api.records.reader.RecordReader recordReader, int batchSize, int labelIndexFrom, int labelIndexTo, boolean regression)
recordReader
- RecordReader to get data fromlabelIndexFrom
- Index of the first regression targetlabelIndexTo
- Index of the last regression target, inclusivebatchSize
- Minibatch sizeregression
- Require regression = true. Mainly included to avoid clashing with other constructors previously defined :/public RecordReaderDataSetIterator(org.datavec.api.records.reader.RecordReader recordReader, org.datavec.api.io.WritableConverter converter, int batchSize, int labelIndex, int numPossibleLabels, int maxNumBatches, boolean regression)
public RecordReaderDataSetIterator(org.datavec.api.records.reader.RecordReader recordReader, org.datavec.api.io.WritableConverter converter, int batchSize, int labelIndexFrom, int labelIndexTo, int numPossibleLabels, int maxNumBatches, boolean regression)
recordReader
- the recordreader to useconverter
- the batch sizemaxNumBatches
- Maximum number of batches to returnlabelIndexFrom
- the index of the label (for classification), or the first index of the labels for multi-output regressionlabelIndexTo
- only used if regression == true. The last index _inclusive_ of the multi-output regressionnumPossibleLabels
- the number of possible labels for classification. Not used if regression == trueregression
- if true: regression. If false: classification (assume labelIndexFrom is apublic org.nd4j.linalg.dataset.DataSet next(int num)
next
in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator
public int totalExamples()
totalExamples
in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator
public int inputColumns()
inputColumns
in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator
public int totalOutcomes()
totalOutcomes
in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator
public boolean resetSupported()
resetSupported
in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator
public boolean asyncSupported()
asyncSupported
in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator
public void reset()
reset
in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator
public int batch()
batch
in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator
public int cursor()
cursor
in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator
public int numExamples()
numExamples
in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator
public void setPreProcessor(org.nd4j.linalg.dataset.api.DataSetPreProcessor preProcessor)
setPreProcessor
in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator
public boolean hasNext()
hasNext
in interface java.util.Iterator<org.nd4j.linalg.dataset.DataSet>
public org.nd4j.linalg.dataset.DataSet next()
next
in interface java.util.Iterator<org.nd4j.linalg.dataset.DataSet>
public void remove()
remove
in interface java.util.Iterator<org.nd4j.linalg.dataset.DataSet>
public java.util.List<java.lang.String> getLabels()
getLabels
in interface org.nd4j.linalg.dataset.api.iterator.DataSetIterator
public org.nd4j.linalg.dataset.DataSet loadFromMetaData(org.datavec.api.records.metadata.RecordMetaData recordMetaData) throws java.io.IOException
loadFromMetaData(List)
recordMetaData
- RecordMetaData to load from. Should have been produced by the given record readerjava.io.IOException
- If an error occurs during loading of the datapublic org.nd4j.linalg.dataset.DataSet loadFromMetaData(java.util.List<org.datavec.api.records.metadata.RecordMetaData> list) throws java.io.IOException
list
- List of RecordMetaData instances to load from. Should have been produced by the record reader provided
to the RecordReaderDataSetIterator constructorjava.io.IOException
- If an error occurs during loading of the data