protected abstract class FileBasedReader.FileBasedIterator extends AbstractBoundedReaderIterator<T>
| Modifier and Type | Field and Description |
|---|---|
protected FileBasedReader.DecompressingStreamFactory |
compressionStreamFactory |
protected ByteArrayOutputStream |
nextElement |
protected long |
offset |
protected ProgressTracker<Integer> |
progressTracker |
protected OffsetRangeTracker |
rangeTracker |
protected com.google.cloud.dataflow.sdk.runners.worker.CopyableSeekableByteChannel |
seeker |
protected PushbackInputStream |
stream |
| Modifier and Type | Method and Description |
|---|---|
void |
close() |
Reader.Progress |
getProgress()
Returns a representation of how far this iterator is through the source.
|
protected boolean |
hasNextImpl() |
protected T |
nextImpl() |
protected abstract ByteArrayOutputStream |
readElement()
Reads the next element.
|
Reader.DynamicSplitResult |
requestDynamicSplit(Reader.DynamicSplitRequest splitRequest)
Attempts to split the input in two parts: the "primary" part and the "residual" part.
|
hasNext, nextcopyprotected final com.google.cloud.dataflow.sdk.runners.worker.CopyableSeekableByteChannel seeker
protected final PushbackInputStream stream
protected final OffsetRangeTracker rangeTracker
protected long offset
protected final ProgressTracker<Integer> progressTracker
protected ByteArrayOutputStream nextElement
protected FileBasedReader.DecompressingStreamFactory compressionStreamFactory
protected abstract ByteArrayOutputStream readElement() throws IOException
ByteArrayOutputStream containing the contents
of the element, or null if the end of the stream
has been reached.IOException - if an I/O error occursprotected boolean hasNextImpl()
throws IOException
hasNextImpl in class AbstractBoundedReaderIterator<T>IOExceptionprotected T nextImpl() throws IOException
nextImpl in class AbstractBoundedReaderIterator<T>IOExceptionpublic Reader.Progress getProgress()
Reader.ReaderIteratorgetProgress in interface Reader.ReaderIterator<T>getProgress in class Reader.AbstractReaderIterator<T>null if no progress measure
can be provided (implementors are discouraged from throwing
UnsupportedOperationException in this case).public Reader.DynamicSplitResult requestDynamicSplit(Reader.DynamicSplitRequest splitRequest)
Reader.ReaderIteratorReader.ReaderIterator keeps processing the primary part, while the residual part
will be processed elsewhere (e.g. perhaps on a different worker).
The primary and residual parts, if concatenated, must represent the same input as the
current input of this Reader.ReaderIterator before this call.
The boundary between the primary part and the residual part is specified in
a framework-specific way using Reader.DynamicSplitRequest: e.g., if the framework
supports the notion of positions, it might be a position at which the input is asked to split
itself (which is not necessarily the same position at which it will split itself);
it might be an approximate fraction of input, or something else.
Reader.DynamicSplitResult encodes, in a framework-specific way, the information
sufficient to construct a description of the resulting primary and residual inputs.
For example, it might, again, be a position demarcating these parts, or it might be a pair of
fully-specified input descriptions, or something else.
After a successful call to Reader.ReaderIterator.requestDynamicSplit(com.google.cloud.dataflow.sdk.util.common.worker.Reader.DynamicSplitRequest), subsequent calls should be
interpreted relative to the new primary.
This call should not affect the range of input represented by the Reader that
produced this Reader.ReaderIterator.
requestDynamicSplit in interface Reader.ReaderIterator<T>requestDynamicSplit in class Reader.AbstractReaderIterator<T>null if the Reader.DynamicSplitRequest cannot be honored
(in that case the input represented by this Reader.ReaderIterator stays the same), or
a Reader.DynamicSplitResult describing how the input was split into a primary
and residual part.public void close()
throws IOException
close in interface Reader.ReaderIterator<T>close in interface AutoCloseableclose in class Reader.AbstractReaderIterator<T>IOException