com.google.cloud.dataflow.sdk.util.common.worker

Class CachingShuffleBatchReader

java.lang.Object

com.google.cloud.dataflow.sdk.util.common.worker.CachingShuffleBatchReader

All Implemented Interfaces:

ShuffleBatchReader

public final class CachingShuffleBatchReader
extends Object
implements ShuffleBatchReader

A ShuffleBatchReader that caches batches as they're read.

Nested Class Summary

Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.util.common.worker.ShuffleBatchReader

ShuffleBatchReader.Batch

Constructor Summary

Constructors

Constructor and Description
CachingShuffleBatchReader(ShuffleBatchReader reader) Constructs a new CachingShuffleBatchReader.

Method Summary

Modifier and Type	Method and Description
ShuffleBatchReader.Batch	read(ShufflePosition startPosition, ShufflePosition endPosition) Reads a batch of data from a shuffle dataset.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CachingShuffleBatchReader

public CachingShuffleBatchReader(ShuffleBatchReader reader)

Constructs a new CachingShuffleBatchReader.

Parameters:

reader - supplies the downstream ShuffleBatchReader this CachingShuffleBatchReader will use to issue reads

Method Detail

read

public ShuffleBatchReader.Batch read(@Nullable
                                     ShufflePosition startPosition,
                                     @Nullable
                                     ShufflePosition endPosition)
                              throws IOException

Description copied from interface: ShuffleBatchReader

Reads a batch of data from a shuffle dataset.

Specified by:

read in interface ShuffleBatchReader

Parameters:

startPosition - encodes the initial key from where to read. This parameter may be null, indicating that the read should start with the first key in the dataset.

endPosition - encodes the key "just past" the end of the range to be read; keys up to endPosition will be returned, but keys equal to or greater than endPosition will not. This parameter may be null, indicating that the read should end just past the last key in the dataset (that is, the last key in the dataset will be included in the read, as long as that key is greater than or equal to startPosition).

Returns:

the first ShuffleBatchReader.Batch of entries

Throws:

IOException