com.google.cloud.dataflow.sdk.util.common.worker

Interface ShuffleBatchReader

All Known Implementing Classes:

CachingShuffleBatchReader

public interface ShuffleBatchReader

ShuffleBatchReader provides an interface for reading a batch of key/value entries from a shuffle dataset.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	ShuffleBatchReader.Batch The result returned by #read.

Method Summary

Modifier and Type	Method and Description
ShuffleBatchReader.Batch	read(ShufflePosition startPosition, ShufflePosition endPosition) Reads a batch of data from a shuffle dataset.

Method Detail

read

ShuffleBatchReader.Batch read(@Nullable
                              ShufflePosition startPosition,
                              @Nullable
                              ShufflePosition endPosition)
                       throws IOException

Reads a batch of data from a shuffle dataset.

Parameters:

startPosition - encodes the initial key from where to read. This parameter may be null, indicating that the read should start with the first key in the dataset.

endPosition - encodes the key "just past" the end of the range to be read; keys up to endPosition will be returned, but keys equal to or greater than endPosition will not. This parameter may be null, indicating that the read should end just past the last key in the dataset (that is, the last key in the dataset will be included in the read, as long as that key is greater than or equal to startPosition).

Returns:

the first ShuffleBatchReader.Batch of entries

Throws:

IOException