com.google.cloud.dataflow.sdk.runners.worker

Class ApplianceShuffleReader

java.lang.Object

com.google.cloud.dataflow.sdk.runners.worker.ApplianceShuffleReader

@ThreadSafe
public final class ApplianceShuffleReader
extends Object

ApplianceShuffleReader reads chunks of data from a shuffle dataset for a position range.

It is a JNI wrapper of an equivalent C++ class.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ShuffleReader.ReadChunkResult Represents a chunk of data read from a shuffle dataset.

Constructor Summary

Constructors

Constructor and Description
ApplianceShuffleReader(byte[] shuffleReaderConfig)

Method Summary

Modifier and Type	Method and Description
void	finalize()
String	getDatasetId()
ShuffleReader.ReadChunkResult	readIncludingPosition(byte[] startPosition, byte[] endPosition) Reads a chunk of data for keys in the given position range.

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ApplianceShuffleReader

public ApplianceShuffleReader(byte[] shuffleReaderConfig)

Parameters:

shuffleReaderConfig - opaque configuration for creating a shuffle reader

Method Detail

finalize

public void finalize()

Overrides:

finalize in class Object

getDatasetId

public String getDatasetId()

readIncludingPosition

public ShuffleReader.ReadChunkResult readIncludingPosition(byte[] startPosition,
                                                           byte[] endPosition)
                                                    throws IOException

Reads a chunk of data for keys in the given position range. The chunk is a sequence of pairs encoded as: <position-size><position><key-size><key> <secondary-key-size><secondary-key><value-size><value> where the sizes are 4-byte big-endian integers.

Parameters:

startPosition - the start of the requested range (inclusive)

endPosition - the end of the requested range (exclusive)

Throws:

IOException