com.google.cloud.dataflow.sdk.runners.worker

Class GroupingShuffleReader<K,V>

java.lang.Object

java.util.Observable

com.google.cloud.dataflow.sdk.util.common.worker.Reader<WindowedValue<KV<K,Reiterable<V>>>>

com.google.cloud.dataflow.sdk.runners.worker.GroupingShuffleReader<K,V>

Type Parameters:

K - the type of the keys read from the shuffle

V - the type of the values read from the shuffle

public class GroupingShuffleReader<K,V>
extends Reader<WindowedValue<KV<K,Reiterable<V>>>>

A source that reads from a shuffled dataset and yields key-grouped data.

Nested Class Summary

Nested classes/interfaces inherited from class com.google.cloud.dataflow.sdk.util.common.worker.Reader

Reader.AbstractReaderIterator<T>, Reader.DynamicSplitRequest, Reader.DynamicSplitResult, Reader.DynamicSplitResultWithPosition, Reader.Position, Reader.Progress, Reader.ReaderIterator<T>

Field Summary

Fields inherited from class com.google.cloud.dataflow.sdk.util.common.worker.Reader

stateSampler, stateSamplerOperationName

Constructor Summary

Constructors

Constructor and Description
GroupingShuffleReader(PipelineOptions options, byte[] shuffleReaderConfig, String startShufflePosition, String stopShufflePosition, Coder<WindowedValue<KV<K,Iterable<V>>>> coder, BatchModeExecutionContext executionContext)

Method Summary

Modifier and Type	Method and Description
Reader.ReaderIterator<WindowedValue<KV<K,Reiterable<V>>>>	iterator() Returns a ReaderIterator that allows reading from this source.

Methods inherited from class com.google.cloud.dataflow.sdk.util.common.worker.Reader

notifyElementRead, setStateSamplerAndOperationName, supportsRestart

Methods inherited from class java.util.Observable

addObserver, clearChanged, countObservers, deleteObserver, deleteObservers, hasChanged, notifyObservers, notifyObservers, setChanged

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GroupingShuffleReader

public GroupingShuffleReader(PipelineOptions options,
                             byte[] shuffleReaderConfig,
                             @Nullable
                             String startShufflePosition,
                             @Nullable
                             String stopShufflePosition,
                             Coder<WindowedValue<KV<K,Iterable<V>>>> coder,
                             BatchModeExecutionContext executionContext)
                      throws Exception

Throws:

Exception

Method Detail

iterator

public Reader.ReaderIterator<WindowedValue<KV<K,Reiterable<V>>>> iterator()
                                                                   throws IOException

Description copied from class: Reader

Returns a ReaderIterator that allows reading from this source.

Specified by:

iterator in class Reader<WindowedValue<KV<K,Reiterable<V>>>>

Throws:

IOException