com.google.cloud.dataflow.sdk.runners.worker

Class PartitioningShuffleReader<K,V>

java.lang.Object

java.util.Observable

com.google.cloud.dataflow.sdk.util.common.worker.Reader<WindowedValue<KV<K,V>>>

com.google.cloud.dataflow.sdk.runners.worker.PartitioningShuffleReader<K,V>

Type Parameters:

K - the type of the keys read from the shuffle

V - the type of the values read from the shuffle

public class PartitioningShuffleReader<K,V>
extends Reader<WindowedValue<KV<K,V>>>

A source that reads from a key-sharded dataset, and returns KVs without any values grouping.

Nested Class Summary

Nested classes/interfaces inherited from class com.google.cloud.dataflow.sdk.util.common.worker.Reader

Reader.AbstractReaderIterator<T>, Reader.DynamicSplitRequest, Reader.DynamicSplitResult, Reader.DynamicSplitResultWithPosition, Reader.Position, Reader.Progress, Reader.ReaderIterator<T>

Field Summary

Fields inherited from class com.google.cloud.dataflow.sdk.util.common.worker.Reader

stateSampler, stateSamplerOperationName

Constructor Summary

Constructors

Constructor and Description
PartitioningShuffleReader(PipelineOptions options, byte[] shuffleReaderConfig, String startShufflePosition, String stopShufflePosition, Coder<WindowedValue<KV<K,V>>> coder)

Method Summary

Modifier and Type	Method and Description
Reader.ReaderIterator<WindowedValue<KV<K,V>>>	iterator() Returns a ReaderIterator that allows reading from this source.

Methods inherited from class com.google.cloud.dataflow.sdk.util.common.worker.Reader

notifyElementRead, setStateSamplerAndOperationName, supportsRestart

Methods inherited from class java.util.Observable

addObserver, clearChanged, countObservers, deleteObserver, deleteObservers, hasChanged, notifyObservers, notifyObservers, setChanged

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PartitioningShuffleReader

public PartitioningShuffleReader(PipelineOptions options,
                                 byte[] shuffleReaderConfig,
                                 String startShufflePosition,
                                 String stopShufflePosition,
                                 Coder<WindowedValue<KV<K,V>>> coder)
                          throws Exception

Throws:

Exception

Method Detail

iterator

public Reader.ReaderIterator<WindowedValue<KV<K,V>>> iterator()
                                                       throws IOException

Description copied from class: Reader

Returns a ReaderIterator that allows reading from this source.

Specified by:

iterator in class Reader<WindowedValue<KV<K,V>>>

Throws:

IOException