com.google.cloud.dataflow.sdk.runners.inprocess

Interface InProcessPipelineRunner.CommittedBundle<T>

Type Parameters:

T - the type of elements contained within this bundle

Enclosing class:

InProcessPipelineRunner

public static interface InProcessPipelineRunner.CommittedBundle<T>

Part of a PCollection. Elements are output to an InProcessPipelineRunner.UncommittedBundle, which will eventually committed. Committed elements are executed by the PTransforms that consume the PCollection this bundle is a part of at a later point.

Method Summary

Modifier and Type	Method and Description
Iterable<com.google.cloud.dataflow.sdk.util.WindowedValue<T>>	getElements() Returns an Iterable containing all of the elements that have been added to this InProcessPipelineRunner.CommittedBundle.
com.google.cloud.dataflow.sdk.runners.inprocess.StructuralKey<?>	getKey() Returns the key that was output in the most recent GroupByKey in the execution of this bundle.
PCollection<T>	getPCollection() Returns the PCollection that the elements of this bundle belong to.
Instant	getSynchronizedProcessingOutputWatermark() Returns the processing time output watermark at the time the producing PTransform committed this bundle.
InProcessPipelineRunner.CommittedBundle<T>	withElements(Iterable<com.google.cloud.dataflow.sdk.util.WindowedValue<T>> elements) Return a new InProcessPipelineRunner.CommittedBundle that is like this one, except calls to getElements() will return the provided elements.

Method Detail

getPCollection

PCollection<T> getPCollection()

Returns the PCollection that the elements of this bundle belong to.

getKey

com.google.cloud.dataflow.sdk.runners.inprocess.StructuralKey<?> getKey()

Returns the key that was output in the most recent GroupByKey in the execution of this bundle.

getElements

Iterable<com.google.cloud.dataflow.sdk.util.WindowedValue<T>> getElements()

Returns an Iterable containing all of the elements that have been added to this InProcessPipelineRunner.CommittedBundle.

getSynchronizedProcessingOutputWatermark

Instant getSynchronizedProcessingOutputWatermark()

Returns the processing time output watermark at the time the producing PTransform committed this bundle. Downstream synchronized processing time watermarks cannot progress past this point before consuming this bundle.

This value is no greater than the earliest incomplete processing time or synchronized processing time timer at the time this bundle was committed, including any timers that fired to produce this bundle.

withElements

InProcessPipelineRunner.CommittedBundle<T> withElements(Iterable<com.google.cloud.dataflow.sdk.util.WindowedValue<T>> elements)

Return a new InProcessPipelineRunner.CommittedBundle that is like this one, except calls to getElements() will return the provided elements. This bundle is unchanged.

The value of the synchronized processing output watermark of the returned InProcessPipelineRunner.CommittedBundle is equal to the value returned from the current bundle. This is used to ensure a PTransform that could not complete processing on input elements properly holds the synchronized processing time to the appropriate value.