com.google.cloud.dataflow.sdk.util

Class WatermarkHold<W extends BoundedWindow>

java.lang.Object

com.google.cloud.dataflow.sdk.util.WatermarkHold<W>

Type Parameters:

W - The kind of BoundedWindow the hold is for.

All Implemented Interfaces:

Serializable

public class WatermarkHold<W extends BoundedWindow>
extends Object
implements Serializable

Implements the logic needed to hold the output watermark back to emit values with specific timestamps.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
WatermarkHold(WindowingStrategy<?,W> windowingStrategy)

Method Summary

Modifier and Type	Method and Description
void	addHold(ReduceFn.ProcessValueContext c, boolean isLate) Update the watermark hold to include the timestamp of the value in c.
StateContents<org.joda.time.Instant>	extractAndRelease(ReduceFn.Context c) Get information from the watermark hold for outputting.
void	holdForFinal(ReduceFn.Context c)
void	holdForOnTime(ReduceFn.Context c)
void	releaseFinal(ReduceFn.Context c)
void	releaseOnTime(ReduceFn.Context c)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WatermarkHold

public WatermarkHold(WindowingStrategy<?,W> windowingStrategy)

Method Detail

addHold

public void addHold(ReduceFn.ProcessValueContext c,
                    boolean isLate)

Update the watermark hold to include the timestamp of the value in c.

If the value was not late, then the input watermark must be less than the timestamp, and we can use WindowFn.getOutputTime(org.joda.time.Instant, W) to determine the appropriate output time.

If the value was late, we pessimistically assume the worst and attempt to hold the watermark to BoundedWindow.maxTimestamp() plus WindowingStrategy.getAllowedLateness(). That allows us to output the result at BoundedWindow.maxTimestamp() without being dropped.

extractAndRelease

public StateContents<org.joda.time.Instant> extractAndRelease(ReduceFn.Context c)

Get information from the watermark hold for outputting.

The output timestamp is the minimum of getOutputTimestamp applied to the non-late elements that arrived in the current pane.

holdForOnTime

public void holdForOnTime(ReduceFn.Context c)

holdForFinal

public void holdForFinal(ReduceFn.Context c)

releaseOnTime

public void releaseOnTime(ReduceFn.Context c)

releaseFinal

public void releaseFinal(ReduceFn.Context c)