com.google.cloud.dataflow.sdk.transforms.windowing

Class Sessions

java.lang.Object

com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn<Object,IntervalWindow>

com.google.cloud.dataflow.sdk.transforms.windowing.Sessions

All Implemented Interfaces:

Serializable

public class Sessions
extends WindowFn<Object,IntervalWindow>

A WindowFn windowing values into sessions separated by gapDuration-long periods with no elements.

For example, in order to window data into session with at least 10 minute gaps in between them:

 
 PCollection<Integer> pc = ...;
 PCollection<Integer> windowed_pc = pc.apply(
   Window.<Integer>into(Sessions.withGapDuration(Duration.standardMinutes(10))));
  

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn

WindowFn.AssignContext, WindowFn.MergeContext

Method Summary

Modifier and Type	Method and Description
Collection<IntervalWindow>	assignWindows(WindowFn.AssignContext c) Given a timestamp and element, returns the set of windows into which it should be placed.
org.joda.time.Duration	getGapDuration()
org.joda.time.Instant	getOutputTime(org.joda.time.Instant inputTimestamp, IntervalWindow window) Returns the output timestamp to use for data depending on the given inputTimestamp in the specified window.
IntervalWindow	getSideInputWindow(BoundedWindow window) Returns the window of the side input corresponding to the given window of the main input.
boolean	isCompatible(WindowFn<?,?> other) Returns whether this performs the same merging as the given WindowFn.
void	mergeWindows(WindowFn.MergeContext c) Does whatever merging of windows is necessary.
Coder<IntervalWindow>	windowCoder() Returns the Coder used for serializing the windows used by this windowFn.
static Sessions	withGapDuration(org.joda.time.Duration gapDuration) Creates a Sessions WindowFn with the specified gap duration.

Methods inherited from class com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn

assignsToSingleWindow, isNonMerging

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

withGapDuration

public static Sessions withGapDuration(org.joda.time.Duration gapDuration)

Creates a Sessions WindowFn with the specified gap duration.

assignWindows

public Collection<IntervalWindow> assignWindows(WindowFn.AssignContext c)

Description copied from class: WindowFn

Given a timestamp and element, returns the set of windows into which it should be placed.

Specified by:

assignWindows in class WindowFn<Object,IntervalWindow>

mergeWindows

public void mergeWindows(WindowFn.MergeContext c)
                  throws Exception

Description copied from class: WindowFn

Does whatever merging of windows is necessary.

See MergeOverlappingIntervalWindows.mergeWindows(com.google.cloud.dataflow.sdk.transforms.windowing.WindowFn<?, com.google.cloud.dataflow.sdk.transforms.windowing.IntervalWindow>.MergeContext) for an example of how to override this method.

Specified by:

mergeWindows in class WindowFn<Object,IntervalWindow>

Throws:

Exception

windowCoder

public Coder<IntervalWindow> windowCoder()

Description copied from class: WindowFn

Returns the Coder used for serializing the windows used by this windowFn.

Specified by:

windowCoder in class WindowFn<Object,IntervalWindow>

isCompatible

public boolean isCompatible(WindowFn<?,?> other)

Description copied from class: WindowFn

Returns whether this performs the same merging as the given WindowFn.

Specified by:

isCompatible in class WindowFn<Object,IntervalWindow>

getSideInputWindow

public IntervalWindow getSideInputWindow(BoundedWindow window)

Description copied from class: WindowFn

Returns the window of the side input corresponding to the given window of the main input.

Authors of custom WindowFns should override this.

Specified by:

getSideInputWindow in class WindowFn<Object,IntervalWindow>

getGapDuration

public org.joda.time.Duration getGapDuration()

getOutputTime

public org.joda.time.Instant getOutputTime(org.joda.time.Instant inputTimestamp,
                                           IntervalWindow window)

Description copied from class: WindowFn

Returns the output timestamp to use for data depending on the given inputTimestamp in the specified window.

The result must be between inputTimestamp and window.maxTimestamp() (inclusive on both sides). If this WindowFn doesn't produce overlapping windows, this can (and typically should) just return inputTimestamp. If this does produce overlapping windows, it is suggested that the result in later overlapping windows is past the end of earlier windows so that the later windows don't prevent the watermark from progressing past the end of the earlier window.

Each KV<K, Iterable<V>> produced from a GroupByKey will be output at a timestamp that is the minimum of getOutputTime applied to the timestamp of all of the non-late KV<K, V> that were used as input to the GroupByKey. The watermark is also prevented from advancing past this minimum timestamp until after the KV<K, Iterable<V>> has been output.

This function should be monotonic across input timestamps. Specifically, if A < B, then getOutputTime(A, window) <= getOutputTime(B, window).

Specified by:

getOutputTime in class WindowFn<Object,IntervalWindow>