com.google.cloud.dataflow.sdk.runners

Interface DirectPipelineRunner.EvaluationContext

All Superinterfaces:

DirectPipelineRunner.EvaluationResults, PipelineResult

Enclosing class:

DirectPipelineRunner

public static interface DirectPipelineRunner.EvaluationContext
extends DirectPipelineRunner.EvaluationResults

The interface provided to registered callbacks for interacting with the DirectPipelineRunner, including reading and writing the values of PCollections and PCollectionViews.

Nested Class Summary

Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.PipelineResult

PipelineResult.State

Method Summary

Modifier and Type	Method and Description
<T> Coder<T>	ensureCoderSerializable(Coder<T> coder) If the evaluation context is testing serializability, ensures that the argument Coder is serializable and deserializable by encoding it and then decoding it, and returning the result.
<T> T	ensureElementEncodable(TypedPValue<T> pvalue, T element) Ensures that the element is encodable and decodable using the TypePValue's coder, by encoding it and decoding it, and returning the result.
<FunctionT extends Serializable> FunctionT	ensureSerializable(FunctionT fn) If the evaluation context is testing serializability, ensures that the argument function is serializable and deserializable by encoding it and then decoding it, and returning the result.
<T> T	ensureSerializableByCoder(Coder<T> coder, T data, String errorContext) If the evaluation context is testing serializability, ensures that the given data is serializable and deserializable with the given Coder by encoding it and then decoding it, and returning the result.
com.google.cloud.dataflow.sdk.util.common.CounterSet.AddCounterMutator	getAddCounterMutator() Returns a mutator, which can be used to add additional counters to this EvaluationContext.
<InputT extends PInput> InputT	getInput(PTransform<InputT,?> transform) Returns the input of the currently being processed transform.
<OutputT extends POutput> OutputT	getOutput(PTransform<?,OutputT> transform) Returns the output of the currently being processed transform.
<T> List<DirectPipelineRunner.ValueWithMetadata<T>>	getPCollectionValuesWithMetadata(PCollection<T> pc) Retrieves the value of the given PCollection, along with element metadata such as timestamps and windows.
DirectPipelineOptions	getPipelineOptions() Returns the configured pipeline options.
String	getStepName(PTransform<?,?> transform) Gets the step name for this transform.
<T> List<T>	randomizeIfUnordered(List<T> elements, boolean inPlaceAllowed) If the evaluation context is testing unorderedness, randomly permutes the order of the elements, in a copy if !inPlaceAllowed, and returns the permuted list, otherwise returns the argument unchanged.
<T> void	setPCollection(PCollection<T> pc, List<T> elements) Shorthand for setting the value of a PCollection where the elements do not have timestamps or windows.
<T> void	setPCollectionValuesWithMetadata(PCollection<T> pc, List<DirectPipelineRunner.ValueWithMetadata<T>> elements) Sets the value of the given PCollection, where each element also has a timestamp and collection of windows.
<ElemT,T,WindowedT> void	setPCollectionView(PCollectionView<T> pc, Iterable<com.google.cloud.dataflow.sdk.util.WindowedValue<ElemT>> value) Sets the value associated with the given PCollectionView.
<T> void	setPCollectionWindowedValue(PCollection<T> pc, List<com.google.cloud.dataflow.sdk.util.WindowedValue<T>> elements) Sets the value of the given PCollection, where each element also has a timestamp and collection of windows.

Methods inherited from interface com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.EvaluationResults

getPCollection, getPCollectionList, getPCollectionView, getPCollectionWindowedValues

Methods inherited from interface com.google.cloud.dataflow.sdk.PipelineResult

getAggregatorValues, getState

Method Detail

getPipelineOptions

DirectPipelineOptions getPipelineOptions()

Returns the configured pipeline options.

getInput

<InputT extends PInput> InputT getInput(PTransform<InputT,?> transform)

Returns the input of the currently being processed transform.

getOutput

<OutputT extends POutput> OutputT getOutput(PTransform<?,OutputT> transform)

Returns the output of the currently being processed transform.

setPCollectionValuesWithMetadata

<T> void setPCollectionValuesWithMetadata(PCollection<T> pc,
                                          List<DirectPipelineRunner.ValueWithMetadata<T>> elements)

Sets the value of the given PCollection, where each element also has a timestamp and collection of windows. Throws an exception if the PCollection's value has already been set.

setPCollectionWindowedValue

<T> void setPCollectionWindowedValue(PCollection<T> pc,
                                     List<com.google.cloud.dataflow.sdk.util.WindowedValue<T>> elements)

Sets the value of the given PCollection, where each element also has a timestamp and collection of windows. Throws an exception if the PCollection's value has already been set.

setPCollection

<T> void setPCollection(PCollection<T> pc,
                        List<T> elements)

Shorthand for setting the value of a PCollection where the elements do not have timestamps or windows. Throws an exception if the PCollection's value has already been set.

getPCollectionValuesWithMetadata

<T> List<DirectPipelineRunner.ValueWithMetadata<T>> getPCollectionValuesWithMetadata(PCollection<T> pc)

Retrieves the value of the given PCollection, along with element metadata such as timestamps and windows. Throws an exception if the PCollection's value hasn't already been set.

setPCollectionView

<ElemT,T,WindowedT> void setPCollectionView(PCollectionView<T> pc,
                                            Iterable<com.google.cloud.dataflow.sdk.util.WindowedValue<ElemT>> value)

Sets the value associated with the given PCollectionView. Throws an exception if the PCollectionView's value has already been set.

ensureElementEncodable

<T> T ensureElementEncodable(TypedPValue<T> pvalue,
                             T element)

Ensures that the element is encodable and decodable using the TypePValue's coder, by encoding it and decoding it, and returning the result.

randomizeIfUnordered

<T> List<T> randomizeIfUnordered(List<T> elements,
                                 boolean inPlaceAllowed)

If the evaluation context is testing unorderedness, randomly permutes the order of the elements, in a copy if !inPlaceAllowed, and returns the permuted list, otherwise returns the argument unchanged.

ensureSerializable

<FunctionT extends Serializable> FunctionT ensureSerializable(FunctionT fn)

If the evaluation context is testing serializability, ensures that the argument function is serializable and deserializable by encoding it and then decoding it, and returning the result. Otherwise returns the argument unchanged.

ensureCoderSerializable

<T> Coder<T> ensureCoderSerializable(Coder<T> coder)

If the evaluation context is testing serializability, ensures that the argument Coder is serializable and deserializable by encoding it and then decoding it, and returning the result. Otherwise returns the argument unchanged.

ensureSerializableByCoder

<T> T ensureSerializableByCoder(Coder<T> coder,
                                T data,
                                String errorContext)

If the evaluation context is testing serializability, ensures that the given data is serializable and deserializable with the given Coder by encoding it and then decoding it, and returning the result. Otherwise returns the argument unchanged.

Error context is prefixed to any thrown exceptions.

getAddCounterMutator

com.google.cloud.dataflow.sdk.util.common.CounterSet.AddCounterMutator getAddCounterMutator()

Returns a mutator, which can be used to add additional counters to this EvaluationContext.

getStepName

String getStepName(PTransform<?,?> transform)

Gets the step name for this transform.