com.google.cloud.dataflow.sdk.runners

## Class DirectPipelineRunner.TestCombineDoFn<K,InputT,AccumT,OutputT>

- java.lang.Object
- - com.google.cloud.dataflow.sdk.transforms.DoFn<KV<K,Iterable<InputT>>,KV<K,OutputT>>
  - - com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.TestCombineDoFn<K,InputT,AccumT,OutputT>

All Implemented Interfaces:
  HasDisplayData, Serializable

Enclosing class:
  DirectPipelineRunner

    public static class DirectPipelineRunner.TestCombineDoFn<K,InputT,AccumT,OutputT>
    extends DoFn<KV<K,Iterable<InputT>>,KV<K,OutputT>>

The implementation may split the `Combine.KeyedCombineFn` into ADD, MERGE and EXTRACT phases ( see `com.google.cloud.dataflow.sdk.runners.worker.CombineValuesFn`). In order to emulate this for the `DirectPipelineRunner` and provide an experience closer to the service, go through heavy serializability checks for the equivalent of the results of the ADD phase, but after the `GroupByKey` shuffle, and the MERGE phase. Doing these checks ensure that not only is the accumulator coder serializable, but the accumulator coder can actually serialize the data in question.

See Also:
  Serialized Form

### Nested Class Summary

### Nested classes/interfaces inherited from class com.google.cloud.dataflow.sdk.transforms.DoFn

`DoFn.Context, DoFn.ProcessContext, DoFn.RequiresWindowAccess`

### Constructor Summary

Constructors
Constructor and Description
`TestCombineDoFn(com.google.cloud.dataflow.sdk.util.PerKeyCombineFnRunner<? super K,? super InputT,AccumT,OutputT> fnRunner, Coder<AccumT> accumCoder, boolean testSerializability, Random rand)`

### Method Summary

All Methods Static Methods Instance Methods Concrete Methods
`static <K,AccumT,InputT> List<AccumT>` `addInputsRandomly(com.google.cloud.dataflow.sdk.util.PerKeyCombineFnRunner<? super K,? super InputT,AccumT,?> fnRunner, K key, Iterable<InputT> values, Random random, DoFn.ProcessContext c)` Create a random list of accumulators from the given list of values.
`static <K,InputT,AccumT,OutputT> DirectPipelineRunner.TestCombineDoFn<K,InputT,AccumT,OutputT>` `create(Combine.GroupedValues<K,InputT,OutputT> transform, PCollection<KV<K,Iterable<InputT>>> input, boolean testSerializability, Random rand)`
`<T> T` `ensureSerializableByCoder(Coder<T> coder, T value, String errorContext)`
`void` `processElement(DoFn.ProcessContext c)` Processes one input element.

### Methods inherited from class com.google.cloud.dataflow.sdk.transforms.DoFn

`createAggregator, createAggregator, finishBundle, getAllowedTimestampSkew, getInputTypeDescriptor, getOutputTypeDescriptor, populateDisplayData, startBundle`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Constructor Detail

#### TestCombineDoFn

    public TestCombineDoFn(com.google.cloud.dataflow.sdk.util.PerKeyCombineFnRunner<? super K,? super InputT,AccumT,OutputT> fnRunner,
                           Coder<AccumT> accumCoder,
                           boolean testSerializability,
                           Random rand)

### Method Detail

#### create

    public static <K,InputT,AccumT,OutputT> DirectPipelineRunner.TestCombineDoFn<K,InputT,AccumT,OutputT> create(Combine.GroupedValues<K,InputT,OutputT> transform,
                                                                                                                 PCollection<KV<K,Iterable<InputT>>> input,
                                                                                                                 boolean testSerializability,
                                                                                                                 Random rand)

#### processElement

    public void processElement(DoFn.ProcessContext c)
                        throws Exception

Description copied from class: `DoFn`
Processes one input element.

The current element of the input `PCollection` is returned by `c.element()`. It should be considered immutable. The Dataflow runtime will not mutate the element, so it is safe to cache, etc. The element should not be mutated by any of the `DoFn` methods, because it may be cached elsewhere, retained by the Dataflow runtime, or used in other unspecified ways.

A value is added to the main output `PCollection` by `DoFn.Context.output(OutputT)`. Once passed to `output` the element should be considered immutable and not be modified in any way. It may be cached elsewhere, retained by the Dataflow runtime, or used in other unspecified ways.

Specified by:
  `processElement` in class `DoFn<KV<K,Iterable<InputT>>,KV<K,OutputT>>`
Throws:
  `Exception`
See Also:
  `DoFn.ProcessContext`

#### addInputsRandomly

    public static <K,AccumT,InputT> List<AccumT> addInputsRandomly(com.google.cloud.dataflow.sdk.util.PerKeyCombineFnRunner<? super K,? super InputT,AccumT,?> fnRunner,
                                                                   K key,
                                                                   Iterable<InputT> values,
                                                                   Random random,
                                                                   DoFn.ProcessContext c)

Create a random list of accumulators from the given list of values.

Visible for testing purposes only.

#### ensureSerializableByCoder

    public <T> T ensureSerializableByCoder(Coder<T> coder,
                                           T value,
                                           String errorContext)