com.google.cloud.dataflow.sdk.runners

Class BlockingDataflowPipelineRunner

java.lang.Object

com.google.cloud.dataflow.sdk.runners.PipelineRunner<BlockingDataflowPipelineRunner.PipelineJobState>

com.google.cloud.dataflow.sdk.runners.BlockingDataflowPipelineRunner

Direct Known Subclasses:

TestDataflowPipelineRunner

public class BlockingDataflowPipelineRunner
extends PipelineRunner<BlockingDataflowPipelineRunner.PipelineJobState>

A PipelineRunner that's like DataflowPipelineRunner but that waits for the launched job to finish.

Prints out job status updates and console messages while it waits.

Returns the final job state, or throws an exception if the job fails or cannot be monitored.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BlockingDataflowPipelineRunner.PipelineJobState Holds the status of a run request.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	BlockingDataflowPipelineRunner(DataflowPipelineRunner internalRunner, com.google.cloud.dataflow.sdk.util.MonitoringUtil.JobMessagesHandler jobMessagesHandler)

Method Summary

Methods

Modifier and Type	Method and Description
<Output extends POutput,Input extends PInput> Output	apply(PTransform<Input,Output> transform, Input input) Applies a transform to the given input, returning the output.
static BlockingDataflowPipelineRunner	fromOptions(PipelineOptions options) Constructs a runner from the provided options.
BlockingDataflowPipelineRunner.PipelineJobState	run(Pipeline p) Processes the given Pipeline, returning the results.
void	setHooks(DataflowPipelineRunnerHooks hooks) Sets callbacks to invoke during execution see DataflowPipelineRunnerHooks.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BlockingDataflowPipelineRunner

protected BlockingDataflowPipelineRunner(DataflowPipelineRunner internalRunner,
                              com.google.cloud.dataflow.sdk.util.MonitoringUtil.JobMessagesHandler jobMessagesHandler)

Method Detail

fromOptions

public static BlockingDataflowPipelineRunner fromOptions(PipelineOptions options)

Constructs a runner from the provided options.

run

public BlockingDataflowPipelineRunner.PipelineJobState run(Pipeline p)

Description copied from class: PipelineRunner

Processes the given Pipeline, returning the results.

Specified by:

run in class PipelineRunner<BlockingDataflowPipelineRunner.PipelineJobState>

apply

public <Output extends POutput,Input extends PInput> Output apply(PTransform<Input,Output> transform,
                                                         Input input)

Description copied from class: PipelineRunner

Applies a transform to the given input, returning the output.

The default implementation calls PTransform.apply(input), but can be overridden to customize behavior for a particular runner.

Overrides:

apply in class PipelineRunner<BlockingDataflowPipelineRunner.PipelineJobState>

setHooks

public void setHooks(DataflowPipelineRunnerHooks hooks)

Sets callbacks to invoke during execution see DataflowPipelineRunnerHooks. Important: setHooks is experimental. Please consult with the Dataflow team before using it. You should expect this class to change significantly in future versions of the SDK or be removed entirely.