com.google.cloud.dataflow.examples.common

## Class DataflowExampleUtils

- java.lang.Object
- - com.google.cloud.dataflow.examples.common.DataflowExampleUtils

    public class DataflowExampleUtils
    extends Object

The utility class that sets up and tears down external resources, starts the Google Cloud Pub/Sub injector, and cancels the streaming and the injector pipelines once the program terminates.

It is used to run Dataflow examples, such as TrafficMaxLaneFlow and TrafficRoutes.

### Constructor Summary

Constructors
Constructor and Description
`DataflowExampleUtils(DataflowPipelineOptions options)`
`DataflowExampleUtils(DataflowPipelineOptions options, boolean isUnbounded)` Do resources and runner options setup.

### Method Summary

All Methods Instance Methods Concrete Methods
`void` `mockUnboundedSource(String inputFile, PipelineResult result)` Start the auxiliary injector pipeline, then wait for this pipeline to finish.
`void` `runInjectorPipeline(Pipeline injectorPipeline)` Runs the provided pipeline to inject data into the PubSubIO input topic.
`void` `runInjectorPipeline(PTransform<? super PBegin,PCollection<String>> readSource, String topic, String pubsubTimestampTabelKey)` Runs a batch pipeline to inject data into the PubSubIO input topic.
`void` `runInjectorPipeline(String inputFile, String topic)` Runs a batch pipeline to inject data into the PubSubIO input topic.
`void` `setup()` Sets up external resources that are required by the example, such as Pub/Sub topics and BigQuery tables.
`void` `setupBigQueryTable()` Sets up the BigQuery table with the given schema.
`void` `setupPubsub()` Sets up the Google Cloud Pub/Sub topic.
`void` `setupResourcesAndRunner(boolean isUnbounded)` Set up external resources, and configure the runner appropriately.
`void` `setupRunner()` Do some runner setup: check that the DirectPipelineRunner is not used in conjunction with streaming, and if streaming is specified, use the DataflowPipelineRunner.
`void` `startInjectorIfNeeded(String inputFile)` If this is an unbounded (streaming) pipeline, and both inputFile and pubsub topic are defined, start an 'injector' pipeline that publishes the contents of the file to the given topic, first creating the topic if necessary.
`void` `waitToFinish(PipelineResult result)` If DataflowPipelineRunner or BlockingDataflowPipelineRunner is used, waits for the pipeline to finish and cancels it (and the injector) before the program exists.

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Constructor Detail

#### DataflowExampleUtils

    public DataflowExampleUtils(DataflowPipelineOptions options)

#### DataflowExampleUtils

    public DataflowExampleUtils(DataflowPipelineOptions options,
                                boolean isUnbounded)
                         throws IOException

Do resources and runner options setup.

Throws:
  `IOException`

### Method Detail

#### setup

    public void setup()
               throws IOException

Sets up external resources that are required by the example, such as Pub/Sub topics and BigQuery tables.

Throws:
  `IOException` - if there is a problem setting up the resources

#### setupResourcesAndRunner

    public void setupResourcesAndRunner(boolean isUnbounded)
                                 throws IOException

Set up external resources, and configure the runner appropriately.

Throws:
  `IOException`

#### setupPubsub

    public void setupPubsub()
                     throws IOException

Sets up the Google Cloud Pub/Sub topic.

If the topic doesn't exist, a new topic with the given name will be created.

Throws:
  `IOException` - if there is a problem setting up the Pub/Sub topic

#### setupBigQueryTable

    public void setupBigQueryTable()
                            throws IOException

Sets up the BigQuery table with the given schema.

If the table already exists, the schema has to match the given one. Otherwise, the example will throw a RuntimeException. If the table doesn't exist, a new table with the given schema will be created.

Throws:
  `IOException` - if there is a problem setting up the BigQuery table

#### startInjectorIfNeeded

    public void startInjectorIfNeeded(String inputFile)

If this is an unbounded (streaming) pipeline, and both inputFile and pubsub topic are defined, start an 'injector' pipeline that publishes the contents of the file to the given topic, first creating the topic if necessary.

#### setupRunner

    public void setupRunner()

Do some runner setup: check that the DirectPipelineRunner is not used in conjunction with streaming, and if streaming is specified, use the DataflowPipelineRunner. Return the streaming flag value.

#### runInjectorPipeline

    public void runInjectorPipeline(String inputFile,
                                    String topic)

Runs a batch pipeline to inject data into the PubSubIO input topic.

The injector pipeline will read from the given text file, and inject data into the Google Cloud Pub/Sub topic.

#### runInjectorPipeline

    public void runInjectorPipeline(PTransform<? super PBegin,PCollection<String>> readSource,
                                    String topic,
                                    String pubsubTimestampTabelKey)

Runs a batch pipeline to inject data into the PubSubIO input topic.

The injector pipeline will read from the given source, and inject data into the Google Cloud Pub/Sub topic.

#### runInjectorPipeline

    public void runInjectorPipeline(Pipeline injectorPipeline)

Runs the provided pipeline to inject data into the PubSubIO input topic.

#### mockUnboundedSource

    public void mockUnboundedSource(String inputFile,
                                    PipelineResult result)

Start the auxiliary injector pipeline, then wait for this pipeline to finish.

#### waitToFinish

    public void waitToFinish(PipelineResult result)

If DataflowPipelineRunner or BlockingDataflowPipelineRunner is used, waits for the pipeline to finish and cancels it (and the injector) before the program exists.