com.google.cloud.dataflow.sdk.runners

Class DataflowPipelineJob

java.lang.Object

com.google.cloud.dataflow.sdk.runners.DataflowPipelineJob

All Implemented Interfaces:

PipelineResult

public class DataflowPipelineJob
extends Object
implements PipelineResult

A DataflowPipelineJob represents a job submitted to Dataflow using DataflowPipelineRunner.

Nested Class Summary

Nested classes/interfaces inherited from interface com.google.cloud.dataflow.sdk.PipelineResult

PipelineResult.State

Constructor Summary

Constructors

Constructor and Description
DataflowPipelineJob(String projectId, String jobId, com.google.api.services.dataflow.Dataflow dataflowClient, com.google.cloud.dataflow.sdk.runners.dataflow.DataflowAggregatorTransforms aggregatorTransforms) Constructs the job.

Method Summary

Modifier and Type	Method and Description
void	cancel() Cancels the job.
<OutputT> AggregatorValues<OutputT>	getAggregatorValues(Aggregator<?,OutputT> aggregator) Retrieves the current value of the provided Aggregator.
com.google.api.services.dataflow.Dataflow	getDataflowClient()
String	getJobId()
String	getProjectId()
DataflowPipelineJob	getReplacedByJob() Returns a new DataflowPipelineJob for the job that replaced this one, if applicable.
PipelineResult.State	getState() Retrieves the current state of the pipeline execution.
PipelineResult.State	waitToFinish(long timeToWait, TimeUnit timeUnit, com.google.cloud.dataflow.sdk.util.MonitoringUtil.JobMessagesHandler messageHandler) Waits for the job to finish and return the final status.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DataflowPipelineJob

public DataflowPipelineJob(String projectId,
                           String jobId,
                           com.google.api.services.dataflow.Dataflow dataflowClient,
                           com.google.cloud.dataflow.sdk.runners.dataflow.DataflowAggregatorTransforms aggregatorTransforms)

Constructs the job.

Parameters:

projectId - the project id

jobId - the job id

dataflowClient - the client for the Dataflow Service

Method Detail

getJobId

public String getJobId()

getProjectId

public String getProjectId()

getReplacedByJob

public DataflowPipelineJob getReplacedByJob()

Returns a new DataflowPipelineJob for the job that replaced this one, if applicable.

Throws:

IllegalStateException - if called before the job has terminated or if the job terminated but was not updated

getDataflowClient

public com.google.api.services.dataflow.Dataflow getDataflowClient()

waitToFinish

@Nullable
public PipelineResult.State waitToFinish(long timeToWait,
                                                   TimeUnit timeUnit,
                                                   com.google.cloud.dataflow.sdk.util.MonitoringUtil.JobMessagesHandler messageHandler)
                                            throws IOException,
                                                   InterruptedException

Waits for the job to finish and return the final status.

Parameters:

timeToWait - The time to wait in units timeUnit for the job to finish. Provide a value less than 1 ms for an infinite wait.

timeUnit - The unit of time for timeToWait.

messageHandler - If non null this handler will be invoked for each batch of messages received.

Returns:

The final state of the job or null on timeout or if the thread is interrupted.

Throws:

IOException - If there is a persistent problem getting job information.

InterruptedException

cancel

public void cancel()
            throws IOException

Cancels the job.

Throws:

IOException - if there is a problem executing the cancel request.

getState

public PipelineResult.State getState()

Description copied from interface: PipelineResult

Retrieves the current state of the pipeline execution.

Specified by:

getState in interface PipelineResult

Returns:

the PipelineResult.State representing the state of this pipeline.

getAggregatorValues

public <OutputT> AggregatorValues<OutputT> getAggregatorValues(Aggregator<?,OutputT> aggregator)
                                                        throws AggregatorRetrievalException

Description copied from interface: PipelineResult

Retrieves the current value of the provided Aggregator.

Specified by:

getAggregatorValues in interface PipelineResult

Parameters:

aggregator - the Aggregator to retrieve values for

Returns:

the current values of the aggregator, which may be empty if there are no values yet

Throws:

AggregatorRetrievalException - if the aggregator values could not be retrieved