com.google.cloud.dataflow.sdk.util.common.worker

Class Operation

java.lang.Object

com.google.cloud.dataflow.sdk.util.common.worker.Operation

Direct Known Subclasses:

ReadOperation, ReceivingOperation

public abstract class Operation
extends Object

The abstract base class for Operations, which correspond to Instructions in the original MapTask InstructionGraph.

Call start() to start the operation.

A read operation's start() method actually reads the data, and in effect runs the pipeline.

Call finish() to finish the operation.

Since both start() and finish() may call process() on this operation's consumers, start an operation after starting its consumers, and finish an operation before finishing its consumers.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Operation.InitializationState The possible initialization states of an Operation.

Field Summary

Fields

Modifier and Type	Field and Description
protected int	finishState
Operation.InitializationState	initializationState The initialization state of this Operation.
protected Object	initializationStateLock The lock protecting the initialization state.
String	operationName The name of this operation.
protected int	processState
OutputReceiver[]	receivers The array of consuming receivers, one per operation output "port" (e.g., DoFn main or side output).
protected int	startState
protected StateSampler	stateSampler

Constructor Summary

Constructors

Constructor and Description
Operation(String operationName, OutputReceiver[] receivers, String counterPrefix, CounterSet.AddCounterMutator addCounterMutator, StateSampler stateSampler)

Method Summary

Modifier and Type	Method and Description
void	finish() Finishes this Operation's execution.
void	start() Starts this Operation's execution.
boolean	supportsRestart() Returns true if this Operation can be started again after it is finished.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

operationName

public final String operationName

The name of this operation.

receivers

public final OutputReceiver[] receivers

The array of consuming receivers, one per operation output "port" (e.g., DoFn main or side output). A receiver might be null if that output isn't being consumed.

initializationState

public Operation.InitializationState initializationState

The initialization state of this Operation.

initializationStateLock

protected final Object initializationStateLock

The lock protecting the initialization state. InitializationState is only written from one thread, but can be read by concurrent threads.

stateSampler

protected final StateSampler stateSampler

startState

protected final int startState

processState

protected final int processState

finishState

protected final int finishState

Constructor Detail

Operation

public Operation(String operationName,
                 OutputReceiver[] receivers,
                 String counterPrefix,
                 CounterSet.AddCounterMutator addCounterMutator,
                 StateSampler stateSampler)

Method Detail

start

public void start()
           throws Exception

Starts this Operation's execution. Called after all successsor consuming operations have been started.

Throws:

Exception

finish

public void finish()
            throws Exception

Finishes this Operation's execution. Called after all predecessor producing operations have been finished.

Throws:

Exception

supportsRestart

public boolean supportsRestart()

Returns true if this Operation can be started again after it is finished.