com.google.cloud.dataflow.sdk.util.common.worker

Class ReadOperation

java.lang.Object

com.google.cloud.dataflow.sdk.util.common.worker.Operation

com.google.cloud.dataflow.sdk.util.common.worker.ReadOperation

public class ReadOperation
extends Operation

A read operation.

Its start() method iterates through all elements of the source and emits them on its output.

Nested Class Summary

Nested classes/interfaces inherited from class com.google.cloud.dataflow.sdk.util.common.worker.Operation

Operation.InitializationState

Field Summary

Fields

Modifier and Type	Field and Description
Reader<?>	reader The Reader this operation reads from.

Fields inherited from class com.google.cloud.dataflow.sdk.util.common.worker.Operation

finishState, initializationState, initializationStateLock, operationName, processState, receivers, startState, stateSampler

Constructor Summary

Constructors

Constructor and Description
ReadOperation(String operationName, Reader<?> reader, OutputReceiver[] receivers, String counterPrefix, CounterSet.AddCounterMutator addCounterMutator, StateSampler stateSampler)

Method Summary

Modifier and Type	Method and Description
protected String	bytesCounterName(String counterPrefix, String operationName)
Reader.Progress	getProgress() Returns a (possibly slightly stale) value of the progress of the task.
Reader<?>	getReader()
Reader.DynamicSplitResult	requestDynamicSplit(Reader.DynamicSplitRequest splitRequest) Relays the split request to ReaderIterator.
protected void	runReadLoop()
void	setProgressUpdatePeriodMs(long millis) Controls the frequency at which progress is updated.
void	start() Starts this Operation's execution.
boolean	supportsRestart() Returns true if this Operation can be started again after it is finished.

Methods inherited from class com.google.cloud.dataflow.sdk.util.common.worker.Operation

finish

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

reader

public final Reader<?> reader

The Reader this operation reads from.

Constructor Detail

ReadOperation

public ReadOperation(String operationName,
                     Reader<?> reader,
                     OutputReceiver[] receivers,
                     String counterPrefix,
                     CounterSet.AddCounterMutator addCounterMutator,
                     StateSampler stateSampler)

Method Detail

setProgressUpdatePeriodMs

public void setProgressUpdatePeriodMs(long millis)

Controls the frequency at which progress is updated. A value of zero means "update progress on each iteration". A value of less than zero means never update progress. Ignored after starting.

bytesCounterName

protected String bytesCounterName(String counterPrefix,
                                  String operationName)

getReader

public Reader<?> getReader()

start

public void start()
           throws Exception

Description copied from class: Operation

Starts this Operation's execution. Called after all successsor consuming operations have been started.

Overrides:

start in class Operation

Throws:

Exception

supportsRestart

public boolean supportsRestart()

Description copied from class: Operation

Returns true if this Operation can be started again after it is finished.

Overrides:

supportsRestart in class Operation

runReadLoop

protected void runReadLoop()
                    throws Exception

Throws:

Exception

getProgress

public Reader.Progress getProgress()

Returns a (possibly slightly stale) value of the progress of the task. Guaranteed to not block indefinitely.

Returns:

the task progress, or null if the source iterator has not been initialized

requestDynamicSplit

public Reader.DynamicSplitResult requestDynamicSplit(Reader.DynamicSplitRequest splitRequest)

Relays the split request to ReaderIterator.