com.google.cloud.dataflow.sdk.runners

Class DataflowPipelineTranslator

java.lang.Object

com.google.cloud.dataflow.sdk.runners.DataflowPipelineTranslator

public class DataflowPipelineTranslator
extends Object

DataflowPipelineTranslator knows how to translate Pipeline objects into Cloud Dataflow Service API Jobs.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DataflowPipelineTranslator.JobSpecification The result of a job translation.
static interface	DataflowPipelineTranslator.TransformTranslator<TransformT extends PTransform> A DataflowPipelineTranslator.TransformTranslator knows how to translate a particular subclass of PTransform for the Cloud Dataflow service.
static interface	DataflowPipelineTranslator.TranslationContext The interface provided to registered callbacks for interacting with the DataflowPipelineRunner, including reading and writing the values of PCollections and side inputs (PCollectionViews).

Method Summary

Modifier and Type	Method and Description
static DataflowPipelineTranslator	fromOptions(DataflowPipelineOptions options) Constructs a translator from the provided options.
<TransformT extends PTransform> DataflowPipelineTranslator.TransformTranslator<TransformT>	getTransformTranslator(Class<TransformT> transformClass) Returns the DataflowPipelineTranslator.TransformTranslator to use for instances of the specified PTransform class, or null if none registered.
static String	jobToString(com.google.api.services.dataflow.model.Job job)
static <TransformT extends PTransform> void	registerTransformTranslator(Class<TransformT> transformClass, DataflowPipelineTranslator.TransformTranslator<? extends TransformT> transformTranslator) Records that instances of the specified PTransform class should be translated by default by the corresponding DataflowPipelineTranslator.TransformTranslator.
String	toString()
DataflowPipelineTranslator.JobSpecification	translate(Pipeline pipeline, List<com.google.api.services.dataflow.model.DataflowPackage> packages) Translates a Pipeline into a JobSpecification.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

fromOptions

public static DataflowPipelineTranslator fromOptions(DataflowPipelineOptions options)

Constructs a translator from the provided options.

Parameters:

options - Properties that configure the translator.

Returns:

The newly created translator.

translate

public DataflowPipelineTranslator.JobSpecification translate(Pipeline pipeline,
                                                             List<com.google.api.services.dataflow.model.DataflowPackage> packages)

Translates a Pipeline into a JobSpecification.

jobToString

public static String jobToString(com.google.api.services.dataflow.model.Job job)

registerTransformTranslator

public static <TransformT extends PTransform> void registerTransformTranslator(Class<TransformT> transformClass,
                                                                               DataflowPipelineTranslator.TransformTranslator<? extends TransformT> transformTranslator)

Records that instances of the specified PTransform class should be translated by default by the corresponding DataflowPipelineTranslator.TransformTranslator.

getTransformTranslator

public <TransformT extends PTransform> DataflowPipelineTranslator.TransformTranslator<TransformT> getTransformTranslator(Class<TransformT> transformClass)

Returns the DataflowPipelineTranslator.TransformTranslator to use for instances of the specified PTransform class, or null if none registered.

toString

public String toString()

Overrides:

toString in class Object