com.google.cloud.dataflow.sdk.io

Class PubsubIO

java.lang.Object

com.google.cloud.dataflow.sdk.io.PubsubIO

public class PubsubIO
extends Object

Read and Write PTransforms for Cloud Pub/Sub streams. These transforms create and consume unbounded PCollections.

Permissions

Permission requirements depend on the PipelineRunner that is used to execute the Dataflow job. Please refer to the documentation of corresponding PipelineRunners for more details.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	PubsubIO.PubsubSubscription Class representing a Cloud Pub/Sub Subscription.
static class	PubsubIO.PubsubTopic Class representing a Cloud Pub/Sub Topic.
static class	PubsubIO.Read A PTransform that continuously reads from a Cloud Pub/Sub stream and returns a PCollection of Strings containing the items from the stream.
static class	PubsubIO.Write A PTransform that continuously writes a PCollection of Strings to a Cloud Pub/Sub stream.

Field Summary

Fields

Modifier and Type	Field and Description
static Coder<String>	DEFAULT_PUBSUB_CODER The default Coder used to translate to/from Cloud Pub/Sub messages.

Method Summary

Modifier and Type	Method and Description
protected static Instant	assignMessageTimestamp(com.google.api.services.pubsub.model.PubsubMessage message, String label, com.google.api.client.util.Clock clock) Returns the Instant that corresponds to the timestamp in the supplied PubsubMessage under the specified ink label.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_PUBSUB_CODER

public static final Coder<String> DEFAULT_PUBSUB_CODER

The default Coder used to translate to/from Cloud Pub/Sub messages.

Method Detail

assignMessageTimestamp

protected static Instant assignMessageTimestamp(com.google.api.services.pubsub.model.PubsubMessage message,
                                                @Nullable
                                                String label,
                                                com.google.api.client.util.Clock clock)

Returns the Instant that corresponds to the timestamp in the supplied PubsubMessage under the specified ink label. See PubsubIO.Read.timestampLabel(String) for details about how these messages are parsed.

The Clock parameter is used to virtualize time for testing.

Throws:

IllegalArgumentException - if the timestamp label is provided, but there is no corresponding attribute in the message or the value provided is not a valid timestamp string.

See Also:

PubsubIO.Read.timestampLabel(String)