com.google.cloud.dataflow.sdk.io

Class PubsubIO.Read.Bound<T>

java.lang.Object

com.google.cloud.dataflow.sdk.transforms.PTransform<PInput,PCollection<T>>

com.google.cloud.dataflow.sdk.io.PubsubIO.Read.Bound<T>

All Implemented Interfaces:

Serializable

Enclosing class:

PubsubIO.Read

public static class PubsubIO.Read.Bound<T>
extends PTransform<PInput,PCollection<T>>

A PTransform that reads from a PubSub source and returns a unbounded PCollection containing the items from the stream.

See Also:

Serialized Form

Field Summary

Fields inherited from class com.google.cloud.dataflow.sdk.transforms.PTransform

name

Method Summary

Modifier and Type	Method and Description
PCollection<T>	apply(PInput input) Applies this PTransform on the given InputT, and returns its Output.
Coder<T>	getCoder()
protected Coder<T>	getDefaultOutputCoder() Returns the default Coder to use for the output of this single-output PTransform.
String	getIdLabel()
int	getMaxNumRecords()
org.joda.time.Duration	getMaxReadTime()
PubsubIO.PubsubSubscription	getSubscription()
String	getTimestampLabel()
PubsubIO.PubsubTopic	getTopic()
PubsubIO.Read.Bound<T>	idLabel(String idLabel) Returns a new PubsubIO.Read PTransform that's like this one but reading unique ids from the given PubSub label.
PubsubIO.Read.Bound<T>	maxNumRecords(int maxNumRecords) Sets the maximum number of records that will be read from Pubsub.
PubsubIO.Read.Bound<T>	maxReadTime(org.joda.time.Duration maxReadTime) Sets the maximum duration during which records will be read from Pubsub.
PubsubIO.Read.Bound<T>	named(String name) Returns a new PubsubIO.Read PTransform that's like this one but with the given step name.
PubsubIO.Read.Bound<T>	subscription(String subscription) Returns a new PubsubIO.Read PTransform that's like this one but reading from the given subscription.
PubsubIO.Read.Bound<T>	timestampLabel(String timestampLabel) Returns a new PubsubIO.Read PTransform that's like this one but reading timestamps from the given PubSub label.
PubsubIO.Read.Bound<T>	topic(String topic) Returns a new PubsubIO.Read PTransform that's like this one but reading from the give topic.
<X> PubsubIO.Read.Bound<X>	withCoder(Coder<X> coder) Returns a new PubsubIO.Read PTransform that's like this one but that uses the given Coder<X> to decode each record into a value of type X.

Methods inherited from class com.google.cloud.dataflow.sdk.transforms.PTransform

getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, toString, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

named

public PubsubIO.Read.Bound<T> named(String name)

Returns a new PubsubIO.Read PTransform that's like this one but with the given step name. Does not modify the object.

subscription

public PubsubIO.Read.Bound<T> subscription(String subscription)

Returns a new PubsubIO.Read PTransform that's like this one but reading from the given subscription. Does not modify the object.

Multiple readers reading from the same subscription will each receive some arbirary portion of the data. Most likely, separate readers should use their own subscriptions.

topic

public PubsubIO.Read.Bound<T> topic(String topic)

Returns a new PubsubIO.Read PTransform that's like this one but reading from the give topic. Does not modify the object.

timestampLabel

public PubsubIO.Read.Bound<T> timestampLabel(String timestampLabel)

Returns a new PubsubIO.Read PTransform that's like this one but reading timestamps from the given PubSub label. Does not modify the object.

idLabel

public PubsubIO.Read.Bound<T> idLabel(String idLabel)

Returns a new PubsubIO.Read PTransform that's like this one but reading unique ids from the given PubSub label. Does not modify the object.

withCoder

public <X> PubsubIO.Read.Bound<X> withCoder(Coder<X> coder)

Returns a new PubsubIO.Read PTransform that's like this one but that uses the given Coder<X> to decode each record into a value of type X. Does not modify this object.

Type Parameters:

X - the type of the decoded elements, and the elements of the resulting PCollection.

maxNumRecords

public PubsubIO.Read.Bound<T> maxNumRecords(int maxNumRecords)

Sets the maximum number of records that will be read from Pubsub.

Setting either this or maxNumRecords will cause the output PCollection to be bounded.

maxReadTime

public PubsubIO.Read.Bound<T> maxReadTime(org.joda.time.Duration maxReadTime)

Sets the maximum duration during which records will be read from Pubsub.

Setting either this or maxNumRecords will cause the output PCollection to be bounded.

apply

public PCollection<T> apply(PInput input)

Description copied from class: PTransform

Applies this PTransform on the given InputT, and returns its Output.

Composite transforms, which are defined in terms of other transforms, should return the output of one of the composed transforms. Non-composite transforms, which do not apply any transforms internally, should return a new unbound output and register evaluators (via backend-specific registration methods).

The default implementation throws an exception. A derived class must either implement apply, or else each runner must supply a custom implementation via PipelineRunner.apply(com.google.cloud.dataflow.sdk.transforms.PTransform<InputT, OutputT>, InputT).

Overrides:

apply in class PTransform<PInput,PCollection<T>>

getDefaultOutputCoder

protected Coder<T> getDefaultOutputCoder()

Description copied from class: PTransform

Returns the default Coder to use for the output of this single-output PTransform.

By default, always throws

Overrides:

getDefaultOutputCoder in class PTransform<PInput,PCollection<T>>

getTopic

public PubsubIO.PubsubTopic getTopic()

getSubscription

public PubsubIO.PubsubSubscription getSubscription()

getTimestampLabel

public String getTimestampLabel()

getCoder

public Coder<T> getCoder()

getIdLabel

public String getIdLabel()

getMaxNumRecords

public int getMaxNumRecords()

getMaxReadTime

public org.joda.time.Duration getMaxReadTime()