# Package com.google.cloud.dataflow.sdk.io

Defines transforms for reading and writing common storage formats, including `AvroIO`, `BigQueryIO`, and `TextIO`.

See: Description

Interface Summary
CompressedSource.DecompressingChannelFactory: Factory interface for creating channels that decompress the content of an underlying channel.
UnboundedSource.CheckpointMark: A marker representing the progress and state of an `UnboundedSource.UnboundedReader`.

Class Summary
AvroIO: `PTransform`s for reading and writing Avro files.
AvroIO.Read: A root `PTransform` that reads from an Avro file (or multiple Avro files matching a pattern) and returns a `PCollection` containing the decoding of each record.
AvroIO.Read.Bound<T>: A `PTransform` that reads from an Avro file (or multiple Avro files matching a pattern) and returns a bounded `PCollection` containing the decoding of each record.
AvroIO.Write: A root `PTransform` that writes a `PCollection` to an Avro file (or multiple Avro files matching a sharding pattern).
AvroIO.Write.Bound<T>: A `PTransform` that writes a bounded `PCollection` to an Avro file (or multiple Avro files matching a sharding pattern).
AvroSource<T>: A `FileBasedSource` for reading Avro files.
AvroSource.AvroReader<T>: A `BlockBasedSource.BlockBasedReader` for reading blocks from Avro files.
BigQueryIO: `PTransform`s for reading and writing BigQuery tables.
BigQueryIO.Read: A `PTransform` that reads from a BigQuery table and returns a `PCollection` of `TableRows` containing each of the rows of the table.
BigQueryIO.Read.Bound: A `PTransform` that reads from a BigQuery table and returns a bounded `PCollection` of `TableRows`.
BigQueryIO.Write: A `PTransform` that writes a `PCollection` containing `TableRows` to a BigQuery table.
BigQueryIO.Write.Bound: A `PTransform` that can write either a bounded or unbounded `PCollection` of `TableRows` to a BigQuery table.
BlockBasedSource<T>: A `BlockBasedSource` is a `FileBasedSource` where a file consists of blocks of records.
BlockBasedSource.Block<T>: A `Block` represents a block of records that can be read.
BlockBasedSource.BlockBasedReader<T>: A `Reader` that reads records from a `BlockBasedSource`.
BoundedSource<T>: A `Source` that reads a finite amount of input and, because of that, supports some additional operations.
BoundedSource.BoundedReader<T>: A `Reader` that reads a bounded amount of input and supports some additional operations, such as progress estimation and dynamic work rebalancing.
CompressedSource<T>: A Source that reads from compressed files.
CompressedSource.CompressedReader<T>: Reader for a `CompressedSource`.
CountingInput: A `PTransform` that produces longs.
CountingInput.BoundedCountingInput: A `PTransform` that will produce a specified number of `Longs` starting from 0.
CountingInput.UnboundedCountingInput: A `PTransform` that will produce numbers starting from `0` up to `Long.MAX_VALUE`.
CountingSource: A source that produces longs.
CountingSource.CounterMark: The checkpoint for an unbounded `CountingSource` is simply the last value produced.
DatastoreIO: Deprecated replaced by `DatastoreIO`
DatastoreIO.DatastoreReader: A `Source.Reader` over the records from a query of the datastore.
DatastoreIO.Sink: A `Sink` that writes a `PCollection` containing `Entities` to a Datastore kind.
DatastoreIO.Source: A `Source` that reads the result rows of a Datastore query as `Entity` objects.
FileBasedSink<T>: Abstract `Sink` for file-based output.
FileBasedSink.FileBasedWriteOperation<T>: Abstract `Sink.WriteOperation` that manages the process of writing to a `FileBasedSink`.
FileBasedSink.FileBasedWriter<T>: Abstract `Sink.Writer` that writes a bundle to a `FileBasedSink`.
FileBasedSink.FileResult: Result of a single bundle write.
FileBasedSource<T>: A common base class for all file-based `Source`s.
FileBasedSource.FileBasedReader<T>: A `reader` that implements code common to readers of `FileBasedSource`s.
OffsetBasedSource<T>: A `BoundedSource` that uses offsets to define starting and ending positions.
OffsetBasedSource.OffsetBasedReader<T>: A `Source.Reader` that implements code common to readers of all `OffsetBasedSource`s.
PubsubIO: Read and Write `PTransform`s for Cloud Pub/Sub streams.
PubsubIO.PubsubSubscription: Class representing a Cloud Pub/Sub Subscription.
PubsubIO.PubsubTopic: Class representing a Cloud Pub/Sub Topic.
PubsubIO.Read: A `PTransform` that continuously reads from a Cloud Pub/Sub stream and returns a `PCollection` of `Strings` containing the items from the stream.
PubsubIO.Read.Bound<T>: A `PTransform` that reads from a Cloud Pub/Sub source and returns a unbounded `PCollection` containing the items from the stream.
PubsubIO.Write: A `PTransform` that continuously writes a `PCollection` of `Strings` to a Cloud Pub/Sub stream.
PubsubIO.Write.Bound<T>: A `PTransform` that writes an unbounded `PCollection` of `Strings` to a Cloud Pub/Sub stream.
PubsubUnboundedSink<T>: A PTransform which streams messages to Pubsub.
PubsubUnboundedSource<T>: A PTransform which streams messages from Pubsub.
Read: A `PTransform` for reading from a `Source`.
Read.Bounded<T>: `PTransform` that reads from a `BoundedSource`.
Read.Builder: Helper class for building `Read` transforms.
Read.Unbounded<T>: `PTransform` that reads from a `UnboundedSource`.
ShardNameTemplate: Standard shard naming templates.
Sink<T>: A `Sink` represents a resource that can be written to using the `Write` transform.
Sink.WriteOperation<T,WriteT>: A `Sink.WriteOperation` defines the process of a parallel write of objects to a Sink.
Sink.Writer<T,WriteT>: A Writer writes a bundle of elements from a PCollection to a sink.
Source<T>: Base class for defining input formats and creating a `Source` for reading the input.
Source.Reader<T>: The interface that readers of custom input sources must implement.
TextIO: `PTransform`s for reading and writing text files.
TextIO.Read: A `PTransform` that reads from a text file (or multiple text files matching a pattern) and returns a `PCollection` containing the decoding of each of the lines of the text file(s).
TextIO.Read.Bound<T>: A `PTransform` that reads from one or more text files and returns a bounded `PCollection` containing one element for each line of the input files.
TextIO.Write: A `PTransform` that writes a `PCollection` to text file (or multiple text files matching a sharding pattern), with each element of the input collection encoded into its own line.
TextIO.Write.Bound<T>: A PTransform that writes a bounded PCollection to a text file (or multiple text files matching a sharding pattern), with each PCollection element being encoded into its own line.
UnboundedSource<OutputT,CheckpointMarkT extends UnboundedSource.CheckpointMark>: A `Source` that reads an unbounded amount of input and, because of that, supports some additional operations such as checkpointing, watermarks, and record ids.
UnboundedSource.UnboundedReader<OutputT>: A `Reader` that reads an unbounded amount of input.
Write: A `PTransform` that writes to a `Sink`.
Write.Bound<T>: A `PTransform` that writes to a `Sink`.
XmlSink: A `Sink` that outputs records as XML-formatted elements.
XmlSink.Bound<T>: A `FileBasedSink` that writes objects as XML elements.
XmlSink.XmlWriteOperation<T>: `Sink.WriteOperation` for XML `Sink`s.
XmlSink.XmlWriter<T>: A `Sink.Writer` that can write objects as XML elements.
XmlSource<T>: A source that can be used to read XML files.

Enum Summary
BigQueryIO.Write.CreateDisposition: An enumeration type for the BigQuery create disposition strings.
BigQueryIO.Write.WriteDisposition: An enumeration type for the BigQuery write disposition strings.
CompressedSource.CompressionMode: Default compression types supported by the `CompressedSource`.
FileBasedSink.FileBasedWriteOperation.TemporaryFileRetention: Options for handling of temporary output files.
FileBasedSource.Mode: A given `FileBasedSource` represents a file resource of one of these types.
TextIO.CompressionType: Possible text file compression types.

## Package com.google.cloud.dataflow.sdk.io Description

Defines transforms for reading and writing common storage formats, including `AvroIO`, `BigQueryIO`, and `TextIO`.

The classes in this package provide `Read` transforms that create PCollections from existing storage:

     PCollection<TableRow> inputData = pipeline.apply(
         BigQueryIO.Read.named("Read")
                        .from("clouddataflow-readonly:samples.weather_stations");

and `Write` transforms that persist PCollections to external storage:

     PCollection<Integer> numbers = ...;
     numbers.apply(TextIO.Write.named("WriteNumbers")
                               .to("gs://my_bucket/path/to/numbers"));