com.google.cloud.dataflow.sdk.io

## Class BoundedSource<T>

- java.lang.Object
- - com.google.cloud.dataflow.sdk.io.Source<T>
  - - com.google.cloud.dataflow.sdk.io.BoundedSource<T>

Type Parameters:
  `T` - Type of records read by the source.

All Implemented Interfaces:
  HasDisplayData, Serializable

Direct Known Subclasses:
  DatastoreIO.Source, OffsetBasedSource

    public abstract class BoundedSource<T>
    extends Source<T>

A `Source` that reads a finite amount of input and, because of that, supports some additional operations.

The operations are:

- Splitting into bundles of given size: `splitIntoBundles(long, com.google.cloud.dataflow.sdk.options.PipelineOptions)`;
- Size estimation: `getEstimatedSizeBytes(com.google.cloud.dataflow.sdk.options.PipelineOptions)`;
- Telling whether or not this source produces key/value pairs in sorted order: `producesSortedKeys(com.google.cloud.dataflow.sdk.options.PipelineOptions)`;
- The accompanying `reader` has additional functionality to enable runners to dynamically adapt based on runtime conditions.
  - Progress estimation (`BoundedSource.BoundedReader.getFractionConsumed()`)
  - Tracking of parallelism, to determine whether the current source can be split (`BoundedSource.BoundedReader.getSplitPointsConsumed()` and `BoundedSource.BoundedReader.getSplitPointsRemaining()`).
  - Dynamic splitting of the current source (`BoundedSource.BoundedReader.splitAtFraction(double)`).

To use this class for supporting your custom input type, derive your class class from it, and override the abstract methods. For an example, see `DatastoreIO`.

See Also:
  Serialized Form

### Nested Class Summary

Nested Classes
`static class` `BoundedSource.BoundedReader<T>` A `Reader` that reads a bounded amount of input and supports some additional operations, such as progress estimation and dynamic work rebalancing.

### Nested classes/interfaces inherited from class com.google.cloud.dataflow.sdk.io.Source

`Source.Reader<T>`

### Constructor Summary

Constructors
Constructor and Description
`BoundedSource()`

### Method Summary

All Methods Instance Methods Abstract Methods
`abstract BoundedSource.BoundedReader<T>` `createReader(PipelineOptions options)` Returns a new `BoundedSource.BoundedReader` that reads from this source.
`abstract long` `getEstimatedSizeBytes(PipelineOptions options)` An estimate of the total size (in bytes) of the data that would be read from this source.
`abstract boolean` `producesSortedKeys(PipelineOptions options)` Whether this source is known to produce key/value pairs sorted by lexicographic order on the bytes of the encoded key.
`abstract List<? extends BoundedSource<T>>` `splitIntoBundles(long desiredBundleSizeBytes, PipelineOptions options)` Splits the source into bundles of approximately `desiredBundleSizeBytes`.

### Methods inherited from class com.google.cloud.dataflow.sdk.io.Source

`getDefaultOutputCoder, populateDisplayData, validate`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Constructor Detail

#### BoundedSource

    public BoundedSource()

### Method Detail

#### splitIntoBundles

    public abstract List<? extends BoundedSource<T>> splitIntoBundles(long desiredBundleSizeBytes,
                                                                      PipelineOptions options)
                                                               throws Exception

Splits the source into bundles of approximately `desiredBundleSizeBytes`.

Throws:
  `Exception`

#### getEstimatedSizeBytes

    public abstract long getEstimatedSizeBytes(PipelineOptions options)
                                        throws Exception

An estimate of the total size (in bytes) of the data that would be read from this source. This estimate is in terms of external storage size, before any decompression or other processing done by the reader.

Throws:
  `Exception`

#### producesSortedKeys

    public abstract boolean producesSortedKeys(PipelineOptions options)
                                        throws Exception

Whether this source is known to produce key/value pairs sorted by lexicographic order on the bytes of the encoded key.

Throws:
  `Exception`

#### createReader

    public abstract BoundedSource.BoundedReader<T> createReader(PipelineOptions options)
                                                         throws IOException

Returns a new `BoundedSource.BoundedReader` that reads from this source.

Throws:
  `IOException`