com.google.cloud.dataflow.sdk.transforms

## Class ApproximateUnique

- java.lang.Object
- - com.google.cloud.dataflow.sdk.transforms.ApproximateUnique

    public class ApproximateUnique
    extends Object

`PTransform`s for estimating the number of distinct elements in a `PCollection`, or the number of distinct values associated with each key in a `PCollection` of `KV`s.

### Nested Class Summary

Nested Classes
`static class` `ApproximateUnique.ApproximateUniqueCombineFn<T>` `CombineFn` that computes an estimate of the number of distinct values that were combined.

### Constructor Summary

Constructors
Constructor and Description
`ApproximateUnique()`

### Method Summary

All Methods Static Methods Concrete Methods
`static <T> com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.Globally<T>` `globally(double maximumEstimationError)` Like `globally(int)`, but specifies the desired maximum estimation error instead of the sample size.
`static <T> com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.Globally<T>` `globally(int sampleSize)` Returns a `PTransform` that takes a `PCollection<T>` and returns a `PCollection<Long>` containing a single value that is an estimate of the number of distinct elements in the input `PCollection`.
`static <K,V> com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.PerKey<K,V>` `perKey(double maximumEstimationError)` Like `perKey(int)`, but specifies the desired maximum estimation error instead of the sample size.
`static <K,V> com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.PerKey<K,V>` `perKey(int sampleSize)` Returns a `PTransform` that takes a `PCollection<KV<K, V>>` and returns a `PCollection<KV<K, Long>>` that contains an output element mapping each distinct key in the input `PCollection` to an estimate of the number of distinct values associated with that key in the input `PCollection`.

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Constructor Detail

#### ApproximateUnique

    public ApproximateUnique()

### Method Detail

#### globally

    public static <T> com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.Globally<T> globally(int sampleSize)

Returns a `PTransform` that takes a `PCollection<T>` and returns a `PCollection<Long>` containing a single value that is an estimate of the number of distinct elements in the input `PCollection`.

The `sampleSize` parameter controls the estimation error. The error is about `2 / sqrt(sampleSize)`, so for `ApproximateUnique.globally(10000)` the estimation error is about 2%. Similarly, for `ApproximateUnique.of(16)` the estimation error is about 50%. If there are fewer than `sampleSize` distinct elements then the returned result will be exact with extremely high probability (the chance of a hash collision is about `sampleSize^2 / 2^65`).

This transform approximates the number of elements in a set by computing the top `sampleSize` hash values, and using that to extrapolate the size of the entire set of hash values by assuming the rest of the hash values are as densely distributed as the top `sampleSize`.

See also `globally(double)`.

Example of use:

     PCollection<String> pc = ...;
     PCollection<Long> approxNumDistinct =
         pc.apply(ApproximateUnique.<String>globally(1000));

Type Parameters:
  `T` - the type of the elements in the input `PCollection`
Parameters:
  `sampleSize` - the number of entries in the statistical sample; the higher this number, the more accurate the estimate will be; should be `>= 16`
Throws:
  `IllegalArgumentException` - if the `sampleSize` argument is too small

#### globally

    public static <T> com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.Globally<T> globally(double maximumEstimationError)

Like `globally(int)`, but specifies the desired maximum estimation error instead of the sample size.

Type Parameters:
  `T` - the type of the elements in the input `PCollection`
Parameters:
  `maximumEstimationError` - the maximum estimation error, which should be in the range `[0.01, 0.5]`
Throws:
  `IllegalArgumentException` - if the `maximumEstimationError` argument is out of range

#### perKey

    public static <K,V> com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.PerKey<K,V> perKey(int sampleSize)

Returns a `PTransform` that takes a `PCollection<KV<K, V>>` and returns a `PCollection<KV<K, Long>>` that contains an output element mapping each distinct key in the input `PCollection` to an estimate of the number of distinct values associated with that key in the input `PCollection`.

See `globally(int)` for an explanation of the `sampleSize` parameter. A separate sampling is computed for each distinct key of the input.

See also `perKey(double)`.

Example of use:

     PCollection<KV<Integer, String>> pc = ...;
     PCollection<KV<Integer, Long>> approxNumDistinctPerKey =
         pc.apply(ApproximateUnique.<Integer, String>perKey(1000));

Type Parameters:
  `K` - the type of the keys in the input and output `PCollection`s
  `V` - the type of the values in the input `PCollection`
Parameters:
  `sampleSize` - the number of entries in the statistical sample; the higher this number, the more accurate the estimate will be; should be `>= 16`
Throws:
  `IllegalArgumentException` - if the `sampleSize` argument is too small

#### perKey

    public static <K,V> com.google.cloud.dataflow.sdk.transforms.ApproximateUnique.PerKey<K,V> perKey(double maximumEstimationError)

Like `perKey(int)`, but specifies the desired maximum estimation error instead of the sample size.

Type Parameters:
  `K` - the type of the keys in the input and output `PCollection`s
  `V` - the type of the values in the input `PCollection`
Parameters:
  `maximumEstimationError` - the maximum estimation error, which should be in the range `[0.01, 0.5]`
Throws:
  `IllegalArgumentException` - if the `maximumEstimationError` argument is out of range