public class AutoComplete extends Object
Concepts: Using the same pipeline in both streaming and batch, combiners, composite transforms.
To execute this pipeline using the Dataflow service in batch mode, specify pipeline configuration:
--project=YOUR_PROJECT_ID
--stagingLocation=gs://YOUR_STAGING_DIRECTORY
--runner=DataflowPipelineRunner
--inputFile=gs://path/to/input*.txt
To execute this pipeline using the Dataflow service in streaming mode, specify pipeline configuration:
--project=YOUR_PROJECT_ID
--stagingLocation=gs://YOUR_STAGING_DIRECTORY
--runner=DataflowPipelineRunner
--inputFile=gs://YOUR_INPUT_DIRECTORY/*.txt
--streaming
This will update the Cloud Datastore every 10 seconds based on the last 30 minutes of data received.
| Modifier and Type | Class and Description |
|---|---|
static class |
AutoComplete.ComputeTopCompletions
A PTransform that takes as input a list of tokens and returns
the most common tokens per prefix.
|
| Constructor and Description |
|---|
AutoComplete() |
public static void main(String[] args) throws IOException
IOException