com.google.cloud.dataflow.examples

Class AutoComplete

java.lang.Object

com.google.cloud.dataflow.examples.AutoComplete

public class AutoComplete
extends java.lang.Object

An example that computes the most popular hash tags for a for every prefix, which can be used for auto-completion.

Concepts: Using the same pipeline in both streaming and batch, combiners, composite transforms.

To execute this pipeline using the Dataflow service in batch mode, specify pipeline configuration: --project= --stagingLocation=gs:// --runner=[Blocking]DataflowPipelineRunner --inputFile=gs://path/to/input*.txt [--outputDataset=]

To execute this pipeline using the Dataflow service in streaming mode, specify pipeline configuration: --project= --stagingLocation=gs:// --runner=DataflowPipelineRunner --inputTopic=/topics/someproject/sometopic [--outputDataset=]

Which will update the datastore every 10 seconds based on the last 30 minutes of data received.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	AutoComplete.ComputeTopCompletions A PTransform that takes as input a list of tokens and returns the most common tokens per prefix.

Constructor Summary

Constructors

Constructor and Description
AutoComplete()

Method Summary

Methods

Modifier and Type	Method and Description
static void	main(java.lang.String[] args)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AutoComplete

public AutoComplete()

Method Detail

main

public static void main(java.lang.String[] args)