public class TrafficRoutes extends Object
Concepts: The batch and streaming runners, GroupByKey, sliding windows, and Google Cloud Pub/Sub topic injection.
This example analyzes traffic sensor data using SlidingWindows. For each window, it calculates the average speed over the window for some small set of predefined 'routes', and looks for 'slowdowns' in those routes. It writes its results to a BigQuery table.
In batch mode, the pipeline reads traffic sensor data from --inputFile.
In streaming mode, the pipeline reads the data from a Pub/Sub topic. By default, the example will run a separate pipeline to inject the data from the default --inputFile to the Pub/Sub --pubsubTopic. It will make it available for the streaming pipeline to process. You may override the default --inputFile with the file of your choosing. You may also set --inputFile to an empty string, which will disable the automatic Pub/Sub injection, and allow you to use separate tool to control the input to this example. An example code, which publishes traffic sensor data to a Pub/Sub topic, is provided in .
The example is configured to use the default Pub/Sub topic and the default BigQuery table from the example common package (there are no defaults for a general Dataflow pipeline). You can override them by using the --pubsubTopic, --bigQueryDataset, and --bigQueryTable options. If the Pub/Sub topic or the BigQuery table do not exist, the example will try to create them.
The example will try to cancel the pipelines on the signal to terminate the process (CTRL-C) and then exits.
| Constructor and Description |
|---|
TrafficRoutes() |
| Modifier and Type | Method and Description |
|---|---|
static void |
main(String[] args)
Sets up and starts streaming pipeline.
|
public static void main(String[] args) throws IOException
IOException - if there is a problem setting up resources