public class StreamingWordExtract extends Object
This pipeline example reads lines of text from a PubSub topic, splits each line into individual words, capitalizes those words, and writes the output to a BigQuery table.
By default, the example will run a separate pipeline to inject the data from the default --inputFile to the Pub/Sub --pubsubTopic. It will make it available for the streaming pipeline to process. You may override the default --inputFile with the file of your choosing. You may also set --inputFile to an empty string, which will disable the automatic Pub/Sub injection, and allow you to use separate tool to control the input to this example.
The example is configured to use the default Pub/Sub topic and the default BigQuery table from the example common package (there are no defaults for a general Dataflow pipeline). You can override them by using the --pubsubTopic, --bigQueryDataset, and --bigQueryTable options. If the Pub/Sub topic or the BigQuery table do not exist, the example will try to create them.
The example will try to cancel the pipelines on the signal to terminate the process (CTRL-C) and then exits.
| Constructor and Description |
|---|
StreamingWordExtract() |
| Modifier and Type | Method and Description |
|---|---|
static void |
main(String[] args)
Sets up and starts streaming pipeline.
|
public static void main(String[] args) throws IOException
IOException - if there is a problem setting up resources