com.google.cloud.dataflow.examples

Class TopWikipediaSessions

java.lang.Object

com.google.cloud.dataflow.examples.TopWikipediaSessions

public class TopWikipediaSessions
extends java.lang.Object

An example that reads Wikipedia edit data from Cloud Storage and computes the user with the longest string of edits separated by no more than an hour within each month.

Concepts: Using Windowing to perform time-based aggregations of data.

It is not recommended to execute this pipeline locally, given the size of the default input data.

To execute this pipeline using the Dataflow service, specify pipeline configuration: --project= --stagingLocation=gs:// --runner=BlockingDataflowPipelineRunner and an output prefix on GCS: --output=gs://

The default input is gs://dataflow-samples/wikipedia_edits/*.json and can be overridden with --input.

The input for this example is large enough that it's a good place to enable (experimental) autoscaling: --autoscalingAlgorithm=BASIC --maxNumWorkers=20 This will automatically scale the number of workers up over time until the job completes.

Constructor Summary

Constructors

Constructor and Description
TopWikipediaSessions()

Method Summary

Methods

Modifier and Type	Method and Description
static void	main(java.lang.String[] args)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TopWikipediaSessions

public TopWikipediaSessions()

Method Detail

main

public static void main(java.lang.String[] args)