com.google.cloud.dataflow.sdk.runners.worker

Class TextSink<T>

java.lang.Object

com.google.cloud.dataflow.sdk.util.common.worker.Sink<T>

com.google.cloud.dataflow.sdk.runners.worker.TextSink<T>

Type Parameters:

T - the type of the elements written to the sink

public class TextSink<T>
extends Sink<T>

A sink that writes text files.

Nested Class Summary

Nested classes/interfaces inherited from class com.google.cloud.dataflow.sdk.util.common.worker.Sink

Sink.SinkWriter<ElemT>

Method Summary

Modifier and Type	Method and Description
static <V> TextSink<V>	create(String filenamePrefix, String shardFormat, String filenameSuffix, int shardCount, boolean appendTrailingNewlines, String header, String footer, Coder<V> coder) Constructs a new TextSink.
static <V> TextSink<WindowedValue<V>>	createForDirectPipelineRunner(String filenamePrefix, String shardFormat, String filenameSuffix, int shardCount, boolean appendTrailingNewlines, String header, String footer, Coder<V> coder) For DirectPipelineRunner only.
static <V> TextSink<WindowedValue<V>>	createForTest(String filename, boolean appendTrailingNewlines, String header, String footer, Coder<V> coder) For testing only.
Sink.SinkWriter<T>	writer() Returns a Writer that allows writing to this Sink.

Methods inherited from class com.google.cloud.dataflow.sdk.util.common.worker.Sink

supportsRestart

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

createForTest

public static <V> TextSink<WindowedValue<V>> createForTest(String filename,
                                                           boolean appendTrailingNewlines,
                                                           @Nullable
                                                           String header,
                                                           @Nullable
                                                           String footer,
                                                           Coder<V> coder)

For testing only.

Used by simple tests that write to a single unsharded file.

createForDirectPipelineRunner

public static <V> TextSink<WindowedValue<V>> createForDirectPipelineRunner(String filenamePrefix,
                                                                           String shardFormat,
                                                                           String filenameSuffix,
                                                                           int shardCount,
                                                                           boolean appendTrailingNewlines,
                                                                           @Nullable
                                                                           String header,
                                                                           @Nullable
                                                                           String footer,
                                                                           Coder<V> coder)

For DirectPipelineRunner only. It wraps the coder with WindowedValue.ValueOnlyCoder.

create

public static <V> TextSink<V> create(String filenamePrefix,
                                     String shardFormat,
                                     String filenameSuffix,
                                     int shardCount,
                                     boolean appendTrailingNewlines,
                                     @Nullable
                                     String header,
                                     @Nullable
                                     String footer,
                                     Coder<V> coder)

Constructs a new TextSink.

Parameters:

filenamePrefix - the prefix of output filenames.

shardFormat - the shard name template to use for output filenames.

filenameSuffix - the suffix of output filenames.

shardCount - the number of outupt shards to produce.

appendTrailingNewlines - true to append newlines to each output line.

header - text to place at the beginning of each output file.

footer - text to place at the end of each output file.

coder - the code used to encode elements for output.

writer

public Sink.SinkWriter<T> writer()
                          throws IOException

Description copied from class: Sink

Returns a Writer that allows writing to this Sink.

Specified by:

writer in class Sink<T>

Throws:

IOException