com.google.cloud.dataflow.sdk.util

Class GcsUtil

java.lang.Object

com.google.cloud.dataflow.sdk.util.GcsUtil

public class GcsUtil
extends Object

Provides operations on GCS.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	GcsUtil.GcsUtilFactory This is a DefaultValueFactory able to create a GcsUtil using any transport flags specified on the PipelineOptions.

Method Summary

Modifier and Type	Method and Description
boolean	bucketExists(GcsPath path) Returns whether the GCS bucket exists.
WritableByteChannel	create(GcsPath path, String type) Creates an object in GCS.
List<GcsPath>	expand(GcsPath gcsPattern) Expands a pattern into matched paths.
long	fileSize(GcsPath path) Returns the file size from GCS or throws FileNotFoundException if the resource does not exist.
boolean	isGcsPatternSupported(String gcsPattern) Returns true if the given GCS pattern is supported otherwise fails with an exception.
SeekableByteChannel	open(GcsPath path) Opens an object in GCS.
protected void	setStorageClient(com.google.api.services.storage.Storage storage)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

isGcsPatternSupported

public boolean isGcsPatternSupported(String gcsPattern)

Returns true if the given GCS pattern is supported otherwise fails with an exception.

setStorageClient

protected void setStorageClient(com.google.api.services.storage.Storage storage)

expand

public List<GcsPath> expand(GcsPath gcsPattern)
                     throws IOException

Expands a pattern into matched paths. The pattern path may contain globs, which are expanded in the result. This function may return non-existent files so this should not be used to validate the existence of files in GCS.

Throws:

IOException

fileSize

public long fileSize(GcsPath path)
              throws IOException

Returns the file size from GCS or throws FileNotFoundException if the resource does not exist.

Throws:

IOException

open

public SeekableByteChannel open(GcsPath path)
                         throws IOException

Opens an object in GCS.

Returns a SeekableByteChannel that provides access to data in the bucket.

Parameters:

path - the GCS filename to read from

Returns:

a SeekableByteChannel that can read the object data

Throws:

IOException

create

public WritableByteChannel create(GcsPath path,
                                  String type)
                           throws IOException

Creates an object in GCS.

Returns a WritableByteChannel that can be used to write data to the object.

Parameters:

path - the GCS file to write to

type - the type of object, eg "text/plain".

Returns:

a Callable object that encloses the operation.

Throws:

IOException

bucketExists

public boolean bucketExists(GcsPath path)
                     throws IOException

Returns whether the GCS bucket exists. If the bucket exists, it must be accessible otherwise the permissions exception will be propagated.

Throws:

IOException