Package com.google.cloud.dataplex.v1

Class Task.SparkTaskConfig

  • All Implemented Interfaces:
    Task.SparkTaskConfigOrBuilder, com.google.protobuf.Message, com.google.protobuf.MessageLite, com.google.protobuf.MessageLiteOrBuilder, com.google.protobuf.MessageOrBuilder, Serializable
    Enclosing class:
    Task
    public static final class Task.SparkTaskConfig
extends com.google.protobuf.GeneratedMessageV3
implements Task.SparkTaskConfigOrBuilder
     User-specified config for running a Spark task.
    Protobuf type google.cloud.dataplex.v1.Task.SparkTaskConfig
    See Also:
    Serialized Form

    • Field Detail

      • MAIN_JAR_FILE_URI_FIELD_NUMBER

        public static final int MAIN_JAR_FILE_URI_FIELD_NUMBER
        See Also:
        Constant Field Values

      • MAIN_CLASS_FIELD_NUMBER

        public static final int MAIN_CLASS_FIELD_NUMBER
        See Also:
        Constant Field Values

      • PYTHON_SCRIPT_FILE_FIELD_NUMBER

        public static final int PYTHON_SCRIPT_FILE_FIELD_NUMBER
        See Also:
        Constant Field Values

      • SQL_SCRIPT_FILE_FIELD_NUMBER

        public static final int SQL_SCRIPT_FILE_FIELD_NUMBER
        See Also:
        Constant Field Values

      • SQL_SCRIPT_FIELD_NUMBER

        public static final int SQL_SCRIPT_FIELD_NUMBER
        See Also:
        Constant Field Values

      • ARCHIVE_URIS_FIELD_NUMBER

        public static final int ARCHIVE_URIS_FIELD_NUMBER
        See Also:
        Constant Field Values

      • INFRASTRUCTURE_SPEC_FIELD_NUMBER

        public static final int INFRASTRUCTURE_SPEC_FIELD_NUMBER
        See Also:
        Constant Field Values

    • Method Detail

      • newInstance

        protected Object newInstance​(com.google.protobuf.GeneratedMessageV3.UnusedPrivateParameter unused)
        Overrides:
        newInstance in class com.google.protobuf.GeneratedMessageV3

      • getDescriptor

        public static final com.google.protobuf.Descriptors.Descriptor getDescriptor()

      • internalGetFieldAccessorTable

        protected com.google.protobuf.GeneratedMessageV3.FieldAccessorTable internalGetFieldAccessorTable()
        Specified by:
        internalGetFieldAccessorTable in class com.google.protobuf.GeneratedMessageV3

      • hasMainJarFileUri

        public boolean hasMainJarFileUri()
         The Cloud Storage URI of the jar file that contains the main class.
 The execution args are passed in as a sequence of named process
 arguments (`--key=value`).
        string main_jar_file_uri = 100;
        Specified by:
        hasMainJarFileUri in interface Task.SparkTaskConfigOrBuilder
        Returns:
        Whether the mainJarFileUri field is set.

      • getMainJarFileUri

        public String getMainJarFileUri()
         The Cloud Storage URI of the jar file that contains the main class.
 The execution args are passed in as a sequence of named process
 arguments (`--key=value`).
        string main_jar_file_uri = 100;
        Specified by:
        getMainJarFileUri in interface Task.SparkTaskConfigOrBuilder
        Returns:
        The mainJarFileUri.

      • getMainJarFileUriBytes

        public com.google.protobuf.ByteString getMainJarFileUriBytes()
         The Cloud Storage URI of the jar file that contains the main class.
 The execution args are passed in as a sequence of named process
 arguments (`--key=value`).
        string main_jar_file_uri = 100;
        Specified by:
        getMainJarFileUriBytes in interface Task.SparkTaskConfigOrBuilder
        Returns:
        The bytes for mainJarFileUri.

      • hasMainClass

        public boolean hasMainClass()
         The name of the driver's main class. The jar file that contains the
 class must be in the default CLASSPATH or specified in
 `jar_file_uris`.
 The execution args are passed in as a sequence of named process
 arguments (`--key=value`).
        string main_class = 101;
        Specified by:
        hasMainClass in interface Task.SparkTaskConfigOrBuilder
        Returns:
        Whether the mainClass field is set.

      • getMainClass

        public String getMainClass()
         The name of the driver's main class. The jar file that contains the
 class must be in the default CLASSPATH or specified in
 `jar_file_uris`.
 The execution args are passed in as a sequence of named process
 arguments (`--key=value`).
        string main_class = 101;
        Specified by:
        getMainClass in interface Task.SparkTaskConfigOrBuilder
        Returns:
        The mainClass.

      • getMainClassBytes

        public com.google.protobuf.ByteString getMainClassBytes()
         The name of the driver's main class. The jar file that contains the
 class must be in the default CLASSPATH or specified in
 `jar_file_uris`.
 The execution args are passed in as a sequence of named process
 arguments (`--key=value`).
        string main_class = 101;
        Specified by:
        getMainClassBytes in interface Task.SparkTaskConfigOrBuilder
        Returns:
        The bytes for mainClass.

      • hasPythonScriptFile

        public boolean hasPythonScriptFile()
         The Gcloud Storage URI of the main Python file to use as the driver.
 Must be a .py file. The execution args are passed in as a sequence of
 named process arguments (`--key=value`).
        string python_script_file = 102;
        Specified by:
        hasPythonScriptFile in interface Task.SparkTaskConfigOrBuilder
        Returns:
        Whether the pythonScriptFile field is set.

      • getPythonScriptFile

        public String getPythonScriptFile()
         The Gcloud Storage URI of the main Python file to use as the driver.
 Must be a .py file. The execution args are passed in as a sequence of
 named process arguments (`--key=value`).
        string python_script_file = 102;
        Specified by:
        getPythonScriptFile in interface Task.SparkTaskConfigOrBuilder
        Returns:
        The pythonScriptFile.

      • getPythonScriptFileBytes

        public com.google.protobuf.ByteString getPythonScriptFileBytes()
         The Gcloud Storage URI of the main Python file to use as the driver.
 Must be a .py file. The execution args are passed in as a sequence of
 named process arguments (`--key=value`).
        string python_script_file = 102;
        Specified by:
        getPythonScriptFileBytes in interface Task.SparkTaskConfigOrBuilder
        Returns:
        The bytes for pythonScriptFile.

      • hasSqlScriptFile

        public boolean hasSqlScriptFile()
         A reference to a query file. This can be the Cloud Storage URI of the
 query file or it can the path to a SqlScript Content. The execution
 args are used to declare a set of script variables
 (`set key="value";`).
        string sql_script_file = 104;
        Specified by:
        hasSqlScriptFile in interface Task.SparkTaskConfigOrBuilder
        Returns:
        Whether the sqlScriptFile field is set.

      • getSqlScriptFile

        public String getSqlScriptFile()
         A reference to a query file. This can be the Cloud Storage URI of the
 query file or it can the path to a SqlScript Content. The execution
 args are used to declare a set of script variables
 (`set key="value";`).
        string sql_script_file = 104;
        Specified by:
        getSqlScriptFile in interface Task.SparkTaskConfigOrBuilder
        Returns:
        The sqlScriptFile.

      • getSqlScriptFileBytes

        public com.google.protobuf.ByteString getSqlScriptFileBytes()
         A reference to a query file. This can be the Cloud Storage URI of the
 query file or it can the path to a SqlScript Content. The execution
 args are used to declare a set of script variables
 (`set key="value";`).
        string sql_script_file = 104;
        Specified by:
        getSqlScriptFileBytes in interface Task.SparkTaskConfigOrBuilder
        Returns:
        The bytes for sqlScriptFile.

      • hasSqlScript

        public boolean hasSqlScript()
         The query text.
 The execution args are used to declare a set of script variables
 (`set key="value";`).
        string sql_script = 105;
        Specified by:
        hasSqlScript in interface Task.SparkTaskConfigOrBuilder
        Returns:
        Whether the sqlScript field is set.

      • getSqlScript

        public String getSqlScript()
         The query text.
 The execution args are used to declare a set of script variables
 (`set key="value";`).
        string sql_script = 105;
        Specified by:
        getSqlScript in interface Task.SparkTaskConfigOrBuilder
        Returns:
        The sqlScript.

      • getSqlScriptBytes

        public com.google.protobuf.ByteString getSqlScriptBytes()
         The query text.
 The execution args are used to declare a set of script variables
 (`set key="value";`).
        string sql_script = 105;
        Specified by:
        getSqlScriptBytes in interface Task.SparkTaskConfigOrBuilder
        Returns:
        The bytes for sqlScript.

      • getFileUrisList

        public com.google.protobuf.ProtocolStringList getFileUrisList()
         Optional. Cloud Storage URIs of files to be placed in the working
 directory of each executor.
        repeated string file_uris = 3 [(.google.api.field_behavior) = OPTIONAL];
        Specified by:
        getFileUrisList in interface Task.SparkTaskConfigOrBuilder
        Returns:
        A list containing the fileUris.

      • getFileUrisCount

        public int getFileUrisCount()
         Optional. Cloud Storage URIs of files to be placed in the working
 directory of each executor.
        repeated string file_uris = 3 [(.google.api.field_behavior) = OPTIONAL];
        Specified by:
        getFileUrisCount in interface Task.SparkTaskConfigOrBuilder
        Returns:
        The count of fileUris.

      • getFileUris

        public String getFileUris​(int index)
         Optional. Cloud Storage URIs of files to be placed in the working
 directory of each executor.
        repeated string file_uris = 3 [(.google.api.field_behavior) = OPTIONAL];
        Specified by:
        getFileUris in interface Task.SparkTaskConfigOrBuilder
        Parameters:
        index - The index of the element to return.
        Returns:
        The fileUris at the given index.

      • getFileUrisBytes

        public com.google.protobuf.ByteString getFileUrisBytes​(int index)
         Optional. Cloud Storage URIs of files to be placed in the working
 directory of each executor.
        repeated string file_uris = 3 [(.google.api.field_behavior) = OPTIONAL];
        Specified by:
        getFileUrisBytes in interface Task.SparkTaskConfigOrBuilder
        Parameters:
        index - The index of the value to return.
        Returns:
        The bytes of the fileUris at the given index.

      • getArchiveUrisList

        public com.google.protobuf.ProtocolStringList getArchiveUrisList()
         Optional. Cloud Storage URIs of archives to be extracted into the working
 directory of each executor. Supported file types: .jar, .tar, .tar.gz,
 .tgz, and .zip.
        repeated string archive_uris = 4 [(.google.api.field_behavior) = OPTIONAL];
        Specified by:
        getArchiveUrisList in interface Task.SparkTaskConfigOrBuilder
        Returns:
        A list containing the archiveUris.

      • getArchiveUrisCount

        public int getArchiveUrisCount()
         Optional. Cloud Storage URIs of archives to be extracted into the working
 directory of each executor. Supported file types: .jar, .tar, .tar.gz,
 .tgz, and .zip.
        repeated string archive_uris = 4 [(.google.api.field_behavior) = OPTIONAL];
        Specified by:
        getArchiveUrisCount in interface Task.SparkTaskConfigOrBuilder
        Returns:
        The count of archiveUris.

      • getArchiveUris

        public String getArchiveUris​(int index)
         Optional. Cloud Storage URIs of archives to be extracted into the working
 directory of each executor. Supported file types: .jar, .tar, .tar.gz,
 .tgz, and .zip.
        repeated string archive_uris = 4 [(.google.api.field_behavior) = OPTIONAL];
        Specified by:
        getArchiveUris in interface Task.SparkTaskConfigOrBuilder
        Parameters:
        index - The index of the element to return.
        Returns:
        The archiveUris at the given index.

      • getArchiveUrisBytes

        public com.google.protobuf.ByteString getArchiveUrisBytes​(int index)
         Optional. Cloud Storage URIs of archives to be extracted into the working
 directory of each executor. Supported file types: .jar, .tar, .tar.gz,
 .tgz, and .zip.
        repeated string archive_uris = 4 [(.google.api.field_behavior) = OPTIONAL];
        Specified by:
        getArchiveUrisBytes in interface Task.SparkTaskConfigOrBuilder
        Parameters:
        index - The index of the value to return.
        Returns:
        The bytes of the archiveUris at the given index.

      • hasInfrastructureSpec

        public boolean hasInfrastructureSpec()
         Optional. Infrastructure specification for the execution.
        .google.cloud.dataplex.v1.Task.InfrastructureSpec infrastructure_spec = 6 [(.google.api.field_behavior) = OPTIONAL];
        Specified by:
        hasInfrastructureSpec in interface Task.SparkTaskConfigOrBuilder
        Returns:
        Whether the infrastructureSpec field is set.

      • getInfrastructureSpec

        public Task.InfrastructureSpec getInfrastructureSpec()
         Optional. Infrastructure specification for the execution.
        .google.cloud.dataplex.v1.Task.InfrastructureSpec infrastructure_spec = 6 [(.google.api.field_behavior) = OPTIONAL];
        Specified by:
        getInfrastructureSpec in interface Task.SparkTaskConfigOrBuilder
        Returns:
        The infrastructureSpec.

      • isInitialized

        public final boolean isInitialized()
        Specified by:
        isInitialized in interface com.google.protobuf.MessageLiteOrBuilder
        Overrides:
        isInitialized in class com.google.protobuf.GeneratedMessageV3

      • writeTo

        public void writeTo​(com.google.protobuf.CodedOutputStream output)
             throws IOException
        Specified by:
        writeTo in interface com.google.protobuf.MessageLite
        Overrides:
        writeTo in class com.google.protobuf.GeneratedMessageV3
        Throws:
        IOException

      • getSerializedSize

        public int getSerializedSize()
        Specified by:
        getSerializedSize in interface com.google.protobuf.MessageLite
        Overrides:
        getSerializedSize in class com.google.protobuf.GeneratedMessageV3

      • equals

        public boolean equals​(Object obj)
        Specified by:
        equals in interface com.google.protobuf.Message
        Overrides:
        equals in class com.google.protobuf.AbstractMessage

      • hashCode

        public int hashCode()
        Specified by:
        hashCode in interface com.google.protobuf.Message
        Overrides:
        hashCode in class com.google.protobuf.AbstractMessage

      • parseFrom

        public static Task.SparkTaskConfig parseFrom​(ByteBuffer data)
                                      throws com.google.protobuf.InvalidProtocolBufferException
        Throws:
        com.google.protobuf.InvalidProtocolBufferException

      • parseFrom

        public static Task.SparkTaskConfig parseFrom​(ByteBuffer data,
                                             com.google.protobuf.ExtensionRegistryLite extensionRegistry)
                                      throws com.google.protobuf.InvalidProtocolBufferException
        Throws:
        com.google.protobuf.InvalidProtocolBufferException

      • parseFrom

        public static Task.SparkTaskConfig parseFrom​(com.google.protobuf.ByteString data)
                                      throws com.google.protobuf.InvalidProtocolBufferException
        Throws:
        com.google.protobuf.InvalidProtocolBufferException

      • parseFrom

        public static Task.SparkTaskConfig parseFrom​(com.google.protobuf.ByteString data,
                                             com.google.protobuf.ExtensionRegistryLite extensionRegistry)
                                      throws com.google.protobuf.InvalidProtocolBufferException
        Throws:
        com.google.protobuf.InvalidProtocolBufferException

      • parseFrom

        public static Task.SparkTaskConfig parseFrom​(byte[] data)
                                      throws com.google.protobuf.InvalidProtocolBufferException
        Throws:
        com.google.protobuf.InvalidProtocolBufferException

      • parseFrom

        public static Task.SparkTaskConfig parseFrom​(byte[] data,
                                             com.google.protobuf.ExtensionRegistryLite extensionRegistry)
                                      throws com.google.protobuf.InvalidProtocolBufferException
        Throws:
        com.google.protobuf.InvalidProtocolBufferException

      • newBuilderForType

        public Task.SparkTaskConfig.Builder newBuilderForType()
        Specified by:
        newBuilderForType in interface com.google.protobuf.Message
        Specified by:
        newBuilderForType in interface com.google.protobuf.MessageLite

      • toBuilder

        public Task.SparkTaskConfig.Builder toBuilder()
        Specified by:
        toBuilder in interface com.google.protobuf.Message
        Specified by:
        toBuilder in interface com.google.protobuf.MessageLite

      • newBuilderForType

        protected Task.SparkTaskConfig.Builder newBuilderForType​(com.google.protobuf.GeneratedMessageV3.BuilderParent parent)
        Specified by:
        newBuilderForType in class com.google.protobuf.GeneratedMessageV3

      • getParserForType

        public com.google.protobuf.Parser<Task.SparkTaskConfig> getParserForType()
        Specified by:
        getParserForType in interface com.google.protobuf.Message
        Specified by:
        getParserForType in interface com.google.protobuf.MessageLite
        Overrides:
        getParserForType in class com.google.protobuf.GeneratedMessageV3

      • getDefaultInstanceForType

        public Task.SparkTaskConfig getDefaultInstanceForType()
        Specified by:
        getDefaultInstanceForType in interface com.google.protobuf.MessageLiteOrBuilder
        Specified by:
        getDefaultInstanceForType in interface com.google.protobuf.MessageOrBuilder