Package com.google.cloud.bigquery

Interface BigQuery

    • Method Detail

      • create

        Dataset create​(DatasetInfo datasetInfo,
               BigQuery.DatasetOption... options)
        Creates a new dataset.

        Example of creating a dataset. 

         {
   @code
   String datasetName = "my_dataset_name";
   Dataset dataset = null;
   DatasetInfo datasetInfo = DatasetInfo.newBuilder(datasetName).build();
   try {
     // the dataset was created
     dataset = bigquery.create(datasetInfo);
   } catch (BigQueryException e) {
     // the dataset was not created
   }
 }
        Throws:
        BigQueryException - upon failure

      • create

        Table create​(TableInfo tableInfo,
             BigQuery.TableOption... options)
        Creates a new table.

        Example of creating a table. 

         {
   @code
   String datasetName = "my_dataset_name";
   String tableName = "my_table_name";
   String fieldName = "string_field";
   TableId tableId = TableId.of(datasetName, tableName);
   // Table field definition
   Field field = Field.of(fieldName, LegacySQLTypeName.STRING);
   // Table schema definition
   Schema schema = Schema.of(field);
   TableDefinition tableDefinition = StandardTableDefinition.of(schema);
   TableInfo tableInfo = TableInfo.newBuilder(tableId, tableDefinition).build();
   Table table = bigquery.create(tableInfo);
 }
        Throws:
        BigQueryException - upon failure

      • create

        Job create​(JobInfo jobInfo,
           BigQuery.JobOption... options)
        Creates a new job.

        Example of loading a newline-delimited-json file with textual fields from GCS to a table. 

         {
   @code
   String datasetName = "my_dataset_name";
   String tableName = "my_table_name";
   String sourceUri = "gs://cloud-samples-data/bigquery/us-states/us-states.json";
   TableId tableId = TableId.of(datasetName, tableName);
   // Table field definition
   Field[] fields = new Field[] { Field.of("name", LegacySQLTypeName.STRING),
       Field.of("post_abbr", LegacySQLTypeName.STRING) };
   // Table schema definition
   Schema schema = Schema.of(fields);
   LoadJobConfiguration configuration = LoadJobConfiguration.builder(tableId, sourceUri)
       .setFormatOptions(FormatOptions.json()).setCreateDisposition(CreateDisposition.CREATE_IF_NEEDED)
       .setSchema(schema).build();
   // Load the table
   Job loadJob = bigquery.create(JobInfo.of(configuration));
   loadJob = loadJob.waitFor();
   // Check the table
   System.out.println("State: " + loadJob.getStatus().getState());
   return ((StandardTableDefinition) bigquery.getTable(tableId).getDefinition()).getNumRows();
 }

        Example of creating a query job. 

         {
   @code
   String query = "SELECT field FROM my_dataset_name.my_table_name";
   Job job = null;
   JobConfiguration jobConfiguration = QueryJobConfiguration.of(query);
   JobInfo jobInfo = JobInfo.of(jobConfiguration);
   try {
     job = bigquery.create(jobInfo);
   } catch (BigQueryException e) {
     // the job was not created
   }
 }
        Throws:
        BigQueryException - upon failure

      • createConnection

        @BetaApi
Connection createConnection​(@NonNull ConnectionSettings connectionSettings)
        Creates a new BigQuery query connection used for executing queries (not the same as BigQuery connection properties). It uses the BigQuery Storage Read API for high throughput queries by default.

        Example of creating a query connection. 

         {
   @code
       ConnectionSettings connectionSettings =
         ConnectionSettings.newBuilder()
             .setRequestTimeout(10L)
             .setMaxResults(100L)
             .setUseQueryCache(true)
             .build();
       Connection connection = bigquery.createConnection(connectionSettings);
 }
        Parameters:
        connectionSettings -
        Throws:
        BigQueryException - upon failure

      • createConnection

        @BetaApi
Connection createConnection()
        Creates a new BigQuery query connection used for executing queries (not the same as BigQuery connection properties). It uses the BigQuery Storage Read API for high throughput queries by default. This overloaded method creates a Connection with default ConnectionSettings for query execution where default values are set for numBufferedRows (20000), useReadApi (true), useLegacySql (false).

        Example of creating a query connection. 

         {
   @code
       Connection connection = bigquery.createConnection();
 }
        Throws:
        BigQueryException - upon failure

      • getDataset

        Dataset getDataset​(String datasetId,
                   BigQuery.DatasetOption... options)
        Returns the requested dataset or null if not found.

        Example of getting a dataset. 

         {
   @code
   String datasetName = "my_dataset";
   Dataset dataset = bigquery.getDataset(datasetName);
 }
        Throws:
        BigQueryException - upon failure

      • getDataset

        Dataset getDataset​(DatasetId datasetId,
                   BigQuery.DatasetOption... options)
        Returns the requested dataset or null if not found.

        Example of getting a dataset. 

         {
   @code
   String projectId = "my_project_id";
   String datasetName = "my_dataset_name";
   DatasetId datasetId = DatasetId.of(projectId, datasetName);
   Dataset dataset = bigquery.getDataset(datasetId);
 }
        Throws:
        BigQueryException - upon failure

      • delete

        boolean delete​(String datasetId,
               BigQuery.DatasetDeleteOption... options)
        Deletes the requested dataset.

        Example of deleting a dataset from its id, even if non-empty. 

         {
   @code
   String datasetName = "my_dataset_name";
   boolean deleted = bigquery.delete(datasetName, DatasetDeleteOption.deleteContents());
   if (deleted) {
     // the dataset was deleted
   } else {
     // the dataset was not found
   }
 }
        Returns:
        true if dataset was deleted, false if it was not found
        Throws:
        BigQueryException - upon failure

      • delete

        boolean delete​(DatasetId datasetId,
               BigQuery.DatasetDeleteOption... options)
        Deletes the requested dataset.

        Example of deleting a dataset, even if non-empty. 

         {
   @code
   String projectId = "my_project_id";
   String datasetName = "my_dataset_name";
   DatasetId datasetId = DatasetId.of(projectId, datasetName);
   boolean deleted = bigquery.delete(datasetId, DatasetDeleteOption.deleteContents());
   if (deleted) {
     // the dataset was deleted
   } else {
     // the dataset was not found
   }
 }
        Returns:
        true if dataset was deleted, false if it was not found
        Throws:
        BigQueryException - upon failure

      • delete

        @Deprecated
boolean delete​(String datasetId,
               String tableId)
        Deprecated.
        Now that BigQuery datasets contain multiple resource types, this invocation is ambiguous. Please use more strongly typed version of #delete that leverages an non-ambiguous resource type identifier such as TableId.
        Deletes the requested table.
        Returns:
        true if table was deleted, false if it was not found
        Throws:
        BigQueryException - upon failure

      • delete

        boolean delete​(TableId tableId)
        Deletes the requested table.

        Example of deleting a table. 

         {
   @code
   String projectId = "my_project_id";
   String datasetName = "my_dataset_name";
   String tableName = "my_table_name";
   TableId tableId = TableId.of(projectId, datasetName, tableName);
   boolean deleted = bigquery.delete(tableId);
   if (deleted) {
     // the table was deleted
   } else {
     // the table was not found
   }
 }
        Returns:
        true if table was deleted, false if it was not found
        Throws:
        BigQueryException - upon failure

      • delete

        boolean delete​(ModelId modelId)
        Deletes the requested model.

        Example of deleting a model. 

         {
   @code
   String projectId = "my_project_id";
   String datasetName = "my_dataset_name";
   String tableName = "my_model_name";
   ModelId modelId = ModelId.of(projectId, datasetName, modelName);
   boolean deleted = bigquery.delete(modelId);
   if (deleted) {
     // the model was deleted
   } else {
     // the model was not found
   }
 }
        Returns:
        true if model was deleted, false if it was not found
        Throws:
        BigQueryException - upon failure

      • delete

        boolean delete​(RoutineId routineId)
        Deletes the requested routine.

        Example of deleting a routine. 

        
 String projectId = "my_project_id";
 String datasetId = "my_dataset_id";
 String routineId = "my_routine_id";
 RoutineId routineId = RoutineId.of(projectId, datasetId, routineId);
 boolean deleted = bigquery.delete(routineId);
 if (deleted) {
   // the routine was deleted
 } else {
   // the routine was not found
 }
        Returns:
        true if routine was deleted, false if it was not found
        Throws:
        BigQueryException - upon failure

      • delete

        boolean delete​(JobId jobId)
        Deletes the requested job.
        Returns:
        true if job was deleted, false if it was not found
        Throws:
        BigQueryException - upon failure

      • update

        Dataset update​(DatasetInfo datasetInfo,
               BigQuery.DatasetOption... options)
        Updates dataset information.

        Example of updating a dataset by changing its description. 

         {
   @code
   // String datasetName = "my_dataset_name";
   // String tableName = "my_table_name";
   // String newDescription = "new_description";

   Table beforeTable = bigquery.getTable(datasetName, tableName);
   TableInfo tableInfo = beforeTable.toBuilder().setDescription(newDescription).build();
   Table afterTable = bigquery.update(tableInfo);

 }
        Throws:
        BigQueryException - upon failure

      • update

        Table update​(TableInfo tableInfo,
             BigQuery.TableOption... options)
        Updates table information.

        Example of updating a table by changing its description. 

         {
   @code
   String datasetName = "my_dataset_name";
   String tableName = "my_table_name";
   String newDescription = "new_description";
   Table beforeTable = bigquery.getTable(datasetName, tableName);
   TableInfo tableInfo = beforeTable.toBuilder().setDescription(newDescription).build();
   Table afterTable = bigquery.update(tableInfo);
 }

        Example of updating a table by changing its expiration. 

         {
   @code
   String datasetName = "my_dataset_name";
   String tableName = "my_table_name";
   Table beforeTable = bigquery.getTable(datasetName, tableName);

   // Set table to expire 5 days from now.
   long expirationMillis = DateTime.now().plusDays(5).getMillis();
   TableInfo tableInfo = beforeTable.toBuilder().setExpirationTime(expirationMillis).build();
   Table afterTable = bigquery.update(tableInfo);
 }
        Throws:
        BigQueryException - upon failure

      • update

        Model update​(ModelInfo modelInfo,
             BigQuery.ModelOption... options)
        Updates model information.

        Example of updating a model by changing its description. 

         {
   @code
   String datasetName = "my_dataset_name";
   String modelName = "my_model_name";
   String newDescription = "new_description";
   Model beforeModel = bigquery.getModel(datasetName, modelName);
   ModelInfo modelInfo = beforeModel.toBuilder().setDescription(newDescription).build();
   Model afterModel = bigquery.update(modelInfo);
 }

        Example of updating a model by changing its expiration. 

         {
   @code
   String datasetName = "my_dataset_name";
   String modelName = "my_model_name";
   Model beforeModel = bigquery.getModel(datasetName, modelName);

   // Set model to expire 5 days from now.
   long expirationMillis = DateTime.now().plusDays(5).getMillis();
   ModelInfo modelInfo = beforeModel.toBuilder().setExpirationTime(expirationMillis).build();
   Model afterModel = bigquery.update(modelInfo);
 }
        Throws:
        BigQueryException - upon failure

      • getTable

        Table getTable​(String datasetId,
               String tableId,
               BigQuery.TableOption... options)
        Returns the requested table or null if not found.

        Example of getting a table. 

         {
   @code
   String datasetName = "my_dataset_name";
   String tableName = "my_table_name";
   Table table = bigquery.getTable(datasetName, tableName);
 }
        Throws:
        BigQueryException - upon failure

      • getTable

        Table getTable​(TableId tableId,
               BigQuery.TableOption... options)
        Returns the requested table or null if not found.

        Example of getting a table. 

         {
   @code
   String projectId = "my_project_id";
   String datasetName = "my_dataset_name";
   String tableName = "my_table_name";
   TableId tableId = TableId.of(projectId, datasetName, tableName);
   Table table = bigquery.getTable(tableId);
 }
        Throws:
        BigQueryException - upon failure

      • getModel

        Model getModel​(ModelId tableId,
               BigQuery.ModelOption... options)
        Returns the requested model or null if not found.

        Example of getting a model. 

         {
   @code
   String projectId = "my_project_id";
   String datasetName = "my_dataset_name";
   String modelName = "my_model_name";
   ModelId modelId = ModelId.of(projectId, datasetName, tableName);
   Model model = bigquery.getModel(modelId);
 }
        Throws:
        BigQueryException - upon failure

      • listPartitions

        List<String> listPartitions​(TableId tableId)
        Parameters:
        tableId -
        Returns:
        A list of the partition ids present in the partitioned table

      • insertAll

        InsertAllResponse insertAll​(InsertAllRequest request)
        Sends an insert all request.

        Example of inserting rows into a table without running a load job. 

         {
   @code
   String datasetName = "my_dataset_name";
   String tableName = "my_table_name";
   TableId tableId = TableId.of(datasetName, tableName);
   // Values of the row to insert
   Map<String, Object> rowContent = new HashMap<>();
   rowContent.put("booleanField", true);
   // Bytes are passed in base64
   rowContent.put("bytesField", "Cg0NDg0="); // 0xA, 0xD, 0xD, 0xE, 0xD in base64
   // Records are passed as a map
   Map<String, Object> recordsContent = new HashMap<>();
   recordsContent.put("stringField", "Hello, World!");
   rowContent.put("recordField", recordsContent);
   InsertAllResponse response = bigquery.insertAll(InsertAllRequest.newBuilder(tableId).addRow("rowId", rowContent)
       // More rows can be added in the same RPC by invoking .addRow() on the
       // builder
       .build());
   if (response.hasErrors()) {
     // If any of the insertions failed, this lets you inspect the errors
     for (Entry<Long, List<BigQueryError>> entry : response.getInsertErrors().entrySet()) {
       // inspect row error
     }
   }
 }
        Throws:
        BigQueryException - upon failure

      • listTableData

        TableResult listTableData​(String datasetId,
                          String tableId,
                          BigQuery.TableDataListOption... options)
        Lists the table's rows.

        Example of listing table rows, specifying the page size. 

         {
   @code
   String datasetName = "my_dataset_name";
   String tableName = "my_table_name";
   // This example reads the result 100 rows per RPC call. If there's no need
   // to limit the number,
   // simply omit the option.
   TableResult tableData = bigquery.listTableData(datasetName, tableName, TableDataListOption.pageSize(100));
   for (FieldValueList row : tableData.iterateAll()) {
     // do something with the row
   }
 }
        Throws:
        BigQueryException - upon failure

      • listTableData

        TableResult listTableData​(TableId tableId,
                          BigQuery.TableDataListOption... options)
        Lists the table's rows.

        Example of listing table rows, specifying the page size. 

         {
   @code
   String datasetName = "my_dataset_name";
   String tableName = "my_table_name";
   TableId tableIdObject = TableId.of(datasetName, tableName);
   // This example reads the result 100 rows per RPC call. If there's no need
   // to limit the number,
   // simply omit the option.
   TableResult tableData = bigquery.listTableData(tableIdObject, TableDataListOption.pageSize(100));
   for (FieldValueList row : tableData.iterateAll()) {
     // do something with the row
   }
 }
        Throws:
        BigQueryException - upon failure

      • listTableData

        TableResult listTableData​(String datasetId,
                          String tableId,
                          Schema schema,
                          BigQuery.TableDataListOption... options)
        Lists the table's rows. If the schema is not null, it is available to the FieldValueList iterated over.

        Example of listing table rows with schema. 

        
 String datasetName = "my_dataset_name";
 String tableName = "my_table_name";
 Schema schema = ...;
 String field = "field";
 TableResult tableData = bigquery.listTableData(datasetName, tableName, schema);
 for (FieldValueList row : tableData.iterateAll()) {
   row.get(field);
 }
        Throws:
        BigQueryException - upon failure

      • listTableData

        TableResult listTableData​(TableId tableId,
                          Schema schema,
                          BigQuery.TableDataListOption... options)
        Lists the table's rows. If the schema is not null, it is available to the FieldValueList iterated over.

        Example of listing table rows with schema. 

         {
   @code
   Schema schema = Schema.of(Field.of("word", LegacySQLTypeName.STRING),
       Field.of("word_count", LegacySQLTypeName.STRING), Field.of("corpus", LegacySQLTypeName.STRING),
       Field.of("corpus_date", LegacySQLTypeName.STRING));
   TableResult tableData = bigquery.listTableData(TableId.of("bigquery-public-data", "samples", "shakespeare"),
       schema);
   FieldValueList row = tableData.getValues().iterator().next();
   System.out.println(row.get("word").getStringValue());
 }
        Throws:
        BigQueryException - upon failure

      • getJob

        Job getJob​(String jobId,
           BigQuery.JobOption... options)
        Returns the requested job or null if not found. If the location of the job is not "US" or "EU", getJob(JobId, JobOption...) must be used instead.

        Example of getting a job. 

         {
   @code
   String jobName = "my_job_name";
   Job job = bigquery.getJob(jobName);
   if (job == null) {
     // job was not found
   }
 }
        Throws:
        BigQueryException - upon failure

      • getJob

        Job getJob​(JobId jobId,
           BigQuery.JobOption... options)
        Returns the requested job or null if not found. If the location of the job is not "US" or "EU", the jobId must specify the job location.

        Example of getting a job. 

         {
   @code
   String jobName = "my_job_name";
   JobId jobIdObject = JobId.of(jobName);
   Job job = bigquery.getJob(jobIdObject);
   if (job == null) {
     // job was not found
   }
 }
        Throws:
        BigQueryException - upon failure

      • listJobs

        com.google.api.gax.paging.Page<Job> listJobs​(BigQuery.JobListOption... options)
        Lists the jobs.

        Example of listing jobs, specifying the page size. 

         {
   @code
   Page<Job> jobs = bigquery.listJobs(JobListOption.pageSize(100));
   for (Job job : jobs.iterateAll()) {
     // do something with the job
   }
 }
        Throws:
        BigQueryException - upon failure

      • cancel

        boolean cancel​(String jobId)
        Sends a job cancel request. This call will return immediately. The job status can then be checked using either getJob(JobId, JobOption...) or getJob(String, JobOption...)).

        If the location of the job is not "US" or "EU", cancel(JobId) must be used instead.

        Example of cancelling a job. 

         {
   @code
   String jobName = "my_job_name";
   boolean success = bigquery.cancel(jobName);
   if (success) {
     // job was cancelled
   } else {
     // job was not found
   }
 }
        Returns:
        true if cancel was requested successfully, false if the job was not found
        Throws:
        BigQueryException - upon failure

      • cancel

        boolean cancel​(JobId jobId)
        Sends a job cancel request. This call will return immediately. The job status can then be checked using either getJob(JobId, JobOption...) or getJob(String, JobOption...)).

        If the location of the job is not "US" or "EU", the jobId must specify the job location.

        Example of cancelling a job. 

         {
   @code
   String jobName = "my_job_name";
   JobId jobId = JobId.of(jobName);
   boolean success = bigquery.cancel(jobId);
   if (success) {
     // job was cancelled
   } else {
     // job was not found
   }
 }
        Returns:
        true if cancel was requested successfully, false if the job was not found
        Throws:
        BigQueryException - upon failure

      • query

        TableResult query​(QueryJobConfiguration configuration,
                  BigQuery.JobOption... options)
           throws InterruptedException,
                  JobException
        Runs the query associated with the request, using an internally-generated random JobId.

        If the location of the job is not "US" or "EU", query(QueryJobConfiguration, JobId, JobOption...) must be used instead.

        This method cannot be used in conjuction with QueryJobConfiguration.dryRun() queries. Since dry-run queries are not actually executed, there's no way to retrieve results.

        Example of running a query. 

         {
   @code
   // BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();
   String query = "SELECT corpus FROM `bigquery-public-data.samples.shakespeare` GROUP BY corpus;";
   QueryJobConfiguration queryConfig = QueryJobConfiguration.newBuilder(query).build();

   // Print the results.
   for (FieldValueList row : bigquery.query(queryConfig).iterateAll()) {
     for (FieldValue val : row) {
       System.out.printf("%s,", val.toString());
     }
     System.out.printf("\n");
   }
 }
        Throws:
        BigQueryException - upon failure
        InterruptedException - if the current thread gets interrupted while waiting for the query to complete
        JobException - if the job completes unsuccessfully

      • writer

        TableDataWriteChannel writer​(WriteChannelConfiguration writeChannelConfiguration)
        Returns a channel to write data to be inserted into a BigQuery table. Data format and other options can be configured using the WriteChannelConfiguration parameter. If the job is not in "US" or "EU", writer(JobId, WriteChannelConfiguration) must be used instead.

        Example of creating a channel with which to write to a table. 

         {
   @code
   String datasetName = "my_dataset_name";
   String tableName = "my_table_name";
   String csvData = "StringValue1\nStringValue2\n";
   TableId tableId = TableId.of(datasetName, tableName);
   WriteChannelConfiguration writeChannelConfiguration = WriteChannelConfiguration.newBuilder(tableId)
       .setFormatOptions(FormatOptions.csv()).build();
   TableDataWriteChannel writer = bigquery.writer(writeChannelConfiguration);
   // Write data to writer
   try {
     writer.write(ByteBuffer.wrap(csvData.getBytes(Charsets.UTF_8)));
   } finally {
     writer.close();
   }
   // Get load job
   Job job = writer.getJob();
   job = job.waitFor();
   LoadStatistics stats = job.getStatistics();
   return stats.getOutputRows();
 }

        Example of writing a local file to a table. 

         {
   @code
   String datasetName = "my_dataset_name";
   String tableName = "my_table_name";
   Path csvPath = FileSystems.getDefault().getPath(".", "my-data.csv");
   String location = "us";
   TableId tableId = TableId.of(datasetName, tableName);
   WriteChannelConfiguration writeChannelConfiguration = WriteChannelConfiguration.newBuilder(tableId)
       .setFormatOptions(FormatOptions.csv()).build();
   // The location must be specified; other fields can be auto-detected.
   JobId jobId = JobId.newBuilder().setLocation(location).build();
   TableDataWriteChannel writer = bigquery.writer(jobId, writeChannelConfiguration);
   // Write data to writer
   try (OutputStream stream = Channels.newOutputStream(writer)) {
     Files.copy(csvPath, stream);
   }
   // Get load job
   Job job = writer.getJob();
   job = job.waitFor();
   LoadStatistics stats = job.getStatistics();
   return stats.getOutputRows();
 }
        Throws:
        BigQueryException - upon failure

      • writer

        TableDataWriteChannel writer​(JobId jobId,
                             WriteChannelConfiguration writeChannelConfiguration)
        Returns a channel to write data to be inserted into a BigQuery table. Data format and other options can be configured using the WriteChannelConfiguration parameter. If the job is not in "US" or "EU", the jobId must contain the location of the job.

        Example of creating a channel with which to write to a table. 

         {
   @code
   String datasetName = "my_dataset_name";
   String tableName = "my_table_name";
   String csvData = "StringValue1\nStringValue2\n";
   String location = "us";
   TableId tableId = TableId.of(datasetName, tableName);
   WriteChannelConfiguration writeChannelConfiguration = WriteChannelConfiguration.newBuilder(tableId)
       .setFormatOptions(FormatOptions.csv()).build();
   // The location must be specified; other fields can be auto-detected.
   JobId jobId = JobId.newBuilder().setLocation(location).build();
   TableDataWriteChannel writer = bigquery.writer(jobId, writeChannelConfiguration);
   // Write data to writer
   try {
     writer.write(ByteBuffer.wrap(csvData.getBytes(Charsets.UTF_8)));
   } finally {
     writer.close();
   }
   // Get load job
   Job job = writer.getJob();
   job = job.waitFor();
   LoadStatistics stats = job.getStatistics();
   return stats.getOutputRows();
 }

      • getIamPolicy

        com.google.cloud.Policy getIamPolicy​(TableId tableId,
                                     BigQuery.IAMOption... options)
        Gets the IAM policy for a specified table.

      • setIamPolicy

        com.google.cloud.Policy setIamPolicy​(TableId tableId,
                                     com.google.cloud.Policy policy,
                                     BigQuery.IAMOption... options)
        Sets the IAM policy for a specified table.

      • testIamPermissions

        List<String> testIamPermissions​(TableId table,
                                List<String> permissions,
                                BigQuery.IAMOption... options)
        Tests whether the caller holds specific permissions on a BigQuery table. The returned list represents the subset of granted permissions.