BigQuery Integration

Stemma extracts table and column metadata including database, schema, table name, table description, column name, and column description from a Bigquery database.

What We Need From You

You will need to create a service account for reading metadata and grant it “BigQuery Metadata Viewer” access to all of your datasets. This can all be done via the BigQUery UI.

Next, collect the information Stemma needs, as follows:

• Authorization/Access Scopes:
  Stemma uses the following scopes to extract metadata and usage information.
  — <https://www.googleapis.com/auth/bigquery.readonly>
  — <https://www.googleapis.com/auth/cloud-platform>
• A JSON Key File: Proceed as follows to create the JSON key file for a service account.

Create a service account (via Cloud Console):

1. In the Cloud Console, go to the Create service account page.
2. Select a project.
3. In the Service account name field, enter a name. The Cloud Console fills in the Service account ID field based on this name.
4. In the Service account description field, enter a description. For example, `Service account for Stemma
   BigQuery Integration
5. Click Create and continue.
6. Click the Select a role field, and make sure to select the following:
   — BigQuery Data Viewer (roles/bigquery.dataViewer) – to extract the metadata, watermarks and usage information. Popular tables will be generated using this information.
   — BigQuery User (roles/bigquery.user) – Needed to map the user’s information with the Tables i.e., Frequent Users, Table Owners, etc.
   — Private Logs Viewer (roles/logging.privateLogViewer) – Needed for enriching table usage.
7. Click Continue, and click Done to finish creating the service account.

Do not close your browser window. You will use it in the next step.

Create a service account Key:

1. In the Cloud Console, click the email address for the service account that you created.
2. Click Keys, click Add key, then click Create new key.
3. Click Create. A JSON key file is downloaded to your computer.
4. Click Close.

Provide the information to Stemma

1. Navigate to the Admin tab of the Stemma UI and choose Connections and Add New Connection
2. In the pop-up window, choose Google Cloud as the Connection type, provide a Connection Name, and paste in the Service Account Key from the JSON file you have just downloaded.
3. When you are finished, click Save Connection.