V2
Get StartedBook a Demo🚀 Cool StuffBlog
Search
⌃K
Links
Comment on page

BigQuery

This guide describes how to connect Aporia to a BigQuery data source in order to monitor your ML Model in production.
We will assume that your model inputs, outputs and optionally delayed actuals can be queried with SQL. This data source may also be used to connect to your model's training set to be used as a baseline for model monitoring.

Create a materialization dataset for Aporia queries

Create a materialization dataset for Aporia to use to perform queries, see instructions here.
A separate materialization dataset location, to which query results will be written, must be designated for each project from which you want to query.

Update the Aporia Service Account for BigQuery access

In order to provide access to BigQuery, you'll need to update your Aporia service account with the necessary API permissions.

Step 1: Obtain your aporia service account

Use the same service account used for the Aporia deployment. If someone else on your team has deployed Aporia, please reach out to them to obtain it.

Step 2: Grant read access to the relevant project

  1. 1.
    Go to the IAM console and login.
  2. 2.
    Find the Aporia service account you obtain in the previous step and click on 🖋 Edit Principle
  3. 3.
    In the "Edit access" window click on ADD ANOTHER ROLE
  4. 4.
    Add the BigQuery Data Viewer and BigQuery Job User roles and click Save

Step 3: Grant access to the materialization dataset

  1. 1.
    Go to the BigQuery console and login.
  2. 2.
    In the left-hand panel, expand the relevant project and find the materialization dataset you created in the previous steps.
  3. 3.
    Click on "..." by the dataset name, then click on Share
  4. 4.
    In the "Share permissions" window click on Add Principal
  5. 5.
    In the "New principal" box, enter the email of the Aporia service account you have obtained. Choose the BigQuery Data Editor role and click Save.
Now Aporia has the permission it needs to connect to the BigQuery datasets and tables you have specified in the policy.

Create a BigQuery data source in Aporia

  1. 1.
    Go to Aporia platform and login to your account.
  2. 2.
    Go to Integrations page and click on the Data Connectors tab
  3. 3.
    Scroll to Connect New Data Source section
  4. 4.
    Click Connect on the BigQuery card and follow the instructions
Bravo!
👏
 now you can use the data source you've created across all your models in Aporia.
​
Data Sources - Previous
Athena
Next - Data Sources
Databricks
Last modified 6mo ago