# Glue Data Catalog

This guide describes how to use the Glue Data Catalog data source in order to monitor your ML Model in production. 

We will assume that your model inputs, outputs and optionally delayed actuals can be found to exist as tables in Glue Data Catalog. This data source may also be used to connect to your model's training set to be used as a baseline for model monitoring.

### Update the Aporia IAM role for Glue Data Catalog access

In order to provide access to Glue Data Catalog, you'll need to update your Aporia IAM role with the necessary API permissions.

#### Step 1: Obtain your aporia IAM role

Use the same role used for the Aporia deployment. If someone else on your team has deployed Aporia, please reach out to them to obtain the role ARN (it should be in the following format: `arn:aws:iam::<account>:role/<role-name-with-path>`).

#### Step 2: Create an access policy

1. In the list of roles, click the role you obtained.
2. Add an inline policy.
3. On the Permissions tab, click **Add permissions** then click **Create inline policy**.&#x20;

   <figure><img src="/files/K7HH6EFbp6rBFs6RJQsk" alt=""><figcaption></figcaption></figure>
4. In the policy editor, click the **JSON** tab.\
   &#x20;

   <figure><img src="/files/mQY5RUpiPIn52OdmQ79c" alt=""><figcaption></figcaption></figure>
5. Copy the following access policy, and make sure to fill your correct region, account ID and restrict access to specific databases and tables if necessary.

   ```json
   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Effect": "Allow",
               "Action": [
                   "glue:GetConnections"
               ],
               "Resource": [
                   "arn:aws:glue:<REGION>:<ACCOUNT_ID>:catalog",
                   "arn:aws:glue:<REGION>:<ACCOUNT_ID>:connection/*"
               ]
           },
           {
               "Effect": "Allow",
               "Action": [
                   "glue:GetDatabase",
                   "glue:GetDatabases"
               ],
               "Resource": [
                   "arn:aws:glue:<REGION>:<ACCOUNT_ID>:catalog",
                   "arn:aws:glue:<REGION>:<ACCOUNT_ID>:database/default",
                   "arn:aws:glue:<REGION>:<ACCOUNT_ID>:database/global_temp",
                   "arn:aws:glue:<REGION>:<ACCOUNT_ID>:database/*"
               ]
           },
           {
               "Effect": "Allow",
               "Action": [
                   "glue:GetTable",
                   "glue:GetTables",
                   "glue:GetPartitions",
                   "glue:GetPartition",
                   "glue:SearchTables"
               ],
               "Resource": [
                   "arn:aws:glue:<REGION>:<ACCOUNT_ID>:catalog",
                   "arn:aws:glue:<REGION>:<ACCOUNT_ID>:database/*",
                   "arn:aws:glue:<REGION>:<ACCOUNT_ID>:table/*"
               ]
           },
           {
               "Effect": "Allow",
               "Action": [
                   "glue:GetUserDefinedFunctions"
               ],
               "Resource": [
                   "*"
               ]
           },
           {
               "Effect": "Allow",
               "Action": [
                   "glue:CreateDatabase"
               ],
               "Resource": [
                   "arn:aws:glue:<REGION>:<ACCOUNT_ID>:catalog",
                   "arn:aws:glue:<REGION>:<ACCOUNT_ID>:database/default",
                   "arn:aws:glue:<REGION>:<ACCOUNT_ID>:database/global_temp"
               ]
           }
       ]
   }
   ```
6. Click **Review Policy**.
7. In the **Name** field, enter a policy name.
8. Click **Create policy**.

Now Aporia has the read permission it needs to connect to the Glue Data Catalog databases and tables you have specified in the policy.

### Create a Glue Data Catalog data source in Aporia

1. Go to [Aporia platform](https://platform.aporia.com/) and login to your account.
2. Go to **Integrations** page and click on the **Data Connectors** tab
3. Scroll to **Connect New Data Source** section
4. Click **Connect** on the Glue Data Catalog card and follow the instructions

Bravo! :clap: now you can use the data source you've created across all your models in Aporia.

### Extracting features from JSON

A common use-case is storing serving data in JSONs on S3 files.

The following is a sample query of how to extract the JSON data to Aporia features:

```
WITH model_data AS (
    SELECT
        prediction_id,
        prediction_timestamp,
        model_version,
        proba,
        actual,
        FROM_JSON(
            features_json,
            "features STRUCT<age FLOAT, state STRING, is_single BOOLEAN>"
        ) AS parsed_json
    FROM
        models_store.test_model
)
SELECT
    prediction_id,
    prediction_timestamp,
    model_version,
    proba,
    actual,
    parsed_json.features.age as age,
    parsed_json.features.state as state,
    parsed_json.features.is_single as is_single
FROM
    model_data
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.aporia.com/data-sources/glue-data-catalog.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.