New Values

Why monitor new values?

Monitoring new values of categorical features / raw inputs helps to locate and examine changes in the model's input. For example, setting the monitor for a feature named state will help us discover a new region for which the model asked to predict results.

Detection Methods

For this monitor type, you can select the following detection methods:

• Change In Percentage - Detects change in a configured percentage or number of new values between the values of the inspected data and the values reported in a time period before the data was collected.
• Compared To Segment - Detects change in a configured percentage or number of new values between the values of the inspected data and the values in a different data segment.
• Compared To Training - Detects change in a configured percentage or number of new values between the values of the inspected data and the values reported in the training set.

Configuration

Start from choosing the features / raw inputs you'd like to monitor. You can select as many as you want :-)

Note that the monitor configuration may vary between the detection method you choose.

Creating this monitor using the REST API

POST https://app.aporia.com/v1beta/monitors
{
    "name": "New region codes",
    "type": "new_values",
    "scheduling": "0 */4 * * *",
    "configuration": {
        "configuration": {
            "focal": {
                "source": "SERVING",
                "timePeriod": "1d"
            },
            "metric": {
                "type": "histogram"
            },
            "actions": [
                {
                    "type": "ALERT",
                    "schema": "v1",
                    "severity": "LOW",
                    "alertType": "new_values",
                    "description": "New values were detected in feature <b>'{field}'</b>. <br /> The values were observed in the <b>{model}</b> model, in version <b>{model_version}</b> for the <b>last {focal_time_period} ({focal_times})</b> <b>{focal_segment}</b> compared to the <b>last {baseline_time_period} ({baseline_times})</b>. <br /><br /> Based on defined limits, the values were expected to be <b>{value_thresholds}</b>, but values <b>{unexpected_values}</b> were received.",
                    "notification": [
                        {
                            "type": "EMAIL",
                            "emails": [
                                "dev@aporia.com"
                            ]
                        }
                    ]
                }
            ],
            "baseline": {
                "source": "SERVING",
                "skipPeriod": "1d",
                "timePeriod": "2w"
            },
            "logicEvaluations": [
                {
                    "name": "VALUES_RANGE",
                    "new_values_count_threshold": 0,
                    "new_values_ratio_threshold": 0.05
                }
            ]
        },
        "identification": {
            "models": {
                "id": "seed-0000-fhpy"
            },
            "segment": {
                "group": null
            },
            "raw_inputs": [
                "categorical_Region_Code"
            ],
            "environment": null
        }
    }
}