Missing Values

Why monitor missing values?

In real world data, there are often cases where particular data element is missing. It is important to monitor the missing values changes to spot and handle cases in which the model has not been trained to deal with.

Causes of missing values include:

• Serving environment fault
• Data store / provider schema changes
• Changes in internal API
• Changes in model subject input

Detection Methods

For this monitor type, you can select the following detection methods:

• Absolute Values - The missing values ratio is lower or higher than a specific value.
• Anomaly Detection - Detects anomalies in the missing values count of the inspected data, compared to the missing values count in a time period before the data was collected.
• Change In Percentage - Detects change in the ratio between the missing values count of the inspected data and the missing values count in a time period before the data was collected.
• Compared To Segment - Detects change in the ratio between the missing values count of the inspected data and the missing values count in a different data segment.

Configuration

Start from choosing the features / raw inputs you'd like to monitor. You can select as many as you want :-)

Note that the monitor configuration may vary between the detection method you choose.

Creating this monitor using the REST API

POST https://app.aporia.com/v1beta/monitors
{
    "name": "Policy_Sales_Channel missing values",
    "type": "missing_values",
    "scheduling": "0 */4 * * *",
    "configuration": {
        "configuration": {
            "focal": {
                "source": "SERVING",
                "timePeriod": "1d"
            },
            "metric": {
                "type": "missing_ratio"
            },
            "actions": [
                {
                    "type": "ALERT",
                    "schema": "v1",
                    "severity": "LOW",
                    "alertType": "feature_missing_values_threshold",
                    "description": "An anomaly in the ratio of missing values of feature <b>'{field}'</b> within the defined limits was detected.<br /> The anomaly was observed in the <b>{model}</b> model, in version <b>{model_version}</b> for the <b>last {focal_time_period} ({focal_times})</b> <b>{focal_segment}</b>. <br /><br /> Based on defined limits, the missing ratio was expected to be below <b>{max_threshold}</b>, but <b>{focal_value}</b> was received.<br /><br /> Missing data can have a significant effect on model behavior and may lead to unexpected results. <br /><br /> Missing data might occur because: <ul><li>Serving environment fault</li><li>Data store / provider schema changes</li><li>Changes in internal API</li><li>Changes in model subject input</li></ul>",
                    "notification": [
                        {
                            "type": "EMAIL",
                            "emails": [
                                "dev@aporia.com"
                            ]
                        }
                    ],
                    "visualization": "value_over_time"
                }
            ],
            "logicEvaluations": [
                {
                    "max": 0.2,
                    "min": null,
                    "name": "RANGE"
                }
            ]
        },
        "identification": {
            "models": {
                "id": "seed-0000-fhpy"
            },
            "segment": {
                "group": null
            },
            "raw_inputs": [
                "numeric_Policy_Sales_Channel"
            ],
            "environment": null
        }
    }
}