New Values
Why monitor new values?
Monitoring new values of categorical features / raw inputs helps to locate and examine changes in the model's input.
For example, setting the monitor for a feature named state
will help us discover a new region for which the model asked to predict results.
Detection Methods
For this monitor type, you can select the following detection methods:
- Change In Percentage - Detects change in a configured percentage or number of new values between the values of the inspected data and the values reported in a time period before the data was collected.
- Compared To Segment - Detects change in a configured percentage or number of new values between the values of the inspected data and the values in a different data segment.
- Compared To Training - Detects change in a configured percentage or number of new values between the values of the inspected data and the values reported in the training set.
Configuration
Start from choosing the features / raw inputs you'd like to monitor. You can select as many as you want :-)
Note that the monitor configuration may vary between the detection method you choose.
Creating this monitor using the REST API
POST https://app.aporia.com/v1beta/monitors
{
"name": "New region codes",
"type": "new_values",
"scheduling": "0 */4 * * *",
"configuration": {
"configuration": {
"focal": {
"source": "SERVING",
"timePeriod": "1d"
},
"metric": {
"type": "histogram"
},
"actions": [
{
"type": "ALERT",
"schema": "v1",
"severity": "LOW",
"alertType": "new_values",
"description": "New values were detected in feature <b>'{field}'</b>. <br /> The values were observed in the <b>{model}</b> model, in version <b>{model_version}</b> for the <b>last {focal_time_period} ({focal_times})</b> <b>{focal_segment}</b> compared to the <b>last {baseline_time_period} ({baseline_times})</b>. <br /><br /> Based on defined limits, the values were expected to be <b>{value_thresholds}</b>, but values <b>{unexpected_values}</b> were received.",
"notification": [
{
"type": "EMAIL",
"emails": [
"dev@aporia.com"
]
}
]
}
],
"baseline": {
"source": "SERVING",
"skipPeriod": "1d",
"timePeriod": "2w"
},
"logicEvaluations": [
{
"name": "VALUES_RANGE",
"new_values_count_threshold": 0,
"new_values_ratio_threshold": 0.05
}
]
},
"identification": {
"models": {
"id": "seed-0000-fhpy"
},
"segment": {
"group": null
},
"raw_inputs": [
"categorical_Region_Code"
],
"environment": null
}
}
}