Comment on page
Custom Metric Syntax
In Aporia, custom metrics are defined using syntax that is similar to python's.
There are three building blocks which can be used in order to create a custom metric expression:
- Constants - a numeric value (e.g.
2
,0.5
, ..) - Functions - out of the builtin function collection you can find below (e.g.
sum
,count
, ...). All those functions return a numeric value. - Binary operation -
+
,-
,*
,/
,**
. Operands can be both constants or function calls.
Before we dive into each of the supported functions, let's take a look at a few examples of custom metric definitions.
// Average annual premium of those with a driving license
sum(column="annual_premium") / count()
// Mean predicted probability
mean(column="proba")
// Model revenue
5 * tp_count(column="will_buy_insurance") -2 * fp_count(column="will_buy_insurance")
// nDCG@4 per step
ndcg_at_k(column="p_views", k=4)
ndcg_at_k(column="p_add_to_cart", k=4)
ndcg_at_k(column="p_purchases", k=4)
// accuracy using custom threshold
accuracy(column="proba", type="numeric", threshold=0.2)
Within Aporia we can always set a segment on our metrics as a whole, but sometimes this is just not enough. Many times we will need to pass a segment of our data to a specific function as part of our metric.
Aporia supports these cases by passing another argument to functions called "filter".
With the "filter" argument you'll be able to set any filtering to the data passed in the "column" argument using the custom segment syntax.
For example:
// Ratio of the annual premium of people above 70 out of the total premium
sum(column="annual_premium", filter="age > 70") / sum(column="annual_premium")
To allow you to set any of your segments upon these metrics as a whole as well, setting a filter within a metric will create behind the scenes, the intersection of the segment within the filter with all of your existing filters. These segments will be counted as any regular segment.
Parameters
- column: the name of the prediction field on which we want to apply the function
- type: the data type of the prediction field we chose. Can be: "numeric" or "boolean".
- threshold: probability threshold according to which we decide the if a class is positive. Required for numeric predictions
Parameters
- column: the name of the prediction field on which we want to apply the function
- type: the data type of the prediction field we chose. Can be: "numeric" or "boolean".
- threshold: probability threshold according to which we decide the if a class is positive. Required for numeric predictions
Parameters
- column: the name of the prediction field on which we want to apply the function
- type: the data type of the prediction field we chose. Can be: "numeric" or "boolean".
- threshold: probability threshold according to which we decide the if a class is positive. Required for numeric predictions
Parameters
- column: the name of the prediction field on which we want to apply the function
- type: the data type of the prediction field we chose. Can be: "numeric" or "boolean".
- threshold: probability threshold according to which we decide the if a class is positive. Required for numeric predictions
Parameters
- column: the name of the prediction field on which we want to apply the function
- type: the data type of the prediction field we chose. Can be: "numeric", "boolean" or "categorical"
- threshold: probability threshold according to which we decide the if a class is positive. Required for numeric predictions
- method: will define the average strategy to use. Can be: "macro", "micro" or "weighted". Required for categorical predictions
Parameters
- column: the name of the prediction field on which we want to apply the function
- type: the data type of the prediction field we chose. Can be: "numeric", "boolean" or "categorical"
- threshold: probability threshold according to which we decide the if a class is positive. Required for numeric predictions
- method: will define the average strategy to use. Can be: "macro", "micro" or "weighted". Required for categorical predictions.
Parameters
- column: the name of the prediction field on which we want to apply the function
- type: the data type of the prediction field we chose. Can be: "numeric", "boolean" or "categorical"
- threshold: probability threshold according to which we decide the if a class is positive. Required for numeric predictions
- method: will define the average strategy to use. Can be: "macro", "micro" or "weighted". Required for categorical predictions.
Parameters
- column: the name of the prediction field on which we want to apply the function
- type: the data type of the prediction field we chose. Can be: "numeric", "boolean" or "categorical"
- threshold: probability threshold according to which we decide the if a class is positive. Required for numeric predictions
- method: will define the average strategy to use. Can be: "macro", "micro" or "weighted". Required for categorical predictions.
Last modified 6mo ago