Custom Metric Syntax
In Aporia, custom metrics are defined using syntax that is similar to python's.
There are three building blocks which can be used in order to create a custom metric expression:
- Constants - a numeric value (e.g.
2
,0.5
, ..) - Functions - out of the builtin function collection you can find below (e.g.
sum
,count
, ...). All those functions return a numeric value. - Binary operation -
+
,-
,*
,/
,**
. Operands can be both constants or function calls.
Before we dive into each of the supported function, let's take a look on few examples for custom metric definitions.
// Average annual premium of those with a driving license
sum(column="annual_premium") / count()
// Mean predicted probability
mean(column="proba")
// Model revenue
5 * tp_count(column="will_buy_insurance") -2 * fp_count(column="will_buy_insurance")
// [email protected] per step
ndcg_at_k(column="p_views", k=4)
ndcg_at_k(column="p_add_to_cart", k=4)
ndcg_at_k(column="p_purchases", k=4)
// accuracy using custom threshold
accuracy(column="proba", type="numeric", threshold=0.2)
Parameters
- column: the name of the prediction field on which we want to apply the function
- type: the data type of the prediction field we chose. Can be: "numeric" or "boolean".
- threshold: probability threshold according to which we decide the if a class is positive. Required for numeric predictions
Parameters
- column: the name of the prediction field on which we want to apply the function
- type: the data type of the prediction field we chose. Can be: "numeric" or "boolean".
- threshold: probability threshold according to which we decide the if a class is positive. Required for numeric predictions
Parameters
- column: the name of the prediction field on which we want to apply the function
- type: the data type of the prediction field we chose. Can be: "numeric" or "boolean".
- threshold: probability threshold according to which we decide the if a class is positive. Required for numeric predictions
Parameters
- column: the name of the prediction field on which we want to apply the function
- type: the data type of the prediction field we chose. Can be: "numeric" or "boolean".
- threshold: probability threshold according to which we decide the if a class is positive. Required for numeric predictions
Parameters
- column: the name of the prediction field on which we want to apply the function
- type: the data type of the prediction field we chose. Can be: "numeric", "boolean" or "categorical"
- threshold: probability threshold according to which we decide the if a class is positive. Required for numeric predictions
- method: will define the average strategy to use. Can be: "macro", "micro" or "weighted". Required for categorical predictions
Parameters
- column: the name of the prediction field on which we want to apply the function
- type: the data type of the prediction field we chose. Can be: "numeric", "boolean" or "categorical"
- threshold: probability threshold according to which we decide the if a class is positive. Required for numeric predictions
- method: will define the average strategy to use. Can be: "macro", "micro" or "weighted". Required for categorical predictions.
Parameters
- column: the name of the prediction field on which we want to apply the function
- type: the data type of the prediction field we chose. Can be: "numeric", "boolean" or "categorical"
- threshold: probability threshold according to which we decide the if a class is positive. Required for numeric predictions
- method: will define the average strategy to use. Can be: "macro", "micro" or "weighted". Required for categorical predictions.
Parameters
- column: the name of the prediction field on which we want to apply the function
- type: the data type of the prediction field we chose. Can be: "numeric", "boolean" or "categorical"
- threshold: probability threshold according to which we decide the if a class is positive. Required for numeric predictions
- method: will define the average strategy to use. Can be: "macro", "micro" or "weighted". Required for categorical predictions.
Last modified 16d ago