Querying metrics

To query metrics from Aporia, initialize a new client and call the query_metrics API:

from datetime import datetime, timedelta
from aporia import (
    Aporia,
    MetricDataset,
    MetricParameters,
    TimeRange,
)
from aporia.sdk.datasets import DatasetType

aporia_token = os.environ["APORIA_TOKEN"]
aporia_account = os.environ["APORIA_ACCOUNT"]
aporia_workspace = os.environ["APORIA_WORKSPACE"]

aporia_client = Aporia(
    base_url="https://platform.aporia.com",  # or "https://platform-eu.aporia.com"
    token=aporia_token,
    account_name=aporia_account,
    workspace_name=aporia_workspace,
)

last_week_dataset = MetricDataset(
    dataset_type=DatasetType.SERVING,
    time_range=TimeRange(
        start=datetime.datetime.now() - datetime.timedelta(days=7),
        end=datetime.datetime.now(),
    ),
)

res = metrics.query_batch(
    model_id=model_id,
    metrics=[
        MetricParameters(
            dataset=last_week_dataset,
            name="count",
        ),
    ],
)

print(f"The model had {metrics[0]} predictions last week")

Parameters

The query_metrics API has the following parameters:

Parameter	Type	Description
model_id	str	Model ID to query metrics for.
metrics	List[MetricParameters]	List of metrics to query.

The API can request values for multiple metrics concurrently.

MetricParameters

Here are different fields for the MetricParameters object:

Field	Type	Description
name	str	Metric name (see Supported functions). Required.
dataset	MetricDataset	Specifies what data to query (training / serving), what segment, and what timeframe. Required.
column	str	Name of the column to calculate the metric for. Required except for the count metric. For performance metrics, this should be the name of the prediction, not the actual.
k	int	K value for ranking metrics such as nDCG. Required only for ndcg_at_k, map_at_k, mrr_at_k, accuracy_at_k, precision_at_k, and recall_at_k.
threshold	float	Threshold to use when calculating binary performance metrics. Required only if the prediction is numeric and the actual is boolean, and the metric is a binary performance metric such as accuracy, recall, precision, f1_score, etc.
custom_metric_id	str	Custom metric ID. Required only if you want to query a custom metric.
baseline	MetricDataset	Specifies what data to use as baseline. Required only for statistical distances such as js_distance, ks_distance, psi, and hellinger_distance.

MetricDataset

The MetricDataset object contains the following fields:

Field	Type	Description
dataset_type	DatasetType	Can be either DatasetType.SERVING or DatasetType.TRAINING. Required.
time_range	TimeRange	Time range (contains start and end fields). Do not pass this for training.
model_version	str	Model version to filter by. Optional.
segment	MetricSegment	Used to query metrics in a specific data segment. Contains id and value fields.