aggregating_kpis

Description

Implementation of KPIs that cannot be computed on a batch level. They aggregate the result over all batches and can return an aggregated result, if requested.

@author: Christian Wirth

Classes

Accuracy	Binary accuracy.
AggregatingKpi	Abstract base class for all aggregating kpis, meaning they can not be computed on batch level, but need to be aggregated over batches.
ApproximateMean	Approximate the mean in the data by binning the values.
ApproximateStdDev	Approximate the standard deviation in the data by binning the values.
ApproximateVariance	Approximate the variance in the data by binning the values.
CalibrationCurve	Calibration Curve: Shows the difference between true positive rate and the average predicted confidence over n_bins.
ConfusionMatrix	Abstract base class for all confusion matrix based metrics.
ECE	Expected calibration error.
Extremum	Find the extreme value of all output tensor entries over all batches (ignoring labels).
F1Score	Binary F1-score.
Histogram	Collect a histogram of the model output values.
Maximum	Find the max value within all batches.
Mean	Calculate the mean of all output tensor entries (ignoring labels).
Minimum	Find the max value within all batches.
NegativePredictiveValue	Binary negative predictive value.
Precision	Binary precision.
PrecisionRecallCurve	Approximated precision / recall curve.
Recall	Binary recall (resp.
SetIoU	Calc set intersection over union (IoU) value for a batch of outputs.
SetIoUThresholdCurve	Create a curve showing the setIoU value for multiple thresholds.
Specificity	Binary specificity (resp.
StandardDev	Calculate the standard deviation of all output tensor entries (ignoring labels).
Variance	Calculate the variance of all output tensor entries (ignoring labels).

Functions

filter_aggregating_kpi_keys(metric_fns)	Given named metric functions return the keys of such behaving like an AggregatingKpi.
save_division(numer, denom[, div_by_zero_val])	Yield sensible values in case of division by zero.