nicetoolbox.evaluation.metrics.evaluate.MetricRunner

class nicetoolbox.evaluation.metrics.evaluate.MetricRunner(loader, eval_cfg: dict)[source]

Bases: object

Drives all metrics: initialize, process samples, evaluate, collect results.

Initializes the metric runner with a data loader and evaluation configuration. Calls the MetricFactory to create all metric handlers.

Parameters:

  • loader – DataLoader that yields batches of data.

  • eval_cfg (dict) – Configuration dictionary for evaluation, including device and metric settings.

Methods

evaluate

Runs the full evaluation process: dispatches batches to metric handlers for processing, computes final metric results, and formats results.
evaluate() → EvalResults[source]

Runs the full evaluation process: dispatches batches to metric handlers for processing, computes final metric results, and formats results.

Returns:

The final structured evaluation results.

Return type:

EvalResults