nicetoolbox.evaluation.metrics.base_metric

Base classes for metrics and metric handlers in the evaluation module. Metric factory to instantiate handlers based on config.

Classes

Metric	Abstract base class for a single, stateful metric computation.
MetricFactory	Static class that instantiates metric handlers based on the evaluation config.
MetricHandler	Abstract base class for a metric handler (e.g., PointCloudMetric).

class nicetoolbox.evaluation.metrics.base_metric.Metric(**kwargs: Any)

Abstract base class for a single, stateful metric computation.

Initializes the metric. Subclasses can use kwargs for specific setup.

abstract compute() → Dict[Tuple[str, str, str], List[BatchResult] | AggregatedResult]

Compute the final metric from the stored state and return as a dictionary.

abstract get_axis3(meta_chunks: List[ChunkWorkItem] | None = None) → List[str]

Get the metrics output description. E.g. bone names or joint names.

abstract reset() → None

Reset the metric’s internal state to prepare for a new run.

abstract update(preds: torch.Tensor, gts: torch.Tensor, meta_chunks: List[ChunkWorkItem], meta_frames: List[FrameInfo]) → None

Update metric state with a batch of data and its corresponding metadata.

class nicetoolbox.evaluation.metrics.base_metric.MetricFactory

Static class that instantiates metric handlers based on the evaluation config.

static create_all(ev_cfg: FinalEvaluationConfig, device: str) → List[MetricHandler]

Create all metric handlers based on the evaluation configuration.

Parameters:

• ev_cfg (EvaluationConfig) – The evaluation config listing metric types.

• device (str) – The device to run the metrics on (e.g., ‘cpu’ or ‘cuda’).

Returns:

A list of metric handlers with instantiated metrics.

Return type:

List[MetricHandler]

class nicetoolbox.evaluation.metrics.base_metric.MetricHandler(cfg: EvaluationMetricType, device: str)

Abstract base class for a metric handler (e.g., PointCloudMetric). A handler creates and manages one or more base Metric instances.

Initialize the metric handler with its config and device, creating its metrics.

Parameters:

• cfg (MetricTypeConfig) – Configuration for this metric type.

• device (str) – Device to run the metrics on (e.g., ‘cpu’ or ‘cuda’).

evaluate() → Dict[Tuple[str, str, str], List[BatchResult] | AggregatedResult]

Compute final results from all managed metrics.

abstract property name: str

The name of the metric handler (e.g., ‘point_cloud_metrics’).

process(batch: Dict[str, Any]) → None

Process a homogeneous, grouped batch of data for all managed metrics.