nicetoolbox.evaluation.data.dataset

Dataset class for evaluation, implemented as an IterableDataset.

Classes

EvaluationDataset

Initialize the EvaluationDataset with found work items, the prediction loader, and optionally the annotation loader.
class nicetoolbox.evaluation.data.dataset.EvaluationDataset(*args: Any, **kwargs: Any)[source]

Initialize the EvaluationDataset with found work items, the prediction loader, and optionally the annotation loader.

Parameters:

  • work_items (List[ChunkWorkItem]) – List of work items (chunks) to process.

  • prediction_loader (PredictionLoader) – Loader for prediction data.

  • annotation_loader (Optional[AnnotationLoader]) – Loader for annotation data.

static collate_fn(batch: List[tuple]) → Dict[str, Any][source]

Custom collate function given to the torch DataLoader to batch the data from the iterable EvaluationDataset.

This function groups the samples by their chunk’s metric type, prediction shape, component, algorithm, and prediction data key. This ensures that each batch contains homogeneous data, allowing for easier handling at metric processing time.

Parameters:

batch (List[tuple]) – A list of tuples (pred_data, gt_data, metadata).

Returns:

A dictionary containing for each group:

  • ’pred’: Batched prediction data as a torch tensor.

  • ’gt’: Batched ground truth data as a torch tensor or None.

  • ’metadata’: A list of metadata dictionaries from each work item.

Return type:

Dict[str, Dict[str, Any]]