nicetoolbox.evaluation.data.discovery

Discovery engine to find all work items for evaluation on disk.

Classes

ChunkWorkItem	Represents a chunk of frames, typically from a single data array within a prediction file.
DiscoveryEngine	Scans file structures and configs to generate a flat list of all WorkItems.
FrameInfo	Holds the per-frame information within a chunk.

class nicetoolbox.evaluation.data.discovery.ChunkWorkItem(dataset_name: str, session: str, sequence: str, component: str, algorithm: str, metric_type: str, pred_path: ~pathlib.Path, pred_data_key: str, annot_path: ~pathlib.Path | None, annot_data_key: str | None, pred_data_description_axis3: ~typing.List[str], pred_reconciliation_map: ~typing.Dict[str, ~typing.Tuple[int, ...]] = <factory>, gt_reconciliation_map: ~typing.Dict[str, ~typing.Tuple[int, ...]] = <factory>, frames: ~typing.List[~nicetoolbox.evaluation.data.discovery.FrameInfo] = <factory>)

Represents a chunk of frames, typically from a single data array within a prediction file. This is the atomic unit of work for the Dataset iterator.

class nicetoolbox.evaluation.data.discovery.DiscoveryEngine(io_manager: IO, run_config: DetectorsRunConfig, dataset_properties: DatasetConfig, evaluation_config: FinalEvaluationConfig)

Scans file structures and configs to generate a flat list of all WorkItems. Crucially, it reconciles the per-clip prediction structure with the dataset-wide annotation structure.

discover_work_items() → List[ChunkWorkItem]

Discover all chunk work items based on the run configuration, dataset properties, and evaluation configuration. This method scans the file structure for predictions and creates reconciliation plans for the ground truth annotations if required, i.e. when pred and GT labels like joints are not aligned.

Returns:

A list of all discovered chunk work items.

Return type:

List[ChunkWorkItem]

class nicetoolbox.evaluation.data.discovery.FrameInfo(frame: int, person: str, camera: str, pred_slicing_indices: Tuple[int, int, int], annot_slicing_indices: Tuple[int, int, int] | None = None)

Holds the per-frame information within a chunk.