nicetoolbox.evaluation.runner.DatasetRunner¶

class nicetoolbox.evaluation.runner.DatasetRunner(io_manager: IO, run_config: DetectorsRunConfig, dataset_properties: DatasetConfig, evaluation_config: FinalEvaluationConfig)[source]¶

Bases: object

Runner for evaluating a single dataset with given configurations.

Initialize the DatasetRunner.

Parameters:

io_manager (IO) – IO manager for file operations.
run_config (RunConfig) – Configuration for the current run.
dataset_properties (DatasetProperties) – Properties of the dataset.
evaluation_config (EvaluationConfig) – Current evaluation task config.

Methods

run

Execute the evaluation process for the dataset.

run() → None[source]¶

Execute the evaluation process for the dataset.

This includes: 1. Initializing the dataset in the IO manager. 2. Discovering work items using the DiscoveryEngine. 3. Setting up the EvaluationDataset and DataLoader. 4. Running the MetricRunner to compute metrics. 5. Saving the evaluation results using the IO manager.