NICE Toolbox
  • Project Overview
  • Changelog
    • 0.3.1
    • 0.3.0
    • 0.2.2
    • 0.2.1
    • 0.2.0
    • 0.1.0
  • Installation
    • System Requirements
    • Docker
    • Prerequisites
      • Python 3.10
      • Conda
      • Cuda 11.8
      • FFmpeg
      • Git
      • On Windows: Microsoft Visual C++
      • On Windows: Make
      • On Windows: Disable path length limit
    • Clone the repository
    • Makefile installation
      • Installing only the detectors you need
    • Hugging Face Access Token
    • Additional notes
  • Getting started
    • 1. Machine-specific and project configs
    • 2. Example dataset
    • 3. Check the dataset’s properties
    • 4. Add an experiment to run
    • 5. Run the NICE Toolbox
    • 6. Visualize the results
    • 7. Run the NICE Toolbox Evaluation
    • 8. Import annotations with NICE Connectors
  • Tutorials
    • Include a dataset with a single camera view
      • 1. Create your config files
      • 2. Prepare the dataset
        • Folder structure
        • Dataset properties
        • Example
      • 3. Create a calibration file
        • Calibration toml file
        • Toml to npz file
      • 4. Define the experiment to run
      • 5. Run the toolbox
    • Include a dataset with multiple camera views
      • 1. Create your config files
      • 2. Prepare the multi-view dataset
        • Update the dataset properties
        • Example
      • 3. Create a multi-view calibration file
      • 4. Define the experiments and run the toolbox
    • Calibration conversion
    • Visualizations in rerun
      • Starting rerun
      • Visualizer config
      • Configuring component data display in rerun windows
        • Configuring algorithm display
        • Configuring appearance
      • Configuring rerun viewer and blueprint in rerun
    • Evaluate Detector Outputs
      • 1. Prerequisites
      • 2. Configure the Evaluation
        • Top-level settings
        • Defining metric instances
        • Example: no ground truth
        • Example: with ground truth annotations
        • Grouping and aggregation
      • 3. Run the Evaluation
      • 4. Read the Results
    • ELAN Connector: Import Gaze Annotations
      • 1. What is ELAN?
      • 2. Prerequisites
      • 3. Prepare and export your annotations
        • Open the example project
        • Tier naming and labels
        • Export as tab-delimited text
      • 4. Configure the connector
        • Sequence settings
        • Subject mapping
      • 5. Run the connector
      • 6. Use the output for evaluation
    • Napari-DeepLabCut Connector: Export and Import Body Joints
      • 1. What is napari-DeepLabCut?
      • 2. Prerequisites
      • 3. Export to napari
        • Sliding window
        • Run the export
      • 4. Load the data in napari
      • 5. Import annotations to NICE Toolbox
        • Camera and subject mapping
        • Run the import
        • Use the result for evaluation
  • Wiki
    • Understanding the config files
      • Placeholders
      • Machine specifics
      • Project config
      • Run file
        • General properties
        • Choosing algorithms to run
        • Defining the experiments
        • Input and output files
      • Dataset properties
        • Minimal shape
        • Filesystem discovery
        • Combining discovery with explicit overrides
      • Detectors config
        • Templates
      • Predictions mapping
      • Visualizer Config
        • Configuring Component Data Display in Rerun Windows
        • Configuring Rerun Viewer and Blueprint in Rerun
    • Understanding the Calibration files
    • Components
      • Output files
        • Numpy arrays (Video based components output format)
        • Json files (Audio based components output format)
        • Data description
        • Python code
      • Body joints
      • Body joints local
      • Body mesh
      • Hand joints
      • Face bounding box
      • Face landmarks
      • Gaze Individual
      • Gaze Multiview
        • Closed Eye Filtering and Interpolation
      • Gaze Interaction
      • Kinematics
      • Proximity
      • Emotion Detection
        • Detector Configuration
      • Head Orientation
      • Eye Closure
        • Eye Aspect Ratio (EAR)
      • Audio Transcription
      • Audio Diarization
      • Speaker Aligned Transcription
    • Algorithms
      • HRNet, ViTPose, & RTMPose (2D Pose Estimation)
        • High-Resolution Networks
        • Vision Transformers
        • Real-Time Pose Estimation
      • MotionBERT (3D body pose lifting)
      • SAM 3D Body
      • ETH-XGaze (Gaze Estimation)
      • UniGaze (Gaze Estimation)
      • Gaze Fusion
      • Gaze Interaction
      • Kinematics
      • Proximity
      • InsightFace (Face Detection)
      • Py-FEAT (Facial Expression Analysis Toolbox)
      • SPIGA (Shape Preserving Facial Landmarks with Graph Attention)
      • Eye Closure
      • WhisperX (Audio Transcription & Speaker Diarization)
      • CrisperWhisper
    • Evaluation Metrics
      • Available Metrics
        • Bone Length
        • Missing Points
        • PCK — Percentage of Correct Keypoints
        • Distance Error
        • Confusion Matrix
        • ROC AUC
        • PR Curve — Average Precision
      • Aggregation
        • Aggregation Functions
    • Connectors
      • ELAN
        • Import gaze
        • Export transcription
        • Import transcription
      • Napari
        • Export body joints
        • Import body joints
    • How to manually label body joints
      • Understanding models and metrics limitation
      • What a key point means
      • Estimating the joint centre using heuristics
      • Correcting existing key points
      • Occlusions
      • A proposal for a practical labeling routine
  • API
    • nicetoolbox.configs
      • nicetoolbox.configs.config_loader
        • nicetoolbox.configs.config_loader.ConfigLoader
        • ConfigLoader
      • nicetoolbox.configs.named_wildcards
        • nicetoolbox.configs.named_wildcards.discover_sequences_for_dataset
        • nicetoolbox.configs.named_wildcards.discovery_to_regex
        • discover_sequences_for_dataset()
        • discovery_to_regex()
      • nicetoolbox.configs.placeholders
        • nicetoolbox.configs.placeholders.get_placeholders
        • nicetoolbox.configs.placeholders.get_placeholders_str
        • nicetoolbox.configs.placeholders.resolve_placeholders
        • nicetoolbox.configs.placeholders.resolve_placeholders_dict
        • nicetoolbox.configs.placeholders.resolve_placeholders_dict_mut
        • nicetoolbox.configs.placeholders.resolve_placeholders_str
        • nicetoolbox.configs.placeholders.resolve_placeholders_str_strict
        • get_placeholders()
        • get_placeholders_str()
        • resolve_placeholders()
        • resolve_placeholders_dict()
        • resolve_placeholders_dict_mut()
        • resolve_placeholders_str()
        • resolve_placeholders_str_strict()
      • nicetoolbox.configs.project_config_handler
        • nicetoolbox.configs.project_config_handler.ProjectConfigHandler
        • ProjectConfigHandler
      • nicetoolbox.configs.utils
        • nicetoolbox.configs.utils.default_auto_placeholders
        • nicetoolbox.configs.utils.default_runtime_placeholders
        • nicetoolbox.configs.utils.dict_to_model
        • nicetoolbox.configs.utils.get_latest_experiment_config_path
        • nicetoolbox.configs.utils.keys_collision_dict
        • nicetoolbox.configs.utils.load_raw_config
        • nicetoolbox.configs.utils.merge_dicts
        • nicetoolbox.configs.utils.model_to_dict
        • nicetoolbox.configs.utils.resolve_filter
        • nicetoolbox.configs.utils.ConfigValidationError
        • ConfigValidationError
        • dict_to_model()
        • keys_collision_dict()
        • load_raw_config()
        • merge_dicts()
        • model_to_dict()
        • resolve_filter()
    • nicetoolbox.connectors
      • nicetoolbox.connectors.config_handler
        • nicetoolbox.connectors.config_handler.ConnectorConfigHandler
        • ConnectorConfigHandler
      • nicetoolbox.connectors.elan
        • nicetoolbox.connectors.elan.cli
        • nicetoolbox.connectors.elan.elan_configs
        • nicetoolbox.connectors.elan.elan_data
        • nicetoolbox.connectors.elan.elan_parser
        • nicetoolbox.connectors.elan.elan_processing
        • nicetoolbox.connectors.elan.elan_time
        • nicetoolbox.connectors.elan.elan_writer
        • nicetoolbox.connectors.elan.gaze_parser
        • nicetoolbox.connectors.elan.labeling_data
        • nicetoolbox.connectors.elan.labeling_from_elan
        • nicetoolbox.connectors.elan.npz_schema
        • nicetoolbox.connectors.elan.tasks
        • nicetoolbox.connectors.elan.toolbox_writer
        • nicetoolbox.connectors.elan.transcript_elan
      • nicetoolbox.connectors.napari
        • nicetoolbox.connectors.napari.cli
        • nicetoolbox.connectors.napari.napari_configs
        • nicetoolbox.connectors.napari.parser
        • nicetoolbox.connectors.napari.tasks
        • nicetoolbox.connectors.napari.writer
    • nicetoolbox.detectors
      • nicetoolbox.detectors.base_detector
        • nicetoolbox.detectors.base_detector.BaseDetector
        • BaseDetector
      • nicetoolbox.detectors.config_handler
        • nicetoolbox.detectors.config_handler.flatten_list
        • nicetoolbox.detectors.config_handler.Configuration
        • Configuration
      • nicetoolbox.detectors.data
        • nicetoolbox.detectors.data.SequenceData
        • SequenceData
      • nicetoolbox.detectors.data_handlers
        • nicetoolbox.detectors.data_handlers.audio_handler
        • nicetoolbox.detectors.data_handlers.handler
        • nicetoolbox.detectors.data_handlers.video_handler
      • nicetoolbox.detectors.detector_inputs
        • nicetoolbox.detectors.detector_inputs.load_detector_inputs
        • nicetoolbox.detectors.detector_inputs.BaseDetectorInput
        • nicetoolbox.detectors.detector_inputs.NpzDetectorInput
        • nicetoolbox.detectors.detector_inputs.ResolvedInput
        • BaseDetectorInput
        • NpzDetectorInput
        • ResolvedInput
        • load_detector_inputs()
      • nicetoolbox.detectors.detector_outputs
        • nicetoolbox.detectors.detector_outputs.BaseDetectorOutput
        • nicetoolbox.detectors.detector_outputs.DetectorOutput
        • nicetoolbox.detectors.detector_outputs.JsonDetectorOutput
        • nicetoolbox.detectors.detector_outputs.NpzDetectorOutput
        • BaseDetectorOutput
        • DetectorOutput
        • JsonDetectorOutput
        • NpzDetectorOutput
      • nicetoolbox.detectors.feature_detectors
        • nicetoolbox.detectors.feature_detectors.base_feature
        • nicetoolbox.detectors.feature_detectors.eye_closure_ear
        • nicetoolbox.detectors.feature_detectors.eye_closure_threshold
        • nicetoolbox.detectors.feature_detectors.gaze_interaction
        • nicetoolbox.detectors.feature_detectors.gaze_multiview
        • nicetoolbox.detectors.feature_detectors.kinematics
        • nicetoolbox.detectors.feature_detectors.proximity
      • nicetoolbox.detectors.in_out
        • nicetoolbox.detectors.in_out.SequenceIO
        • SequenceIO
      • nicetoolbox.detectors.main
        • nicetoolbox.detectors.main.entry_point
        • nicetoolbox.detectors.main.get_algo_components
        • nicetoolbox.detectors.main.main
        • entry_point()
        • get_algo_components()
        • main()
      • nicetoolbox.detectors.method_detectors
        • nicetoolbox.detectors.method_detectors.base_method
        • nicetoolbox.detectors.method_detectors.eth_xgaze
        • nicetoolbox.detectors.method_detectors.insight_face
        • nicetoolbox.detectors.method_detectors.mmpose
        • nicetoolbox.detectors.method_detectors.py_feat
        • nicetoolbox.detectors.method_detectors.sam_3d_body
        • nicetoolbox.detectors.method_detectors.spiga
        • nicetoolbox.detectors.method_detectors.unigaze
        • nicetoolbox.detectors.method_detectors.whisperx
      • nicetoolbox.detectors.subsequence_context
        • nicetoolbox.detectors.subsequence_context.SubsequenceContext
        • SubsequenceContext
      • nicetoolbox.detectors.utils
        • nicetoolbox.detectors.utils.draw_2d
        • nicetoolbox.detectors.utils.filters
        • nicetoolbox.detectors.utils.keypoints_post_processing
        • nicetoolbox.detectors.utils.keypoints_triangualation
        • nicetoolbox.detectors.utils.pose_utils
    • nicetoolbox.evaluation
      • nicetoolbox.evaluation.config_handler
        • nicetoolbox.evaluation.config_handler.ConfigHandler
        • ConfigHandler
      • nicetoolbox.evaluation.data
        • nicetoolbox.evaluation.data.input_loader
        • nicetoolbox.evaluation.data.plots
        • nicetoolbox.evaluation.data.results_saver
        • nicetoolbox.evaluation.data.summary
        • nicetoolbox.evaluation.data.transcript_loader
      • nicetoolbox.evaluation.main
        • nicetoolbox.evaluation.main.entry_point
        • nicetoolbox.evaluation.main.main
        • entry_point()
        • main()
      • nicetoolbox.evaluation.metrics
        • nicetoolbox.evaluation.metrics.audio
        • nicetoolbox.evaluation.metrics.base_metric
        • nicetoolbox.evaluation.metrics.categorical
        • nicetoolbox.evaluation.metrics.joints
        • nicetoolbox.evaluation.metrics.metric_result
    • nicetoolbox.utils
      • nicetoolbox.utils.base_detectors
        • nicetoolbox.utils.base_detectors.flatten_inference_config
        • nicetoolbox.utils.base_detectors.input_map_to_string_keys
        • nicetoolbox.utils.base_detectors.input_map_to_tuple_keys
        • nicetoolbox.utils.base_detectors.string_key_to_tuple
        • nicetoolbox.utils.base_detectors.tuple_key_to_string
        • flatten_inference_config()
        • input_map_to_string_keys()
        • input_map_to_tuple_keys()
        • string_key_to_tuple()
        • tuple_key_to_string()
      • nicetoolbox.utils.calibration_gui
        • nicetoolbox.utils.calibration_gui.calib_io
        • nicetoolbox.utils.calibration_gui.calibration_converter
        • nicetoolbox.utils.calibration_gui.constants
        • nicetoolbox.utils.calibration_gui.initialization
        • nicetoolbox.utils.calibration_gui.make_forms
        • nicetoolbox.utils.calibration_gui.matrix_utils
      • nicetoolbox.utils.check_and_exception
        • nicetoolbox.utils.check_and_exception.check_options
        • nicetoolbox.utils.check_and_exception.check_token_in_filepath
        • nicetoolbox.utils.check_and_exception.check_value_bounds
        • nicetoolbox.utils.check_and_exception.check_zeros
        • nicetoolbox.utils.check_and_exception.error_log_and_raise
        • nicetoolbox.utils.check_and_exception.file_exists
        • check_options()
        • check_token_in_filepath()
        • check_value_bounds()
        • check_zeros()
        • error_log_and_raise()
        • file_exists()
      • nicetoolbox.utils.comparison
        • nicetoolbox.utils.comparison.compare_npz_files
        • nicetoolbox.utils.comparison.DifferentKeysError
        • nicetoolbox.utils.comparison.DifferentShapeError
        • nicetoolbox.utils.comparison.DifferentValuesError
        • DifferentKeysError
        • DifferentShapeError
        • DifferentValuesError
      • nicetoolbox.utils.config
        • nicetoolbox.utils.config.default
        • nicetoolbox.utils.config.save_config
        • save_config()
      • nicetoolbox.utils.dependency_sort
        • nicetoolbox.utils.dependency_sort.sort_detectors_order
        • sort_detectors_order()
      • nicetoolbox.utils.error_handling
        • nicetoolbox.utils.error_handling.manage_error_scope
        • manage_error_scope()
      • nicetoolbox.utils.filehandling
        • nicetoolbox.utils.filehandling.find_json_files
        • nicetoolbox.utils.filehandling.find_npz_files
        • nicetoolbox.utils.filehandling.load_json_file
        • nicetoolbox.utils.filehandling.load_toml
        • nicetoolbox.utils.filehandling.read_npz_file
        • nicetoolbox.utils.filehandling.resolve_single_file
        • find_json_files()
        • find_npz_files()
        • load_json_file()
        • load_toml()
        • read_npz_file()
        • resolve_single_file()
      • nicetoolbox.utils.git_utils
        • nicetoolbox.utils.git_utils.try_get_toolbox_git_metadata
        • nicetoolbox.utils.git_utils.GitMetadata
        • GitMetadata
        • try_get_toolbox_git_metadata()
      • nicetoolbox.utils.graph
        • nicetoolbox.utils.graph.topological_sort
        • topological_sort()
      • nicetoolbox.utils.hf_token
        • nicetoolbox.utils.hf_token.effective_hf_hub_token
        • effective_hf_hub_token()
      • nicetoolbox.utils.linear_algebra
        • nicetoolbox.utils.linear_algebra.distance_line_point
        • distance_line_point()
      • nicetoolbox.utils.logging_utils
        • nicetoolbox.utils.logging_utils.abbrev_list
        • nicetoolbox.utils.logging_utils.assert_and_log
        • nicetoolbox.utils.logging_utils.init_console_logging
        • nicetoolbox.utils.logging_utils.init_file_logging
        • nicetoolbox.utils.logging_utils.log_banner
        • nicetoolbox.utils.logging_utils.log_main_banner
        • nicetoolbox.utils.logging_utils.log_with_underscore
        • abbrev_list()
        • assert_and_log()
        • init_console_logging()
        • init_file_logging()
      • nicetoolbox.utils.srt
        • nicetoolbox.utils.srt.SrtWriter
        • SrtWriter
      • nicetoolbox.utils.system
        • nicetoolbox.utils.system.check_long_path_support
        • nicetoolbox.utils.system.detect_os_type
        • nicetoolbox.utils.system.normalize_ffmpeg_filter_path_in_windows
        • nicetoolbox.utils.system.system_capability_check
        • check_long_path_support()
        • detect_os_type()
        • system_capability_check()
      • nicetoolbox.utils.to_csv
        • nicetoolbox.utils.to_csv.convert_json_to_csv_files
        • nicetoolbox.utils.to_csv.convert_npz_to_csv_files
        • nicetoolbox.utils.to_csv.results_to_csv
        • convert_json_to_csv_files()
        • convert_npz_to_csv_files()
        • results_to_csv()
      • nicetoolbox.utils.triangulation
        • nicetoolbox.utils.triangulation.project_points_to_camera
        • nicetoolbox.utils.triangulation.triangulate_stereo
        • nicetoolbox.utils.triangulation.undistort_points_pinhole
        • project_points_to_camera()
        • triangulate_stereo()
        • undistort_points_pinhole()
      • nicetoolbox.utils.video
        • nicetoolbox.utils.video.frames_to_video
        • nicetoolbox.utils.video.get_ffmpeg_base_args
        • nicetoolbox.utils.video.json_to_video_info
        • nicetoolbox.utils.video.probe_video
        • nicetoolbox.utils.video.render_subtitled_track_video
        • nicetoolbox.utils.video.split_into_frames
        • nicetoolbox.utils.video.VideoInfo
        • VideoInfo
        • frames_to_video()
        • get_ffmpeg_base_args()
        • json_to_video_info()
        • probe_video()
        • render_subtitled_track_video()
        • split_into_frames()
      • nicetoolbox.utils.visual_utils
        • nicetoolbox.utils.visual_utils.gaze_arrow_pixel_length
        • nicetoolbox.utils.visual_utils.get_cam_para_studio
        • nicetoolbox.utils.visual_utils.load_calibration
        • nicetoolbox.utils.visual_utils.reproject_gaze_to_camera_view_vectorized
        • nicetoolbox.utils.visual_utils.return_2d_vector
        • nicetoolbox.utils.visual_utils.vector_to_pitchyaw
        • gaze_arrow_pixel_length()
        • reproject_gaze_to_camera_view_vectorized()
    • nicetoolbox.version
    • nicetoolbox.visual
      • nicetoolbox.visual.config_handler
        • nicetoolbox.visual.config_handler.Configuration
        • Configuration
      • nicetoolbox.visual.in_out
        • nicetoolbox.visual.in_out.IO
      • nicetoolbox.visual.media
        • nicetoolbox.visual.media.components
        • nicetoolbox.visual.media.main
        • nicetoolbox.visual.media.viewer
NICE Toolbox
  • Wiki

Wiki¶

The wiki provides detailed information about certain parts of the NICE Tolbox.

  • Understanding the config files
  • Understanding the Calibration files
  • Components
  • Algorithms
  • Evaluation Metrics
  • Connectors
  • How to manually label body joints
Previous Next

© Copyright 2026, Max Planck Society / Optics and Sensing Laboratory - Max Planck Institute for Intelligent Systems.

Built with Sphinx using a theme provided by Read the Docs.