nicetoolbox.detectors.feature_detectors.gaze_interaction.gaze_distance.GazeDistance¶

class nicetoolbox.detectors.feature_detectors.gaze_interaction.gaze_distance.GazeDistance(io, data, sequence_context, algorithm_instance: str)[source]¶

Bases: BaseFeature

The GazeDistance class is a feature detector that computes the gaze_interaction component.

The GazeDistance feature detector accepts two primary inputs: the gaze_individual and face_landmarks components. These components are computed using the gaze_individual and body_joints method detectors, respectively. This feature detector calculates the smallest distance between a gaze direction vector and face landmarks within a 2-person context. Additionally, it has the ability to determine whether the gaze is directed at the face and if the gaze interaction is mutual.

Initialize base detector with references.

Subclasses should call super().__init__() and set inference_config.

Methods

`compute`	This method computes the gaze_interaction component and saves the results as a compressed .npz file.
`compute_output_folders`	Compute extra output folders for all components.
`compute_result_folders`	Compute result folders for all components.
`compute_viz_folders`	Compute visualization folders for all components.
`get_input_file`	Get the input file path for a specific upstream detector.
`run`	Execute feature detector: compute() + post_compute().
`visualization`	Creates visualizations for the computed gaze interaction features.

Attributes

`algorithm_type`
`components`
`predictions_mapping`	Access predictions mapping from runtime config.
`requires_out_folder`
`data`
`io`
`sequence_context`
`detector_config`
`algorithm_instance`
`inference_config`
`visualize`

compute()[source]¶

This method computes the gaze_interaction component and saves the results as a compressed .npz file.

It calculates the Euclidean distance between gaze direction vectors and face landmarks within a 2-person context. The distance is calculated between adjacent frames, measuring the change from t to t-1. The first frame will be empty.

The method also determines whether the gaze is directed at the face (look_at) and if the gaze interaction is mutual.

The results are saved as a compressed .npz file with the following structure:

distance_gaze: smallest distances from the gaze vector (of person A) to the
face (of person B), and vice versa.
gaze_look_at: a boolean array indicating whether the gaze is directed at the
face
gaze_mutual: a boolean array indicating whether the gaze is mutual
data_description: A dictionary containing the data description for all of the
above output numpy arrays. See the documentation of the output for more details.

Returns:: A list containing the distances from the gaze to the face, a boolean array indicating whether the gaze is directed at the face, and a boolean array indicating whether the gaze is mutual.
Return type:: visualization_data (list)

compute_output_folders(requires_out_folder: bool) → Dict[str, str]¶: Compute extra output folders for all components.

compute_result_folders() → Dict[str, str]¶: Compute result folders for all components.

compute_viz_folders(visualize: bool) → Dict[str, str]¶: Compute visualization folders for all components.

get_input_file(component: str, algorithm: str) → Path¶

Get the input file path for a specific upstream detector.

Parameters:

component – Component name (e.g., ‘body_joints’)
algorithm – Algorithm name (e.g., ‘hrnetw48’)

Returns:

Path to the .npz result file

property predictions_mapping¶: Access predictions mapping from runtime config.

run() → Any¶

Execute feature detector: compute() + post_compute().

Returns computed data for visualization.

visualization(data)[source]¶

Creates visualizations for the computed gaze interaction features.

This method generates line graphs showing the distance between gaze points and face landmarks, binary graphs indicating whether the gaze is directed at the face, and binary graphs indicating whether the gaze is mutual. Additionally, it creates videos of these line graphs evolving over time.

Parameters:: data (tuple) – Output data from the compute method containing: - the distances of the gaze to the face - a boolean array indicating whether the gaze is directed at the face - a boolean array indicating whether the gaze is mutual.
Returns:: None