nicetoolbox.detectors.feature_detectors.gaze_interaction.gaze_distance

GazeDistance feature detector class for 2 person gaze interaction components.

Classes

GazeDistance

The GazeDistance class is a feature detector that computes the gaze_interaction component.
class nicetoolbox.detectors.feature_detectors.gaze_interaction.gaze_distance.GazeDistance(io, data, sequence_context, algorithm_instance: str)[source]

The GazeDistance class is a feature detector that computes the gaze_interaction component.

The GazeDistance feature detector accepts two primary inputs: the gaze_individual and face_landmarks components. These components are computed using the gaze_individual and body_joints method detectors, respectively. This feature detector calculates the smallest distance between a gaze direction vector and face landmarks within a 2-person context. Additionally, it has the ability to determine whether the gaze is directed at the face and if the gaze interaction is mutual.

Initialize base detector with references.

Subclasses should call super().__init__() and set inference_config.

compute()[source]

This method computes the gaze_interaction component and saves the results as a compressed .npz file.

It calculates the Euclidean distance between gaze direction vectors and face landmarks within a 2-person context. The distance is calculated between adjacent frames, measuring the change from t to t-1. The first frame will be empty.

The method also determines whether the gaze is directed at the face (look_at) and if the gaze interaction is mutual.

The results are saved as a compressed .npz file with the following structure:

  • distance_gaze: smallest distances from the gaze vector (of person A) to the

    face (of person B), and vice versa.

  • gaze_look_at: a boolean array indicating whether the gaze is directed at the

    face

  • gaze_mutual: a boolean array indicating whether the gaze is mutual

  • data_description: A dictionary containing the data description for all of the

    above output numpy arrays. See the documentation of the output for more details.

Returns:

A list containing the distances from the gaze to the face, a boolean array indicating whether the gaze is directed at the face, and a boolean array indicating whether the gaze is mutual.

Return type:

visualization_data (list)

visualization(data)[source]

Creates visualizations for the computed gaze interaction features.

This method generates line graphs showing the distance between gaze points and face landmarks, binary graphs indicating whether the gaze is directed at the face, and binary graphs indicating whether the gaze is mutual. Additionally, it creates videos of these line graphs evolving over time.

Parameters:

data (tuple) – Output data from the compute method containing: - the distances of the gaze to the face - a boolean array indicating whether the gaze is directed at the face - a boolean array indicating whether the gaze is mutual.

Returns:

None