nicetoolbox.detectors.feature_detectors.gaze_interaction.gaze_distance

GazeDistance feature detector class for 2 person gaze interaction components.

Classes

GazeDistance

The GazeDistance class is a feature detector that computes the gaze_interaction component.
class nicetoolbox.detectors.feature_detectors.gaze_interaction.gaze_distance.GazeDistance(config, io, data)[source]

The GazeDistance class is a feature detector that computes the gaze_interaction component.

The GazeDistance feature detector accepts two primary inputs: the gaze_individual and face_landmarks components. These components are computed using the gaze_individual and body_joints method detectors, respectively. This feature detector calculates the smallest distance between a gaze direction vector and face landmarks within a 2-person context. Additionally, it has the ability to determine whether the gaze is directed at the face and if the gaze interaction is mutual.

Component: gaze_interaction

components

A list containing the name of the component this class is

Type:

list

responsible for
gaze_interaction:
  • distance_gaze , distances from the gaze (of person 1) to the face

    (of person 2)

  • gaze_look_at , boolean array indicating whether the gaze is directed

    at the face

  • gaze_mutual , boolean array indicating whether the gaze is mutual

algorithm

The name of the algorithm used to compute the components (gaze_interaction).

Type:

str

gaze_detector_file_list

A list of file paths for the gaze detector output.

Type:

list

threshold_look_at

The threshold value for determining whether the gaze is directed at the face.

Type:

float

Setup the GazeDistance feature detector and extract gaze component from method detector output.

This method initializes the GazeDistance class by setting up the necessary configurations, input/output handler, and data. It supports handling of multiple cameras.

Parameters:

  • config (dict) – The configuration settings for the feature detector. It should include ‘input_detector_names’ key which contains gaze component and algorithm.

  • io (class) – The input/output handler , including ‘get_detector_output_folder’ method which returns the output folder for the gaze detector.

  • data (class) – The data class.

compute()[source]

This method computes the gaze_interaction component and saves the results as a compressed .npz file.

It calculates the Euclidean distance between gaze direction vectors and face landmarks within a 2-person context. The distance is calculated between adjacent frames, measuring the change from t to t-1. The first frame will be empty.

The method also determines whether the gaze is directed at the face (look_at) and if the gaze interaction is mutual.

The results are saved as a compressed .npz file with the following structure:

  • distance_gaze: smallest distances from the gaze vector (of person A) to the

    face (of person B), and vice versa.

  • gaze_look_at: a boolean array indicating whether the gaze is directed at the

    face

  • gaze_mutual: a boolean array indicating whether the gaze is mutual

  • data_description: A dictionary containing the data description for all of the

    above output numpy arrays. See the documentation of the output for more details.

Returns:

A list containing the distances from the gaze to the face, a boolean array indicating whether the gaze is directed at the face, and a boolean array indicating whether the gaze is mutual.

Return type:

visualization_data (list)

post_compute(data)[source]

Post-processing after computation.

This method is intended to perform any necessary post-processing tasks after the main computation method (compute) has been executed. It is designed to be overridden in derived classes to provide specific post-processing logic.

visualization(data)[source]

Creates visualizations for the computed gaze interaction features.

This method generates line graphs showing the distance between gaze points and face landmarks, binary graphs indicating whether the gaze is directed at the face, and binary graphs indicating whether the gaze is mutual. Additionally, it creates videos of these line graphs evolving over time.

Parameters:

data (tuple) – Output data from the compute method containing: - the distances of the gaze to the face - a boolean array indicating whether the gaze is directed at the face - a boolean array indicating whether the gaze is mutual.

Returns:

None