nicetoolbox.detectors.method_detectors.body_joints.mmpose_inference

Run the MMPose inference algorithm and save the results as npz files.

Functions

calculate_iou	Calculate the Intersection over Union (IoU) of two bounding boxes.
check_correct_and_sort_person_detections	Check the person detections, correct and sort them from Left to Right (based on image 2d bbox coords).
convert_output_to_numpy	Convert the output data from a pose estimation model to numpy arrays.
filter_overlapping_bboxes	Filter out bounding boxes that have high intersection-over-union (IoU) values with others, retaining only the bounding boxes with the highest confidence scores among those that overlap significantly.

nicetoolbox.detectors.method_detectors.body_joints.mmpose_inference.calculate_iou(box1, box2)

Calculate the Intersection over Union (IoU) of two bounding boxes.

Parameters:

• box1 (array_like) – Bounding box coordinates of the first box in the format [x1, y1, x2, y2]. (x1, y1) represents the top-left coordinate and (x2, y2) represents the bottom-right coordinate.

• box2 (array_like) – Bounding box coordinates of the second box in the format [x1, y1, x2, y2]. (x1, y1) represents the top-left coordinate and (x2, y2) represents the bottom-right coordinate.

Returns:

The Intersection over Union (IoU) overlap between the two bounding boxes.

Returns 0 if there is no overlap.

Return type:

float

Notes

IoU = Area of Overlap / Area of Union

nicetoolbox.detectors.method_detectors.body_joints.mmpose_inference.check_correct_and_sort_person_detections(data, num_subjects, bbox_conf_threshold=0.7, bbox_overlapping_threshold=0.9)

Check the person detections, correct and sort them from Left to Right (based on image 2d bbox coords).

1. Check bounding box confidence score of each the person detected person in each

   frame if the confidence score is bbox_conf_threshold delete this detection

2. Check if number of detected person (after the correction in previous step) is

   equal to num_subjects in dataset config subjects descriptions. if not: check if there is any overlapping bbox and delete them if exists.

3. Check again if number of detected subjects is correct.

   if yes: sort the person detections from left to right in image 2d coordinates (based on bbox top_left_x coord) if no: save the results of previous frame

Args

data (list of dict): A list where each element is frame results. MMpose

inference result. ### Explanation about results structure of mmpose ### ### frame[‘predictions’][0] # list of dict. Each detect is a detected ### person. ### the keys for each person dictionary: # keypoints: [[x1,y1], [x2,y2], [x3,y3], …, [xn,yn]], coords. of # keypoints, where n = # of keypoints (i.e., n=133 for coco-wholebody) # keypoint_score: [c1, c2, c3, …, cn], confidence score of keypoints # (min=0.0, max=1.0), n = # of keypoints # bbox: ([x1,y1,x2,y2]), corners of bbox, x1y1 is top left corner # bbox_score: int, confidence score of bbox (person detection

num_subjects (integer): An Integer that defines the number of expected subjects

in dataset.

bbox_conf_threshold (float): Threshold < 1. The person detections whose

bounding boxesconfidence level is below this threshold, will be removed. The default is 0.7.

bbox_overlapping_threshold (float): Threshold < 1.0. If the IoU of two bounding

boxes exceeds this threshold, the bounding box with the lower confidence score is marked for removal. The default is 0.8.

Returns:

A list of corrected and sorted frame results.

Return type:

updated_frame_predictions_list (list of dict)

nicetoolbox.detectors.method_detectors.body_joints.mmpose_inference.convert_output_to_numpy(data, num_persons)

Convert the output data from a pose estimation model to numpy arrays.

The output has the following structure: - 2d: Numpy array of shape

(num_persons, num_frames, num_keypoints, [coordinate_x, coordinate_y, confidence_score])

• bbox_2d: Numpy array of shape

  (num_persons, num_frames, 1, [top_left_x, top_left_y, bottom_right_x, bottom_right_y, confidence_score])

• data_description: A dictionary containing the description of the data.

Parameters:

• data (list) – The output data from the pose estimation model.

• num_persons (int) – The number of persons detected in the data.

Returns:

A tuple containing the keypoints array, bbox array, and data description.

Return type:

tuple

nicetoolbox.detectors.method_detectors.body_joints.mmpose_inference.filter_overlapping_bboxes(bboxes, confidence_scores, overlapping_threshold)

Filter out bounding boxes that have high intersection-over-union (IoU) values with others, retaining only the bounding boxes with the highest confidence scores among those that overlap significantly.

Parameters:

• bboxes (list of lists) – A list where each element is a bounding box defined as a list of four integers [x1, y1, x2, y2], where (x1, y1) are the coordinates of the top-left corner, and (x2, y2) are the coordinates of the bottom-right corner of the bounding box.

• confidence_scores (list of float) – A list of confidence scores corresponding to each bounding box in bboxes.

• overlapping_threshold (float) – A float number (< 1) which defines the threshold. If the IoU of two bounding boxes exceeds this threshold, the bounding box with the lower confidence score is marked for removal. The default is 0.8.

Returns:

The indices of the bounding boxes that are kept,

referring to their positions in the original bboxes list.

Return type:

keep_indices (list of int)